Multi-tenancy: How One Server Hosts Many Customers Safely

Multi-tenancy is a fundamental concept behind every hosting business, including cloud services. It involves running multiple customer websites or applications on a single physical machine while ensuring they are isolated from each other to prevent any one tenant from disrupting the others.

Overview

The goal of multi-tenancy is to provide a safe and fair environment for all customers. This is achieved by setting up isolation boundaries, which can vary in strength:

• Process/Account Isolation (Classic Shared Hosting): Each customer has their own Unix user account with a home directory, and the web server runs each site as its respective user.
• Containers: Each tenant operates within their own namespace jail, having their own filesystem, process space, and resource limits (CPU/RAM quotas).
• Virtual Machines (VMs): Each tenant gets an entire virtual computer with its own kernel.
• Dedicated Hardware: The isolation is enforced by the physical hardware itself.

Isolation Boundaries

1. Process/Account Isolation

• Description: Every customer has their own Unix user account, and the web server runs each site as that user.
• Pros: Cheapest option with high density.
• Cons: Weakest isolation; a kernel exploit or misconfigured permissions could allow one tenant to affect others.

2. Containers

• Description: Each tenant is in their own namespace jail, providing their own filesystem, process space, and resource limits (CPU/RAM quotas).
• Pros: Stronger isolation than shared hosting but still shares the host kernel.
• Cons: Still vulnerable to kernel exploits and can be impacted by noisy neighbors.

3. Virtual Machines

• Description: Each tenant gets a complete virtual computer with its own kernel.
• Pros: Strongest isolation, hardware-enforced boundaries at the cost of additional resource overhead.
• Cons: Higher costs due to the additional layer of virtualization.

4. Dedicated Hardware

• Description: The boundary is defined by physical hardware itself, providing the strongest possible isolation.
• Pros: Maximum security and compliance.
• Cons: Highest cost due to dedicated resources per tenant.

Ensuring Fairness and Security

Beyond setting up these boundaries, several other measures are necessary to ensure a safe and fair environment:

Resource Fairness

• Description: Implement quotas and monitoring to prevent one tenant from overusing resources and causing outages.
• Example: Monitoring tools can alert administrators if a tenant is using too much CPU or RAM.

Network Isolation

• Description: Ensure tenants cannot sniff or reach each other's traffic. This involves VLANs, private bridges, and per-tenant firewall rules.
• Example: Use virtual LANs (VLANs) to separate network segments for different tenants.

Data Isolation

• Description: Maintain separate databases or at least separate credentials for each tenant.
• Example: Multi-tenant applications might have bugs that allow one tenant to query another's data, so careful database design is crucial.

Blast Radius Planning

• Description: Plan how a single host failure will impact tenants. This involves designing the system to minimize disruptions and ensuring redundancy where necessary.
• Example: Designing the system to handle a single host failure without affecting other hosts or tenants.

Practical Example: Our Homelab

At BreadLab, we started with process/account isolation for our initial shared hosting setup. As we scaled and began hosting more diverse services, we transitioned to container-based isolation. Our next phase (Phase 4) will involve virtual machines on Proxmox, providing stronger boundaries for strangers while maintaining the flexibility of a homelab environment.

By understanding these concepts, you can better appreciate the challenges and solutions involved in multi-tenancy hosting, whether you're a small operator or a large cloud provider.