AI Infrastructure

Multi-Tenant GPU Hosting: Shared Infrastructure for AI Workloads

GPU servers are expensive. An eight-GPU H100 SXM5 node costs between $250,000 and $400,000 to purchase and draws 10-12 kW of power. Running one at full capacity for AI training makes economic sense. Running one at 15% utilization for sporadic inference requests does not. Multi-tenant GPU hosting solves this by allowing multiple customers to share the same physical GPU infrastructure, each receiving an isolated slice of compute resources -- paying only for what they use while the hosting provider maximizes hardware utilization and amortizes costs across tenants.

How Multi-Tenant GPU Hosting Works

Multi-tenant GPU hosting divides physical GPU resources among multiple independent customers using one or more isolation technologies. The goal is to give each tenant guaranteed, predictable performance without interference from other tenants' workloads -- the same principle that makes cloud virtual machines reliable, applied to GPU compute.

NVIDIA MIG (Multi-Instance GPU)

NVIDIA's Multi-Instance GPU technology is the gold standard for GPU-level multi-tenancy. Available on A100 and H100 GPUs, MIG physically partitions a single GPU into up to seven independent instances, each with its own dedicated compute units (Streaming Multiprocessors), memory bandwidth, and L2 cache. The isolation is enforced in hardware, not software -- a tenant cannot access another tenant's memory or interfere with their compute.

An H100 with 80 GB of HBM3 memory can be partitioned into seven instances of approximately 10 GB each, or fewer larger instances (for example, two 40 GB instances or a combination of sizes). Each instance behaves like a separate, smaller GPU from the tenant's perspective, with its own CUDA device ID, error isolation, and QoS guarantees.

Time-Sharing Schedulers

Before MIG, multi-tenant GPU access relied on time-sharing: a scheduler allocates GPU time slices to different tenants in round-robin or priority-based order. This provides weaker isolation than MIG -- a tenant's workload may experience latency spikes when other tenants' tasks are running -- but works on older GPU hardware that does not support MIG (V100, T4, consumer GPUs).

Kubernetes-based GPU schedulers (like Run:ai, NVIDIA GPU Operator, and custom kube-scheduler plugins) implement time-sharing at the orchestration layer, assigning fractional GPU requests to pods. This is practical for development environments and non-latency-sensitive batch workloads but generally unsuitable for production inference where consistent response times matter.

Virtual GPU (vGPU)

NVIDIA's vGPU software creates virtual GPUs that can be assigned to virtual machines, providing VM-level isolation with GPU pass-through. Each vGPU receives a fixed allocation of GPU memory and compute, managed by a hypervisor. This approach integrates with existing VM-based infrastructure (VMware, KVM) and is well-suited for enterprises that already run virtualized environments. The trade-off is higher overhead (5-15% performance penalty from the virtualization layer) compared to bare-metal MIG.

Isolation: The Make-or-Break Factor

Multi-tenant GPU hosting only works if tenants cannot interfere with each other. Isolation must operate at three levels simultaneously.

Compute Isolation

Each tenant's GPU workload must receive guaranteed compute resources that are not affected by other tenants. MIG achieves this by assigning dedicated Streaming Multiprocessors -- if your instance has 14 SMs, those 14 SMs run your workload exclusively, regardless of what the other six instances are doing. Time-sharing provides weaker guarantees: your workload gets its full time slice, but may experience context-switching overhead.

Memory Isolation

GPU memory must be strictly partitioned. In a correctly configured MIG setup, tenant A cannot read, write, or even detect tenant B's memory allocation. This is critical for workloads handling proprietary models, medical imaging data, financial records, or any other sensitive information. Memory isolation prevents side-channel attacks that could leak model weights or inference data.

Network Isolation

At the data center level, each tenant's network traffic must be segmented using VLANs, VXLANs, or network micro-segmentation to prevent one tenant from sniffing another's API calls, model weights, or training data in transit. Providers typically assign dedicated virtual network interfaces per tenant with firewall rules enforced at the switch level.

Pricing Models for Multi-Tenant GPU

Model	How It Works	Best For	Typical Cost Range
Per-GPU-Hour	Pay for fractional GPU time used	Inference, batch processing, sporadic workloads	$0.50 - $3.50/GPU-hr (MIG slice)
Reserved Instance	Commit to a fixed MIG partition for a term	Steady-state inference, production APIs	30-50% discount vs. on-demand
Per-Request	Pay per inference call or token processed	Variable-traffic APIs, startups	$0.001 - $0.05 per 1K tokens
Subscription	Monthly fee for a guaranteed GPU partition	SMBs with predictable workloads	$300 - $2,000/month per slice

The economic advantage of multi-tenant hosting is straightforward: a customer running inference at 20% GPU utilization pays for 20% of the GPU (or a proportional MIG slice), while the provider fills the remaining 80% with other tenants. Both sides win -- the customer pays less than a dedicated server, and the provider earns more total revenue per GPU than a single dedicated tenant would generate.

For a detailed comparison of GPU pricing models, see our colocation pricing models guide.

When Multi-Tenant Makes Sense (and When It Does Not)

Good Fit: Multi-Tenant GPU

• AI inference serving with moderate traffic (under 100 requests/second per model). Most inference workloads use a fraction of a full GPU's capacity.
• Fine-tuning small models (under 7B parameters) where a MIG slice with 10-20 GB memory is sufficient for LoRA or QLoRA adapters.
• Development and prototyping where engineers need GPU access for testing but not sustained full-GPU training.
• Batch inference with scheduled windows -- process overnight, release resources during the day.
• Startups and SMBs that cannot justify $250K+ for a dedicated GPU node but need production-grade GPU compute.

Better as Dedicated

• Large-scale training (70B+ parameter models) that requires full GPU memory, NVLink interconnects between multiple GPUs, and sustained 90%+ utilization for days or weeks.
• High-throughput inference where response latency is critical and any resource contention is unacceptable (real-time trading, autonomous systems).
• Strict compliance workloads where regulations explicitly prohibit shared hardware (some government and defense contracts).
• Workloads that need the full 80 GB of H100 HBM3 memory for large model weights or large batch sizes.

Choosing a Multi-Tenant GPU Hosting Provider

Not all multi-tenant GPU offerings are equal. When evaluating providers, focus on these criteria:

1. Isolation technology: MIG-based isolation is the standard for production. Time-sharing is acceptable only for dev/test. Ask explicitly how GPU resources are partitioned.
2. SLA guarantees: Verify that the SLA specifies per-tenant performance guarantees, not just aggregate uptime. A noisy-neighbor problem that degrades your inference latency by 3x is effectively downtime.
3. Security certifications: SOC 2 Type II, ISO 27001, and per-tenant encryption at rest and in transit. Ask for third-party audit reports, not self-assessments.
4. GPU hardware generation: A100 and H100 support MIG natively. Older hardware relies on weaker isolation mechanisms. Newer GB200 systems offer enhanced partitioning capabilities.
5. Network architecture: Dedicated virtual network per tenant, with firewall rules enforced at the infrastructure level (not just application-level security groups).
6. Scaling path: Can you start with a MIG slice and scale to a full dedicated node as your workload grows? Providers with both multi-tenant and dedicated options offer the smoothest scaling path.

For a comprehensive provider evaluation checklist, refer to our AI hosting provider selection guide.

The Economics: Multi-Tenant vs. Dedicated vs. Cloud

For a typical inference workload using 20% of an H100's capacity:

Option	Monthly Cost	Effective $/GPU-hr	Utilization
Cloud GPU (on-demand)	$2,500 - $4,000	$3.50 - $5.50	20% (paying for 100%)
Dedicated colocation	$1,800 - $3,000	$2.50 - $4.00	20% (paying for 100%)
Multi-tenant (MIG slice)	$400 - $800	$0.55 - $1.10	~100% of allocated slice

The multi-tenant option costs 70-80% less for the same effective compute because you are paying for a right-sized slice rather than an entire GPU you cannot fully utilize. The savings compound at scale: an organization running 10 inference models that each need 10% of an H100 would pay for 10 full GPUs in a dedicated model versus 1-2 GPUs worth of MIG slices in a multi-tenant model.

For a deeper dive into the build-vs-buy analysis, see our GPU-as-a-Service economics guide and bare-metal vs. cloud comparison.