High-Density Colocation: Power Per Rack for AI and HPC Workloads

Why Power Density Has Become the Defining Metric in Colocation

For decades, data center capacity was measured in square footage. A rack drew 3–5 kW, cooling was handled by raised-floor air conditioning, and planning focused on how many racks could physically fit in a room. That model is obsolete for anyone deploying AI training clusters, large-scale inference, or high-performance computing workloads.

Today, the limiting factor is power per rack. A single NVIDIA DGX H100 system draws roughly 10 kW. A full rack of GPU servers can exceed 60 kW. The latest NVIDIA GB200 NVL72 platform pushes a single rack past 120 kW — more than an entire row of traditional servers. Colocation providers that cannot deliver this density simply cannot serve AI customers, regardless of how much floor space they have available.

This shift has split the colocation market into two distinct categories: standard-density facilities serving traditional enterprise IT, and high-density facilities purpose-built for GPU colocation and compute-intensive workloads. Understanding where your requirements fall — and what infrastructure each tier demands — is essential for choosing the right facility.

Power Density Tiers: From Standard IT to Extreme AI

Not every workload needs 100 kW per rack. Power density requirements vary enormously depending on the type of compute being deployed. The following tiers represent the practical categories that operators and enterprises work with in 2026.

Density Tier	Power per Rack	Typical Workloads	Cooling Method
Standard	3–10 kW	Web servers, storage, enterprise apps	Air cooling (CRAH/CRAC)
Medium	10–20 kW	AI inference, database clusters, CDN	Enhanced air or rear-door heat exchangers
High	20–50 kW	Multi-GPU servers, HPC, AI fine-tuning	Direct-to-chip liquid cooling
Ultra-High	50–100 kW	Dense GPU training clusters (H100/H200)	Liquid cooling or immersion
Extreme	100–150+ kW	GB200 NVL72, frontier AI training	Full liquid cooling infrastructure

The average rack density across the colocation industry has risen to approximately 27 kW as of early 2026, reflecting the rapid shift toward GPU-heavy deployments. However, averages obscure the reality: many new builds are designed for 40 kW and above from day one, while legacy facilities struggle to retrofit beyond 15 kW per rack.

What Drives Rack-Level Power Consumption

Understanding where the watts go inside a high-density rack helps explain why these numbers are so much higher than traditional IT.

GPU Accelerators

GPUs are the dominant power consumer. A single NVIDIA H100 SXM module draws up to 700 W. An 8-GPU server (such as a DGX H100 or HGX H100 platform) therefore draws roughly 5–6 kW in GPU power alone, before accounting for CPUs, memory, storage, and networking. The H200 draws a similar envelope, while the B200 and GB200 push individual GPU power toward 1,000 W and beyond.

High-Speed Networking

InfiniBand switches and host channel adapters add meaningful power overhead. A 64-port NDR InfiniBand switch draws approximately 2,500 W. In a dense GPU cluster, one or two such switches per rack add 2.5–5 kW to the total. High-speed Ethernet (400GbE) switches draw similarly.

CPUs, Memory, and Storage

Host CPUs, DRAM, and NVMe storage add another 1–3 kW per server depending on configuration. While modest relative to GPU power, these components collectively push the total rack draw higher — and they generate heat that must be managed alongside the GPUs.

Power Distribution Overhead

Every watt that reaches the server passes through power distribution units (PDUs), UPS systems, and transformers, each introducing conversion losses. In a well-designed facility, this overhead adds 8–15% to the IT load. In older facilities with less efficient power paths, losses can exceed 20%.

Infrastructure Requirements for High-Density Racks

Delivering 40, 60, or 100+ kW to a single rack is not simply a matter of running larger power cables. It requires coordinated upgrades across five infrastructure domains.

Power Distribution

Standard colocation racks are fed by 20–30 A circuits at 208 V, delivering roughly 4–6 kW per circuit. High-density racks require 60–100 A circuits at 415 V three-phase, with multiple feeds per rack for redundancy. Busway distribution systems running above the racks provide the flexibility to tap off high-amperage feeds without re-cabling. PDUs must be rated for the full rack load with per-outlet monitoring to prevent overloading individual circuits.

Cooling

Air cooling hits practical limits around 20–25 kW per rack, even with hot-aisle containment and high-flow CRAH units. Above that threshold, liquid cooling becomes necessary. Direct-to-chip (cold plate) cooling is the most common approach for GPU servers, where chilled water circulates through plates attached directly to GPUs and CPUs. Rear-door heat exchangers capture remaining air-borne heat. For extreme densities above 80 kW, some operators turn to full immersion cooling, submerging servers in dielectric fluid.

Structural Floor Loading

High-density racks are heavy. A fully populated GPU rack with liquid cooling manifolds can weigh 1,500–2,500 kg (3,300–5,500 lbs). Standard raised floors designed for 250 lbs/sq ft cannot support this. Purpose-built facilities use slab-on-grade construction with reinforced concrete rated for 500+ lbs/sq ft.

Fire Suppression

Liquid cooling introduces water into the data hall, which changes fire-suppression requirements. Clean-agent gas systems (such as FM-200 or Novec 1230) remain the standard for IT equipment, but liquid-cooled environments must also account for potential water leaks with leak-detection systems under raised floors and within cooling distribution units.

Monitoring and Management

At 100 kW per rack, a single point of failure — a stuck valve in a cooling loop, a tripped breaker, a failed PDU — can take down hundreds of thousands of dollars of compute in seconds. Data center infrastructure management (DCIM) systems must provide real-time visibility into power draw, coolant temperatures, flow rates, and environmental conditions at the individual rack level, with automated alerting and, where possible, automated failover.

How to Evaluate a High-Density Colocation Provider

Not all providers claiming "high-density" capabilities deliver the same quality of service. When evaluating a facility for AI or HPC colocation, these are the questions that matter most.

Power Questions

• What is the maximum committed power per rack? Is it contractually guaranteed?
• What voltage and phase configuration is available? (415 V three-phase is optimal for high density.)
• What redundancy configuration is in place — N+1 or 2N?
• What is the facility's total available power, and how much is already committed?

Cooling Questions

• What cooling methods are deployed at the rack level? (Ask specifically about liquid cooling infrastructure.)
• What is the coolant supply temperature, and can it be adjusted for GPU-optimized operating ranges?
• Is the cooling capacity matched to the power capacity per rack, or is there an air gap?

Network and Connectivity

• Is InfiniBand supported within the facility?
• What cross-connect options exist for connecting to cloud on-ramps, internet exchanges, and private peering?
• Can the provider support fat-tree or rail-optimized network topologies for GPU clusters?

Operational Maturity

• What tier certification does the facility hold?
• Does the provider have experience deploying GPU clusters at the density you require?
• What SLA commitments are offered for power and cooling uptime?

The UAE Advantage for High-Density Colocation

The UAE has emerged as a strategic location for high-density colocation, driven by several structural factors that align particularly well with AI and HPC infrastructure needs.

Competitive energy pricing provides a direct cost advantage for power-intensive deployments. Since electricity is the single largest operating expense for high-density compute, a lower per-kWh rate translates directly to better unit economics. The Rax Energy division works to optimize power procurement specifically for high-density workloads.

The country's strategic position between Europe, Asia, and Africa offers low-latency connectivity to multiple major markets. For enterprises serving global AI inference workloads, a UAE data center can reach billions of users with competitive round-trip times.

National investment in AI infrastructure, combined with a clear regulatory framework, has attracted data center operators and hyperscalers to build new capacity in the region. Much of this new build is designed from day one for high-density AI workloads, avoiding the retrofit limitations that constrain legacy facilities elsewhere.

Matching Your Workload to the Right Density Tier

Overprovisioning power is expensive. Underprovisioning leaves performance on the table or prevents deployment entirely. Matching your workload to the right density tier requires an honest assessment of your current and near-future compute requirements.

• AI inference at moderate scale: Most inference deployments run comfortably at 10–20 kW per rack, using GPUs like the L40S or A100 with lower power envelopes than training-class hardware.
• AI fine-tuning and small-scale training: Fine-tuning large language models or training smaller models typically requires 20–50 kW per rack with 4- or 8-GPU servers.
• Large-scale AI training: Training frontier models demands 50–100+ kW per rack, with dense GPU clusters connected by InfiniBand and served by high-performance parallel storage.
• ASIC mining and blockchain compute: ASIC hosting deployments vary widely, from 10 kW per rack for efficient next-generation miners to 30+ kW for older, power-hungry models.
• Traditional HPC (simulation, rendering, genomics): CPU-heavy HPC workloads typically fall in the 15–30 kW range, though GPU-accelerated HPC pushes higher.

The Cost Equation: Power Density and Total Cost of Ownership

Higher density typically reduces total cost of ownership (TCO) per unit of compute, even though the per-rack cost is higher. The math is straightforward: a rack delivering 60 kW of GPU compute in a purpose-built facility costs less per GPU-hour than spreading the same compute across six 10 kW racks in a standard facility — because you need fewer racks, less floor space, fewer network connections, and less operational overhead.

Colocation pricing for high-density deployments is typically structured per kW per month, ranging from $100 to $200 depending on location, contract length, and density tier. At scale, power cost (the per-kWh electricity charge) often exceeds the colocation fee itself, making energy pricing the single most important variable in long-term TCO. For a detailed analysis of power costs and their impact on operations, see our guide on electricity costs and ROI.

FAQ: High-Density Colocation Power Requirements

How many kW per rack does an AI GPU server need?

A single 8-GPU server based on NVIDIA H100 SXM draws approximately 10 kW. A full rack containing multiple servers typically ranges from 30 kW to 60 kW. Next-generation platforms like the NVIDIA GB200 NVL72 push a single rack to 120–140 kW. The exact figure depends on the GPU model, server configuration, and networking equipment in the rack.

What cooling is required for high-density colocation above 40 kW per rack?

Above approximately 40 kW per rack, air cooling alone becomes insufficient. Direct-to-chip liquid cooling or rear-door heat exchangers are standard at 40–80 kW. Above 80 kW, most deployments use direct liquid cooling or full immersion cooling to manage the thermal load safely and efficiently. Our liquid cooling guide covers these technologies in detail.

What is the difference between high-density and standard colocation?

Standard colocation supports 5–10 kW per rack and uses traditional air cooling, suitable for general enterprise IT. High-density colocation supports 20 kW and above per rack, with reinforced power distribution, advanced cooling (liquid or hybrid), stronger floor loading, and higher-capacity circuit breakers designed for GPU, AI, and HPC workloads.

How much does high-density colocation cost per kW?

High-density colocation typically costs between $100 and $200 per kW per month, depending on location, density tier, contract length, and included services. Facilities in regions with lower power costs, such as the UAE or parts of the US, often sit at the lower end of this range.