Why Power Density Matters
Power density per rack is the single variable that determines more about a data center's design than any other factor. It dictates the cooling architecture, the electrical distribution topology, the structural engineering of the floor, the fire suppression strategy, and ultimately the cost per kilowatt of deployed IT capacity. A facility designed for 5 kW racks and a facility designed for 100 kW racks are fundamentally different buildings, even if they deliver the same total megawatts of IT power.
The industry has moved through these density tiers over decades, but the arrival of GPU-accelerated computing has compressed what would normally be a 15-year evolution into a 3-year sprint. Facilities built in 2022 for 10 to 15 kW racks are already being retrofitted or replaced to handle 60+ kW racks for AI training workloads. Understanding each tier, its infrastructure demands, and the transition points between them is essential for any operator planning colocation capacity today.
This guide walks through five distinct density classifications, the workloads that drive each one, the cooling and power infrastructure each tier requires, and the cost implications of building or procuring capacity at each level.
Tier 1: Standard Density (5–8 kW Per Rack)
Typical Workloads
Standard density racks host the traditional enterprise IT footprint: 1U and 2U servers with one or two CPUs per node, storage arrays, network switches, and appliances. A typical 42U rack at this tier might contain 15 to 20 servers, each drawing 300 to 500 watts, plus networking and cabling overhead. This is the density profile of legacy enterprise data centers, managed hosting providers, and most multi-tenant colocation facilities built before 2020.
Cooling Requirements
Air cooling handles this tier comfortably. Perimeter CRAC (Computer Room Air Conditioning) or CRAH (Computer Room Air Handler) units supply chilled air through a raised floor plenum, with perforated tiles distributing airflow to the cold aisle. Hot-aisle/cold-aisle containment improves efficiency but is not strictly required at these densities. A standard raised-floor design with 18-inch plenum depth and perimeter cooling delivers adequate airflow for 5 to 8 kW racks without any supplemental cooling at the rack level.
Power Distribution
Single-phase 120V or 208V power circuits are standard. Each rack receives one or two 20A or 30A branch circuits from the floor PDU (Power Distribution Unit), providing 3.3 to 5.7 kW per circuit. Dual-corded servers connect to A and B power feeds from separate PDUs for redundancy. Cable management is straightforward, with standard whip cables running from overhead or underfloor ladder trays to each rack position.
Cost Profile
This is the most widely available and least expensive colocation tier. Infrastructure buildout costs range from $8,000 to $12,000 per kW. Colocation pricing in mature markets runs $100 to $180 per kW per month, reflecting the commodity nature of this capacity. The facilities are plentiful, the designs are well-understood, and the supply chain is mature.
Tier 2: Medium Density (8–15 kW Per Rack)
Typical Workloads
Medium density racks reflect modern CPU-intensive computing: current-generation dual-socket servers with Intel Xeon Scalable or AMD EPYC processors drawing 600 to 1,200 watts per node, high-performance storage (NVMe all-flash arrays), virtualization clusters running at high utilization, and database servers with large memory configurations. A rack at 10 to 15 kW might contain 10 to 14 high-performance 2U servers, each with two 300W+ CPUs, 512 GB or more of DDR5 memory, and local NVMe storage.
This tier also includes single-GPU inference servers, such as systems with one NVIDIA L40S or A30 accelerator per node, where the GPU adds 150 to 300 watts per server. These edge-inference and light-AI workloads push rack density into the upper range of this tier without requiring the full GPU infrastructure of higher tiers.
Cooling Requirements
Air cooling remains viable but requires more deliberate design. Hot-aisle/cold-aisle containment transitions from optional to mandatory. In-row cooling units (CRAHs placed between racks rather than at the room perimeter) become the preferred approach because they shorten the air path and reduce the mixing of hot and cold air streams. Raised-floor plenum depth may need to increase to 24 to 36 inches to provide adequate static pressure for higher airflow volumes. Variable-speed fan drives on CRAH units improve efficiency by matching airflow to actual heat load rather than running at full speed continuously.
At the upper end of this tier (13 to 15 kW), operators begin to encounter hot spots where individual racks or rack positions exceed the local cooling capacity. Blanking panels in empty rack units, cable management that does not obstruct airflow, and careful placement of high-power equipment within the rack become important operational practices.
Power Distribution
The transition from single-phase to three-phase power distribution occurs in this tier. Racks at 10+ kW benefit from 208V three-phase circuits, which deliver more power per conductor and reduce the number of circuits required per rack. A single 30A three-phase circuit provides approximately 8.6 kW, and a 60A three-phase circuit provides approximately 17.3 kW. Intelligent PDUs with per-outlet monitoring become valuable for tracking actual power consumption and identifying capacity headroom per rack.
Cost Profile
Medium density colocation is available in most major markets but requires facilities with more capable cooling and power distribution infrastructure. Buildout costs range from $7,000 to $11,000 per kW. Monthly colocation pricing ranges from $120 to $200 per kW, reflecting the need for containment, in-row cooling, and three-phase power distribution.
Tier 3: High Density (15–30 kW Per Rack)
Typical Workloads
High density racks mark the entry point for meaningful GPU compute. This tier hosts multi-GPU inference servers (2 to 4 GPUs per node), dense CPU clusters for HPC and scientific computing, and mixed AI/ML workloads that combine GPU acceleration with substantial CPU and memory resources. A rack at 20 to 30 kW might contain four to six 4U GPU servers, each with two to four NVIDIA A100 PCIe or L40S accelerators drawing 300 watts each, plus the associated CPU, memory, and networking power draw.
This tier also includes the upper range of ASIC cryptocurrency mining racks, dense storage racks with dozens of high-speed NVMe drives, and telecommunications equipment for network core and edge deployments.
Cooling Requirements
This is the tier where air cooling reaches its practical limits. At 15 to 20 kW per rack, aggressive hot-aisle containment with in-row cooling can still work, but the airflow volumes required start to create noise issues, increase fan energy consumption, and demand very precise airflow management. Above 20 kW, the economics and physics of air cooling deteriorate rapidly.
Rear-door heat exchangers (RDHx) become the bridge technology in this tier. An RDHx unit mounts on the back of a standard rack and uses chilled water circulating through a coil to intercept the hot exhaust air before it enters the return plenum. A well-sized RDHx can neutralize 20 to 40 kW of heat per rack, effectively allowing existing air-cooled infrastructure to support significantly higher densities without a full cooling system redesign. The trade-off is the need for a chilled water distribution loop to each rack row, which may not exist in legacy facilities.
For a deeper technical analysis of cooling approaches at these densities, see our complete guide to direct liquid cooling for GPU data centers.
Power Distribution
Three-phase power distribution at 208V is standard throughout this tier. At the upper end (25 to 30 kW), operators begin deploying 415V three-phase distribution to further reduce current and cable gauge requirements. Intelligent rack PDUs with network management cards provide real-time power monitoring, remote outlet control, and integration with data center infrastructure management (DCIM) platforms. Power redundancy requirements increase: 2N power feeds (fully redundant A+B paths) become the norm rather than the exception.
Cost Profile
High density colocation is less widely available than standard or medium density. Not all colocation providers can support 20+ kW racks, and those that can charge a premium for the specialized infrastructure. Buildout costs range from $7,000 to $10,000 per kW, with the water distribution infrastructure for RDHx units adding $500 to $1,500 per kW. Monthly pricing ranges from $150 to $250 per kW.
Tier 4: Ultra-High Density (30–60 kW Per Rack)
Typical Workloads
Ultra-high density racks are purpose-built for GPU-accelerated computing at scale. This tier hosts NVIDIA HGX H100 and H200 systems (8-GPU SXM servers drawing 10 to 12 kW per node), dense AI inference clusters with multiple GPU nodes per rack, and high-performance computing systems with GPU co-processors. A typical configuration at 40 to 50 kW per rack might contain four HGX H100 nodes in a 4U form factor, plus top-of-rack networking (InfiniBand or 400GbE switches), power distribution overhead, and management infrastructure.
This is also the density range for purpose-built AI inference racks running workloads like large language model serving, recommendation systems, and real-time computer vision processing. Inference deployments at this tier prioritize throughput per rack unit over training cluster interconnect topology.
Cooling Requirements
Air cooling is no longer viable as the primary cooling mechanism at these densities. The transition to liquid cooling is mandatory. Direct-to-chip liquid cooling is the standard approach: cold plates mounted directly on GPU and CPU dies circulate water or water-glycol solution through an in-rack manifold connected to a coolant distribution unit (CDU). The CDU exchanges heat between the rack-side coolant loop and the facility water loop.
At 30 to 60 kW per rack, direct-to-chip cooling handles 70 to 80 percent of the total heat load through the liquid loop. The remaining 20 to 30 percent (from memory, VRMs, SSDs, and other components not on the cold plate) is still rejected to air and must be managed by supplemental air cooling, typically in-row CRAH units or overhead fan-coil units. This hybrid approach (liquid for compute components, air for everything else) is the dominant design pattern for H100-class deployments.
CDU footprint planning becomes a critical facility design consideration. Each CDU occupies 2 to 4 rack footprints and serves 4 to 8 compute racks depending on capacity. The facility must provide chilled water supply and return piping rated for the total CDU heat rejection load, with redundancy to allow CDU maintenance without disrupting compute operations.
Power Distribution
Three-phase 415V distribution is standard. At 40+ kW per rack, the current draw at lower voltages would require prohibitively large conductors and generate excessive I2R losses. Overhead busway distribution systems begin to replace traditional whip cables in new builds and major retrofits. A busway runs above the rack row, with tap-off boxes providing power connections at each rack position. This simplifies cable management, enables rapid reconfiguration of power allocation between racks, and reduces the number of panel boards required in the electrical room.
Intelligent PDUs at this tier provide per-circuit and per-outlet power metering, environmental monitoring (temperature, humidity, airflow), and automated load balancing across phases. Power factor correction becomes important as the switching power supplies in GPU servers can introduce harmonic distortion on three-phase circuits.
For more on power redundancy design at these levels, see our article on data center power redundancy architectures.
Cost Profile
Ultra-high density colocation is a specialized market served by a smaller number of providers with purpose-built or significantly retrofitted facilities. Buildout costs range from $6,000 to $10,000 per kW, with liquid cooling infrastructure (CDUs, piping, manifolds, leak detection) representing 15 to 25 percent of total mechanical costs. Monthly colocation pricing ranges from $140 to $220 per kW, with the lower per-kW pricing reflecting the efficiency gains of amortizing fixed facility costs over a higher IT load per rack.
Tier 5: Hyperscale GPU Density (60–100+ kW Per Rack)
Typical Workloads
The hyperscale tier exists almost exclusively for large-scale AI training clusters. The defining system is the NVIDIA GB200 NVL72, a liquid-cooled rack-scale system that integrates 36 Grace CPUs and 72 Blackwell GPUs into a single rack drawing approximately 120 kW in reference configuration and up to 132 kW with full networking and storage accessories. Other systems in this tier include dense H200 configurations at 60 to 70 kW per rack and custom AI training platforms built by hyperscale operators for specific model architectures.
These racks are the compute engines behind the largest foundation model training runs. A single GB200 NVL72 rack delivers approximately 1.4 exaFLOPS of FP4 tensor performance. Training runs for frontier models deploy hundreds to thousands of these racks, creating data center power demands measured in hundreds of megawatts.
Cooling Requirements
Direct-to-chip liquid cooling is a hard engineering requirement, not an optimization choice. The GB200 NVL72 ships with integrated cold plates and rack manifolds, and its thermal design assumes facility water supply at 25 to 45 degrees Celsius. No air-cooled configuration exists. The liquid cooling loop handles 85 to 90 percent or more of total heat rejection, with minimal residual air cooling for ancillary components.
At these densities, the facility-side cooling infrastructure becomes the primary design constraint. A 10 MW training cluster with 80+ racks at 120 kW each requires approximately 8.5 to 9.5 MW of heat rejection capacity from the liquid cooling plant. Dry coolers, adiabatic coolers, or district cooling connections must be sized for this load with N+1 redundancy. The coolant distribution network (headers, branch piping, isolation valves, and monitoring) is a major infrastructure system comparable in complexity and cost to the electrical distribution.
Immersion cooling enters the conversation as an alternative at the extreme end of this tier. Single-phase immersion systems can handle 100 to 250+ kW per tank, providing total heat capture to liquid and eliminating all fans from the IT equipment. Two-phase immersion, while more expensive, achieves PUE values of 1.02 to 1.03 and is particularly attractive in hot-climate deployments where air-side economizer hours are limited. However, immersion requires custom server form factors and has implications for serviceability that make it less mainstream than direct-to-chip for most GPU deployments today.
Power Distribution
The electrical distribution for hyperscale GPU racks is a specialized discipline. Three-phase 415V power via overhead busway is the standard topology. Each rack position requires 150+ amperes of three-phase capacity to support the 120+ kW IT load plus PDU losses. The electrical room must provide medium-voltage to low-voltage transformation, UPS capacity, and switchgear rated for the aggregate load, with static transfer switches (STS) or distributed redundant UPS architectures providing fault tolerance.
At this scale, the power distribution system itself becomes a significant source of heat. Transformer losses, UPS inefficiency, and cable I2R heating add 8 to 15 percent to the total facility power draw. High-efficiency UPS systems (97 to 99 percent at load) and short cable runs between transformation and IT load are essential to minimize these losses.
For more detail on the technology infrastructure supporting these deployments, see our technology overview.
Cost Profile
Hyperscale GPU colocation is the most capital-intensive tier but offers the lowest cost per kW at scale. Purpose-built greenfield facilities designed for 100+ kW racks achieve buildout costs of $5,000 to $9,000 per kW by eliminating air-cooling infrastructure entirely, designing floor loading for heavy liquid-cooled racks from the start, and optimizing the electrical distribution for a single high-density use case. Monthly colocation pricing in this tier is typically negotiated as custom contracts based on total deployment size, term length, and power commitment, with rates of $100 to $180 per kW per month for multi-megawatt commitments.
Cooling Transitions Across Density Tiers
The progression from air cooling to liquid cooling is not a single threshold but a series of transitions, each unlocking the next density tier:
| Density Range | Primary Cooling | Supplemental Cooling | Key Limitation |
|---|---|---|---|
| 5–15 kW | Raised-floor air, CRAC/CRAH | None required | Airflow volume and mixing |
| 15–20 kW | In-row air with containment | Blanking panels, airflow management | Fan energy, noise, hot spots |
| 20–40 kW | Rear-door heat exchangers | Chilled water loop to rack rows | Water distribution infrastructure |
| 40–100 kW | Direct-to-chip liquid cooling | Residual air cooling (20–30%) | CDU footprint, piping complexity |
| 100+ kW | Direct-to-chip or immersion | Minimal air for ancillary | Facility water capacity, structural loading |
Each transition introduces new infrastructure requirements that cannot be easily retrofitted. The water distribution piping for RDHx units, the CDU floor space and piping for direct-to-chip cooling, and the structural reinforcement for heavy liquid-cooled racks all require either upfront design or expensive retrofit work. Operators planning for growth should design their mechanical infrastructure at least one tier ahead of their current deployment density.
Power Distribution Architecture by Tier
Electrical distribution evolves in parallel with cooling as density increases. The key transitions are driven by physics: higher power per rack means higher current at a given voltage, and excessive current creates cable management problems, excessive heat generation in conductors, and increased distribution losses.
Single-Phase (5–10 kW)
Standard branch circuits from floor-mounted PDUs. 120V or 208V single-phase, 20A to 30A per circuit. Simple, inexpensive, and universally available. Two circuits per rack (A+B feeds) provide basic redundancy. Cable management uses standard overhead or underfloor ladder trays with whip cables to each rack.
Three-Phase 208V (10–30 kW)
Three-phase distribution delivers 73 percent more power per conductor than single-phase at the same voltage and current. A 30A three-phase 208V circuit provides 8.6 kW versus 5.0 kW for a single-phase 208V 30A circuit. This reduces the number of circuits per rack and simplifies cable management. Intelligent rack PDUs with per-phase current monitoring ensure balanced loading across all three phases.
Three-Phase 415V (30–60+ kW)
Doubling the voltage halves the current for the same power delivery, which halves the conductor cross-section required and reduces I2R losses by 75 percent. At 40+ kW per rack, 415V three-phase is the standard. This voltage level requires PDUs with built-in step-down transformers for equipment that expects 208V input, adding cost and complexity but enabling the power densities required for GPU racks.
Overhead Busway (60+ kW)
At the highest densities, traditional whip cables from remote panel boards become impractical. Overhead busway systems run continuous copper or aluminum conductors above the rack row, with tap-off boxes at each rack position providing plug-in power connections. Busway eliminates the cable congestion of running dozens of high-amperage whips per row, enables hot-swappable tap-off boxes for maintenance without shutting down adjacent racks, and allows flexible power reallocation between rack positions as workloads change.
Infrastructure Cost Implications
The relationship between density and cost is counterintuitive: higher density racks cost more per rack to support but less per kilowatt of IT capacity. This is because fixed facility costs (land, building shell, security, fire suppression, network connectivity, staffing) are amortized over a higher IT load per unit of floor space.
| Density Tier | kW Per Rack | Buildout $/kW | Colo $/kW/mo | Racks Per MW |
|---|---|---|---|---|
| Standard | 5–8 | $8,000–$12,000 | $100–$180 | 125–200 |
| Medium | 8–15 | $7,000–$11,000 | $120–$200 | 67–125 |
| High | 15–30 | $7,000–$10,000 | $150–$250 | 33–67 |
| Ultra-High | 30–60 | $6,000–$10,000 | $140–$220 | 17–33 |
| Hyperscale GPU | 60–132 | $5,000–$9,000 | $100–$180 | 8–17 |
The economics shift decisively at the ultra-high and hyperscale tiers. A 1 MW deployment in standard 7 kW racks requires approximately 143 racks, 143 rack positions of floor space, 143 sets of power whips, and the associated cooling for a large hall. The same 1 MW in 120 kW racks requires approximately 8 racks, freeing over 130 rack positions for additional compute or reducing the required facility footprint by 90 percent. The infrastructure savings from this density improvement more than offset the liquid cooling investment.
Review our colocation pricing for current rate structures across density tiers.
Choosing the Right Density Tier
The optimal density tier for a given deployment depends on four factors:
- Workload profile: CPU-only workloads typically fall in the 5 to 15 kW range. Mixed CPU/GPU inference workloads land at 15 to 40 kW. GPU training clusters require 40 to 132 kW. Match the tier to the actual hardware you are deploying, not a theoretical maximum.
- Growth trajectory: If your current workload is 10 kW per rack but you expect to deploy GPU servers within 18 months, selecting a facility designed for 30+ kW avoids a disruptive migration later. Paying a modest premium for higher-density-capable infrastructure now is almost always cheaper than migrating a production deployment.
- Facility availability: Standard and medium density colocation is available in every major market. High density (20+ kW) availability narrows significantly. Ultra-high and hyperscale density (40+ kW) is concentrated in a small number of purpose-built facilities. Lead times for new high-density capacity can be 12 to 24 months.
- Total deployment scale: For small deployments (under 100 kW total), the overhead of dedicated liquid cooling infrastructure may not justify the density gains. For multi-megawatt deployments, the per-kW economics of high-density are compelling.
The industry trajectory is unambiguous. Every major GPU platform on current and announced roadmaps increases power per device. NVIDIA's roadmap from H100 (700W TDP) through B200 (1,000W TDP) to the Rubin platform (projected 1,200W+ TDP) means that rack densities will continue to rise. Facilities designed for today's maximum density will be mid-range within two product generations. Plan accordingly.
Explore the full range of AI compute hosting solutions available at Rax, designed to support every density tier from medium through hyperscale GPU.