NVIDIA GB300 & Vera Rubin NVL144: Hosting Next-Gen AI Infrastructure

The Next Leap in GPU Compute: From Blackwell to Vera Rubin

NVIDIA's roadmap has never been more aggressive. While the GB200 NVL72 is still being deployed across hyperscale data centers globally, the company has already unveiled its successor: the GB300 superchip built on the Vera Rubin architecture, configured in the NVL144 rack-scale system. For data center operators, colocation providers, and AI infrastructure planners, the GB300 NVL144 represents both a massive opportunity and a significant infrastructure challenge.

This guide covers what we know about the GB300 NVL144 so far, what it means for facility planning, and what operators should be doing right now to prepare their data centers for the next generation of AI compute.

GB300 NVL144: Architecture and Specifications

What Is the Vera Rubin Architecture?

Vera Rubin is NVIDIA's next-generation GPU architecture following Blackwell. Named after the astronomer who provided foundational evidence for dark matter, the architecture introduces several critical improvements for AI workloads:

• HBM4 memory: The GB300 is expected to be among the first GPUs to use HBM4, offering roughly 1.5 to 2 times the memory bandwidth of the HBM3e used in Blackwell GPUs. This directly accelerates large language model inference and training by reducing memory-bandwidth bottlenecks.
• NVLink 6: The next-generation interconnect doubles the per-GPU bandwidth to approximately 3.6 TB/s bidirectional, enabling tighter coupling between GPUs in the NVL144 configuration and improving collective communication performance for distributed training.
• Advanced process node: The Vera Rubin GPU die is manufactured on a more advanced process (expected TSMC 3nm or 2nm class), allowing higher transistor density and improved performance per watt.
• Doubled GPU count: The NVL144 rack configuration packs 144 GPUs into a single rack system, compared to 72 in the current NVL72, effectively doubling the rack-scale compute density.

GB300 vs. GB200: Key Differences

Specification	GB200 NVL72 (Blackwell)	GB300 NVL144 (Vera Rubin)
GPUs per rack	72	144
Memory type	HBM3e	HBM4
Memory bandwidth per GPU	~8 TB/s	~12-16 TB/s (est.)
NVLink generation	NVLink 5 (1.8 TB/s)	NVLink 6 (~3.6 TB/s)
Rack power draw	120-132 kW	200-240 kW (est.)
AI training throughput	~1.4 EFLOPS (FP8)	~4+ EFLOPS (FP8, est.)
Cooling requirement	Liquid (mandatory)	Liquid (mandatory, higher capacity)
Rack weight	~1,360 kg	~2,000+ kg (est.)
Availability	Shipping (2025-2026)	Late 2026 / H1 2027

Power Infrastructure Requirements

The most immediate challenge for operators is power. At 200 to 240 kW per rack, the NVL144 demands power infrastructure that most data centers simply do not have today. Even facilities that were purpose-built for GB200 NVL72 deployments at 120 kW per rack will need significant upgrades.

Electrical Infrastructure Checklist

• Utility feed capacity: A single NVL144 rack consumes roughly the same power as 20 to 30 traditional enterprise server racks. A 10 MW data center that could host ~80 NVL72 racks can only support ~40 to 50 NVL144 racks at full load.
• Transformer and switchgear sizing: Medium-voltage switchgear, step-down transformers, and automatic transfer switches must be rated for the concentrated loads. Many existing facilities have 2N or N+1 redundancy at lower per-rack capacities that will need re-evaluation.
• Power distribution units (PDUs): Standard 20-30 kW rack PDUs are inadequate. Operators will need overhead busway systems or custom high-density PDUs rated for 250+ kW per rack position. Learn more about PDU specifications.
• UPS considerations: Uninterruptible power supply systems must handle the concentrated load profiles. Lithium-ion UPS batteries offer higher energy density for the compact footprints these deployments demand.

Cooling Infrastructure: Beyond Current Liquid Cooling

If the GB200 NVL72 made liquid cooling mandatory, the GB300 NVL144 makes advanced liquid cooling an engineering imperative. Removing 200+ kW of heat from a single rack requires cooling infrastructure that goes well beyond today's standard CDU deployments.

Cooling Design Considerations

• Higher-capacity CDUs: Coolant distribution units must handle approximately double the thermal load per rack compared to NVL72 deployments. Purpose-built CDUs rated for 250+ kW per rack will be necessary.
• Facility water plant upgrades: The chilled water plant (chillers, cooling towers, pumps) must be sized for the aggregate heat rejection. A 10 MW NVL144 deployment generates roughly 10 MW of heat that must be rejected to the environment.
• Immersion cooling potential: At these power densities, some operators are evaluating single-phase or two-phase immersion cooling as an alternative or complement to direct-to-chip liquid cooling. Immersion can handle higher heat fluxes and simplifies the per-rack plumbing.
• Hot-climate advantages: Counterintuitively, the UAE and Gulf region may have advantages for NVL144 deployments. District cooling networks in Dubai and Abu Dhabi can provide chilled water at scale, offloading the heat rejection burden from the data center's own mechanical plant.

Structural and Physical Requirements

The NVL144 rack's estimated weight of 2,000+ kg (over 4,400 lbs) creates structural requirements that go beyond what most data centers were designed for:

• Floor loading: Traditional raised-floor data centers designed for 500 to 750 kg per rack position cannot support NVL144 racks. Purpose-built slab-on-grade construction rated for 2,500+ kg per rack position is required.
• Rack dimensions: The NVL144 may require wider or deeper rack footprints than standard 42U enclosures. Operators should plan for non-standard rack spacing and aisle widths.
• Liquid plumbing routing: Supply and return piping for liquid cooling must be routed to each rack position, typically through overhead distribution or under-floor manifolds. The piping diameter and flow rates for 200+ kW racks are significantly larger than NVL72 installations.
• Seismic considerations: In seismically active regions, the concentrated mass of NVL144 racks requires additional seismic bracing and anti-tip restraints.

Networking at Rack Scale and Beyond

The NVL144's 144-GPU unified compute domain creates both opportunities and networking challenges:

Intra-Rack Communication

Within the rack, NVLink 6 provides approximately 3.6 TB/s bidirectional bandwidth per GPU, allowing all 144 GPUs to function as a single unified compute domain with roughly 43 TB of aggregated HBM4 memory. This makes the NVL144 capable of training models with hundreds of billions of parameters without any inter-rack communication overhead.

Inter-Rack Fabric

For multi-rack training clusters, the NVL144 uses NVIDIA ConnectX-8 or next-generation SuperNIC adapters supporting 800 Gbps per port. Building a multi-rack training fabric for NVL144 systems requires:

• 800G spine switches: The inter-rack fabric must support 800 Gbps per link, requiring next-generation switches and optical transceivers.
• Ultra-low latency: Collective operations (all-reduce, all-gather) across racks demand sub-microsecond switch latency. InfiniBand remains the preferred fabric for the most demanding training workloads.
• Cable density: Each NVL144 rack will have dozens of 800G optical connections to the fabric. Structured cabling and cable management become critical infrastructure. Cable management best practices must be followed rigorously.

Financial Planning and TCO

The economics of NVL144 hosting differ fundamentally from traditional colocation:

Capital Cost Considerations

Cost Category	Traditional Colo (10 kW/rack)	NVL144 (200+ kW/rack)
Power infrastructure per rack	$15,000 - $25,000	$150,000 - $300,000
Cooling infrastructure per rack	$8,000 - $15,000	$80,000 - $150,000
Structural reinforcement	Minimal	$20,000 - $50,000 per position
Network infrastructure	$5,000 - $10,000	$50,000 - $100,000
Total facility cost per rack position	$30,000 - $55,000	$300,000 - $600,000

However, the revenue per rack position scales proportionally. An NVL144 rack delivers 20 to 30 times the AI compute of a traditional rack, justifying premium pricing that can yield strong returns on the higher infrastructure investment.

Operational Cost Factors

Electricity costs dominate the operating expense profile for NVL144 deployments. At a competitive power rate, operators in the UAE can offer attractive all-in hosting costs while maintaining healthy margins, particularly when leveraging free-zone incentives and long-term power purchase agreements.

Preparing Your Data Center: A Timeline

For operators planning to host NVL144 systems, the preparation timeline should look something like this:

Now (Q3-Q4 2026)

• Conduct a facility readiness assessment against NVL144 requirements
• Engage with NVIDIA's partner program for early access and technical specifications
• Begin electrical engineering designs for 200+ kW per rack power delivery
• Evaluate cooling plant capacity and identify upgrade requirements

Q1 2027

• Complete structural assessments and begin reinforcement if needed
• Order long-lead electrical equipment (transformers, switchgear, high-density PDUs)
• Design and begin installing liquid cooling distribution infrastructure
• Finalize network fabric architecture and order 800G switching equipment

Q2-Q3 2027

• Complete infrastructure installation and commissioning
• Conduct thermal and electrical load testing
• Begin accepting NVL144 rack deliveries and conducting burn-in testing

How Rax Data and Energy Is Preparing

At Rax Data, we are already engineering our facilities to support next-generation GPU densities beyond 200 kW per rack. Our approach includes:

• Purpose-built high-density halls: New construction designed from the ground up for 250+ kW per rack, with slab-on-grade floors rated for 3,000 kg per position.
• Scalable liquid cooling: Modular CDU architecture that can be expanded as GPU densities increase, with pre-plumbed rack positions ready for NVL144 connections.
• UAE power advantages: Through our Rax Energy division, we secure competitive power rates and can provision utility-scale feeds for high-density AI deployments.
• NVLink fabric expertise: Our engineering team has deployed GB200 NVL72 systems and is working with NVIDIA on NVL144 readiness, including 800G fabric design and AI compute hosting optimization.

Frequently Asked Questions

Can existing GB200 NVL72 facilities host NVL144 racks?

In most cases, not without significant upgrades. The power density roughly doubles, cooling capacity must increase proportionally, and floor loading requirements may exceed what existing facilities were designed for. However, facilities that were designed with future density growth in mind may be able to accommodate NVL144 with targeted upgrades to power distribution and cooling.

Is the GB300 backward-compatible with NVL72 infrastructure?

NVIDIA has historically maintained some degree of backward compatibility, but the NVL144 is a fundamentally different rack-scale architecture with different physical dimensions, cooling connections, and power requirements. It should be treated as new infrastructure, not an upgrade to existing NVL72 installations.

What workloads justify NVL144 over NVL72?

The NVL144's 144-GPU unified domain with NVLink 6 is particularly valuable for training the largest foundation models (1 trillion+ parameters), where the increased GPU count and memory bandwidth within a single rack eliminate inter-rack communication bottlenecks. For inference workloads, the NVL72 may remain more cost-effective per query in many scenarios.

Conclusion

The NVIDIA GB300 NVL144 represents the most significant leap in AI compute density since the original DGX systems. For data center operators, colocation providers, and AI hosting companies, the message is clear: start planning now. The facilities that can host NVL144 racks at scale will command premium positioning in the rapidly growing AI infrastructure market.

The window between announcement and availability is shrinking with each GPU generation. Operators who treated the GB200 NVL72 as a wake-up call are now best positioned for the GB300 era. Those who are still catching up on liquid cooling and high-density power delivery have an even more urgent timeline ahead.