Custom AI Accelerators: Groq, Cerebras, and Specialized Silicon for Data Centers

For most of the past decade, AI infrastructure has been nearly synonymous with NVIDIA GPUs. The CUDA ecosystem, the A100, H100, H200, and now Blackwell architectures have defined the hardware landscape for both training and inference. That dominance is not ending, but it is being joined by a growing ecosystem of custom AI accelerators designed to handle specific parts of the AI workload more efficiently than general-purpose GPUs. For data center operators, hosting providers, and enterprises planning infrastructure investments, this diversification has real implications for facility design, power delivery, cooling, and how hosting contracts are structured.

This guide surveys the major custom AI accelerator architectures that are moving from laboratory and limited deployment into production data center environments, explains what makes each one different from conventional GPU infrastructure, and lays out what data center operators need to plan for as the hardware mix diversifies.

Why Custom AI Silicon Is Emerging Now

Three forces are driving the custom accelerator wave.

The Inference Cost Problem

As AI models have moved from research into production, the cost of inference at scale has become the dominant operational expense. Training a model is a one-time (or periodic) capital expenditure. Serving that model to millions of users continuously is an ongoing operating cost. For many organizations, inference now accounts for 80-90% of their AI compute spending. This creates enormous economic pressure to find hardware that delivers more inference throughput per dollar and per watt than general-purpose GPUs.

GPU Supply Constraints and Pricing

NVIDIA GPU demand has consistently outstripped supply since the generative AI boom began in 2023. Lead times for H100 and H200 clusters have stretched to months, and pricing reflects scarcity. This supply-demand imbalance gives custom accelerator vendors a market opening: if they can deliver competitive inference performance with available hardware, customers have a reason to consider alternatives even if the software ecosystem is less mature.

Architectural Specialization Advantages

GPUs are inherently general-purpose. They can run training, inference, scientific simulation, rendering, and many other workloads because they are programmable parallel processors. That generality comes at an efficiency cost. A chip designed from the ground up for a single task, like serving transformer-based language models, can eliminate unused hardware, optimize memory hierarchy for the specific access patterns involved, and deliver fundamentally better performance per watt. This is the same logic that drove Bitcoin mining from CPUs to GPUs to ASICs: specialization wins on efficiency when the workload is well-defined and high-volume.

Major Custom AI Accelerator Platforms

Groq Language Processing Units (LPUs)

Groq's LPU (Language Processing Unit) is built around a deterministic, compiler-scheduled architecture called the Tensor Streaming Processor (TSP). Unlike GPUs, which rely on hardware schedulers and caches to manage workload execution dynamically, Groq's compiler maps the entire computation graph onto the chip at compile time. Every memory access, data movement, and arithmetic operation is scheduled statically, which eliminates the unpredictability and overhead of dynamic scheduling.

The result is extremely low and predictable inference latency, which is Groq's primary selling point. For large language model serving, where users experience latency as time-to-first-token and tokens-per-second, Groq's architecture delivers measurably faster responses than GPU-based inference for many model sizes. Groq has deployed GroqCloud as a public inference API and has begun placing its GroqRack systems in partner data centers.

From a data center perspective, GroqRack systems use a proprietary form factor, require specific power and cooling configurations, and do not drop into standard GPU server chassis. Hosting providers need to accommodate these non-standard physical requirements.

Cerebras Wafer-Scale Engine (WSE)

Cerebras takes the most physically radical approach in the accelerator market. Instead of cutting a silicon wafer into hundreds of individual chips, Cerebras fabricates a single processor across an entire wafer, currently the WSE-3, which contains approximately 4 trillion transistors and 900,000 AI-optimized compute cores. The CS-3 system that houses the WSE-3 is designed for both training and inference, with particular strength in training large models without the inter-node communication overhead that dominates multi-GPU cluster training.

Because the entire model can fit on a single wafer for models up to certain parameter counts, Cerebras eliminates the latency and bandwidth penalties of splitting a model across multiple discrete chips connected by network fabric. For training workloads where inter-chip communication is the bottleneck, this is a fundamental architectural advantage.

The CS-3 draws significant power per unit (over 20 kW) and requires dedicated liquid cooling. Facilities hosting Cerebras hardware need high-density power delivery, typically 30+ kW per rack position, and direct liquid cooling infrastructure, which is beyond what many conventional colocation environments provide.

Google Tensor Processing Units (TPUs)

Google's TPUs are the longest-running custom AI accelerator program, now in their sixth generation (Trillium / TPU v6). TPUs are available to external customers through Google Cloud and are optimized for TensorFlow and JAX workloads. While not available for on-premises or third-party colocation deployment (they exist only in Google's own data centers), TPUs are a significant part of the custom accelerator landscape because they demonstrate the viability and performance advantages of purpose-built AI silicon at hyperscale.

For data center operators outside the hyperscaler ecosystem, TPUs matter as a competitive reference point: when a cloud customer can run inference on TPUs at a lower cost-per-token than on GPU instances, it puts pricing pressure on GPU-based GPU-as-a-Service offerings and colocation hosting that relies on GPU hardware alone.

Intel Gaudi (Habana Labs)

Intel's Gaudi accelerator line, developed through the Habana Labs acquisition, targets both training and inference with a focus on cost-competitive performance against NVIDIA's mid-range offerings. Gaudi 3 uses standard OCP (Open Compute Project) form factors and works within conventional data center infrastructure, making it the most facility-friendly of the major custom accelerators. It does not require exotic cooling or non-standard rack configurations.

Gaudi's market positioning is price-performance: Intel offers the hardware at lower price points than equivalent NVIDIA GPUs, targeting customers who are willing to invest in software porting (from CUDA to Intel's software stack) in exchange for lower hardware costs. For hosting providers, Gaudi represents the least disruptive custom accelerator to accommodate from a facility perspective.

Amazon Trainium and Inferentia

AWS has developed two custom chip families: Trainium for training and Inferentia for inference. Like Google's TPUs, these are available only through the cloud (AWS instances) and are not sold for on-premises deployment. They matter to the hosting market because they create cost-competitive cloud alternatives that siphon workloads away from colocation GPU deployments when cloud economics are favorable.

Infrastructure Requirements: How Custom Accelerators Differ from GPUs

Requirement	Standard GPU (H100/H200)	Groq LPU	Cerebras CS-3	Intel Gaudi 3
Form Factor	Standard 4U-8U rack server	Proprietary GroqRack	Custom CS-3 enclosure	Standard OCP/rack server
Power per Unit	5-10 kW per server	Varies by config	20+ kW per CS-3	3-6 kW per server
Cooling	Air or liquid (recommended for H200+)	Liquid recommended	Liquid required	Air-cooled standard
Network Fabric	InfiniBand / RoCE	Proprietary interconnect	Ethernet (external), on-wafer (internal)	Ethernet / RoCE
Software Ecosystem	CUDA (dominant)	Groq compiler + API	Cerebras SDK + PyTorch	Intel Gaudi SDK + PyTorch

What This Means for Data Center Operators and Hosting Providers

Facility Flexibility Is Now a Competitive Advantage

A colocation facility designed exclusively around standard GPU server form factors, air cooling, and 10-15 kW per rack will struggle to host Cerebras, Groq, or next-generation custom accelerators that demand liquid cooling, non-standard chassis, and 30-60+ kW per rack. Facilities that invest in flexible power delivery, liquid cooling readiness, and accommodating diverse hardware form factors are better positioned to capture the full range of AI hosting demand, not just GPU tenants.

Multi-Vendor Hardware Strategies Are Emerging

Sophisticated AI operators are beginning to deploy heterogeneous hardware fleets: GPUs for training and general-purpose inference, Groq or similar accelerators for latency-sensitive inference serving, and potentially Cerebras for large-model training where inter-node communication is the bottleneck. Hosting providers that can support mixed hardware within a single facility, with appropriate power, cooling, and network infrastructure for each type, can serve these multi-vendor strategies more effectively than single-hardware-type environments.

Export Controls Apply Regardless of Vendor

US export controls on advanced computing chips apply based on performance thresholds, not brand name. Custom AI accelerators from US-based vendors (Groq, Cerebras, Intel) are subject to the same Bureau of Industry and Security regulations as NVIDIA GPUs. Data center operators in the UAE and broader Middle East region need to verify the export license status of each hardware vendor and ensure their facility and end-use certifications support procurement of controlled items.

Software Ecosystem Lock-In vs. Flexibility

One of NVIDIA's strongest competitive moats is the CUDA software ecosystem. Every custom accelerator requires its own SDK, compiler, and model porting effort. For hosting providers, this means that tenant workloads running on custom accelerators are less portable than GPU workloads, which can affect tenant retention but also increases switching costs. Providers should understand the software ecosystem implications when planning which hardware to offer and how to structure managed hosting agreements.

The Economics of Custom Accelerators vs. GPUs

The economic case for custom accelerators rests on a simple proposition: deliver equal or better performance on a target workload at lower total cost of ownership. Total cost includes hardware acquisition, power consumption (ongoing), cooling infrastructure (capital and operating), network fabric, software engineering effort for porting, and operational complexity.

For pure inference workloads at scale, several custom accelerators already demonstrate favorable economics versus GPU-based inference, particularly on throughput per watt and throughput per dollar metrics. The challenge is that these advantages are workload-specific. An accelerator optimized for large language model serving may offer limited advantage for computer vision, recommendation systems, or multimodal workloads where the computation pattern differs significantly.

For training, GPUs remain dominant because training workloads are diverse, iterative, and require the flexibility that general-purpose hardware provides. Cerebras is the notable exception for certain large-model training scenarios, but even there, the addressable market is a subset of total training demand.

Implications for Inference-as-a-Service in the UAE

The UAE is positioning itself as a regional hub for AI compute, and the emerging custom accelerator ecosystem creates an opportunity for UAE-based hosting providers to offer differentiated inference services. An operator who can host Groq, Cerebras, or similar hardware alongside traditional GPU clusters gives regional AI companies and government entities access to best-in-class inference performance without the latency penalty of routing to US or European cloud regions.

The key constraint is export control compliance: UAE operators need to work with each accelerator vendor's export licensing team to confirm availability and end-use requirements. Where procurement is feasible, the combination of UAE industrial power rates, purpose-built facility infrastructure, and proximity to Middle Eastern and South Asian markets creates a compelling value proposition.

Planning for Hardware Diversity

Data center operators evaluating how to prepare for a more diverse AI hardware landscape should focus on several infrastructure capabilities:

• Modular power delivery: Rack-level power capacity that can scale from 10 kW to 60+ kW per rack without major electrical infrastructure changes, using busway or overhead power distribution systems.
• Liquid cooling readiness: Piped coolant distribution to every row or rack position, even if not all positions use it initially. Retrofitting liquid cooling into an air-cooled facility is expensive and disruptive.
• Flexible floor loading: Some custom accelerator systems are heavier than standard servers. Raised floors and structural slab capacity should accommodate loads above typical 2,000 lbs/rack thresholds.
• Network fabric diversity: Support for both InfiniBand and high-speed Ethernet (400G/800G) within the same facility, since different accelerator platforms use different interconnect technologies.
• Vendor-neutral operations: Monitoring, management, and operational procedures that are not locked to a single hardware vendor's tooling, so that mixed-hardware deployments can be managed consistently.

How Rax Approaches AI Hardware Diversity

Rax builds UAE data center infrastructure with hardware flexibility as a design principle. That means high-density power delivery, liquid cooling capability across facility footprints, and network infrastructure that supports the interconnect requirements of both GPU clusters and emerging custom accelerators. As the AI chip landscape diversifies, the hosting providers best positioned to serve the market are those whose facilities can accommodate whatever hardware their tenants bring, whether it is the latest NVIDIA platform, a Groq inference cluster, or hardware from a vendor that does not exist yet.

For operators evaluating how GPU virtualization and custom accelerators fit together in a hosting strategy, the answer increasingly is both: GPUs for flexibility and training, custom silicon for cost-optimized inference at scale, all within a facility designed to support heterogeneous hardware.