Tools For The Job

The biggest AI hardware vendors are splitting chips apart to build a new ecosystem around inference.

Between December 2025 and July 2026, four separate hardware partnerships reorganized AI inference around a single architectural idea. NVIDIA acquired Groq for $20 billion, AMD partnered with Cerebras, and Intel built a demonstration rack from three vendors’ silicon. Each arrangement separates the two phases of inference, prompt processing and token generation, onto different hardware. The industry has begun calling this approach disaggregated inference.

Inefficiencies revealed

AI inference involves two phases with opposing hardware demands. Prefill processes the entire input prompt in a single parallel pass, saturating a processor’s floating-point units across thousands of simultaneous operations. Decode generates output tokens one at a time, reading sequentially from a cache of previously computed values in a loop governed almost entirely by memory bandwidth.

The gap between these two profiles has widened as models have grown more capable. Reasoning models and agentic workflows routinely generate tens of thousands of output tokens per request, which magnifies decode’s share of total inference time relative to earlier chatbot-style interactions that rarely exceeded a few hundred tokens. At that ratio, the hardware serving decode determines latency for every downstream task.

A general-purpose GPU can handle both phases, and for most of the AI boom it has. At production scale, that arrangement wastes capacity in both directions. During prefill, the GPU’s large memory subsystem goes largely unused while the compute units work at full load. During decode, the compute units wait while the processor reads from memory. Operators running large models at high concurrency pay for silicon that spends much of its time idle, a cost that compounds with every additional user and every reasoning trace that extends to thousands of tokens.

Industrial response

NVIDIA acknowledged the split in December 2025 when it acquired Groq’s inference unit for $20 billion. Groq’s Language Processing Units store model weights in on-chip SRAM rather than external high-bandwidth memory, a design that trades capacity for speed. A single LPU cannot hold a large model, but a rack of them can, and the sequential token generation that defines decode maps naturally to SRAM’s low-latency access patterns. At GTC in March, NVIDIA unveiled the LPX rack, pairing its GPUs for prefill with Groq’s LPUs for decode in a single system connected through NVLink.

AMD and Cerebras announced a partnership on July 23 built on the same principle. AMD’s Helios rack, packing 72 MI455X GPUs with 31 terabytes of HBM4 memory, serves as the prefill engine. The Cerebras Wafer-Scale Engine, the largest chip in production, handles decode. The companies claim that the combination delivers five times the tokens per second per watt compared with a Cerebras-only configuration, and Cerebras plans to deploy Helios systems in its data centers with the joint offering available through Cerebras Cloud in the second half of this year.

Cerebras had already partnered with AWS earlier this year on a comparable arrangement, pairing Trainium chips for prefill with CS-3 systems for decode. In each of these three partnerships, a memory-bandwidth specialist handles decode while a compute-dense engine handles prefill, and neither side attempts to do both.

A new ecosystem

Intel demonstrated at Computex in June what disaggregated inference looks like when no single vendor supplies every component. The demonstration used Xeon processors for orchestration and scheduling, NVIDIA Blackwell GPUs for prefill, and SambaNova RDUs for decode. Intel supplied no accelerator of its own in that configuration, positioning itself in the orchestration and systems-integration layer that grows more valuable as the number of specialized components increases.

Disaggregation makes the infrastructure around the chips more consequential than any individual processor. Scheduling decisions determine which requests reach which hardware pool, and networking governs how quickly the key-value cache transfers between prefill and decode nodes. Major inference-serving frameworks including vLLM and NVIDIA’s Dynamo now offer native support for disaggregated serving, allowing operators to route each phase to separate pools and scale them independently. The competitive question in inference is shifting from who builds the fastest chip to who assembles the most efficient system from purpose-built parts.

Applying lessons learned

Seven months separated NVIDIA’s acquisition of Groq from AMD’s partnership with Cerebras, with Intel’s multi-vendor demonstration at Computex falling between them. The speed of that convergence suggests that each vendor reached the same conclusion independently, under the same production-scale pressures. Prefill and decode impose different demands on hardware, and serving both on a single architecture leaves capacity idle in both directions. As inference workloads grow with every deployed agent and every reasoning trace, the system-level integration of specialized components will matter as much as the performance of any individual chip.