Nvidia Groq 3 LPX: $20B Inference Chip Goes Live

Nvidia said on August 24, 2026 that its Groq 3 LPX inference accelerator has entered full production and will come online before the end of the year — the first tangible product of the roughly $20 billion deal Nvidia struck in December 2025 to license Groq’s low-latency inference technology and bring founder Jonathan Ross on board. The launch is aimed squarely at the workload the industry is racing to serve: fast, cheap, long-context inference for AI agents.

What Nvidia announced

The Groq 3 LPX is a dedicated inference accelerator — a language processing unit (LPU) design that Nvidia acquired the rights to and has now integrated into its rack-scale systems. According to Nvidia, each liquid-cooled LPX rack packs 256 LPUs alongside Vera Rubin GPUs, and delivers up to 35 times more inference throughput per megawatt than the systems it replaces. That power-efficiency figure matters as much as raw speed: data-center capacity is increasingly rationed by the megawatt, not the square foot.

Nvidia paired the announcement with a benchmark from the third-party firm Artificial Analysis. Running Gemma 4 31B, an open-source model sized for agentic workloads, a Groq 3 LPX rack produced 3,400 output tokens per second on a 100,000-token long-context task — a figure Nvidia said was roughly 4x faster than the nearest alternative platform. The neocloud Nebius will be the first AI cloud to adopt the system, bringing it to its Nebius Token Factory inference platform later this year.

The timing is fast by hardware standards. The Groq 3 LPX reached full production roughly eight months after the December 24, 2025 agreement — a compressed schedule that reflects both Groq’s existing silicon and Nvidia’s manufacturing muscle.

The prefill-and-decode split

The most important idea in the launch is architectural, not financial. Nvidia is no longer treating inference as one undifferentiated workload. It is splitting the job in two.

Large-language-model inference has two distinct phases. Prefill is the compute-heavy work of reading a prompt and its context — the “thinking about what you said” step, which scales with how much context you feed the model. Decode is the memory-bandwidth-bound work of generating output one token at a time — the “typing the answer” step, which determines how fast words stream back to the user. The two phases stress hardware in opposite ways, and a chip tuned for one is rarely optimal for the other.

Nvidia’s answer is to use the right silicon for each. The Vera Rubin NVL72 rack-scale system, built on its newest Rubin platform, keeps handling large-scale context processing. The Groq 3 LPX takes over latency-critical token decoding, where the LPU’s deterministic, low-latency design shines. In effect, Rubin reads and reasons; Groq talks. Splitting the two lets an operator scale each independently — adding decode capacity for a chatty agent workload without over-provisioning expensive prefill compute, or vice versa.

That disaggregation is the whole thesis. Agentic systems — models that call tools, hold long conversations, and chain many short generations together — punish slow decode. Every extra millisecond per token compounds across thousands of steps. A rack purpose-built to spit out tokens quickly at long context is, in Nvidia’s framing, the missing piece for making agents feel responsive at scale.

Why Nvidia bought its way in

Nvidia already dominates AI training. Its GPUs are the default for building frontier models, and demand is still outrunning supply — the company’s most recent quarterly results showed $89 billion in data-center revenue and a guide well above expectations. But inference is a different, and arguably larger, market. Training a model happens once; serving it happens billions of times. And inference is exactly where a wave of specialized challengers has been trying to undercut the GPU on cost and latency.

Groq was one of the most credible of those challengers. Its LPU architecture was designed from the ground up for deterministic, low-latency token generation rather than the general-purpose parallelism of a GPU. By licensing that technology and hiring its founder, Nvidia did two things at once: it added a best-in-class decode engine to its portfolio, and it neutralized an independent competitor before the inference market fully formed. The move rhymes with the broader scramble in custom inference silicon, from OpenAI and Broadcom’s Jalapeño chip to a crowded field of startups all promising cheaper tokens.

The distinction between training and serving is where much of 2026’s chip strategy is being fought. For a primer on why the two workloads demand different hardware, see our explainer on batch versus real-time inference.

The wider Vera Rubin buildout

Groq 3 LPX is one component of a larger rack-scale story Nvidia has been assembling around Vera Rubin. Alongside it, the company is shipping Spectrum-X Multiplane networking — which the neocloud CoreWeave has already deployed into production to lash Vera Rubin racks together over high-bandwidth, lossless fabrics — and NVLink Fusion, which lets custom third-party accelerators and CPUs plug into Nvidia-based rack infrastructure.

Read together, these pieces describe a strategy: Nvidia wants to sell not a chip but an AI factory — a full system in which context processing, token generation, networking, storage, and management each get purpose-built hardware, all inside one Nvidia-orchestrated rack. Groq 3 LPX is the decode engine in that factory. NVLink Fusion is the hedge that even customers who bring their own silicon still build on Nvidia’s rails.

What we don’t know yet

Nvidia’s numbers are, as always, Nvidia’s numbers. The 3,400 tokens per second figure comes from a single benchmark on a single open-source model at a specific context length; real-world throughput will vary with model size, batch strategy, and quantization. The 35x throughput-per-megawatt claim is measured against unspecified prior systems. And pricing — the figure that ultimately decides whether the LPX wins inference share — has not been disclosed.

Availability is also staged. “In full production” and “online before the end of 2026” via a single launch partner is a rollout, not a flood. How quickly Groq 3 LPX racks reach the broader base of clouds and enterprises will determine whether it reshapes inference economics in 2027 or merely promises to.

What it means

The Groq 3 LPX launch is a signal about where the AI hardware fight is moving. Training built Nvidia’s empire; inference is where the volume — and the competition — now lives. By absorbing Groq’s technology and its founder, Nvidia has converted one of the most serious threats to its inference position into a product line of its own, and wrapped it in the Vera Rubin system that ties customers deeper into its stack.

The winners, if the specs hold, are the clouds and agent builders who get dramatically faster long-context decode without leaving Nvidia’s ecosystem — Nebius first, others to follow. The pressure lands on the independent inference-chip startups, whose core pitch was cheaper, faster tokens than a GPU; Nvidia has now answered that pitch with a purpose-built decode engine of its own and the manufacturing scale to ship it.

The number to watch is not tokens per second but cost per token in production, once independent clouds run the LPX against merchant GPUs and custom ASICs on their own workloads. If the power-efficiency gains translate into real savings, the prefill-and-decode split becomes the default way to build an inference stack, and Nvidia extends its lead into the half of the AI market it did not yet own. If they don’t, the LPX is a strong niche accelerator rather than a category-definer. Either way, the strategic logic of the $20 billion deal is now clear: Nvidia intends to own inference the way it owns training — and it was willing to buy the best challenger to make sure of it.