Samsung LPDDR5X-PIM: In-Memory Compute for AI

At the Hot Chips 2026 conference, Samsung detailed LPDDR5X-PIM, a version of its low-power mobile memory that performs part of an AI model’s math inside the DRAM itself rather than shipping every value out to a separate processor. The pitch is unusually concrete for a research disclosure: Samsung says the part slots into the same 561-ball package as standard LPDDR5X, meaning it can act as a drop-in replacement without redesigning the board or the system around it. The company presented the design across server, client and mobile targets, with edge AI on phones and laptops as a headline use case.

“PIM” stands for processing-in-memory, and the idea has been circulating in Samsung’s high-bandwidth-memory research for years. What is new here is bringing it to LPDDR5X — the low-power DRAM that already sits in hundreds of millions of phones and thin laptops — and doing it in a way that, on paper, does not force device makers to rearchitect anything.

What Samsung actually built

The core problem LPDDR5X-PIM attacks is the memory wall: on inference workloads, especially the token-by-token generation that dominates large language models, the processor spends much of its time waiting on data to arrive from memory rather than computing. Moving that data back and forth burns time and power. Processing-in-memory tries to shorten the trip by doing some of the arithmetic where the data already lives.

Inside each chip, Samsung places 16 PIM blocks within the DRAM banks. Those blocks contain MAC (multiply-accumulate) trees that run in parallel — multiply-accumulate being the fundamental operation behind the matrix math in neural networks — plus an ALU that Samsung says handles both floating-point and integer data types. The company describes it as the first LP-PIM part with multi-precision support, which matters because modern inference mixes formats: weights are often quantized to low-precision integers while some operations stay in floating point. A memory device that only spoke one numeric format would be far less useful in a real serving stack.

The bandwidth figures are the eye-catching part. Samsung quotes PIM bandwidth of 614 GB/s, which it frames as roughly eight times the ~76.8 GB/s a conventional LPDDR5X interface delivers. The gain comes from operating on data internally across those parallel banks rather than squeezing it through the external memory bus. Capacity in the demonstrated configuration is 16 GB.

The performance claims

Samsung backed the design with a workload rather than a synthetic benchmark. Running Meta’s Llama 3.1 model, the company reported token throughput rising from 27 to 81.3 tokens per second — about a 3x improvement — while the time to complete the task fell from 12.3 seconds to 5.4 seconds. In other framings of the same results, Samsung cited roughly 2.28x faster model runtime and 3.01x greater token throughput versus conventional LPDDR5X.

Two caveats are worth stating up front, because vendor-run inference numbers rarely travel intact into production. First, these are Samsung’s own measurements on a configuration it chose, not independent third-party benchmarks. Second, PIM accelerates the memory-bound portions of inference; a workload that is compute-bound, or one whose model does not fit the access patterns PIM is optimized for, will see far less benefit. The 3x figure is best read as a ceiling for the workloads PIM is built for, not a blanket speedup.

Still, the drop-in packaging is the detail that separates this from a lab curiosity. Much of the reason exotic memory architectures stall is that adopting them means redesigning the system. A part that keeps the same ball-out and can be validated as a substitute lowers that barrier considerably, at least in principle.

Why now: the HBM cost problem

The timing is not an accident. The industry’s answer to AI’s memory appetite has been high-bandwidth memory — expensive, capacity-constrained, and stacked directly onto accelerators like data-center GPUs. HBM supply has been one of the tightest links in the AI hardware chain, and its price has climbed through the memory supercycle that has reshaped the whole DRAM market this year. Samsung has been raising prices across its memory lines amid that demand; its foundry side lifted quotes by double digits citing AI orders.

LPDDR5X-PIM is a different bet. Instead of competing head-on with HBM for the data center’s most demanding training jobs, it targets inference — and especially inference that runs on-device or at the edge, where power budgets are tight and HBM is not an option. If a phone or laptop can run a meaningful chunk of a model’s generation loop inside its main memory, it can serve tokens faster and use less energy doing it, without a costly accelerator or an HBM stack. That is a large and growing market as more AI moves from cloud APIs toward local execution.

It also fits a broader industry pattern: the compute problem is increasingly a memory and data-movement problem. Custom inference silicon has been chasing the same efficiency-per-watt prize from the logic side — OpenAI’s Jalapeño inference chip, built with Broadcom, made its efficiency case at the same Hot Chips gathering. PIM comes at it from the memory side. The two approaches are not mutually exclusive; a future system could pair custom accelerators with smarter memory.

What it means

The strategic logic is sound even if the headline numbers deserve caution. Samsung is trying to turn its memory dominance into a compute story at the exact moment the market is paying a premium for anything that moves data less. A drop-in LPDDR5X part that accelerates on-device inference is aimed squarely at where AI is heading — toward local execution on phones and laptops — rather than fighting for scraps in the HBM-hungry data center, where demand already runs hot enough to lift results across the sector, as Nvidia’s blowout quarter underscored.

The winners, if this ships, are device makers and Samsung’s memory business. Being able to offer better local AI performance without a new accelerator or a board redesign is exactly the kind of incremental, low-friction upgrade that OEMs adopt. For Samsung, PIM is a way to sell memory with a margin story attached rather than competing purely on commodity price.

The open questions are software and standardization. Processing-in-memory has failed to break through before for a reason: it needs the software stack — compilers, runtimes, model formats — to know how to offload the right operations, and it needs enough industry alignment that developers can target it without bespoke work per device. Multi-precision support and standard packaging address part of that, but a single vendor’s part is not an ecosystem. Watch for whether Samsung can get PIM-aware paths into the frameworks that actually run foundation models, and whether rival memory makers converge on a compatible approach. Absent that, LPDDR5X-PIM risks being an impressive demo that stays a demo.

For now, the disclosure is a real signal about where memory is going: not just faster and denser, but compute-aware. Whether that becomes a product category or another PIM footnote depends less on the benchmark slides and more on the far slower work of getting software to use it.