Samsung Puts Compute Inside the DRAM: LPDDR5X-PIM Takes On-Device Bandwidth from 76.8 GB/s to 614 GB/s, and Llama 3.1 8B from 27 to 81.3 tok/s

1 views

Samsung presented LPDDR5X-PIM at Hot Chips 2026 on August 25 — the industry's first LPDDR-based processing-in-memory solution. On LPDDR5X-9600, PIM bandwidth reaches 614 GB/s against 76.8 GB/s through the conventional memory interface, roughly eight times, achieved by placing 16 PIM blocks across all 16 banks of the chip with MAC trees running in parallel and an ALU handling both FP and INT. Measured with Llama 3.1 8B on an edge AI accelerator, generation went from 27 to 81.3 tokens per second and task completion from 12.3 to 5.4 seconds. It uses the same 561-ball package as standard LPDDR5X and works with a conventional memory controller.

Where the Eight-Fold Bandwidth Comes From

The bottleneck in on-device inference is usually not arithmetic but movement — the trip weights take from memory to the compute units. PIM's answer is to skip the trip and compute inside the memory. Samsung places 16 PIM blocks into all 16 banks of the chip, with MAC trees inside each running in parallel alongside an ALU that handles both floating-point and integer data. The result on LPDDR5X-9600 is 614 GB/s of usable PIM bandwidth against 76.8 GB/s through the conventional interface on the same 16 GB chip. The measured numbers come from Llama 3.1 8B running on an edge AI accelerator: generation from 27 to 81.3 tokens per second, an output gain of 3.01x, and task completion from 12.3 to 5.4 seconds, a 2.28x improvement. The two multipliers differ because one measures throughput and the other end-to-end latency — do not use them interchangeably.

Compatibility Is the Most Practical Part

More notable than the performance figures is how it deploys. LPDDR5X-PIM uses the same 561-ball package as standard LPDDR5X and works with a conventional memory controller, with the spec targeting server, mobile and client at 16 GB across four dies per rank. It does not demand a redesigned board and controller — historically where processing-in-memory proposals have stalled. Samsung traced its PIM timeline from the Aquabolt-XL HBM2-PIM proof of concept in 2021 to LPDDR5X-PIM productization in 2026. The path to shipping is named too: the in-development AI PC chipset GAIA is expected to incorporate it, which would be the first PIM integration into an actual AI PC chipset, with mass production as early as 2027. The chip made its public debut at FMS 2026 in Santa Clara in July, winning a Best of Show award.

The Motivation Is Written on the Cost Line

Samsung is blunt about it: HBM is expensive. By its figures, HBM's share of AI chip component spending grew from 52% in Q1 2024 to 63% by Q4 2025. Presenter Hwang Garam framed the need as a new memory form factor, as AI shifts toward inference-centric workloads running on smartphones and AI PCs. The boundary is worth stating: Samsung positions LPDDR5X-PIM as a complement to HBM, not a replacement, with HBM still the better fit for data-center workloads. For teams working on on-device inference, the practical takeaway is the schedule — actually using it means waiting for a chipset like GAIA, with mass production no earlier than 2027. What you can do now is factor it into on-device performance expectations for the next two years.

via: Tom's Hardware, ServeTheHome, TrendForce