Samsung Unveils Three-Phase HBM Roadmap: From 'Space-Saving' to 3D zHBM Eliminating Interposer, Ultimate Architecture to Cut Power by 70%, Boost Bandwidth by 230%

At the recent Hot Chips technology conference, Sangwook Han from Samsung's DRAM design team officially unveiled Samsung's three-phase roadmap for HBM evolution, with the ultimate goal being a new architecture called zHBM—directly stacking DRAM vertically on top of compute chips such as GPUs and TPUs, completely eliminating the 2.5D interposer.

As reported by U.S. tech media Wccftech on Sunday (23rd), compared to standard HBM4E, zHBM claims to achieve a 70% reduction in power consumption and a 230% increase in DRAM bandwidth, saving 100W per DRAM module while freeing up an additional 8.3% power budget for the GPU.

HBM consists of two types of chips: the C-die (core chip), which contains the DRAM storage units and can be vertically stacked up to 16 layers; and the B-die (base chip), located at the bottom of the entire stack, responsible for handling various DRAM control functions and communicating with compute chips like GPUs via the PHY (physical interface layer). The two are connected through TSVs (Through-Silicon Vias), vertical conductive channels that penetrate the chip. Current HBM4 stacks already exceed 3TB/s of bandwidth, with HBM4E advancing into the 4TB/s range, and HBM5 expected to double HBM4's bandwidth and exceed 60GB in capacity.

However, continued bandwidth expansion faces two major constraints: the physical limits of TSV count and pitch, and the upper limits on the number and speed of I/Os in the PHY interface within the base chip. Meanwhile, the process node gap between the base chip and the compute chip (xPU SoC) is narrowing with each generation, presenting both a challenge and an opportunity for Samsung's roadmap.

The core objective of the first phase is to reclaim silicon area from the xPU. Samsung has applied its D1c and 4nm logic processes to the HBM4 base die, primarily to reduce power and shrink effective area—marking the true starting point of DRAM's integration with advanced logic processes. Specific measures include replacing traditional HBM PHY with a D2D interface to shorten channel length and directly improve energy efficiency, thereby freeing up valuable silicon area on the xPU; offloading the memory controller from the xPU to the base die (B-die) of cHBM, expected to free up 5–10% area on the xPU, corresponding to a 10–20% performance gain; and introducing a fine-grained repair scheme based on SRAM, utilizing idle space on the B-die to deploy repair resources.

The side effect of area reduction is hotspot issues, which Samsung addresses with its Heat Path Block (HPB) technology, built into the cHBM4 solution, reducing peak temperatures by over 35% and covering 50% of the PHY area.

The second phase shifts from 'space-saving' to 'function expansion,' which Samsung defines as the functional enhancement stage. First, memory capacity is expanded: as AI large models dramatically increase context window sizes, demand for KV cache (a memory area used to store intermediate states during model inference) grows exponentially. Samsung plans to integrate memory expansion controllers and PHYs into the idle silicon area of the base die, expanding system-available memory capacity via external LPDDR or HBM solutions. Second, processing units (PEs) are integrated onto the base die, offloading some computations originally performed on the xPU to the memory side, reducing D2D bandwidth demand, power consumption, and thermal load—this form is known as AHBM (Advanced HBM).

Additionally, reliability and test capabilities are enhanced: advanced RAS (Reliability, Availability, Serviceability) sensors and real-time telemetry functions, along with on-chip self-test (ATIP) capabilities, are integrated into the base die to improve yield and test coverage.

The third phase represents the ultimate form of the entire roadmap. Current mainstream AI systems use a 2.5D packaging architecture, with GPUs and HBM placed side-by-side on the same interposer, transmitting data via lateral interconnects. The concept of zHBM is to 'stand this structure upright'—directly pressing the DRAM stack onto the xPU chip, forming a true 3D vertical integration.

Key characteristics described by Samsung's DRAM design team include: distributed I/O (minimizing data transmission distance within the HBM stack), 3D structure (eliminating traditional 2D interfaces, greatly improving system efficiency), I/O power target of approximately 0.5 pJ/bit (achieved by removing redundant modules like SerDes), over 2.3x bandwidth improvement, and 100W of system thermal headroom. Samsung's demonstrated configuration features a four-stack zHBM layered atop a single xPU.

To achieve these goals, Samsung is developing two key packaging technologies: WoW (Wafer-on-Wafer) and HCB (Hybrid Cube Bonding), aiming for ultra-high I/O density and ultimately building a unified SoC-DRAM co-design system.

The core logic behind the roadmap is to reposition the HBM's base die from a 'passive data relay station' to an 'intelligent partner with active computing capabilities.' The three-phase evolution is clear: first, compress area and free up xPU space through process upgrades (Phase 1); then, leverage the freed space to integrate more functions, expand capacity, and enhance computing power (Phase 2); finally, completely restructure the system architecture through 3D vertical integration, achieving simultaneous breakthroughs in power, bandwidth, and thermal management (Phase 3).

Samsung's Sangwook Han concluded his presentation: 'By mastering advanced packaging and unified SoC-DRAM co-design, we will overcome the power, area, and capacity bottlenecks constraining AI systems, paving the way for higher efficiency, higher performance, and greater scalability in the coming years.'