TSMC: AI Power Consumption to Increase 6-Fold, Power Delivery and Thermal Management as Two Key Challenges for Next-Gen Systems

As AI computing power continues to expand, high power consumption has rapidly emerged as a critical issue. TSMC (2330-TW)(TSM-US) Director Chen Yen-ming stated today (1) at a forum that with ongoing improvements in AI computing capabilities and deeper 3D stacking and heterogeneous integration, the total power consumption of a single system will increase approximately sixfold over the next five years. Power delivery and thermal management will become the two key challenges for the continued expansion of next-generation AI systems, and the industry will shift from past chip-level optimization toward System Technology Co-Optimization (STCO).

Chen noted that from 2024 to 2029, CoWoS package size will expand from 3.3 reticles to over 14 reticles. As more computing logic is integrated, the scale of computing transistors within a single package is expected to increase by about 48 times. On the memory side, during the same period, the total bandwidth of HBM in a single package is projected to increase over 34 times, with I/O count rising about 2 times and per-I/O transmission speed increasing approximately 6 times.

As chips advance toward heterogeneous integration, power consumption rises significantly, creating two major challenges: thermal dissipation and power delivery. To address power delivery challenges, TSMC is moving components such as capacitors and voltage regulators closer to computing chips, shortening the distance power travels from the circuit board into the package.

Chen pointed out that currently, high-performance computing (HPC) and AI systems mostly rely on voltage regulators (VRs) on PCBs to step down higher input voltages to the lower voltages required by chips. However, as AI chip power increases and operating voltages continue to drop, the current required by the system rises, leading to more severe IR drop and power loss issues in PCB traces and power delivery.

He illustrated with an example: when system power increases from 600W to 1,400W, power consumption rises over 5 times, not only reducing power efficiency but also converting additional losses into heat, further increasing overall thermal burden. This shows that power delivery and thermal management can no longer be treated separately in the future.

Regarding power delivery bottlenecks, Chen stated that the primary direction is to reduce impedance. Currently, this can be achieved by integrating high-density capacitors into SoCs to lower impedance and voltage fluctuations in high-frequency regions. However, due to chip area limitations, scalability is constrained. Therefore, the next step is to integrate high-density capacitors into the interposer to achieve broader impedance improvement.

Still, capacitors alone are insufficient to meet future high-power AI demands. Chen noted that the industry is seeking higher-efficiency power delivery architectures, moving power sources progressively from PCBs toward the package—such as onto substrates, or ultimately integrating Integrated Voltage Regulators (IVRs) directly into the interposer—to shorten high-current transmission distances and reduce impedance and power loss.

On thermal management, Chen stated that to prevent AI systems from hitting a 'Thermal Wall,' future designs must simultaneously optimize mechanical structures, materials, and design. TSMC has already demonstrated up to approximately 40% improvement in thermal resistance through system co-design.

Chen further emphasized that thermal performance enhancement cannot be achieved by a single material alone but involves lid design and other factors. As chip power continues to rise, the entire industry will rapidly shift from air cooling to liquid cooling and continue to minimize thermal interface resistance along the heat transfer path. Liquid cooling architectures are evolving rapidly; beyond traditional cold plates, TSMC is collaborating with system partners to advance microchannel, jet impingement, and two-phase cooling technologies. The ultimate goal is to further integrate packaging and cooling structures, even completely eliminating thermal interface materials (TIMs).

Chen explained that while TIMs fill contact gaps between chips, lids, and cold plates, they themselves are sources of thermal resistance. Therefore, the long-term ideal is to shorten or even eliminate interfaces between chips and coolant, directly integrating cooling systems with packaging or chip structures to further improve thermal efficiency for high-heat-flux AI chips.

In addition to power delivery and thermal management, Chen emphasized that process technology alone is no longer sufficient for highly complex AI system design. Future designs must incorporate standardized methodologies and EDA tools to jointly optimize chips, packaging, memory, power delivery, and thermal management. AI-driven EDA should also be introduced to enhance design automation and system optimization capabilities.

Chen concluded that AI will continue to drive the evolution of transistors, heterogeneous integration, and 3D packaging technologies. STCO has become an inevitable direction for next-generation AI systems, requiring collaboration among design, architecture, materials, EDA tools, and supply chain partners to continuously advance AI platform performance and energy efficiency.