Is the market overly worried about Kimi K3? SemiAnalysis: AI hardware demand for GPUs and memory may grow even stronger

Although China's artificial intelligence (AI) company Moonshot AI recently launched its next-generation large language model Kimi K3, which uses a linear attention mechanism and has raised market concerns about potentially weakening demand for NVIDIA (NVDA-US), HBM, and networking equipment, semiconductor research firm SemiAnalysis believes that K3's massive parameter scale and unique inference architecture could instead make demand for high-end AI hardware even tighter.

SemiAnalysis points out that K3's massive parameter scale and inference architecture require extremely high computing power, and are expected not only to fail to suppress demand for high-end AI hardware, but could instead drive continued growth in demand for NVIDIA's high-end GPUs, HBM, and high-speed interconnect solutions.

According to the firm's analysis, K3 has over 2.8 trillion parameters, and just storing the model weights requires more than 1.5TB of HBM capacity.

Even when the number of concurrent users is not high, the KV cache generated during inference must still be heavily offloaded to the server's DDR5 memory and NVMe solid-state drives, indicating that HBM capacity constraints have not been alleviated.

More notably, Moonshot AI has previously revealed that to run K3 efficiently, a large-scale expansion domain architecture consisting of at least 64 chips is required—a hardware specification that aligns perfectly with the design philosophy of NVIDIA's rack-level AI servers such as GB200 and GB300 NVL72.

SemiAnalysis therefore argues that the market's initial assumption that linear attention mechanisms would weaken GPU demand may be misguided.

The real effect could be exactly the opposite: as model inference efficiency improves and costs decline, more enterprises and application scenarios will adopt AI, thereby increasing long-term demand for GPUs, HBM, DRAM, and network infrastructure.

KDA mechanism saves bandwidth, but gains offset by expert parallel computation

Market concerns primarily stem from the Kimi Delta Attention (KDA) mechanism used in K3. Compared to traditional Transformer architecture attention computation, KDA can significantly reduce the data volume required for KV cache transmission, lowering network bandwidth pressure by up to tenfold.

This technological breakthrough reminded many investors of the panic during the launch of DeepSeek R1, when fears arose that AI hardware demand might shrink.

However, SemiAnalysis emphasizes that such views overlook another key variable in large model inference: the computational and interconnect pressure caused by the parameter scale itself.

K3's parameter count exceeding 2.8 trillion means the model deployment must rely on large-scale distributed computing systems. Additionally, K3 adopts WideEP, an advanced parallel computing strategy designed specifically for Mixture-of-Experts (MoE) models, distributing 896 expert modules across multiple GPUs so that each GPU only handles a portion of the expert weights, thereby improving computational efficiency.

However, WideEP also brings new challenges: frequent data exchange between expert modules requires stronger network interconnect capabilities to support it.

SemiAnalysis analysis indicates that the copper backplane interconnect architecture used in NVIDIA's GB200/GB300 NVL72 achieves up to 18 times the rack-level bandwidth of traditional DGX B200 systems, making it ideally suited to handle such large-scale expert parallel inference tasks.

In other words, the bandwidth saved by the KDA mechanism in KV cache transmission is likely partially offset by the bandwidth required for weight exchange in the WideEP architecture, meaning the overall pressure on AI infrastructure has not truly been reduced.

64-chip expansion domain not exclusive to NVIDIA—Huawei Ascend can also benefit

Notably, not all industry experts agree that NVIDIA will be the sole winner.

Semiconductor and AI industry analyst GDP pointed out that the '64-chip expansion domain' architecture mentioned by Moonshot AI does not necessarily equate to NVIDIA's NVL72 solution. Huawei's Ascend 950 SuperPod also uses a 64-chip configuration and features unified bus memory expansion capabilities similar to NVLink.

In terms of scalability, the Ascend 950 SuperPod can even span 16 racks and scale up to 1,024 NPUs, giving it competitive strength in handling large-scale model inference demands.

This means that the growth in hardware demand driven by K3 may not exclusively benefit NVIDIA, but the overall market trend toward stronger demand for high-end AI interconnect systems remains clear.

Efficiency gains stimulate demand—industry invokes 'Jevons Paradox'

SemiAnalysis further cites the economic 'Jevons Paradox' to explain this trend in AI infrastructure development. The theory argues that when a technology improves resource efficiency and lowers unit costs, overall demand often increases rather than decreases, as the expanded application scope drives broader adoption.

Applied to the AI industry, once linear attention mechanisms reduce inference costs, more enterprises may deploy AI applications, leading to continuous expansion of global AI inference scale, ultimately driving synchronized growth in demand for GPUs, HBM, DRAM, and high-speed networking equipment.

However, GDP maintains a more cautious view. He acknowledges the long-term validity of the Jevons Paradox but also notes that the KDA mechanism genuinely optimizes state storage for long-context tasks.

He analyzes that even though K3's model weight scale is massive, due to its 4-bit quantization and highly sparse design, the actual memory pressure may be lower than commonly perceived.

GDP believes the truly important variable for long-term market attention is whether global AI leaders with the highest inference demands—such as OpenAI, Anthropic, and Google (GOOGL-US) DeepMind—have already or will soon adopt similar linear attention approaches like KDA or DeepSeek's CSA/HCA.

He points out that once these leading AI companies widely adopt such architectures, demand for memory and interconnect resources in long-context inference could significantly decline, becoming a key factor shaping future AI hardware demand structures.

Analysts suggest that looking ahead, the real question for the industry may no longer be whether linear attention mechanisms can reduce resource consumption at a single point, but whether the new demand generated by the continuous expansion of AI applications can long-term outpace the resource savings achieved through architectural efficiency improvements.