AI Data Volume Soars: NVIDIA Drives GPU Direct Storage Access, Speed Approaching Memory

NVIDIA (NVDA-US) unveiled multiple advancements in AI storage technology at this week's Future of Memory and Storage (FMS) conference. The core concept is enabling GPUs to directly read data from storage devices, reducing reliance on CPU processing and bringing storage access speeds closer to memory levels.

As AI models grow increasingly large, they demand not only greater computational power but also the ability to process vast amounts of data. Applications like AI agents and long-context inference are rapidly evolving, requiring data volumes that exceed the capacity of standard system memory, significantly elevating the importance of storage in AI infrastructure.

NVIDIA emphasized that the next phase of AI development depends not only on GPU compute power but also on whether storage systems can deliver data to GPUs quickly, reliably, and securely. If data delivery cannot keep pace, even large GPU clusters may remain underutilized due to data starvation.

As AI agents handle thousands of concurrent tasks, GPUs may generate thousands of storage requests simultaneously, placing immense pressure on storage systems. Beyond fast data transfer, storage must also perform encryption, compression, verification, and data reconstruction—processes that can become performance bottlenecks in AI systems.

To address these challenges, NVIDIA is integrating accelerated computing into storage systems. According to NVIDIA's test results, the Vera CPU on the NVIDIA Vera BlueField-4 STX platform achieves up to 3.21 times higher data processing throughput than traditional x86 CPUs when performing simultaneous data compression and encryption.

This means storage platforms equipped with Vera CPUs can handle more AI data with fewer compute resources, reducing infrastructure costs and improving overall data transfer efficiency.

Historically, storage devices were primarily for long-term data retention, with data moved to faster, smaller, and more expensive memory when computation was needed. However, advances in GPU, networking, and storage technologies are transforming this model. NVIDIA aims to make storage devices active participants in AI data processing, not just passive repositories.

In the future, AI applications won't need to preload massive datasets into memory. Instead, they can directly and rapidly access specific data from storage on demand, balancing storage capacity, cost, and access speed.

NVIDIA announced the open-sourcing of the cuFile application programming interface (API) and its underlying storage software architecture. cuFile is a software suite for NVIDIA's GPUDirect Storage technology, enabling GPUs to directly read from or write to storage devices.

In traditional architectures, data from storage must pass through the CPU and system memory before GPU processing. With cuFile, this detour is minimized, shortening the data path and reducing CPU overhead.

Leveraging GPU parallelism, high-bandwidth memory, and software optimizations, cuFile enables systems to access storage data in microseconds.

NVIDIA stated that open-sourcing cuFile will simplify integration across different storage, software, and hardware platforms, enabling broader developer participation and fostering a unified, interoperable AI storage ecosystem.

Beyond speed, cuFile incorporates security by design. Rapid data access is critical for AI security systems—for example, real-time analysis of vast security logs, threat intelligence, and system data to detect attacks or anomalies early.

Open access to cuFile is expected to empower security AI with faster data access and support initiatives like the Open Secure AI Alliance.

Future APIs will be delivered via a new open platform with community-driven development. Founding maintainers include Google, Intel, NVIDIA, and Meta. Industry players can optimize for various platforms, lowering the barrier to adopting AI storage technologies.

In addition to open-sourcing cuFile, NVIDIA is advancing next-generation AI storage through the Storage-Next initiative, collaborating with storage and memory industry partners.

Storage-Next brings together storage manufacturers, controller suppliers, thermal and cooling specialists, resource orchestration providers, and standards bodies to jointly define how GPU-driven storage systems should operate and to transform these technologies into open, interoperable industry standards.

Over 40 storage and flash memory companies, including DDN, KIOXIA, and Micron, have joined the initiative to co-develop next-gen AI storage technologies.

The program's focus is enabling GPUs to rapidly locate and retrieve only the necessary data from large datasets, eliminating the need to transfer entire datasets into GPU memory.

To achieve this, NVIDIA introduced the SCADA framework—Scaled, Accelerated Data Access.

SCADA allows GPUs with massive parallel processing capabilities to directly read specific application data from storage and place it into high-speed GPU memory.

For instance, when an AI model needs only a small portion of a large database, traditional systems may waste bandwidth and compute resources by loading unnecessary data. SCADA retrieves only the required portion, minimizing redundant data transfers.

Storage vendor DDN plans to integrate SCADA into its Infinia platform, a software-defined data platform designed to reduce storage bottlenecks in large-scale AI workloads.

Sven Oehme, DDN's CTO, stated that enterprise AI success hinges not just on infrastructure scale but on effective resource utilization. The collaboration with NVIDIA aims to create a direct, efficient link between GPUs and data, reducing GPU idle time and enhancing AI investment returns.

Building AI-Native Data Platforms

Storage-Next and SCADA will be further integrated with NVIDIA's existing AI storage infrastructure, including the NVIDIA Vera BlueField-4 STX.

Vera BlueField-4 STX is a modular rack-scale platform integrating the NVIDIA Vera Rubin computing platform, Vera BlueField-4 storage processors, and Spectrum-X Ethernet technology.

NVIDIA positions STX as a next-generation AI-native data platform, enabling continuous enforcement of access controls and security policies during AI data transfers via the unified NVIDIA DOCA security software framework.

Additionally, NVIDIA is building the CMX Context Memory Storage platform on STX, targeting long-context, multi-turn dialogue, and agent-based AI inference.

Context memory allows AI to retain and quickly retrieve past conversations, work logs, or related data, avoiding redundant processing. As AI agents handle increasingly complex tasks, such storage platforms will become increasingly vital.

NVIDIA noted that while direct application-to-storage communication boosts speed, it also introduces security risks. Poor access control could allow applications to read unauthorized data or overwrite memory used by other processes. Thus, direct access technologies must balance performance with robust security.

To address this, SCADA divides the system into two parts: one for high-speed data access at the application level, and a separate, higher-privilege component dedicated to security configuration and access control.

During system initialization and setup, the privileged component verifies which storage resources an application can access and establishes protected data channels. Thereafter, the application can rapidly read data within its authorized scope, while