GPU Direct Storage Cuts AI Memory Latency

GPUs can now fire off storage requests on their own, no CPU middleman required, and that single change is rewriting how AI systems handle data. At the Future of Memory and Storage conference this week, NVIDIA detailed why that shift matters: AI agents are pulling in enormous volumes of data, and a GPU issuing its own storage calls can generate thousands of concurrent operations in the process. Every one of those operations needs encrypting, compressing, verifying and reconstructing on the fly. When thousands of agents hit a storage system at once, those data services turn into chokepoints fast.

NVIDIA's answer, unveiled at FMS, centers on new storage hardware built for this kind of load, alongside a broader argument that the next jump in AI performance depends on storage infrastructure as much as raw compute power. The company frames this as a break from how data centers have long thought about the divide between memory and storage, a divide built around speed tradeoffs that accelerated computing is starting to erase. What happens to that old calculus, and what NVIDIA's own hardware benchmarks show, gets laid out next.