The bottleneck of agentic AI is no longer the GPU: it's the hard drive that feeds it

By SiliconANGLE (theCUBE) · July 8, 2026.

At the RAISE Summit 2026, Greg Matson, vice president of marketing and product at Solidigm —SK Hynix's brand for enterprise NAND—, gave an interview to theCUBE with a simple thesis: storage has stopped being cheap plumbing to become what he calls the 'intelligence layer.' According to Matson, a couple of years ago the focus was on training, which required high-capacity storage attached to the GPU. Now, with agentic inference, the pressure has surged: the new storage layer acts as an extension of the system's memory, saving and serving the context an agent accumulates as it reasons across multiple steps.

The numbers he offers are illustrative, though they should be taken as the source presents them —an executive of Solidigm itself, in coverage that SiliconANGLE expressly acknowledges as sponsored by the event—: a prompt of barely 15 words can generate up to 40,000 output tokens, that is, between 5 and 10 gigabytes of context data. Multiplied by thousands of employees using agents at once, the storage need scales to petabytes. Solidigm's commercial answer is SSD drives of up to 122 terabytes per unit and what it says is the first cold-plate-cooled enterprise SSD, designed for fanless Nvidia GPU servers, at a time when AI racks are abandoning air cooling in favor of liquid cooling.

It is worth separating the fact from the wrapping. That an executive of a memory and storage manufacturer says storage is the most strategic piece of the AI stack is, by definition, a declared commercial interest; it is not an independent source. But the underlying phenomenon he describes is well documented in the sector by other means: models with ever-longer context windows (SiliconANGLE itself reports in the same news batch a 10-million-token model) and agentic workflows that chain dozens or hundreds of calls per task generate a volume of intermediate data —context caches, session states, tool histories— that no longer fits comfortably in the GPU's memory and needs a fast intermediate tier between the chip and traditional storage.

Our reading is that this news, though born from a product pitch, points to something we already sensed: the real cost of agentic AI is generated not by people writing prompts, but by agents orchestrating thousands of automatic calls per task, and it is that internal data traffic —not just the compute— that is rewriting the infrastructure investment map. During the training phase, the money and the narrative concentrated on the GPU and HBM; in the agentic inference phase, a growing share of the spending shifts toward layers that were previously invisible: high-capacity storage, liquid cooling, internal networks. It is the same logic we already saw with the Google-Microsoft dispute over the 'plumbing' of agents: whoever controls the boring but indispensable layer of the system also captures value, even if it doesn't appear in the headlines about the latest model.

In the short term, this translates into a capex cycle that gives no respite: hyperscalers have spent between 12 and 18 months replacing storage infrastructure more than a decade old, and that replacement is accelerating just as the deployment of sovereign AI by regional governments adds another layer of simultaneous demand. It is a real business for memory manufacturers like SK Hynix, but also a risk of overbuilding if the adoption of enterprise agents does not grow at the pace being financed; we have seen that pattern before with GPU capacity.

In the long term, however, this kind of infrastructure —however unglamorous it may be— is precisely what lowers the cost per token and makes it viable for agentic inference to stop being a laboratory luxury and become an accessible utility. Every improvement in how context is served without saturating the GPU is, ultimately, an improvement in how much intelligence can be distributed per dollar spent, and that is exactly the kind of efficiency that sustains the abundance thesis: not the promise of a smarter model, but the certainty that running already-capable models gets cheaper and cheaper for more and more people.