ICMSP Could Drive Additional 100EB Of AI Storage

Introduction to Nvidia’s Inference Context Memory Storage Platform

Nvidia’s 2026 CES announcements included the introduction of the Rubin Vera AI architecture and various open-source AI foundational models. One notable announcement was the Inference Context Memory Storage Platform (ICMSP), which utilizes the Nvidia BlueField-4 Network Intelligent Interface Card (NIC). This innovative platform is designed to accelerate and scale agentic AI by providing a new kind of AI-native storage infrastructure for gigascale inference.

Understanding Context Memory

Context memory refers to the data generated during interactions with an AI system. This data can be saved to make future interactions more consistent, coherent, and personalized. By storing context information in long-term storage, AI systems can remember details across conversations, learn unique patterns, and understand complex, multi-turn interactions. This is achieved by storing relevant data beyond the immediate prompt, allowing for more efficient use of GPU processing and memory.

Benefits of Context Memory Storage

Besides enhancing the user experience, context memory storage can also reduce the new calculation requirements for individual queries. By recovering data from storage rather than regenerating or keeping it in expensive and limited High-Bandwidth Memory (HBM), context memory storage can save energy and enable more efficient use of GPU processing and memory. This data is stored in the form of key-value (KV) cache, which can be regenerated if needed, unlike traditional enterprise storage systems.

Nvidia’s ICMSP and BlueField-4 NIC

The ICMSP is a network storage system that can consist of solid-state drives (SSDs) and/or hard disk drives (HDDs) for storing and accessing context memory data. This new tier of storage sits between traditional enterprise storage and HBM DRAM, which holds the data being processed by GPUs. The Nvidia ICMSP will store 16TB of storage per GPU, enabling petabytes of shared context across a GPU cluster to support large workloads. Throughput is targeted at 800Gb/s through the BlueField-4 board.

Industry Partners and Availability

Nvidia has partnered with several industry leaders, including AIC, Cloudian, DDN, Dell Technologies, HPE, Hitachi Vantara, IBM, Nutanix, Pure Storage, Supermicro, VAST Data, and WEKA, to offer ICMSP products in the second half of 2026. These products will provide a premium inference experience for customers, with the ability to store large amounts of context memory data and reduce the requirements for new calculations.

Expert Insights and Industry Trends

Experts from VAST and Micron shared their insights on the ICMSP and its potential impact on the industry. Phil Manez, VP of GTM Execution at VAST, discussed the shortages in memory and storage, while Jeremy Werner, SVP & GM Core Data Center Business Unit at Micron, highlighted the company’s investments in additional production capacity and innovations like Storage Next. The ICMSP is expected to drive greater demand for storage, particularly solid-state NAND storage, and will likely lead to higher prices and some constraints on AI data center buildouts.

Future Developments and Innovations

Micron is introducing 245TB E3L form factor SSDs this year, which will help meet the large storage requirements for context KV cache storage. The company is also working on additional innovations, such as Storage Next, and has demonstrated 230M IOPS in a single storage server. As the demand for AI inference experiences continues to grow, Nvidia’s ICMSP and the BlueField-4 NIC are poised to play a crucial role in driving the development of more efficient and scalable AI systems.