Memory And Design Advances From The AI Infra Summit

The 2025 AI Infra Summit, held in Santa Clara, CA, brought together industry leaders to discuss the latest advancements in memory and chip design. Among the notable talks were those from Kove, Pliops, and Cadence, each showcasing innovative solutions to enhance AI performance. John Overton from Kove presented their Linux-based memory software, which enables sharing memory between servers to increase memory utilization, CPU, and GPU utilization. This is particularly important, as GPUs and CPUs have been scaling, but conventional memory systems have not, leading to overprovisioning and processing bottlenecks in servers.

Kove’s SDM software can be installed in just 15 minutes and allows for unlimited memory access from virtualized elastic memory pools across servers, supporting up to 64PiB of DRAM per process. Additionally, the software can hide latency, making memory appear local to a CPU, even when it isn’t. This capability can work across Infiniband and RoCE fabrics, and Kove claims it can hide latency between memory that is over 150m away. The resulting AI performance improvements are significant, with Kove stating that AI inference can run 3-5X faster. This has been demonstrated through partnerships with companies like Redhat and SuperMicro, showcasing key-value cache at scale using benchmarks like Redis and Valkey.

Pliops was also at the Summit, showcasing its XDP LightningAI, a GenAI native memory stack designed to power inference and retrieval workloads for hyperscale and enterprise applications. The product consists of an ASIC, a software stack, and distributed nodes, using a GPU-initiated Key-Value I/O interface. According to Pliops, deploying XDP LightningAI in data centers can offer significant cost savings, including a 67% optimization in rack space, a 66% reduction in power consumption, 58% annual OpEx savings, and a 69% decrease in initial investment costs. Pliops is collaborating with Tensormesh, an inference optimization software company, to combine LightningAI memory acceleration with Tensormesh’s shared KV cache architecture, resulting in fast time-to-first token and GPU savings across multi-GPU clusters.

Charles Alpert from Cadence, an electronic design software company, gave an interesting talk about the challenges in AI infrastructure, including energy consumption, thermal management, and the time to operationalize infrastructure. He discussed how adding AI for design can improve these challenges, creating a virtuous cycle of continuous improvements in data centers and devices. Cadence has tools for data center design, as well as traditional semiconductor design, and their tools can work with 3D stacks of die, including Multiphysics digital twin simulation. Alpert stated that over half the chips built today use AI technology, and this will accelerate to 90% in the next few years using agentic AI.

Cadence’s agentic AI is expected to lead to levels of autonomous design, similar to those used in autonomous driving. The company’s EDA tools with multi-physics capability will enable designing 3D devices made from stacking semiconductor die, often called heterogeneous integration. This requires massive system-level integration and is resource-intensive. Cadence has also expanded its technology beyond chip design, creating digital twins of data centers, including all functional components. Their Millennium M2000 system enables faster chip design and system design, and is part of Cadence’s Digital Twin Ecosystem for data center design.

In summary, the 2025 AI Infra Summit saw Kove, Pliops, and Cadence present innovative solutions to enhance AI performance. Kove’s massive memory sharing, Pliops’ AI-native memory stack, and Cadence’s complete digital twin data center design all aim to address the challenges in AI infrastructure. As the industry continues to evolve, it’s clear that advancements in memory and chip design will play a crucial role in shaping the future of AI.