CXL discussions have often started with the technology: how the interconnect works, how much memory a device can attach, and what pooling might eventually enable. Penguin Solutions brought a systems perspective to its Endless Memory with CXL session. As a supplier of AI infrastructure and memory products, Penguin asked a more practical question: how should customers assemble memory to meet the capacity, bandwidth, power, reliability, and cost requirements of an AI workload?

David Selle, Business Development Manager for Advanced Memory Products, and Arthur Sainio, Director of DRAM, described a memory hierarchy that extends from conventional DDR5 to MRDIMMs, SOCAMM2, and CXL. Their presentation illustrated an important evolution for CXL. It is becoming one option in a system design, chosen for a specific job rather than discussed in isolation.

Register to attend Episode 1: The Imperative for Memory Innovation in the AI Era

AI is putting pressure on every part of memory

Selle opened with Penguin’s central argument: memory has become a limiting factor in AI. He identified four pressures facing infrastructure teams at once: capacity, bandwidth, supply, and cost.

Compute performance has grown much faster than the bandwidth available to feed it. Penguin illustrated an approximately 1,000-fold divergence between compute growth and memory bandwidth growth since 1980. The figures are an order-of-magnitude comparison, but the consequence is familiar to system operators: costly compute resources can sit idle while waiting for data.

At the same time, AI workloads need several kinds of memory simultaneously. Model weights must be available to serve inference. KV cache grows with context length and concurrent requests. Agentic applications add persistent context and state across multiple steps. Penguin’s illustrative projection puts the combined memory requirement per AI system at roughly 80 times its 2020 level by 2030. Selle stressed that these demands stack rather than replace one another.

Supply and price add urgency. Selle argued that rapid growth in AI usage and model capabilities has outpaced increases in available memory, while the low memory prices of 2023 are a poor basis for future infrastructure plans. His conclusion was that organizations should treat memory as an architectural decision, with a direct effect on how efficiently they use their AI investment.

Register to attend Episode 1: The Imperative for Memory Innovation in the AI Era

Agentic AI changes the host system

Sainio showed why the memory discussion now reaches beyond GPUs and HBM. A straightforward inference request moves through a CPU to a GPU that generates tokens. An agentic workflow may also plan a sequence of tasks, interact with business systems, retain context, and return to a model repeatedly.

That additional orchestration increases demand for host CPUs and system memory. Penguin illustrated a possible shift in the fleet-level CPU-to-GPU ratio from roughly 1:4 toward 1:1. It also showed system memory per GPU rising from approximately 512 GB toward 1–2 TB as working sets and persistent context grow. These are directional planning figures, not requirements for every deployment. They make the system-level point clear: expanding accelerator capacity alone will not answer every memory need.

The design challenge is to put each workload’s data in memory that provides the required speed and capacity at a workable cost.

A memory hierarchy built around workloads

Sainio organized Penguin’s portfolio around four jobs:

• DDR5 RDIMMs provide core capacity. They remain the foundation for CPU-attached memory in mainstream AI, compute, and storage servers.
• MRDIMMs increase host bandwidth. Penguin’s DDR5-12800 MRDIMM offers up to twice the peak bandwidth of DDR5-6400 for supported platforms and bandwidth-intensive workloads.
• SOCAMM2 targets bandwidth per watt. This detachable LPDDR5X form factor offers a compact, serviceable option for emerging AI systems where power and space matter.
• CXL expands capacity. It creates options for memory beyond conventional DIMM slots, including tiering, pooling, and reuse of existing modules.

Sainio also showed that component selection involves operating details. Penguin offers optional Zefr screening intended to identify weak DDR5 modules before deployment. Conformal-coated RDIMMs address reliability concerns in liquid-cooled systems, while very-low-profile modules fit dense blades where height and airflow are constraints. These examples reinforced the presentation’s systems theme: a memory specification is only useful when it fits the server and its operating environment.

MRDIMM illustrates the economic choice within the hierarchy. HBM remains suited to the most bandwidth-critical data near an accelerator. MRDIMM can increase CPU-attached bandwidth while retaining the configurable capacity and serviceability of socketed memory. Neither serves every part of the workload.

Register to attend Episode 1: The Imperative for Memory Innovation in the AI Era

CXL adds capacity and a path to DDR4 reuse

Within that hierarchy, CXL addresses expansion. It can attach additional memory when a server needs capacity beyond its conventional DIMM configuration. Penguin also discussed pooling as a way to make memory available more flexibly across systems.

Sainio cited TechInsights projections that CXL-attached DRAM per server could grow from 402 GB in 2025 to 1,741 GB by 2031. The same source projects that the share of servers using memory pooling could rise from 4% in 2027 to 49% in 2031. These are forecasts rather than measured deployments, but they explain why suppliers are preparing products for both near-term expansion and future pooling.

DDR4 reuse makes the near-term opportunity especially concrete. Sainio described removing DDR4 RDIMMs from decommissioned systems, testing and qualifying them, then installing suitable modules on a CXL add-in card for use in a newer server. The modules sit behind a CXL controller, giving organizations a way to redeploy memory they already own even when the new server does not accept DDR4 directly.

Penguin showed several form factors for different constraints: an eight-DIMM DDR5 card offering up to 1 TB, a narrower four-DIMM DDR5 card for space-constrained servers, an eight-DIMM DDR4 card for reuse, and an E3.S CXL memory module designed for front-access serviceability. The product choice depends on the server, workload, available space, and qualification requirements.

Register to attend Episode 1: The Imperative for Memory Innovation in the AI Era

The system decision comes first

Penguin’s presentation showed why the next phase of CXL adoption will be shaped by deployment decisions. More capacity has value when it serves a workload effectively, fits the platform, and improves the economics of the whole system. The same principle applies across the memory hierarchy: DDR5 for core capacity, MRDIMM for host bandwidth, SOCAMM2 for efficient modular memory, and CXL for expansion and reuse.

Selle closed by pointing to the next topics in Penguin’s Memory Innovation series: KV cache servers on October 28, DDR4 reuse with CXL on December 2, and photonics on January 13. Together, they extend the discussion from selecting memory components to placing, expanding, and sharing memory across AI infrastructure.

Register to attend Episode 1: The Imperative for Memory Innovation in the AI Era