The next AI bottleneck is context

The Next AI Bottleneck Isn’t Compute. It’s Context.

by Frank Berry | Sep 3, 2026 | AI Leadership, Industry Analysis

Key Takeaways

Getting the right data to models and agents is as critical as GPU performance.
Data movement is already consuming significant AI resources.
The context layer is critical enterprise AI infrastructure.
The solution is both moving data faster and moving less data.
Dell is uniquely equipped to attack the problem end-to-end.

Getting The Right Data To The Right Model At The Right Time

For the past several years, the AI infrastructure race has largely been a race for compute. More GPUs. Bigger clusters. Faster networks. More power. But as enterprises move from AI pilots to production, and especially toward agentic AI, I think another bottleneck is becoming just as important:

Getting the right enterprise data to the right AI model at the right time. In other words, context.

I recently interviewed Varun Chhabra, SVP of Product Marketing for Dell Technologies’ Infrastructure Solutions Group, about the results of our Enterprise AI Infrastructure Brand Leader surveys. When I asked which AI infra categories could become more important, he talked about the data layer. With 5,000 AI deployments behind him, I believe he’s right.

Varun said, “You can call it a semantic data layer or a knowledge layer or a context layer. This is becoming increasingly important as we talk to customers. And I think as we head first into agentic AI, this is going to be even more important.”

The Next AI Infrastructure Battleground: The Enterprise Context Layer

A Great Model With Bad Context Is Still Bad AI

The industry’s focus on models and GPUs can obscure a simple reality. Most enterprises don’t differentiate through access to a foundation model. Their competitors can buy access to many of the same models and GPUs. What competitors don’t have is their data.

Customer histories. Product information. Contracts. Engineering knowledge. Transactions. Support records. Intellectual property. Business processes. Decades of accumulated institutional knowledge. That’s where enterprise AI should derive much of its unique value.

The problem is that enterprise data wasn’t created for AI. It’s fragmented across databases, file systems, object stores, SaaS applications and countless documents. Some is current. Some is obsolete. Some is authoritative. Some isn’t.

Dell says this is already showing up in real-world AI deployments. Chhabra told me that one of the key determinants of AI workload success isn’t simply access to GPUs or the latest hardware. It’s often fragmented enterprise data.

That should get the infrastructure industry’s attention.

The Context Layer Is the Missing Link

A new infrastructure layer between enterprise data and AI models is emerging fast. Its purpose is easy to take all the proprietary information an enterprise possesses and find, prepare and assemble the specific information an AI model or agent needs to accomplish a task.

The context layer turns proprietary enterprise data into the right information for AI.

Underneath that simple diagram is a lot of technology and expertise. Enterprise data has to be discovered and prepared. Unstructured information may need to be converted into embeddings and indexed. Retrieval systems have to find relevant information. Permissions and governance have to be enforced. Context has to be assembled for the particular model, user, agent and task. And all of this has to happen fast.

That’s why Varun’s comments suggest that context is more than a software feature, it’s critical enterprise AI infrastructure

A 12X Performance Gain Tells Us Something

Dell’s latest AI Data Platform provides an interesting example. Dell and NVIDIA are using NVIDIA CUDA-X libraries to GPU-accelerate portions of the data pipeline. NVIDIA cuVS accelerates vector indexing and search, while cuDF accelerates structured-data processing. Dell reports up to 12X faster vector indexing with the architecture.

I’m less interested in the benchmark itself, than in what is being accelerated. Vector indexing is part of the process of making enterprise information searchable by meaning so AI systems can find relevant context. In other words, GPUs are moving upstream.

We spent the first phase of generative AI using GPUs primarily to accelerate models. Now we’re using them to accelerate the data and context those models consume. That’s a signal of where the infrastructure bottleneck is moving.

The GPU Can’t Compute on Data It Doesn’t Have

Another number from my conversation with Chhabra may be even more telling. Dell says 20% to 50% of AI processing time can be spent moving data rather than computing.

AI Processing Time can be Dominated by Data Movement

Think about the implications. The industry can continue building increasingly powerful GPUs but realized AI performance depends on keeping those GPUs supplied with useful information. An idle GPU is an extraordinarily expensive piece of infrastructure. And agentic AI could magnify the problem.

A chatbot might retrieve several documents before answering a question. An enterprise agent could need customer history, transactions, policies, product information, previous interactions and data from multiple applications before making a decision or taking an action.

The agent doesn’t simply need more data. It needs the right context.

AI Infrastructure Is Expanding

None of this means compute becomes less important. Demand for accelerated computing is likely to continue growing enormously. What changes is our definition of AI infrastructure.

The first phase centered heavily on Compute + Networking + Storage. The next phase increasingly adds Data + Knowledge + Context + Memory.

That’s going to create some interesting competitive collisions. Storage companies are moving toward data intelligence. Database and data-platform vendors are moving toward AI. Vector databases, knowledge graphs and AI memory platforms are attacking pieces of the problem from another direction. NVIDIA is accelerating not only AI computation but increasingly the processing, retrieval and movement of AI data.

Dell calls its implementation an AI Data Platform. Other vendors use terms such as semantic layer, knowledge layer, RAG infrastructure and context engineering. I don’t think the industry has settled on the eventual product taxonomy. I prefer context layer because it describes the outcome rather than the underlying technology.

The objective isn’t merely to store more data or retrieve documents faster. It’s to give an AI model or agent the information it needs to understand the task in front of it.

Analyst Take: Dell Is Uniquely Equipped to Solve the Context Bottleneck

If context is becoming the next major AI infrastructure bottleneck, Dell is uniquely equipped to address it because it can optimize virtually the entire path from enterprise data to GPU to AI response. Dell combines PowerScale and ObjectScale for enterprise data, Lightning and Exascale for high-performance AI storage, high-speed networking, NVIDIA technologies for GPU-accelerated data processing and KV-cache offload, the AI Data Platform for turning fragmented data into usable context, and services to integrate and optimize the complete environment.

Few vendors have that breadth. More importantly, Dell says it now has experience from roughly 5,000 AI deployments, giving it a growing body of real-world knowledge about where AI infrastructure actually bottlenecks.

The next phase of AI infrastructure will be won by keeping GPUs fed with the right data and context, while minimizing the time and resources spent moving it. Dell has assembled the compute, storage, networking, data platform, software ecosystem, services, and vast experience in the field, to attack that problem as a system rather than as a collection of components.

About IT Brand Pulse

IT Brand Pulse is an independent research and analysis firm focused on the technologies, companies and market shifts shaping the future of AI. Its research practices include:

AI Industry Analysis – examining emerging technologies and market trends.

AI Brand Leader – measuring AI / IT professionals’ perceptions of market and innovation leadership.

Industry First – documenting and validating technology innovations and the companies that pioneered them.

Personal and Organizational AI Memory (POAM) – tracking the emerging infrastructure and platforms that enable persistent memory for AI models and agents.

AI Leadership Forum – IT Brand Pulse manages the AI Leadership Forum on behalf of the AI infrastructure community, bringing technology leaders, practitioners and innovators together through its LinkedIn group, industry events and YouTube channel to share insights and advance the conversation around enterprise AI.