The enterprise AI bottleneck is about context, not capability

Nearly four years after the generative AI wave began, enterprise deployments have settled into a familiar pattern: models are capable, pilots are convincing, but production rollouts plateau on accuracy, drift on outputs, and fail to compound productivity gains the way anyone expected. The diagnosis is increasingly consistent: The bottleneck is the context layer.

Major platform vendors are now staking a claim to the enterprise context layer, analysts are beginning to name it as a category, and a new class of infrastructure companies is being built around it, yet what a context layer actually is, why it’s genuinely hard to build, and how it differs from prior metadata catalogs and knowledge graphs remains poorly understood.

Managing Director Lonne Jaffe sat down with Prukalpa Sankar, cofounder and co-CEO of Atlan, to work through the fundamentals: What context actually means in an enterprise setting, what it takes to build a layer that AI agents can reliably consume, and why the hardest part of all may have nothing to do with technology. This conversation came from a webinar hosted by Atlan.

Jaffe: You’ve been one of the most cited voices on this topic, but the term “context layer” means different things to different people. Before we get into what Atlan is building, can you just ground us on the definition? What is context, actually?

Sankar: I always say, let’s forget the jargon and start with what context actually means. I think of it as three things: knowledge, expertise, and norms.

Let me illustrate with an example most people understand. Say you call a contact center and tell them your order was messed up and you want a refund. For an agent (human or AI) to handle that well, the first thing it needs is knowledge: who is this customer, what happened to their order, was the mistake ours or theirs? That’s the map of the business.

The second thing it needs is expertise: how do I triage competing issues? The customer mentioned social media, which triggers a de-escalation procedure. This is what companies call learning on the job.

And then there are norms: I’m not allowed to authorize more than $50 of a refund without escalating to my manager. Some norms are explicit. A lot are implicit. That’s where human judgment gets created.

“Performance is a function of intelligence and context. Cognitive intelligence only has about a 10% impact on job performance. Everything else is this context: the knowledge, and what they call learning on the job.”

Jaffe: That framing redefines the problem in a useful way. Anyone who’s used AI in an enterprise context has noticed that as the model improves, work often gets re-bottlenecked by the lack of different kinds of context. The missing knowledge, expertise, and norms get in the way.

Sankar: Exactly. And we saw it earlier than most, because we were trying to ship AI products ourselves. When we launched Atlan’s first AI features, this was three or four months after ChatGPT. We were probably among the first 20 or 30 companies globally to ship a real AI product. We could never get accuracy above 70%. And 70% was genuinely better than not having it. But it wasn’t anywhere close to the autonomous AI ecosystem we wanted to build.

What we realized was that the accuracy ceiling wasn’t a model problem but a context problem.

Why incumbents haven’t solved this

Jaffe: You might have expected the large pre-genAI incumbents to own this layer. They have existing data relationships, brands, distribution, products, and customers already using and storing their data inside, and huge balance sheets. The AI part is delivered like electricity from frontier labs. So you’d think they just wire it into their products and win.

Sankar: And our ability to move that fast came directly from living the problem. The only reason Atlan exists is because we built data platforms at scale before we built a product. We spent years running data teams for the United Nations, the World Bank, and large governments. We built India’s national data platform, which became the largest public sector data lake of its kind. We experienced, firsthand, how hard it is to share context across interdisciplinary teams where everyone speaks a different language and no one can see anyone else’s work.

The only reason we’ve grown as fast as we have is that we lived the problem.

What a real context layer does

Jaffe: Atlan came to market as a data catalog. You called it the third-generation catalog, built around active metadata. That product grew fast. But when generative AI arrived, you did something more ambitious than bolting on AI features. You basically rebuilt the fundamental premise of the product. Can you walk through what the context platform actually does today?

Sankar: The core insight was that our catalog was solving a human-consumption problem. It was designed to help data engineers and analysts find and trust data. But the future primary consumer of this context was going to be an AI agent. Agents have completely different requirements.

So the platform now works in three stages.

The first is context mining. We have our own agentic harness that reads across hundreds of connected systems: data warehouses, BI tools, code repositories, systems of record. It reverse engineers accurate context from all of them. I want to stress the word accurate here, because this is where a lot of people underestimate the problem. You can take tables and throw them into any large language model today and ask it to generate an ontology. It’ll actually look pretty good. The problem is that it won’t be accurate.

“You can throw tables into an LLM, and it’ll generate an ontology that looks pretty good. The problem is it’s not going to be accurate. We’ve built a context quality pipeline, similar to how you’d build a data quality pipeline.”

We’ve invested in making this deterministic. We read lineage, we read code, we track how metrics are used across systems, we see how your BI tools consume data. We build descriptions from verifiable sources, not inference. Then we can reverse-construct metrics, relationships, and eventually the ontology: the map of your business.

The second stage is embedding into agent development. Getting an agent live takes minutes. Getting it accurate enough to trust and actually ship takes months, because context engineering is genuinely hard. We embed into that development lifecycle so builders can pull from the context layer, test against it, and push learnings back.

The third is trace mining. Every agentic interaction is itself context. When a human overrides an AI decision, that override is information. We capture those traces (they’re big data now, at scale) and feed them back into the context layer. I’ll have customers say, “We launched this agent at 50% autonomy, and we want to take it to 90%. How do I do that?” You capture more traces of how humans are making decisions and overriding the system, and you improve your context over time. Every interaction makes the next one smarter.

“It takes five minutes to build an agent. It takes many months to give it the context you can actually trust and ship. We embed ourselves into that agent development lifecycle so that that gap closes.”

Where the context layer fits in the existing stack

Jaffe: The questions we got from the audience were quite sophisticated on this: the relationship to traditional metadata repositories, semantic graph systems, data lineage, and lakehouse governance. Where does the context layer sit relative to all of that?

Sankar: The way I’d frame it is that a context layer manages several types of context. The first is the ontology, the map of how things in your business connect. Customer is related to order, is related to contract, is related to renewal date. That’s what any AI agent needs to reason across the business, not just within a single system. The second is the semantic model: the metrics, measures, and dimensions associated with your data, the source of truth definitions, what counts as ARR versus what doesn’t. The third is the AI-ready data itself: descriptions, relationships, how to run a join, how to trust a number.

And then separately, there’s expertise and norms: skills, procedures, what people call agent operating procedures, the equivalent of standard operating procedures for agents.

As for where it sits relative to existing tooling: if you have a metadata catalog, we can pull from it. You don’t need to rip it out. The context layer sits above all of these systems, catalogs, data management, reference data, business process tools, and converts their contents into a single machine-readable interface that every agent in your stack can consume.

Jaffe: The coding agent analogy is useful here. Coding agents don’t only use your source code repository directly. They can use a knowledge graph built from that code, one that understands what each piece of code was for and how it relates to everything else. The agent is the consumer, not the human. The enterprise context layer is a similar concept, but applied to the whole company.

“Really good companies get better over time as they interact with more surfaces in the world. The context layer can become the core brain of the company as a learning system.”

The final frontier is human

Jaffe: You’ve been unusually transparent about Atlan’s own AI transformation. You have a whole site called Frontier documenting your experiments, including some that didn’t work. The one that got a lot of attention was telling your engineers they weren’t allowed to code anymore. What’s the real lesson there?

Sankar: The lesson is that the final frontier in becoming an AI-native company is human, not technical. All of us, especially if you’ve built expertise in something, have these fixed mental models about what we know how to do. And those models are exactly what you need to break.

I had a moment earlier this year where I realized I was still using AI as a way to improve my own work. I’d come up with something and ask AI to make it better. That’s a very natural first mode. But I wasn’t letting it replace my first move.

Then one day I said, just go do it. Go look at how comparable companies have positioned this, read our context, write the landing page. And it came back with something that was so good. And I remember having this moment of genuine insecurity. Positioning is one of the few things in my professional life where I thought I was genuinely top 1% in the world.

“In that moment, people have different responses. You fight, you flee, you freeze. But you can also choose to play and bring back childlike curiosity. What you choose to do in that moment is what changes.”

That moment of insecurity is what every individual and every team is going to go through. The companies that figure out AI-native operations aren’t winning on technology. They’re winning because they’ve found ways to get their teams to move through that moment and keep going.

The context infrastructure is genuinely buildable. What Atlan is building, and what a growing number of companies are starting to recognize they need, is tractable. The harder problem is the one Shankar describes: Getting people to let go of what they already know how to do. That’s where the gap between companies that get AI right and those that don’t will ultimately show up.

Editor’s note: Insight Partners has invested in Atlan.