The Enterprise AI Context Gap: Why Business Questions Remain Hard to Answer

Niyati Srivastava

July 21, 2026

The reason your AI keeps guessing, and what your people know that it does not.

Niyati Srivastava | AI Practice Head – UK & Europe

‍

One conversation from the Databricks Data and AI Summit 2026 keeps coming back to me. In conversation with a data leader, I learnt how quickly her team had built an AI agent. Then, almost in the same breath she admitted she would not hand it a real question from her board just yet. That same sentiment surfaced repeatedly across the three days.

Teams can stand up an impressive AI agent in a matter of days. Pointing it at a genuine business question, but trusting the answer is where things get tricky

That gap between the first demos and the board-level queries is the real story of enterprise AI right now, and it is worth being honest about how wide it is.

‍

The paradox in the numbers

Most organisations have already adopted AI in some measure. McKinsey pegs that number to 78 percent AI use in atleast one business function of the organisation. The difficulty shows up at the next stage. Only about 6 percent firms qualify as genuine high performers with meaningful impact on earnings.

95 percent of generative AI pilots deliver little to no measurable effect on the P&L as per MIT's NANDA study.

‍

So, we have near universal adoption sitting alongside near universal disappointment.

Gartner expects more than 40 percent of agentic AI projects to be cancelled by the end of 2027, citing runaway cost, unclear value, and weak controls. The technology dazzles in the demo and then stalls on the way to the P&L

This is a key value problem the industry cannot ignore. The reflex is to blame the models. Now with newer, larger, cheaper, faster models over the past year, the value gap has barely moved. It’s pointing to a bottleneck somewhere else.

The real bottleneck is business context—and it’s fragmented across the enterprise.

Getting insight out of data is still hard, even with the best models available. Data is the ground truth of the business, and meaning sits a layer above it, in context: what a field represents, which table is authoritative, how your business defines net sales, why a particular exception exists. That context is the difference between a number and an actionable answer.

Here is where the difficulty lies. In most enterprises, that context is scattered. It lives across dashboard KPIs, SQL queries, pipelines, wikis, SOPs, tickets, documents, and chat threads, and a fair amount of it survives only in the heads of a few experienced people. Analysts estimate that roughly 90% of enterprise information sits in unstructured places like documents, emails, and logs rather than in tidy tables.

When an AI agent cannot easily find the context it needs, it does what a confident junior employee might do. It fills the gap with inference. It guesses. The result is an answer that is generic at best and a red herring at worst. And crucially, it is wrong with total confidence, delivered at machine speed, with no one in the room to catch it.

The current generation of agents tries to compensate by probing iteratively, making call after call to work out what a term means or whether a source can be trusted. That process is slow, expensive, and still forces a compromise on quality. Every one of those extra steps is a tax you pay because the context was missing to begin with.

The consequence is the one that matters most to any leader betting on AI: unacceptably poor performance on exactly the data driven decisions and actions the investment was meant to improve.

‍

Without shared meaning, enterprise data remains dissconnected

The fix is to give your AI the same shared understanding of the business that your most experienced people carry in their heads, in a form in which a machine can act on.

This is what an enterprise ontology, or semantic layer, provides. It captures the nouns of your business (customer, order, supplier, store) and the verbs that connect them (a customer places an order, a promotion applies to a product), together with the rules that remove ambiguity, such as which definition of revenue is authoritative. It turns data access into data understanding, so a person and an agent work from the same picture of the things that matter.

‍

The Open Semantic Interchange specification reached its first full version in early 2026 with backing from Snowflake, dbt Labs, Salesforce, and more than thirty partners, and Microsoft added ontology support to Fabric. Shared business meaning is now becoming the standard infrastructure.

The semantic layer enables AI agents and applications to understand business context, generate more accurate SQL, retrieve relevant knowledge, and reason consistently across structured and unstructured data—turning enterprise data into machine-understandable, business-aware intelligence.

‍

Understanding the business is only half the job

There is a second theme that ran through the Summit. Once an agent can understand your business, it can also act on it, and understanding without control quickly becomes a liability.

An agent acting on your data needs clear limits: what it can see, what it can decide on its own, and where a human signs off. You cannot govern an autonomous agent the way you governed a static model, because the risk now runs across a whole chain of decisions and actions taken at speed. Trust, traceability, and human oversight are what turn a clever pilot into something you can put into production.

‍

How we approach this at Exponentia.ai

Our view is straightforward. Treat the business context as the real project. Once that foundation is in place, the model becomes the comparatively easy part.

In practice that means grounding agents in a governed layer of business meaning, built on the platforms our clients already run, so the context sits close to the data rather than bolted on afterwards. It means building the semantic foundation once and reusing it, so every future agent starts smarter than the last. And it means wrapping the whole thing in governance from the outset through propreity framework GenTrust^TM, our framework for agent registry, evaluation, monitoring and retirement- so visibility, risk tiering, and human approval are designed in well before anything reaches production (add link to GenTrust page / video)

‍

What stayed with me

What stayed with me from the Summit was how much the tone had shifted. Last year the buzz was about what these models could do. This year the conversation was about whether we can trust them with the business. The teams I met who were furthest ahead were quietly getting on with the patient, less glamorous work of giving their AI a real understanding of how their business processes run. That is where I would put my energy - get that foundation right, and the questions your board cares about, start getting answered.

If you were at the Summit too, I’d love to hear your perspective. What were your biggest takeaways?

https://www.linkedin.com/in/niyatisrivastava/

‍

Sources:

McKinsey, The State of AI 2025 (78 percent adoption; roughly 6 percent high performers; 39 percent enterprise EBIT impact), as reported at https://www.punku.ai/blog/state-of-ai-2024-enterprise-adoption

MIT NANDA, The GenAI Divide: State of AI in Business 2025 (around 95 percent of pilots deliver no measurable P&L impact), via Fortune: https://fortune.com/2025/08/18/mit-report-95-percent-generative-ai-pilots-at-companies-failing-cfo/

Gartner (more than 40 percent of agentic AI projects cancelled by end 2027), as reported at https://www.thedataexperts.us/writing/enterprise-ai-failure-crisis-95-percent-failure-rate.html

Snowflake engineering experiment on context grounding accuracy (roughly 10 to 31 percent without context against 94 to 99 percent with it), reported by Atlan: https://atlan.com/know/context-layer-enterprise-ai/

Share of enterprise data that is unstructured (around 90 percent) and Open Semantic Interchange v1.0 plus Microsoft Fabric ontology support, Oracle: