How AI Agents Query Structured Data

How Agents Decide to Query Data in the First Place: Tool Calls and Function Routing

Agents don't issue SQL spontaneously. They receive a task, reason about which tools are available, and emit a structured tool call to a registered data interface: a function name, a parameter schema, an expected return type. The agent is choosing from a menu, not improvising. That distinction matters more than it appears.

Before Model Context Protocol, introduced by Anthropic in late 2024, teams building agentic data access were largely doing it by hand. Every agent needed bespoke connectors to every data interface, and integration work was consuming the majority of AI project time. Not exactly where you want your best engineers spending their cycles. MCP standardized the interface so that any agent runtime, whether Claude, ChatGPT, or an in-house model, can invoke the same governed data tool without custom glue code. The ecosystem grew from roughly 100,000 downloads in November 2024 to more than 97 million monthly SDK downloads by 2026. That kind of adoption curve doesn't happen because something is theoretically elegant; it happens because people were exhausted by the alternative.

Agent-to-agent protocols extend the same logic. When one agent delegates a data sub-task to a specialist agent, the same tool-call model applies, and the interface stays consistent even as the system grows more compositional.

Why an Agent Can't Just Receive a Raw Schema and Write Correct SQL

Handing an agent a raw schema is not the same as giving it the data. A schema gives structure; it does not give meaning. Column names like "rev\adj\net" or "cust\tier\cd" carry no embedded definition, and the agent has to infer what they mean. Inference is where the pipeline starts to fail, and it fails quietly.

Researchers identify two distinct failure modes in text-to-SQL systems. The first is schema-based hallucination: the model selects the wrong tables or columns during schema linking, the process of matching natural-language terms to database objects. The second is logic-based hallucination: the model selects the right schema elements but constructs the wrong SQL logic. Inaccurate schema linking is the single most significant predictor of overall failure. When a model is presented with hundreds of irrelevant columns, hallucination rates climb. Benchmarks like BIRD-SQL have shown that limiting the agent's context to relevant schema elements, rather than flooding it with the full schema, improves accuracy by a material margin. More context is not always better context.

Unlike a human analyst, an agent cannot turn to a colleague and ask what a vague column name means. Worse, a stale or wrong answer looks exactly as confident as a correct one. There is no hesitation, no hedge. The hallucination problem in text-to-SQL is structural: without embedded business semantics, every query regenerates the same inference problem from scratch. A more sophisticated model doesn't resolve that; it just produces more polished errors.

What a Semantic Layer Gives an Agent That a Schema Cannot

A semantic layer maps raw tables and columns to named business entities, metric definitions, dimensions, and relationships. Instead of reasoning from scratch about which tables to join and how to aggregate revenue, the agent picks from pre-defined measures and forms a structured request: these measures, by these dimensions, with these filters, over this time range. The semantic layer compiles that request to SQL. The hard, error-prone inference work is solved once by the data team, not re-guessed on every query.

A 522-query enterprise benchmark conducted by Snowflake Intelligence and Atlan found reliability above 95% when text-to-SQL was grounded in glossaries, semantic layers, and quality metadata, compared to schema-only approaches. Gartner elevated the semantic layer to essential infrastructure in the 2025 Hype Cycle for Business Intelligence and Analytics. These are not promotional findings; they reflect what practitioners have been learning through failed pilots.

There is a distinction worth making carefully here. A BI semantic layer ensures consistent metric definitions for human users, valuable, but not sufficient. An agent-ready semantic layer must do more: it must also carry table descriptions, entity relationships, and enough contextual scaffolding that an agent with no institutional memory can reason correctly across sources. The agent cannot phone a data steward. So the semantic layer has to be one that serves both purposes.

MCP makes this concrete rather than aspirational. The same governed semantic layer can serve multiple agent runtimes without bespoke integration, and that portability is what makes semantic context actually durable at scale rather than something you rebuild every time you adopt a new model.

How Query Execution Differs When an Agent Is the Consumer

Human analysts run dozens of queries per session. Agents can issue thousands per hour. Execution infrastructure designed for human query patterns does not hold up under that volume, and this is typically the first thing that breaks when agents move from pilot to production.

Agents do not distinguish between a cached pre-aggregation and a live table scan, nor should they have to. The semantic layer must route intelligently, serving cached results where freshness tolerances allow and hitting live sources where they don't. Freshness is use-case-dependent in ways that are genuinely consequential: a snapshot that is perfectly acceptable for a weekly dashboard is already stale for an agent making a real-time inventory decision. The execution layer must know which is which.

Service-account-based access, one broad credential shared across all agent queries, breaks the identity model in ways that become governance problems quickly. Permissions must be evaluated at query time under the actual end user's identity. An agent acting on behalf of a marketing analyst should not inherit the data access of a finance executive simply because they share the same service account. The scenario sounds unlikely until it happens.

Write actions compound the problem further. Agents are increasingly being asked to take them, and writes carry a fundamentally different risk profile than reads. Stricter governance at the execution layer is required for writes, not just at the model layer. Platforms that validate generated SQL against actual schemas before execution report that this step catches a substantial share of potential errors before they reach users or downstream systems.

What Governed Execution Requires: Permissions, Audit, and Lineage at Agent Scale

Governance for agents is not a compliance layer you bolt on after the query runs. It must be enforced at the point where data is combined and returned, because sensitivity can emerge from combination even when individual fields are benign in isolation. A query that joins salary data with department and location is a privacy event, regardless of whether any of those individual columns are flagged as restricted.

A 2025 survey by Precisely and Drexel University found that 62% of organizations cite lack of data governance as their primary challenge inhibiting AI initiatives. That bottleneck exists before agent-scale query volume is even a factor. Add agents to an under-governed data estate and the bottleneck doesn't bend; it snaps.

Audit logs must capture identity, intent, and lineage together, at agent query volume. A log that records only "service account ran a query" is operationally useless. You cannot answer "why did this decision get made" from that log. You cannot investigate an anomaly. A regulator asking to trace a decision to its underlying data will not be satisfied by it either.

The regulatory stakes are specific. The EU AI Act's high-risk AI obligations, including mandatory log retention, human oversight requirements, and Fundamental Rights Impact Assessments, become enforceable in August 2026. An agentic architecture that cannot produce auditable lineage from agent decision to specific data product and semantic definition will not meet those requirements.

Access must also be revocable at the workload level. Revoking a shared service account to cut off one misbehaving agent breaks every other agent that depends on it. Granular revocability is a requirement. Security risks specific to agent-data interfaces deserve explicit acknowledgment as well: prompt injection, credential theft through MCP tool calls, and unauthorized data modification. These cannot be addressed solely at the model layer; they require read/write separation and robust access controls embedded in the data layer itself.

Why Most Enterprise Data Infrastructure Wasn't Designed for This Pipeline

Enterprise data systems accumulated faster than they were integrated. Each source has its own schema, its own internal logic, its own history, and those histories often contradict each other in ways nobody fully documented. Relational models freeze early design decisions into table structures. Traditional formats store structure but not meaning. The result is an estate that holds enormous information but obscures the semantic context agents need to reason across it. This isn't a failure of ambition; it's just what happens when systems are built to solve today's problem without anticipating next decade's consumer.

Only 7% of enterprises say their data is completely ready for AI, per a March 2026 report from Cloudera and Harvard Business Review Analytic Services. Seven percent. That number has been sitting in various forms of infamy at data conferences for a few years now, and it keeps being roughly accurate.

The failure pattern is consistent. A pilot succeeds in a controlled environment with a narrow, well-documented slice of data. Then it moves to production and encounters the full enterprise data landscape: conflicting definitions across business units, lineage that disappears at system boundaries, and access rules that were never designed with non-human consumers in mind. By 2025, a significant share of enterprises had abandoned most of their AI initiatives, up sharply from the prior year, according to an S&P Global survey of more than 1,000 enterprises. The timing correlates with agents reaching production and finding the data layer unprepared. The models were fine; the infrastructure was not.

This is a structural mismatch, not a tuning problem. Prompt engineering doesn't close a governance gap. Better model selection doesn't resolve conflicting metric definitions. The pipeline requires infrastructure changes, and the sooner organizations stop treating it as a configuration problem, the sooner they stop cycling through failed pilots.

What Purpose-Built Infrastructure for Agent Data Access Actually Looks Like

The architecture that can support this pipeline has several properties that aren't optional, and it's worth being direct about what they are.

It starts with a unified query interface that sits on top of existing warehouses, SaaS tools, and operational systems without requiring a full data migration. Most enterprises cannot rip and replace their data estate to reach an agent-ready state. Good infrastructure meets them where they are, which is an obvious point that a surprising number of vendor pitches ignore.

Semantic context must be embedded persistently at the layer between agent and data: table descriptions, metric definitions, entity relationships, business rules that survive across query sessions. No single agent query should have to re-infer them. The institutional memory lives in the layer, not in the model's context window, which has limits and doesn't persist.

Runtime permission enforcement must resolve to the actual end user's identity, not a service account. Every query evaluates access rules at execution time for the human on whose behalf the agent is acting. Full audit lineage is required: every query logged with who asked, under what identity, which semantic definitions were applied, and which sources were touched, in enough detail to satisfy both internal governance and the regulatory obligations already on the horizon.

Freshness routing belongs in the infrastructure layer, not in the agent's reasoning. The layer knows which use cases tolerate cached results and which require live source queries. Agents should not be making that judgment call; they lack the operational context to make it consistently.

The vendor landscape is consolidating around this model. Snowflake's Open Semantic Interchange initiative, backed by dbt Labs; dbt's open-sourced MetricFlow; Databricks' Metric Views; and Salesforce's Tableau Semantics: all of these point toward semantics as a portable, above-warehouse layer that persists independent of any single agent runtime. MCP as the agent-facing protocol completes the stack.

The clearest test for whether infrastructure is agent-ready is simple, even if passing it is not: can a net-new agent, with no pre-loaded institutional knowledge, issue a governed query against the right data, under the right user's permissions, with a full audit trail, without a human stepping in to explain the schema? If a human still has to intervene, the pipeline isn't ready. It's just an analyst doing the same work with an extra layer of indirection sitting in front of them.