Descriptive Metadata as AI Agent Context

Agents don't browse a catalog the way an analyst does on a Tuesday afternoon. They retrieve context at runtime, in the moment they are formulating a query, and they reason from whatever that retrieval returns. Table descriptions, column definitions, metric logic, relationship maps: whatever exists in the metadata layer at that moment is what the agent works from. Stale definitions produce confident-sounding wrong answers. Absent definitions produce the same.

The categories of metadata agents depend on break down roughly like this. Technical metadata covers schemas, data types, and lineage, the basic map of what exists and where it came from. Business metadata is where meaning lives: glossary terms, KPI definitions, ownership, the human-language layer that converts a field name into something actionable. Operational metadata tells the agent whether the data is trustworthy right now, through freshness indicators and pipeline run history. Compliance metadata encodes what the agent is actually permitted to use and under whose identity.

Here is the part that makes metadata gaps systematically dangerous rather than just annoying: the agent's output looks identical whether the metadata it retrieved was accurate and current or stale and incomplete. There is no confidence signal that propagates from a missing definition to the final response. The agent generates fluent, valid-looking SQL and produces a well-formed answer. Nothing in that output signals it reasoned from an undocumented column or a definition that changed six months ago.

The Model Context Protocol, increasingly adopted as a standard for connecting agents to enterprise tools and catalogs, makes this dynamic more consequential. An MCP-connected catalog can expose governed metadata, including definitions, lineage, certification status, and access controls, directly to an agent at query time. But the protocol delivers whatever the catalog actually stores. Weak governance underneath means MCP delivers weak context at scale, efficiently and automatically.

Why "Clean Data" and "AI-Ready Data" Are Solving Two Different Problems

Clean data, as most enterprises have defined it operationally, means consistent formats, no duplicates, referential integrity, validated fields. Those standards were designed for human analysts and BI tools that arrive with their own interpretive context. A person looking at a dashboard labeled "Revenue" brings institutional knowledge about which revenue figure it is, whether refunds are excluded, which business unit's definition applies. The validation pipeline doesn't need to encode that. The human already knows.

An agent knows none of it. For data to be AI-ready, every field must carry its own meaning as an explicit, machine-readable attribute. The agent cannot infer that "Revenue" in the finance schema excludes refunds while "Revenue" in the CRM includes them; someone has to state that distinction in the metadata. Relationships between assets must be explicit and traversable, rather than implied by naming conventions a human would recognize in passing. Quality and freshness signals must be attached and current. Permissions must be scoped to the actual end user's identity, not inherited from a broad service account whose access reflects no particular person's authorization level.

Most organizations are in genuinely good shape by human-analyst standards. The same organizations commonly have undocumented columns, inconsistent metric definitions across business units, and lineage that stops at the warehouse boundary. Those gaps were tolerable when a person could paper over them with a quick conversation or a reasonable inference. They become systematic failures when an agent is the primary consumer, because the agent has no institutional memory and no mechanism for a follow-up question.

Closing this gap requires an active metadata layer, one that updates in response to events rather than on a nightly crawl. A definition that changed last week but won't propagate until tonight's batch job is already stale for an agent acting on it this afternoon.

What a Semantic Layer Does That Schema Documentation Cannot

Schema documentation tells an agent that a column is named "revadjnet" and typed as a decimal. It does not tell the agent what that field means in the context of a business decision, which exclusions apply, which time zone the adjustment reflects, or how it relates to the "revenue" figure referenced in a boardroom presentation. A human reads the surrounding context and figures it out. An agent needs it stated.

A semantic layer bridges that gap through mechanisms schema documentation structurally cannot provide. Business concept mapping connects terms like "customer" or "region," as an agent or user understands them, to the specific tables, joins, and filters that produce those concepts in the actual data. Metric definitions encode agreed logic: what counts as "churn," what is included in "revenue," what time zone "last updated" refers to. Synonym resolution maps the natural language a user might employ, whether "sales," "bookings," or "ARR," to the canonical technical field. Domain scoping disambiguates terms that mean different things across business units, so the agent uses the finance definition of "margin" when answering a finance question and the operations definition when context calls for it.

The governance problem most existing semantic layers haven't fully solved is that they were built to ensure SQL consistency for BI tools. They govern queries. They were designed for tools operating at fixed query volumes and access patterns, not for agents, which operate at different volumes, with different access patterns, and with the ability to combine context from multiple sources in ways a fixed query cannot.

Without a universal semantic layer, you end up with many agents each carrying their own bespoke definitions, producing inconsistent answers to the same business question depending on which agent a user happens to query. The industry has begun to recognize this explicitly. Initiatives like dbt's MetricFlow, Snowflake Semantic Views, Databricks Metric Views, Salesforce Tableau Semantics, and the Open Semantic Interchange effort all reflect a shared understanding that portability matters. Definitions built once and consumable by any agent stack are durable as frameworks evolve. Definitions embedded in a single tool's proprietary layer are not.

The Real Pattern Behind AI Initiative Failures and What the Data Layer Explains

The failure rate for enterprise AI initiatives is not a fringe concern. It reflects the majority experience. A large share of proofs-of-concept are abandoned before reaching production, and among those that do reach production, only a small fraction scale meaningfully. This is the general pattern reported across multiple independent surveys conducted in 2025, and it is consistent enough that the outliers, the organizations achieving meaningful returns, are worth examining for what they actually did differently.

Practitioners consistently identify data quality and readiness as the primary obstacle, ahead of model capability, ahead of tooling, ahead of talent. The CDO Insights 2025 survey reflects this directly. The models are fine. The data layer underneath them isn't ready for machine consumption.

The specific failure mechanism deserves a precise name: when an agent hallucinates in production, the proximate cause is usually a stale definition, an undocumented column, or an access policy that never propagated to the metadata layer. These are not model failures. They are metadata failures the model reproduces as confident output. The model is doing exactly what it was designed to do, reasoning from what it was given. What it was given was wrong.

The most common reason data quality programs stall without closing the gap is that teams focus on fixing data values before aligning on what those values are supposed to mean. Cleaning a field whose definition is disputed or undocumented produces a clean field that still means different things to different consumers.

The ROI signal from organizations that solved the infrastructure problem first is consistent. IDC research shows substantially stronger revenue and cost outcomes from AI among organizations with mature, adaptive governance programs. McKinsey's 2025 research found that organizations achieving significant AI returns were roughly twice as likely to have redesigned data workflows before selecting models. Infrastructure first, model second.

What Governance Means When Agents Are Making Queries Instead of People

The core difference between human query governance and agent query governance is scale and velocity. A human analyst runs queries at a pace that allows for manual review, after-the-fact auditing, periodic reconciliation. An agent can issue orders of magnitude more queries in the same window. Governance architectures designed for human-scale query volume don't degrade gradually at agent scale; they break categorically.

An audit log that captures what query ran, but not who authorized it, under whose identity it executed, and what data it combined, tells regulators and security teams almost nothing useful. Authorization and identity must be captured at query time, not reconstructed after.

Permissions must be evaluated under the actual end user's identity. A shared service account grants every agent that uses it whatever the account can see. Revoking access for one misbehaving workload breaks all downstream workloads sharing that account.

Sensitivity cannot be evaluated only at the field level. Two individually non-sensitive fields can become sensitive when joined. An agent assembling context from multiple sources can cross that threshold without any single retrieval step appearing problematic. Governance must evaluate combinations, not just components.

Write actions require stricter governance than reads. Agents that can modify records, trigger workflows, or initiate transactions operate at a qualitatively different risk level than query-only agents, and the governance architecture should reflect that distinction explicitly.

Regulatory frameworks are moving in the same direction. NIST's AI RMF, ISO/IEC 42001, and the EU AI Act all point toward demonstrable documentation, lineage, and accountability as present-tense requirements for production AI systems in regulated contexts, not aspirational roadmap items.

The only architecture that scales to agent-level query volume is declarative, metadata-enforced policy applied automatically at query time, covering classification, access control, and audit logging together. Manual stewardship cannot operate at the speed or volume agentic systems demand.

How the Data Catalog's Job Changed When Agents Became the Primary Consumer

The traditional data catalog was a directory. Analysts browsed it, found datasets, read the documentation, and then went somewhere else to actually query the data. The catalog lived adjacent to the pipeline, useful for discovery, irrelevant to execution.

That architecture fails when agents are the primary consumer. Agents call the catalog at runtime and expect structured, governed context in response. The catalog must be queryable infrastructure, not a browsable UI. Metadata must be kept current through event-driven updates; an agent acting on yesterday's freshness signal or a definition that changed last week is already operating on stale context. Access control must be enforced and audit records emitted at the point of query, not as a downstream process applied after the fact.

Salesforce's trajectory illustrates what this shift looks like at enterprise scale. Metadata has always underpinned their multi-tenant architecture, ensuring different customers' data stays correctly separated and governed within a shared platform. The push toward large-scale agent deployment through Agentforce has required that metadata layer to evolve from platform foundation into real-time operational infrastructure.

For a catalog to function as agent infrastructure, it needs to support a few distinct capabilities that weren't part of the original design. A semantic layer providing consistent definitions across the enterprise. Transparency into how an answer was generated and from which sources. Access controls enforced at query time under real user identity. Auditability that captures identity, intent, and lineage together per query. And scalability through metadata-driven automation that doesn't require a human steward for every asset as the number of agents and queries grows.

Data Products as the Unit of Trusted Context an Agent Can Reliably Consume

A raw dataset, even one sitting beneath a strong metadata layer, exposes structure. A data product exposes meaning, purpose, quality status, ownership, and access terms as a bundled, self-describing artifact. That distinction matters for agent consumption. An agent querying a data product doesn't need to infer what the dataset is for or whether it has been approved for its intended use; that context is part of the product itself.

For a data product to be genuinely agent-ready, it needs to be discoverable through a governed marketplace with machine-readable metadata, not just findable by a human who knows where to look. It needs to be certified against quality, ownership, and compliance standards before it is ever published. It needs to be self-describing: the purpose, limitations, and usage boundaries an agent needs to use it correctly should be intrinsic to the product, not stored in a separate document someone may or may not have updated recently. And it needs to be interoperable, accessible via governed APIs that aren't locked to a single platform.

Governance enforcement through metadata at the product level propagates automatically to every agent or application that consumes the product. Versioning and deprecation controls prevent agents from acting on data that has drifted from its documented definition. Audit logs capture who accessed a product, when, and in what combination with other products, producing the lineage that makes agent-level queries accountable to the same standards as human-level queries.

The marketplace model is what makes self-service governance operationally viable. Agents, and the humans authorizing them, can discover, request access to, and consume approved data products without involving data engineering for each individual use case. Only certified products are published, and the marketplace enforces the governance baseline structurally rather than through manual review of every request.

Platforms that sit above existing infrastructure and expose a single governed interface, combining semantic context, query-time permissions, and access controls across heterogeneous underlying systems, represent one practical approach to this model. Peaka takes that approach: rather than requiring organizations to rebuild their data infrastructure, it provides a governed, semantically enriched interface over what already exists, so agents can retrieve context at query time without the underlying data estate needing to be homogenized first.

The goal is the same regardless of architecture. An agent should be able to retrieve exactly the context it needs, confirm it is accurate and currently certified, and operate within access boundaries that reflect the actual end user's permissions, without any of that requiring human intervention in the query path. The metadata layer either supports that at runtime or it does not. Right now, for most organizations, it does not.