The Intent Gap: Why Your AI Agents Are Optimizing for the Wrong Thing

Klarna's AI customer service agent processed 2.3 million conversations in its first month across 23 markets and 35 languages. Resolution times dropped from 11 minutes to under two. The company banked $60 million in savings by Q3 2025.

Then CEO Sebastian Siemiatkowski went on Bloomberg and walked it back. Cost, he admitted, had been "too predominant evaluation factor." The result: "lower quality." Klarna quietly began rehiring the human agents it had cut months earlier.

The AI worked brilliantly. That was the problem. It optimized for exactly what it was told to optimize for. The objectives that actually mattered to the business, relationship quality, brand trust, lifetime value, had never been expressed in any form the agent could act on.

This pattern shows up in every large enterprise I work with. A customer service agent that drives resolution speed when the actual priority is retention. A content agent that maximizes output volume when brand consistency matters more. Capable AI systems pointed at the wrong target because nobody built the infrastructure to aim them.

No One Owns the Translation

Here's the question I keep coming back to: who in your organization is responsible for translating strategic intent into something an AI agent can actually act on?

In most organizations, the answer is nobody.

The executives who understand what the company is trying to accomplish don't build agents. The engineers who build agents rarely have deep visibility into strategic priorities. I've watched this disconnect play out repeatedly. A CTO describes transformation goals in language that means nothing to the platform team. The platform team builds something technically impressive that serves none of those goals. Both sides assume alignment happened somewhere in between. It didn't.

Nate Jones, a product leader and former Amazon VP, recently gave this problem a name that I think is exactly right. In a YouTube video and Substack article that have been circulating in enterprise circles, Jones calls it intent engineering: encoding what an organization actually values, its goals, tradeoffs, and decision boundaries, into forms that autonomous systems can act on. His core framing is sharp:

"The distinction between AI that fails and AI that succeeds at the wrong thing is the most important unsolved problem in enterprise AI right now."

Jones traces a progression. Prompt engineering was individual and session-based: one person, one chat window, one set of instructions. Context engineering is where the industry works now: wiring up RAG pipelines, standing up MCP servers, structuring company knowledge so agents can reach it. Anthropic formalized the concept in September 2025. Harrison Chase of LangChain called it "everything we've done at LangChain without knowing the term existed" in a Sequoia Capital interview. Intent engineering is the third discipline: where context engineering gives agents access to what they need to know, intent engineering encodes what the organization actually needs them to prioritize.

Jones's analysis is the clearest articulation I've seen of where the gap lives. What I want to do in this series is go further: lay out the organizational machinery required to close it. Because naming the gap is not the same as building the infrastructure to fix it.

The Scale of the Disconnect

The investment numbers and the failure numbers aren't in tension. They describe the same phenomenon from different angles.

Enterprises are spending aggressively. PwC found 88% of senior US executives plan to increase AI budgets in the next 12 months. Deloitte's 2026 report shows organizations putting 21 to 50% of digital transformation budgets into AI automation. And the results are failing at roughly the same rate the checks are clearing. MIT's GenAI Divide report found roughly 95% of generative AI pilots fail to deliver measurable business impact. S&P Global reported companies abandoning the majority of their AI initiatives surged from 17% to 42% year over year. Gartner predicts over 40% of agentic AI projects will be canceled by end of 2027.

Microsoft's Copilot tells the same story at a single-product scale. Nearly 70% of Fortune 500 companies adopted it. Then only 5% moved past pilot. Only 3.3% of the total Microsoft 365 user base became paid Copilot users. The standard explanation points to UX and model quality. Those are real issues, but they don't explain why adoption collapses even when the technology works.

Most organizations have answered the first question: "Can AI perform this task?" The answer is usually yes. The question they haven't touched: "Can AI perform this task in a way that serves what we're actually trying to accomplish as a company?" That second question is the intent engineering question. Almost nobody is working on it systematically.

Three Layers of the Gap

In his video, Jones breaks the intent gap into three layers. The framework holds up well against what I see in enterprise engagements, though the organizational reality at each layer is messier than any framework suggests. Here's how I'd characterize what's actually happening.

Layer 1: Context Infrastructure

This is the layer the industry talks about most, and it's still mostly absent in practice.

In every large enterprise I work with, the pattern repeats. Each team building agents assembles its own context stack from scratch. The sales engineering team has a RAG pipeline pulling from Slack. The product team built a custom integration with their wiki. A platform team stood up an MCP server for CRM data but never connected it to the project management system. And a fourth group across the building is duplicating all of it without knowing the others exist.

Anyone who lived through the shadow IT era of early cloud adoption will recognize this pattern. But the stakes are materially higher. Shadow IT meant employees spinning up unauthorized SaaS tools. Shadow agents means unauthorized autonomous systems accessing customer PII, financial records, and regulated data, then making decisions based on it. Security and compliance teams know this is a problem. Without sanctioned infrastructure that's easy enough to actually use, they can't stop it.

The Model Context Protocol (MCP), which Anthropic introduced in late 2024 and donated to the Linux Foundation in December 2025, is the strongest protocol-level move toward standardization so far. OpenAI, Google, Microsoft, and dozens of enterprise partners have committed to it. But having a standard protocol and having a functioning organizational implementation are completely different problems. A shared connector standard is useless if nobody in your company has decided which systems to connect, who governs the connections, or what data flows through them.

The hard questions at this layer aren't technical. Does the finance team's data become visible to a marketing agent? Who draws those boundaries? How do you keep agents from operating on last quarter's strategy when this quarter's priorities have changed? These are governance decisions that disguise themselves as infrastructure problems.

Layer 2: Workflow Architecture

Walk through any floor of a large enterprise and you'll find a dozen different AI workflows. None documented. None shared. An engineer has a sophisticated agent chain. A product manager is copy-pasting between three chat interfaces. A director built a custom Slack integration that nobody else uses. Each person has optimized for their own productivity, and none of that optimization is transferable.

The distinction that matters here is between individual AI productivity and organizational AI capability. In my experience, the teams getting real value aren't the ones with the most AI tools. They're the ones that rethought how work actually flows. You can't train individuals into that kind of fluency. It requires shared infrastructure that no amount of personal productivity hacks will replace.

Deloitte's 2026 report found that workforce access to sanctioned AI tools expanded by 50% in a year. But access isn't the bottleneck. Organizations are distributing tools without distributing the organizational context that makes those tools useful. The models work. The organizational layer connecting them to purpose doesn't exist.

Layer 3: The Intent Layer Itself

This is the layer that doesn't exist yet. It's also the one that matters most.

Think about how alignment works for human employees. Your OKRs are written in natural language for people who can read between the lines. When a manager says "here's what matters this quarter," the team interprets that through everything they've absorbed: which priorities leadership actually enforces under pressure, which tradeoffs get rewarded and which get punished, the unwritten rules about when to follow process and when to break it. That interpretation builds over months of watching real decisions get made.

Agents don't have that learning curve, and they never will. An agent won't pick up on the difference between your stated values and your revealed preferences. If your OKRs say "customer satisfaction" but your incentive structure rewards throughput, a human employee figures that out within a quarter. An agent will optimize for whatever you told it to optimize for, literally and indefinitely.

This means organizations need to build something most have never attempted: explicit, structured expressions of what they actually want, encoded in a form that machines can act on.

Jones calls this a "cascade of specificity." A goal like "increase customer satisfaction" is a human aspiration, not something an agent can execute against. To make it actionable, you need to decompose it into measurable terms, locate the signals in your systems, and define what the agent is empowered to do versus what requires escalation. Most organizations have never had to answer these questions with this level of precision because human employees filled the gaps with judgment. Jones also introduces "delegation frameworks," your organizational principles translated into decision logic that an agent can follow. I'll build on this concept extensively in the third post in this series when I introduce Agent Actionable Objectives.

And underneath all of it, you need feedback mechanisms. When agents are making hundreds of decisions per hour, how do you detect drift from what the organization intended? Without closed-loop measurement, misalignment compounds silently until it surfaces as a Klarna-sized problem.

Why This Function Doesn't Exist Yet

MIT's GenAI Divide research found that the dominant barrier to enterprise AI value isn't technology or budget. It's organizational design. Companies that succeed empower domain leaders and cross-functional teams to drive adoption rather than centralizing it under a single technical function. As long as most enterprises treat AI as a centralized technology initiative instead of a cross-functional business capability, the gap I'm describing is inevitable.

CIOs can build infrastructure. But the intent layer requires input from the entire leadership team: strategy, operations, product, legal, finance. The emerging CAIO role (present in roughly 26% of organizations according to IBM's 2025 survey) focuses on governance, compliance, and high-level strategy. The nascent Chief Agent Officer concept focuses on agent lifecycle management. Neither is specifically building the translation layer between what the organization wants and what agents actually optimize for.

This translation function requires something unusual in most enterprises: cross-functional authority spanning strategy, engineering, operations, and data. It's not a VP-level role tucked under a CTO or COO. It needs direct executive sponsorship and the mandate to work across every domain that touches how the organization makes decisions.

The most honest framing: it's a capability the leadership team must build together, with someone accountable who has direct access to the CEO and authority to work across all domains. Deloitte's own data reinforces this. Their 2026 report found that enterprises where senior leadership actively shapes AI governance achieve significantly greater business value than those delegating the work to technical teams alone.

The technology moves too fast for organizational thinkers to keep up. The technologists don't think alignment is their problem. And in that gap between the two groups, agents are making thousands of decisions a day with no connection to what the organization actually values.

What I'm Building Toward

I'm going to go deep on two frameworks in the next two posts.

The first is what I'm calling an Organizational Capability Map for AI: a living document that maps which of your organization's workflows are agent-ready, which require human-in-the-loop oversight, and which remain human-only. Not a static document that gets filed in Confluence. A system that evolves as agent capabilities mature and organizational trust grows. Without this map, you get Klarna: agents deployed into workflows they had no business owning autonomously.

The second is Goal Translation Infrastructure and the concept of Agent Actionable Objectives (AAOs): structured, machine-readable expressions of what your organization actually wants agents to optimize for. These are not OKRs pasted into a prompt. They're a new artifact type, designed for systems that can't interpret ambiguity or fill in gaps with judgment. OKRs gave Intel a way to align thousands of humans around shared objectives in the 1970s. Agent Actionable Objectives are the equivalent management innovation for aligning thousands of agents in 2026.

The Organizational Infrastructure Race

For three years, the AI conversation has been an intelligence race. Which model tops the benchmarks? Whose context window is biggest?

That framing made sense when model quality was the limiting factor. It's not anymore. The current generation of frontier models are all remarkably capable for most enterprise use cases. The bottleneck has moved. Klarna didn't fail because GPT-4 wasn't smart enough. Copilot didn't stall because the model couldn't write emails. They failed because the organizational infrastructure to aim those models didn't exist.

The instinct is to solve AI adoption by improving the AI. Better models, better prompts, better context pipelines. All of that matters. But none of it addresses the core question: does the AI know what your organization is actually trying to accomplish, and can it make decisions accordingly?

The companies that build that infrastructure, the intent layer, the capability maps, the goal translation systems, will compound their advantage over every competitor still buying licenses and hoping for the best.

This is the first post in a three-part series on Intent Engineering and organizational readiness for agentic AI. Next up: the Organizational Capability Map, the framework for deciding which workflows your agents should own, which they should augment, and which should stay human.

I'm working through these frameworks with enterprise leaders who are facing this challenge right now. If this resonates, I'd like to hear how your organization is approaching it. Find me on LinkedIn or reach out at jonathan@jonathangardner.io.