Learned in Crossing the Chasm
When looking at the adoption of AI in the workplace, the issue isn’t people building agents – almost everyone is. It’s getting the use and deployment to stick with in companies. New research from MIT shows that 95% of gen AI pilots fail to produce measurable financial impact (MIT / NANDA Initiative, 2025). 80% of deployment effort is consumed by data, governance, and workflow integration, not AI. That single figure should reframe how every team approaches this work.
What this statistic is showing is the whole story. While it’s easy to blame model quality, and assume that the final interaction point is where the problem lies, it simply isn’t true. The real issue lies in poor workflow integration, missing governance, and metrics that were never aligned to a business outcome in the first place. After deploying agents in real enterprise and industrial environments, here are the eight lessons that separate the pilots that quietly die from the agents that actually run in production.
Start with the Right Problem
The fastest way to kill an agentic project is to fall in love with the word “agentic” before you have defined the outcome you are chasing. We see teams stand up ten agents at once, convinced that breadth signals ambition. More often it signals ten simultaneous failures, because none of them was given the chance to prove value first.
The discipline that works is almost boring in its simplicity: pick one high-impact, low-risk workflow and earn the right to scale by demonstrating value there. Before you commit, get honest about process variance — the places where human judgment still has to enter the loop — because those are exactly the points an agent cannot paper over. And ask the question most teams skip in their excitement: could a simple if-then rule handle this? If the answer is yes, use the rule. Agents are expensive, opaque, and hard to govern; reserve them for genuine complexity, not problems that deterministic logic already solves.
Data Is the Real Bottleneck
If 80% of deployment effort goes to data engineering, stakeholder alignment, and workflow integration, then the honest conclusion is that the bottleneck was never the model. It is data readiness and on this point, practitioners are in near-unanimous agreement.
In practice this plays out in three ways. The first is standardization: agents need data converted into uniform formats they can interpret reliably across sources, and that work has to happen before you scale, not after the cracks show. The second is breaking down silos, because an agent operating on partial information will fail in ways that are difficult to detect and embarrassing to explain — fragmentation is the single biggest barrier we encounter.
The third is continuous validation. Robust API management, version control, and pipeline checks are not glamorous, but they are what keep agents running once the launch-day attention fades.
Build Governance Before You Build Agents
Governance is not a phase you bolt on after the demo impresses someone. It is the foundation you pour before anything else stands on it, and in our experience it comes down to six commitments.
Every agent needs a named owner, a written job description, and a defined scope settled before deployment — not negotiated after something goes wrong. Promotion to production should move through staged gates, from dev to UAT to pre-prod to prod, with a human signing off at each one. Escalation paths have to be mapped explicitly, so it is clear which decisions the agent makes on its own and which it must hand to a person. You should plan for retirement from the start, defining exit criteria before an agent becomes so entrenched that no one dares turn it off. Accountability must be assigned to a human, because “the LLM said so” is never an acceptable answer when something breaks. And finally, patterns need to be standardized — common permission tiers, connector policies, and reusable structures — or you will wake up one day to find agent sprawl you can no longer reason about.
Onboarding an Agent Is More Like Hiring Than Deploying Software
“Onboarding agents is more like hiring a new employee versus deploying software.” — Business leader interviewed by McKinsey
That reframing changes the work in concrete ways. You give an agent a job description the same way you would a new hire — clear scope, the tools it is allowed to use, and the behavior you expect of it. You build feedback loops that verify performance at each step of the workflow and let you refine continuously after deployment, rather than treating launch as the finish line. And you invest in evaluations: benchmarks that codify what good looks like for each task the agent performs, so that “is it working?” becomes a measurable question instead of a gut feeling.
Observability and Hallucination Control
A 2% hallucination rate is perfectly fine for a chatbot and catastrophic for a claims-processing or financial agent.
The acceptable failure rate is set entirely by what the agent touches, and that should shape how much observability you build in. It starts with logging far more than conversations. You need full traceability of inputs, tool calls, reasoning steps, and outputs, because the day a regulator or an auditor asks why an agent made a particular decision, the answer has to be reconstructable. Beyond logging, you have to evaluate at every step rather than only inspecting the final output, that is how you catch errors early and refine logic continuously, even after the agent is live. And you need real guardrails against prompt drift, model drift, and scope creep: defined escalation triggers and rollback procedures established before you ever touch production, not improvised during an incident.
Keep Humans in the Loop — By Design
Autonomy is something an agent earns, not something you grant on day one. The path runs through four deliberate stages, and skipping any of them tends to end badly.
It begins with a pilot, where you define the critical decision points at which the agent must escalate to a human and let it earn autonomy incrementally from there. Rollout comes next, and the goal is collaboration rather than substitution — workflows redesigned so people and agents work together, with handoffs mapped explicitly so nothing falls through the gaps.
Only once performance is proven and auditable do you scale, expanding autonomous operation while tracking accountability clearly the whole way. And running through all of it is governance: a clear delineation of who bears responsibility when the agent makes an error or causes harm, documented before an incident forces the question, not after.
The Model Is Not the Competitive Advantage
This lesson tends to surprise people, because it contradicts where most of the industry’s attention goes: 42% of enterprise implementations found their model choice to be fully interchangeable (Stanford, 2026).
If the model is not the moat, where is the durable advantage? It lives in the quality of your orchestration layer, the strength of your governance frameworks, the depth of your workflow integration, and the institutional knowledge of operating agents at enterprise scale that you can only accumulate over time. So the smarter investment is not the latest foundation model — it is the infrastructure underneath it: reusable agent components, validated architectural patterns, shared permission tiers, and connector policies. Get that foundation right and almost any capable model will perform well on top of it.
Security Enables — It Doesn’t Just Block
The most counterintuitive lesson is also one of the most reliable: the security requirements that look like project killers at the outset are usually the very things that make it possible to put agents near sensitive data in production at all.
Done well, security operates as four reinforcing layers. Prompt filtering blocks injection attempts, jailbreaks, and scope violations before the agent ever acts on them. Data protection classifies information by sensitivity and enforces least privilege, so an agent can reach only what its role genuinely requires. External access control puts every tool call, API hit, and outbound action through a defined permission gate. And response enforcement validates the agent’s outputs before they reach downstream systems or users — because the output is itself a trust boundary, and treating it as one is what keeps a single bad generation from propagating through everything connected to it.
The Enterprise AI Agent Readiness Checklist
Before you ship, you should be able to answer yes to all of the following:
✓ Have you defined a specific, measurable business outcome before building?
✓ Have you fixed data silos, standardized formats, and validated your pipelines?
✓ Have you built governance — ownership, gates, escalation — before agents go live?
✓ Have you established full observability across inputs, tool calls, reasoning steps, and outputs?
✓ Have you kept humans in the loop at critical points and let autonomy be earned incrementally?
✓ Have you invested in the orchestration layer rather than the newest foundation model?
✓ Have you applied all four security layers: prompt, data, access, and response?
✓ Have you planned for agent retirement from day one?
The Bottom Line
The agents themselves are rarely the hard part. The hard parts are the orchestration, the governance, the data foundation, and the integration into enterprise infrastructure — the unglamorous work that determines whether anything reaches production at all.
The enterprises that pull ahead will be the ones that stop treating agentic AI as an experiment and start treating it as operational infrastructure. That is the shift underway right now: from automating tasks to entrusting decisions.
SOURCE: Somya Kapoor (2026 June 30) Productizing AI Agents: Lessons Learned in Crossing the Chasm. IFS Blog. https://blog.ifs.com/productizing-ai-agents-lessons-learned-in-crossing-the-chasm
