The handoff is the hard part, not the model

Where agents fail

Every agentic AI system we have shipped at Zaphrix has failed its first internal review for the same reason, and it is never the reason the client expects. It is not hallucination. It is not latency. It is not even accuracy on the underlying task. It is that the agent did not know when to stop and ask a human.

Clients come to us wanting an agent that can do something end to end: triage a support ticket, reconcile an invoice, draft a contract clause, pull a lead list and start outreach. The demo version of that agent is easy to build. LLMs are genuinely good at the underlying reasoning now. The gap between a working prototype and a system you can put in front of real customers is not model capability. It is the handoff — the moment where the agent decides its own confidence has run out and control needs to move back to a person, cleanly, with enough context that the human does not have to redo the agent's work to pick it up.

We learned this the expensive way on an early agentic AI project: a lead-qualification agent that read inbound form submissions, scored intent, and drafted a first reply. In testing, it looked excellent. In production, it occasionally sent a confident, well-written reply to a submission that was actually a support complaint mis-routed into the sales form. The agent was not wrong about the words on the page — it was wrong about not noticing that its own classification confidence had dropped below a threshold that should have triggered a human review instead of a send. Nothing in the architecture asked it to notice that. We had built a very capable agent and forgotten to build the off-ramp.

Three kinds of stop: Act, Propose, Hand off

Since then, escalation has become the first thing we design, not the last. Concretely, that means three things go into every agentic AI system before a single production workflow is wired up.

That decision is not a single accuracy number — it is a confidence and risk model that is separate from the task model. The agent scores not just "what should I do" but "how sure am I, and what happens if I am wrong," and those two numbers jointly decide which of the three lanes the action takes.

The escalation contract

None of this is exotic engineering. It is mostly discipline: instrumenting the agent's own uncertainty, and building the human interface for escalation with the same care as the agent's own interface. When an agent hands off, it does not just ping a human — it hands over its reasoning trace, the data it used, and a specific question, so the human is making a decision, not starting an investigation. A minimal version of that contract looks like this:

interface Escalation {
  confidence: number
  risk: "low" | "medium" | "high"
  reasoningTrace: string[]
  data: Record<string, unknown>
  question: string
}
 
function shouldEscalate(e: Escalation): boolean {
  if (e.confidence < 0.7) return true
  if (e.risk === "high") return true
  return false
}

The reason this gets skipped is that it does not show up in a demo. A demo is a curated set of inputs where the agent looks decisive. Production is an uncurated set of inputs where the agent needs to look humble on the ten percent it cannot handle. Clients evaluating an agentic AI vendor should ask to see the escalation path, not just the happy path.

What it costs later

The cost difference between building this in from the start and retrofitting it later is not small. Retrofitting escalation into a live agent means re-architecting the decision layer while the system is already handling real customer interactions, usually under pressure because something has already gone wrong. We have done both, and the build-it-first version is consistently a fraction of the engineering time and carries none of the reputational risk of a bad autonomous action reaching a customer.

This is also why we resist scoping agentic AI projects purely around the model. Model selection matters, prompt design matters, but the part of the system that determines whether a client trusts the agent enough to expand its scope over time is the boring middle layer: the confidence scoring, the escalation UI, the audit log. We build that layer first, prove it with a narrow, low-risk workflow, and only then widen what the agent is trusted to do on its own. It is a slower start and a much faster path to an agent that is still running, unsupervised, a year later.

If you are evaluating whether an agentic AI system is the right next step for your team — or you already have one that keeps needing manual cleanup — we are happy to look at where the handoffs are breaking down. Get in touch and we will walk through it on a call.