Building production agents

Demo-perfect, production-fragile

Almost every agent demo works. That’s the trap. A scripted path, a clean input, a forgiving audience — anything looks autonomous for ten minutes. Production is the opposite: messy inputs, half-failed tool calls, state that has to survive a restart, and nobody watching at 3am. The gap between the two isn’t the model. It’s the architecture around it.

That architecture is what I build.

The three things that make an agent dependable

• Memory that’s deliberate. What the agent remembers, for how long, and how it forgets. Short-term working state, long-term grounded knowledge, and a clear boundary between them. Memory without decay turns one bad inference into a permanent belief; memory without grounding is just a confident guess. Both are designed, not left to a framework default.
• State that survives a restart. A production agent can be killed mid-task and resume exactly where it left off — pause, hand off, recover. If the whole run lives in one heroic prompt, a dropped connection takes the entire job with it. Externalised, checkpointed state is the difference between an agent and a long prompt holding its breath.
• Orchestration with a definition of done. Multi-step work needs a plan, typed tool contracts, retries, and a verifiable “done” the agent can’t fake. “I closed the ticket” has to mean the ticket is closed and the check passed — not that the model narrated a plausible completion.

The harness, not the prompt

I think of this as harness engineering: the engineered loop the model runs inside — plan, gather context, reason, act, observe, learn — going around a thousand times a night across a fleet. The model owns one beat. The other five are architecture, and they’re where projects live or die.

Get the harness right and the model becomes a config value: you swap it in an afternoon, and everything you built around it keeps working. Get it wrong and you have a clever demo that falls over the first time a tool call times out.

Security is a property of the build

Because agents act, the threat model is real — prompt injection, over-broad tokens, irreversible tool calls. I treat every input as hostile, give each agent its own scoped identity and short-lived credentials, classify every tool by blast radius, and put high-impact actions behind approval. But this is built into the architecture, not bolted on afterwards. A dependable agent is a secure agent, because both come from the same discipline: knowing exactly what the system can do, and proving it.

What it unlocks

When the architecture is right, autonomy stops being scary. You can give an agent more scope because you can see what it’s doing, stop it when you need to, and trust that it will pick up cleanly when something fails. Dependable beats impressive every time the demo ends and the real work starts.