Harness Fundamentals: Long-Running Agents

The workspace was green. The file had the right hash. A fresh run asked to call it done, and the CLI returned exit code 2. Nothing was broken. That was exactly why it had to fail.

Run one watches a check fail, changes the file, watches it pass, and earns a valid red-to-green acceptance. Run two starts against the same already-green workspace and calls the same gate.

The correct result is still blocked.

The regression is called test_prior_run_evidence_cannot_satisfy_a_new_run. I wrote it while combining a governed, durable runtime I’d built with ATV-Phoenix, a failure-first verification layer. I’ll call the two halves the runtime and the verifier from here. The runtime owns policy, replay, leases, and final receipts. The verifier owns sensing, snapshots, bounded recovery, and red-to-green proof.

The receipt belongs to the run that earned it. The new run has no failure-first trace of its own, no acceptance authenticated to its own run ID, and no right to turn inherited state into a fresh claim of completion. If that second run returned 0, any new worker could borrow an old victory without doing one new step.

That was the moment the long-running-agent problem stopped looking like “how do I give the model more memory?” A long-running agent isn’t one mind working for hours. It’s a shift system — fresh contexts arriving at the same worksite, each capable of repeating work, racing another worker, inheriting stale evidence, or deciding the site looks finished.

The model gets a shift. The harness owns the handover.

Anthropic frames the same problem as engineers arriving for shifts with no memory of the shift before. Its Effective harnesses for long-running agents turns that image into an operating pattern: one initialiser session builds the environment, then fresh coding sessions complete one feature at a time and leave clear artifacts behind. The autonomous coding demo persists a feature list, progress notes, an init.sh, and Git history so the next context can resume.

I took that split and put it under the runtime. Four more failure classes showed up almost immediately: two workers arriving together; a process dying after the side effect but before the receipt; a proof going stale after it was issued; and a new run trying to inherit an old run’s victory.

This is the next layer under my earlier post about a run killed at 2 am and resumed at 2:01. That one was about surviving a process restart. This one is about surviving a change of worker without letting the truth change with it.

The short version:

• The first shift builds the worksite. Later shifts do one bounded piece of work.
• The conversation is working memory, not run state. Anything load-bearing lives outside it.
• One worker owns the run. A lease without fencing is not enough.
• An authorised side effect with no recorded outcome makes the run fail closed until the outcome is resolved.
• “Done” needs proof scoped to this run, verified again against the state that exists now.

The shift changes. The worksite doesn’t.

Every context window is temporary. The repository, feature contract, progress record, event log, snapshots, and acceptance trace are not.

That sounds obvious until you watch a fresh session inherit a beautifully written summary and the wrong reality. Summaries compress uncertainty. They turn “the check timed out” into “testing was attempted.” They turn “I think the tool ran” into “tool completed.” The prose gets cleaner as the truth gets weaker.

So the conversation can’t be the source of truth. It’s a temporary view assembled from the source of truth.

Compaction helps. Larger context windows help. Better summaries help. None of them should be load-bearing. If you can drop a fresh process into the directory and it can’t answer four questions, the run isn’t durable:

1. What is the finish line?
2. What has actually been proven?
3. What is the next smallest useful move?
4. Does somebody else own the run right now?

Miss one and the new shift starts by guessing.

The first shift builds the worksite

Anthropic’s initialiser/coding split fixes a failure I’ve seen repeatedly: the first context tries to build the whole product in one pass, runs out of room halfway through, and leaves a large ambiguous change for the next context to reverse-engineer. Later, another context sees a mostly working application and quietly promotes “substantial progress” to “done.”

The initialiser has a different job. It creates:

• a comprehensive list of testable features, initially marked failing;
• an init.sh that can bring the environment up the same way every time;
• a progress file the next shift can scan without replaying the run;
• an initial Git commit that defines known starting state;
• a workspace boundary that stops the agent wandering outside the project.

In Anthropic’s demo, that feature list can hold more than 200 testable entries. The quickstart warns that the first session can take several minutes while it writes them. Good. The first shift is spending its budget turning a fuzzy request into a worksite that later shifts can trust.

Every coding shift is smaller: orient, pick one unfinished feature, implement it, test it end to end, commit it, and leave the site clean.

The first shift’s output isn’t the product — it’s a site the second shift can enter without asking what happened.

Five contracts have to survive the shift

Once I put the Anthropic handoff pattern beside the runtime and verifier, I ended up with five contracts.

Contract	The question it answers	Failure it prevents
Outcome	What exactly counts as done?	The agent drifts, builds everything at once, or stops early.
Handoff	What happened, what is true now, and what comes next?	Every fresh context rediscovers the project and repeats mistakes.
Ownership	Who may advance this run right now?	Two workers duplicate work or race the same side effect.
Recovery	What was the last state we know was good?	One bad step contaminates every shift after it.
Proof	What evidence permits this run to say complete?	The model grades itself or reuses stale evidence.

Outcome: freeze the finish line

The finish line exists before implementation. Anthropic uses a structured feature list. The verifier freezes a top-level acceptance check and gives every backlog item its own runnable check.

The rule is more important than the file format: the agent may satisfy the check; it may not quietly weaken it.

For a bug fix, write the reproduction and observe it fail. For a feature, add the acceptance check first and observe the missing behaviour. If you can’t write a meaningful check, you don’t yet have an unattended task. You have a task that still needs a human to decide what “good” means.

Handoff: externalise the working memory

A progress file is cheap to read. Git history records real changes. An append-only event log is stronger again because the runtime can rebuild the model’s view from completed model and tool turns instead of trusting a polished recap.

The handoff should tell the next shift, in under a minute:

• the current goal and frozen acceptance check;
• the last completed slice and its evidence;
• anything unknown or blocked;
• the next failing item worth attempting.

If the next context needs the old transcript to reconstruct any of that, the handoff is carrying a story instead of state.

Ownership: one run, one worker

The moment a scheduler retries, “resume” can quietly become “run twice.” A browser refreshes. A worker times out while another one picks up the session. Suddenly two agents believe they own the next step.

The runtime implementation I tested uses a process-local exclusive lease. A worker must acquire it before replaying and advancing the event log:

with self.store.lease(session_id):
    session = self.store.get(session_id)
    events = self.store.events(session_id)
    # replay completed turns, then advance exactly once

A second local worker fails fast with SessionBusyError — and the regression test asserts that only one model event was written.

That lock is enough for one process. It isn’t the production answer for a distributed service. A service-backed store needs atomic acquisition and renewal plus a fencing token that changes with each owner and is checked at every mutation boundary. A production store would need a mutation fence like this:

def append_event(session_id, event, fence):
    if fence != store.current_fence(session_id):
        raise StaleOwner("lease moved to another worker")
    store.append(session_id, event)

Expiry alone doesn’t stop the old worker waking up late and writing after ownership has moved.

Resumable doesn’t mean concurrently replayable.

Recovery: save green, not merely recent

A recent checkpoint can still be broken. The useful checkpoint is the last state you proved was good.

The verifier snapshots only after an objective check passes. If the next step stays red, recovery is bounded: return to the blessed snapshot, retry within a fixed budget, then escalate. The loop never advances because the model says the failure “looks unrelated.”

That keeps damage close to the change that caused it. Without that boundary, one bad edit survives three context windows and turns into archaeology.

Proof: make completion current and run-scoped

Anthropic pushes the coding agent to test the application through the browser as a human would, not stop at unit tests or a successful curl. The verifier adds a stricter completion predicate. Normalised into Python, the logic is literal:

def accept(check, trace) -> bool:
    return (
        trace.intact()
        and trace.saw_red(check)
        and trace.green_after_red(check)
        and check.run_now().green
    )

The runtime adds one more binding: the successful acceptance receipt is authenticated to the current run. The signed payload includes the gateway-normalised target and expected digest:

{
  "nonce": "fresh-run-nonce",
  "check": {
    "kind": "file_sha256",
    "target": ["/workspace/green"],
    "expect": "b24d6d33736ecd5604a4b17bc9c6481039fac362bb7df044ef1c10a2bfd21db6"
  },
  "result": {
    "ok": true,
    "check_digest": "cd83d20950c593f00cacb40503728216bfeca2cf39bf89cb09f6e5018f471e98",
    "trace_intact": true,
    "saw_red": true,
    "green_after_red": true,
    "currently_green": true,
    "reason": "failure-first satisfied: red→green in an intact trace, currently green"
  },
  "mac": "hmac-sha256(nonce, check, result)"
}

A prior run’s proof can’t satisfy a new run, even when the workspace is still green.

Red-to-green isn’t magic — it records that the check saw a failing state before the change and a passing state after it. That evidence only means what you think it means if the red failure happened for the right reason and the check measures the real outcome. A strong proof of a weak check is still a weak result.

The check is part of the trusted system. Treat it like production code.

The crash after the action is where the lie starts

In May, I wrote that idempotency made the external world see each effect exactly once.

That sentence was too neat.

It’s true only when the downstream system actually enforces the idempotency key or lets you query the action by that key. The event log by itself can’t tell whether a side effect finished after the process died.

Here is the ugly sequence:

1. The harness authorises issue_refund(order_4012).
2. It persists the authorised tool intent.
3. The payment system may process the refund.
4. The process dies before the harness records the result.

On restart, the log proves the refund was allowed. It doesn’t prove whether money moved.

Run it again and you may pay twice. Skip it and you may pay nothing. The honest state is not success or failure.

It’s unknown.

The runtime doesn’t persist UNKNOWN as a successful or resumable terminal state. It marks the session FAILED because the external outcome is unknown:

status = "failed"
reason = "incomplete_tool_turn"
error  = "authorized tool outcome is unknown; reconciliation required"

On resume, it makes zero new model calls and zero tool calls. It fails closed before either can guess.

The fix is tool-specific. Retry with the same idempotency key if the downstream system genuinely enforces it, or query the action by that key and reconcile the result. If the system supports neither, stop for a human — that side effect is not safe to run unattended.

This is where the shift metaphor stops being cute: the model has forgotten; the world is still holding the side effect.

Proof can tell you the dangerous thing worked

When I combined the runtime with the verifier, I initially wanted the pair to answer every question through one gate. The rollback tool broke that idea.

The verifier can prove that a rollback restored the check to green. That doesn’t answer whether the agent was allowed to roll the workspace back in the first place.

Different jobs:

• Governance decides whether this identity may attempt this action here.
• Verification observes whether the required state exists.
• Recovery chooses the bounded path after verification stays red.

The safe profile allows sensing, snapshots, trace verification, and acceptance. Mutation-capable healing escalates for review. Every verification call crosses policy before and after the tool runs. Final completion still needs an authenticated acceptance receipt, an independently verified trace, and a final run of that exact accepted check.

A verifier can prove that a dangerous action succeeded. It can’t grant permission. A governor can authorise a useless action. It can’t prove the outcome. Put one in charge of both and it will eventually answer the wrong question with total confidence.

Six failure classes I trust more than a success message

I ran the focused suites while writing this: six completion-gate tests and four durable-loop tests. 10 passed. That’s not an autonomy benchmark — and I won’t dress it up as one. It’s a deterministic contract suite for the six failure classes this article claims the harness handles.

$ python -m pytest tests/test_completion_gate.py -q
......                                                                   [100%]
6 passed

$ python -m pytest -q \
    tests/test_loop.py::test_store_appends_monotonically_and_resumes_from_cursor \
    tests/test_loop.py::test_resume_does_not_repeat_taken_turns \
    tests/test_loop.py::test_resume_with_ambiguous_tool_outcome_fails_closed \
    tests/test_loop.py::test_concurrent_runner_fails_fast_without_duplicate_work
....                                                                     [100%]
4 passed

Failure injected	Required result
A new run tries to accept evidence from the prior run	Blocked: no authenticated receipt for this run
State changes after a valid acceptance	Blocked: accepted check is no longer green
The model exhausts its turn budget	Failure, never success-shaped completion
A second worker enters the same session	SessionBusyError; no duplicate model work
A persisted tool intent has no observation	incomplete_tool_turn; reconcile before continuing
A resumed session already contains completed turns	Replay them; don’t repeat the model or tool call

That table is the practical definition of “long-running” I use now. Not “the process stayed alive for eight hours.” The run can change contexts, restart, reject stale proof, refuse duplicate ownership, and stop when the external truth is unknowable.

Steal this: the shift protocol

You don’t need either implementation behind this article to apply the pattern. Here is the design target I would start from — not a schema from either shipped system:

.harness/
  goal.json          # frozen outcome and top-level acceptance check
  backlog.json       # bounded slices, each with its own check
  progress.md        # append-only handoff for the next shift
  trace.jsonl        # observed actions, results, and check evidence
  snapshots/         # last-known-good states only
init.sh              # deterministic startup and smoke test

Then make every shift follow the same checklist:

[ ] Confirm the workspace and acquire the run lease.
[ ] Read goal.json, backlog.json, progress.md, Git history, and the latest trace.
[ ] Stop if a prior tool intent has no recorded outcome.
[ ] Run init.sh and the basic end-to-end smoke check.
[ ] Select one highest-priority failing backlog item.
[ ] Snapshot only if the current state is green.
[ ] Make one bounded change and run that item's objective check.
[ ] On red: recover within budget or escalate. Never drift forward.
[ ] On green: commit, append the evidence and handoff, then release the lease.
[ ] Report done only when the top-level proof is intact, current, and scoped to this run.

Add three stops the model can’t negotiate away: a turn or wall-clock budget, a no-progress limit, and a broken-trace or lost-lease circuit breaker.

If you can’t write the finish line, keep a human in the decision. If you can’t tell whether the side effect happened, reconcile it. If the proof is stale, run it again.

The bill: your checks are now production code

The handoff doesn’t come for free — and the bill gets larger as the run gets longer.

Your feature contract can be wrong. Your progress file can drift from the event log. Your lease store needs fencing and recovery. Every irreversible tool needs its own idempotency or reconciliation story. The acceptance check becomes part of the trusted computing base, which means a weak check can produce beautifully authenticated nonsense.

Fresh contexts also pay an orientation tax. The whole design works because each shift spends time reading the worksite and running the smoke test before it touches the next feature. Remove that cost and you get speed. You also get duplicate work, stale assumptions, and false completion.

That’s the standing bargain: less faith in the model, more engineering in the handoff.

Pay it, and the prize isn’t a model that can think for eight straight hours. It’s a run that can lose a process, a context, or a worker without losing the truth.

You don’t make an agent long-running by stretching one context until it breaks. You make every context replaceable, every side effect reconcilable, and every proof current.

The model gets the shift.

You own what survives it.

---

Try the handover test: if this context vanished now, could a fresh one continue from files and receipts alone, without trusting a summary or repeating an action? If not, you have found the next part of the harness to build.