Loop Engineering: State, Recovery and the Right to Continue

The most dangerous question after a crash is not “where were we?” It is “did the action already happen?”

Imagine a loop that creates an invoice, opens a pull request, sends a message, or deploys a service. The external system accepts the action. Before the loop records the result, the worker dies.

The next worker arrives to an honest blank:

• Retry, and you may do it twice.
• Assume success, and you may skip work that never happened.
• Start over from the chat summary, and you are trusting prose written before the crash.

This is where loop engineering stops being a clever prompt pattern and becomes systems engineering.

A loop earns the right to continue only when it can reconstruct the past well enough to make the next action safe.

The conversation is not the state

Long context windows are useful. Summaries are useful. Neither should be the source of truth for a durable loop.

Conversation is a working view. It is shaped for the model, compacted for cost, and sometimes disconnected from the world between turns. Durable state has a different job: preserve the facts another process needs to resume.

A fresh worker should be able to answer:

1. What goal is this loop pursuing?
2. Which steps definitely completed?
3. Which side effects are confirmed?
4. What evidence did the checker return?
5. What budget remains?
6. Is the run active, waiting, done, or failed?

If any answer exists only in the previous model’s memory, the loop is not resumable. It is merely hopeful.

Store events, not a polished story

The most useful primitive I found while building the NEXUS loop runtime was an append-only event log.

Every meaningful step becomes an event. The exact names vary by runtime; across the NEXUS loop and the Frontier human-approval rail, the useful vocabulary includes:

• grounding
• model
• tool_requested
• tool_result
• check
• retry
• compaction
• approval_requested
• approval_decided
• learning
• error
• status

Each event gets a monotonically increasing sequence number. The sequence becomes a cursor: “I have safely observed everything through event 41.”

This design gives you two things a summary cannot:

Replay

A new process reconstructs the run from facts in order. It does not need the old process or its private memory.

Resume

A client can disconnect, reconnect with its last cursor, and receive only the events it missed. A hard refresh becomes an inconvenience, not a lost run.

The event log should be the source of truth. Live notifications are only a wake-up mechanism. If a notification is missed, the cursor still finds the event.

Idempotency answers “may I try again?”

An append-only log tells you what was recorded. It does not automatically tell you whether an unrecorded external action happened.

For side effects, the loop needs an idempotency key: a stable identity for one intended action.

If the worker retries create-payment:order-1842, the external system should either:

• return the original result; or
• report the current status of that same action.

It should not create a second payment because the network response was lost.

Where the external system does not support idempotency, add a status lookup before retry. The unsafe option is blind repetition.

This is why retries belong inside the loop contract. “Retry three times” is not resilience unless the operation is safe to repeat.

Retry transient failure, not every failure

A rate limit, timeout, or temporary service error may deserve exponential backoff. An invalid request, denied permission, or failed policy check usually does not.

The loop should record every retry:

{
  "type": "retry",
  "operation": "model_step",
  "attempt": 2,
  "delay_seconds": 4,
  "error": "429 rate limited"
}

That record matters later. It shows whether the system recovered from a transient condition or spent its budget repeating a deterministic failure.

The breaker can then make a better decision:

• transient and improving: continue;
• deterministic and unchanged: hand off;
• ambiguous side effect: resolve status;
• budget exhausted: stop.

Context must be managed separately from history

The durable event log grows. The model’s context window cannot grow forever with it.

Do not solve that by deleting history. Keep the full record durable, then build a bounded working context for each model step.

A practical context manager:

1. keeps the newest events verbatim;
2. compacts older events into a summary;
3. preserves unresolved decisions and critical evidence;
4. records that compaction happened;
5. stays inside a fixed token budget.

The model sees a useful working set. The system keeps the full truth.

That separation is easy to miss. If the compacted prompt becomes the only history, every summary mistake becomes permanent. If the full event log is always sent to the model, cost and context eventually explode.

Durable history and model context are different products.

Learning needs an outcome, not just activity

Once a loop is durable, it is tempting to make it self-improving: count what the agent used, reinforce it, and feed it into future runs.

Be careful. Activity is not success.

In NEXUS, this problem appeared in retrieval. A document could be boosted simply because search returned it often. That creates a popularity loop:

returned often -> ranked higher -> returned more often

Nothing in that cycle proves the document helped.

The fix is to record a trajectory:

query -> retrieved evidence -> action -> outcome

Then reinforce only from a positive, independent outcome: the source was cited, the answer was grounded, the check passed, or the task succeeded.

This principle applies beyond retrieval:

• Do not learn from a draft merely because it was generated.
• Do not learn from a tool merely because it was called.
• Do not learn from a plan merely because the agent sounded confident.
• Learn from work that survived the checker and produced a real outcome.

A loop that learns from its own activity can become confidently worse. A loop that learns from verified outcomes has a chance to improve.

The minimum durable loop

You do not need a distributed event platform to begin. A local JSONL file or small database can carry the fundamentals.

For each run, persist:

State	Minimum field
Identity	stable session ID
Purpose	goal and acceptance condition
Progress	append-only sequenced events
Side effects	idempotency keys and confirmed results
Judgment	checker verdicts and evidence
Control	attempt, cost, time, and remaining budget
Disposition	running, waiting, done, or failed
Learning	outcome reported separately from the attempt

Then prove one recovery scenario before adding scale:

1. Start the loop.
2. Let it take one real but reversible action.
3. Kill the worker between steps.
4. Start a fresh worker.
5. Confirm it replays the record, avoids duplicate action, and resumes from the correct point.

If that test is hard, it is telling you where the hidden state lives.

A loop is a promise across time

The maker-checker-breaker pattern gives a loop bounded authority. Durable state gives that authority continuity without turning memory into guesswork.

The event log says what happened. The cursor says what has been seen. Idempotency says what may be retried. The checker says what passed. The outcome says what is worth learning.

Together they answer the only question that matters after interruption:

Does this loop have the evidence to continue?

If yes, resume. If no, stop honestly and hand the uncertainty to a person.

That is not a limitation of autonomy. It is what makes autonomy trustworthy enough to repeat.

---

Start with the series: The Repeat Is the Product explains the six-part loop contract, followed by Maker, Checker, Breaker for bounded autonomy.