Implement, Verify, Review, Refactor: AI Agent Retry Loop

The problem with "just try again"

You ask an AI coding agent to add webhook handling to a TypeScript service. It creates a new pub/sub layer instead of using the existing EventBus in src/core/events.ts. Tests fail because the new layer doesn't integrate with the existing event subscribers. You re-run the agent. It sees its own broken pub/sub code and, instead of questioning the approach, starts patching it. It wraps the broken handler in a try-catch, adds a special case for the failing test, and imports a compatibility shim. By iteration three, the code is worse than if the agent had never touched it.

The underlying problem is context pollution. A naive retry feeds the agent its own failed code, and it tries to fix that code in place. Stale snippets, debug logs from previous attempts, and half-applied fixes pile on more noise. By iteration three, the agent spends most of its context window navigating artifacts from earlier attempts instead of understanding the actual codebase.

PithyCyborg put this well: "A retry that doesn't change the conditions is usually just a more expensive way to fail." Feeding the agent its own failed output as context doesn't change the conditions.

The LLMLOOP study (ICSME 2025) confirmed the flip side: structured feedback loops measurably improve LLM code generation quality on benchmark tasks. Whether iteration is productive or just expensive randomness comes down to how the loop is designed.

Three design decisions that separate convergence from spiraling

A productive retry loop needs three mechanisms that most implementations skip.

Checks as a gate before review

Run lint, test, and build after every implementation attempt. If any check fails, skip review entirely and loop back to the implementer immediately.

Most retry loops send every attempt to a reviewer (human or LLM) regardless. That wastes the reviewer's token budget on surface-level problems. A reviewer analyzing code that doesn't compile will point out missing imports and syntax errors, not whether the approach is correct. The check gate ensures the reviewer only sees code that meets a minimum bar: it compiles, the linter passes, and the existing tests still work.

The check gate also creates a fast inner loop. Failed checks restart the cycle in seconds (the time it takes to run npm test), not minutes (the time it takes for a reviewer agent to read, analyze, and produce structured feedback). For common first-attempt mistakes like missing imports or type errors, this fast loop resolves the issue without ever invoking the reviewer.

Three-strategy review, not pass/fail

After checks pass, a reviewer evaluates the implementation and chooses one of three strategies: pass (exit the loop), continue (approach is sound, needs more work), or refactor (approach is fundamentally wrong).

"Continue" and "refactor" describe two different situations that binary pass/fail conflates.

Take the webhook example from the opening. On the first attempt, the agent creates its own pub/sub layer instead of using EventBus. The reviewer identifies this as a refactor situation: the foundation is wrong. No amount of patching will make a custom pub/sub layer integrate correctly with the existing event subscribers. The right response is to revert and start fresh with the instruction "use EventBus in src/core/events.ts."

On the second attempt, the agent uses EventBus correctly but forgets to clean up the handler registration in UserService.teardown(). The reviewer identifies this as a continue situation: the foundation is sound. The agent used the right abstraction, integrated with existing subscribers, and just missed one detail. The next iteration should build on this work, not throw it away.

Treating these the same way is why retry loops feel like expensive randomness. When a "continue" situation gets the "refactor" treatment, you throw away sound work. When a "refactor" situation gets the "continue" treatment, you spend iterations patching code that should have been reverted on iteration one.

This three-strategy model is a design choice, not a discovered law. You could add a fourth strategy ("abort and escalate to a human") or collapse it into two with a severity parameter. The point is that binary pass/fail isn't enough. The reviewer needs a way to distinguish "keep building" from "start over."

The reviewer’s feedback is all the next iteration has to work with. It needs to describe what went wrong, what approach would work better, which parts of the codebase matter, and what patterns the next implementer should follow.

Refactor means undo, not fix

When the reviewer chooses "refactor," a dedicated agent undoes the work. It does not attempt to fix or improve the code. It reverts.

The next iteration's implementer starts fresh: the original request plus the reviewer's feedback about what went wrong and what to try instead. The workspace is clean, with no artifacts from the failed approach to work around.

Throwing away code feels wasteful. But patching fundamentally wrong code usually makes things worse. The implementer is better off starting clean with good feedback than inheriting a workspace full of wrong-approach artifacts. The reviewer's diagnosis is what carries forward, not the code.

The reviewer isn't an oracle

The entire model depends on the reviewer making correct strategy calls. LLM reviewers get it wrong: they call "continue" on a fundamentally wrong approach, or call "refactor" on code that just needs a small fix. Two mechanisms limit the damage.

First, max_retries (default: 3) acts as a circuit breaker. Even if the reviewer consistently misclassifies, the loop terminates after a bounded number of iterations. When the loop exits at the retry limit, the workspace contains whatever state the last iteration left. The user reviews from there.

Second, yield: true pauses the loop after each iteration for human review. This is the default. The user sees the reviewer's strategy and feedback, and can override: continuing when the reviewer said refactor, or reverting manually when the reviewer said continue. Fully autonomous mode (yield: false) removes this safety net and is only appropriate when you trust the reviewer's judgment on the specific codebase.

What each iteration actually sees

In a naive retry, each iteration inherits everything from previous attempts: code, debug logs, failed approaches, reviewer comments. Context grows linearly with each attempt. JetBrains Research (2025) found that agent-generated context quickly becomes noise, and that effective context size is often much smaller than the window size. Stale implementation details from earlier iterations crowd out the agent's ability to reason about the actual problem.

A structured retry loop inverts this. Only the reviewer's feedback crosses the iteration boundary. Implementation narratives, check logs, and refactoring details stay in isolated thread forks that are discarded after each iteration.

Concretely, after three iterations the main thread contains:

Three messages. Not three full implementation transcripts with diffs, debug logs, and reviewer analyses.

The third implementer receives three feedback messages instead of 50,000 tokens of accumulated implementation history. Martin Fowler's overview of context engineering for coding agents arrives at the same conclusion: the challenge is curating what the model sees, not fitting more in.

The workflow implementation

Get It Right, an open-source Reliant workflow, implements these three decisions in a single workflow.yaml.

The workflow runs an attempt loop: implement → [lint, test, build] → review → [refactor?] → loop back. A CEL expression controls the loop condition:

The loop continues when the reviewer says "continue" or "refactor," or when any check fails. It exits on "pass" or when max_retries is reached. When the loop hits the retry limit without a "pass," the workflow exits with the current workspace state. The user takes over from there.

Lint, test, and build run in parallel after each implementation. A join_checks node waits for all three, then an edge condition decides whether to proceed to review:

If any check has a non-zero exit code, the edge goes to [] (no next node), ending the iteration. The reviewer never runs on broken code.

The reviewer produces a structured JSON response ({grade, strategy, feedback}), and its save_message directive writes only the strategy and feedback to the main thread. The reviewer's full analysis, the code diffs it examined, and the commands it ran stay in the reviewer's forked thread and are discarded.

The edge from review to refactor fires only on the "refactor" strategy:

Every agent fork uses memo: false, starting a fresh thread each iteration. Only the reviewer's save_message writes back to the main thread. The result: workspace state and reviewer feedback are the only data that persists across iterations.

When structured retry loops pay off

Retry loops with structured review cost roughly 2-4x a single agent pass in tokens and time: multiple implementation attempts, reviewer analysis, and conditional refactoring. That overhead is justified in specific situations.

Complex brownfield codebases with implicit conventions are the clearest fit — the kind where human PR reviews consistently say "we don't do it that way here." When the agent keeps generating code that "should" work but violates undocumented patterns, a retry loop with a reviewer that discovers and communicates those patterns is worth the token cost. Cross-cutting changes that touch multiple interconnected systems are another good case: higher odds of a fundamentally wrong approach, and the conditional undo prevents iteration debt from accumulating.

Greenfield projects and simple one-file bug fixes don't need this. No existing conventions to discover, no architecture to fight against. Run the agent once. If it fails, re-prompt. For well-documented codebases where the architecture is obvious from the code, the value of the three-strategy distinction drops because the agent is less likely to choose a fundamentally wrong approach in the first place.

There's a gray area for medium-complexity tasks where we're less sure. A multi-file refactor in a codebase you understand well might benefit from the check gate alone (run tests before re-prompting) without the full three-strategy review. The overhead depends on how often the agent picks wrong approaches, and that varies by codebase, task, and model.

These principles apply regardless of tooling. Running Claude Code manually? Run tests before re-prompting. When you review the output, distinguish "keep going" from "start over." When starting over, git checkout . before re-running the agent. These concepts work whether you use Get It Right, build your own loop, or do it by hand.

Get It Right as a workflow

Get It Right is an open-source Reliant workflow that implements this retry pattern as an automated pipeline. The workflow.yaml shown throughout this post is the complete implementation:

The workflow accepts configurable check commands, a max retry count (default 3), and an execution mode. Set yield: true to pause after each iteration for manual review, or yield: false to run the full loop autonomously.

The trade-off is real: Reliant replaces your current tool. If you're using Claude Code or Cursor, adopting Get It Right means switching to a different development environment. That's a legitimate switching cost, and it only makes sense if you're working on codebases complex enough to justify structured iteration.

For readers who won't switch, the three design decisions transfer to any setup. The check gate, three-strategy review, and conditional undo are patterns, not product features. Build them into your own scripts, apply them manually, or use them as a framework for evaluating other tools.

Get It Right is one workflow in a broader library. One Ring adds planning, parallel verification, and multi-reviewer code review around the same retry loop. The workflow.yaml and documentation are on GitHub. The Reliant platform documentation covers the workflow primitives (loop nodes, edge conditions, thread forking, structured output) used throughout this post.