A one-shot pipeline commits to its first try: parse once, retrieve once, generate once, return whatever comes out. When retrieval comes back empty or the answer is half-formed, there is no second chance. A loop gives the pipeline one: notice the miss, adjust, and run the weak step again before the user ever sees it.

This article is a standalone companion to Enterprise Document Intelligence, a series that builds an enterprise RAG system from four bricks. It sits next to Article 7bis (context engineering for single-document RAG) and treats the layer just above it.

Article 13 (the composite pipeline) glued the bricks together and named the dispatcher. This article zooms into what the dispatcher does when a single call is not enough. Volume 4 develops the agentic version of the same discipline. This piece stays at single-document scope. The rest is the map: what loop engineering is, the anatomy of one loop (trigger, termination, recovery), the two scales it runs at, the failure modes to guard against, and the outer loop the human holds.

🧭 New to the series? Start with the map: Prompt, Context, Loop sets out the three engineering layers every RAG system is built on, the prompt (the call itself), the context (what fills the model’s window), the loop (when the next call fires and when it stops), and walks the whole series through that lens, article by article. It is the shortest way to see what is covered and where this one sits.

📓 The runnable companion shows the loops firing: you trip llm_parse on a simulated timeout and watch the backoff schedule count down, then flip complete_answer_found to false on a listing answer and watch the dispatcher widen the retrieval scope and regenerate. On GitHub: doc-intel/notebooks-vol1.

Three disciplines stack on top of an LLM call. Prompt engineering writes the call: system message, user message, schema. Context engineering chooses what enters and exits the model’s context window between calls (see Article 7bis). Loop engineering decides when the next call happens, what triggers it, when the loop stops, and how the system recovers when something goes wrong.

A polished prompt with a clean retrieval upstream produces a correct answer most of the time. The rest of the time the call fails in one of the ways the opening listed: an invalid JSON, a self-flagged incomplete listing, a 429 at the deployment’s rate cap. Loop engineering is the discipline of designing what happens next.

The shortest critique of bad loop engineering, paraphrasing the practitioner literature, is this: a loop that retries the same action on the same error is not learning, it is spinning. The difference between a loop that helps and a loop that wastes tokens is whether each iteration changes something the previous iteration did not address.

Bounded retry is the oldest pattern in the catalogue: Erlang’s let it crash model has carried some version of it since the 1980s. ReAct (Princeton and Google, October 2022) put it inside an LLM call for the first time: reason about the result, decide to act again or stop. AutoGPT (March 2023) made the autonomous version public. Reflexion (NeurIPS 2023) added self-evaluation. Plan-and-Execute separated planning from execution. Geoffrey Huntley’s Ralph Loop (July 2025) put the goal on disk so a context reset never lost track. Anthropic released the /goal command in Claude Code (May 2026) and packaged the whole stack two weeks later as Dynamic Workflows. The single-document case in this article uses a much smaller subset: retry-with-backoff, schema-fail retry, completeness check, dispatcher branching. None of the heavier orchestration primitives bite at single-document scope. They will, in V1 Part IV (corpus) and V4 (agentic).

Before cataloguing the loops in a RAG pipeline, it helps to fix the parts of a single one. Every loop, small or big, has the same three control surfaces and answers to the same one rule.

A loop has three control surfaces that are easier to design separately than together.

Trigger. The condition under which a new call fires after the first one returned. Three triggers show up in the single-document case. Schema validation failed: the model produced JSON that the Pydantic schema rejected. The answer carries a self-flag: the listing schema’s complete_answer_found field is false, the answer schema’s confidence is below 0.6, the model wrote needs_clarification. Transient API failure: an APITimeoutError, an APIConnectionError, a 429, a 5xx. Each trigger is a single named predicate. The trace store logs which one fired; the dispatcher (Article 13) decides which one to handle and how.

Termination. The condition under which the loop stops. Three classes show up in the single-document case. Loop-until-done terminates when the predicate is finally satisfied (the schema validates, complete_answer_found is true, the confidence is above threshold). Loop-until-budget terminates when a quota is exhausted (six retries reached, sixty seconds of wall-clock burned). Hard cap terminates unconditionally after N attempts (the llm_parse wrapper hard-stops at six retries with RuntimeError). A loop without an explicit termination is the canonical failure mode the Anthropic Cookbook flags first.

Recovery. What happens when a call fails. The library this series provides does the simplest answer in its LLM wrapper: retry-with-backoff. Six attempts, delays of 2 / 4 / 8 / 16 / 32 / 60 seconds plus jitter, on a fixed list of retriable exceptions (APITimeoutError, APIConnectionError, RateLimitError, 429, 500, 502, 503, 504). Three richer answers fit different cases: fallback to a different model (the 3B failed schema, the 7B might), escalate to a human reviewer (any answer below 0.6 confidence on a high-stakes question), or skip the item and return what was computed (the batch result carries the list of successes and the list of failures, the consumer decides whether to retry the failures later).

These three controls describe most of what a loop does between LLM calls.

The one control surface the library provides as a ready-made primitive is recovery, in the shape of retry-with-backoff. It lives in llm_parse and wraps every parse call: bounded attempts, an exponential delay with jitter, a hard cap that raises rather than spins.

```

the library's LLM wrapper - retry-with-backoff, built into every parse call

def llm_parse(, input, text_format, max_retries=6, cache=True, *opts):
for attempt in range(max_retries):
try:
return client.responses.parse(input=input, text_format=text_format)
except Exception as err:
if not is_retriable(err): # timeout/429/5xx retry ;
raise # context-length never does
if attempt == max_retries - 1:
raise # hard cap -> propagate
delay = _compute_delay(attempt, base=2, cap=60, jitter=0.3,
retry_after=extract_retry_after(err)) # 2,4,..,60s
time.sleep(delay) # honour Retry-After floor
```
A loop that fires its trigger, retries with the same payload, gets the same failure, and retries again is not a loop. It is a denial-of-service against the company’s API budget. The rule that prevents this is the simplest one in the discipline.

A loop should only retry when something has changed between the previous attempt and the next. Three legitimate things to change.

The payload: the dispatcher widened the retrieval scope (50 lines became 200 lines), the system prompt got an extra constraint (“the previous answer did not validate; produce ONLY a JSON matching this schema with no prose around it”), the question was rephrased (the user clarified what they meant by “in Q3”).

The model: the small model returned garbage, the large model gets the second shot. Article 8quater (adaptive model selection) builds this loop: start with the cheapest model the job allows, escalate only when validation rejects.

The strategy: the keyword retrieval missed, the second attempt uses dense retrieval. This is the dispatcher’s job (Article 13) and the V4 catalogue calls it classify-and-act.

Retry that does not change one of the three is spinning. The implementation rule is operational: every retry should carry a trace block that names what changed. If the trace block is empty, the retry should not have fired. The fix is to bound retries at a small N (the library uses six) and to log the changed-since-last field on every iteration. The day a user complains that “the same question now costs four times as much”, the trace tells you whether the loop was working or spinning.

A loop is not one magic mechanism bolted onto the pipeline. Each one is a named, bounded response to a specific way a brick can fall short, and they sort into two scales: small loops that run inside a single brick, and big loops that cross bricks when generation distrusts its own input.

Small loops live inside a single brick. Document parsing runs the image cascade: filter the page’s images cheaply, classify what each one is, and pay a vision model to describe only the ones worth reading (Article 5). It also rebuilds a missing outline from body typography, looping until the outline stabilises (Article 5octies), and dispatches each page to the parser its signals call for (Article 5nonies). Retrieval walks the table of contents down to the right subsection (Article 7quater). Question parsing asks one clarifying question and learns the default (Article 6bis). Generation retries when the JSON fails the schema, and reruns retrieval and generation when the listing comes back incomplete (Articles 8 and 12).

Each of these has the same three control surfaces as any loop, a trigger, a termination, and a bounded depth; it just iterates over its own brick’s material rather than reacting to a downstream result. Retry-with-backoff wraps every API call underneath all of them, the one loop that is pure plumbing rather than a brick’s own reasoning.

Big loops cross bricks. Generation reads its own input, decides it was not good enough to answer from, and sends the pipeline back to an earlier brick. The single-document case has three: adaptive parsing (generation’s context_structured=false re-parses the offending page with a deeper parser, Articles 10A and 10B), reference resolution (generation reports pending_references, the orchestrator resolves the pointer against the parsing tables, re-retrieves the target region, regenerates, Article 11), and scope feedback (generation’s complete_answer_found=false widens retrieval, Article 13). Same shape each time: a typed flag on the answer, a bounded trip back upstream, a second generation pass that re-checks itself.

The check is a typed flag, the escalation is bounded to the flagged page, and the loop stops the moment the re-checked answer trusts its context.

Looking at the existing V1 code with the 2026 loop-engineering vocabulary on hand: most of the named primitives have a counterpart already in the code, under a less catchy name.

Loop-until-done: the V1 Article 12 listing completeness loop. The generator answers, the schema carries the complete_answer_found flag, the loop reruns retrieval and generation if the flag is false. The pattern came with the listing schema and predates the name.

Retry-with-backoff: the library’s llm_parse wrapper. Six attempts with the exponential schedule. The cost analysis in Article 21 Section 4.6 details when this triggers and what the operational impact looks like.

Generate-and-filter: V1 Article 8’s structured-output schema. The LLM returns JSON, the Pydantic validator rejects anything that does not match, the loop retries if validation fails. The 2026 vocabulary makes the loop layer explicit; the discipline has been in the series since the schema-first generation article.

Classify-and-act: V1 Article 13’s composite dispatcher. The parsed question carries a shape (single-value, listing, aggregation, comparison), the dispatcher routes to the matching handler. Anthropic’s classify-and-act applies the same shape to free-form classification rather than to a parsed-question taxonomy; the underlying machinery is the same.

Feedback-loop-between-bricks: V1 Article 13’s between-bricks feedback. Generation flags the retrieval scope as insufficient, the dispatcher widens the scope, generation reruns. The 2026 vocabulary calls this evaluator-optimiser; the series called it the dispatcher’s feedback loop and built it in 2024.

What is not yet in V1: adversarial verification (a separate agent that tries to refute the answer), the tournament pattern (N candidate answers, judges pick the winner), explicit loop-until-dry termination (K consecutive empty rounds). Those are V4-scope; the single-document case rarely needs them.

Every loop in the single-document pipeline, small or big, wears the same three control surfaces from Section 2. Read down the trigger column and the shape is the same each time: the loop fires on one specific, named signal, terminates on a bound, and changes exactly one thing before it retries.

Three failure modes are routine in production single-document RAG and each one maps to a specific loop-engineering rule.

Infinite-loop on transient failure. The retry layer has no hard cap. The deployment hits a 429, the loop retries, the next 429 arrives within the rate-limit window, the loop retries again, the API budget burns through the night. Fix: a hard cap on retries (the library uses six), an explicit RuntimeError after the cap, and an alert on the trace store when the cap is hit.

Spinning on a deterministic failure. The Pydantic schema rejects the output because the model wrote a field name with a typo. The loop retries with the same prompt. The model produces the same typo. Six retries, six rejections, the user sees a RuntimeError. Fix: on schema-fail retry, modify the prompt to include the validator error message verbatim (“your previous answer failed validation with: field_name should be field_name_actual. Produce a JSON matching the schema exactly.”). The next attempt has new information.

Confidence-flag ignored. The answer comes back with confidence = 0.3 and the dispatcher returns it to the user without flagging anything. Fix: the dispatcher applies a confidence threshold, low-confidence answers either escalate to a richer retrieval pass (widen scope, switch from keyword to dense) or carry an explicit low_confidence warning to the consumer.

Loop engineering as covered so far is one loop: the dispatcher fires a call, reads the result, decides to retry, terminates when satisfied. Zoom out and there are three loops running at different cadences around the same system, borrowing the frame Andrew Ng laid out in his “Three Key Loops for Building Great Software” letter:

The inner loop, seconds to a minute.The dispatcher itself. Retry-with-backoff on a 502, regenerate on a schema fail, expand the anchor set on- complete_answer_found: false. No human, bounded budget, code decides. Sections 2 to 4 of this article.
The dev loop, minutes to hours.An engineer reads the audit trail, spots a class of failures, edits- concept_keywords_df, adds a synonym pair, tightens a Pydantic schema, deploys. The dispatcher then runs the corrected version on the next call. Article 6’s expert dictionary and Article 20’s evaluation dashboard are the surfaces this loop reads from.
The outer loop, hours to weeks.A user edits an extracted field the system got wrong. The correction lands in the storage layer with a citation, becomes ground truth for the next evaluation pass, feeds into the concept catalog through a review queue. Or a claims handler pushes back on a question template the system was answering literally, and the template gets rephrased for everyone downstream. The refresh-field pattern from Article 19 and the question-template review from Article 20 sit here.

Ng’s point is that as long as the human knows something the AI does not, that outer loop cannot be closed by more automation, and treating it as noise to be optimised away is the mistake. “As long as the human knows something the AI does not, human-in-the-loop is needed to inject that knowledge into the system.” This is the series’ amplify-the-expert thesis in 2026 vocabulary. The dispatcher owns the inner loop, the engineer owns the dev loop, the expert owns the outer loop, and none of them can substitute for the others.

The corollary for design: keep the outer loop cheap for the expert. If correcting a wrong field takes ten clicks, the outer loop breaks and the system’s error rate stops improving. The refresh-field UX, the review queue, and the audit trail keep the outer loop fast enough to compound.

Three siblings stay open here.

The corpus case.Single-document loops are bounded by the size of one document. Corpus loops add cascade-with-partial-recovery (twelve documents in a batch, three fail, keep the nine successes and let the consumer retry the failures later) and refresh-field on user edit. V1 Article 17quater treats the corpus loop engineering.
The agentic case.Single-document loops are bounded in turns (the dispatcher fires at most a handful of retries before terminating). Agentic loops keep the budget open across dozens of turns, with sub-agents, memory tools, and the- /compactlifecycle. V4 Article 9bis covers context engineering for the agentic loop; V4 Article 9ter covers loop engineering for the same.
Adversarial verification.This article focuses on bounded retry and self-flagged completeness checks. The richer discipline of- independent agents whose job is to refute the answeris V4-scope. V1 single-document RAG rarely needs it; the V4 article develops it.

Loop engineering is the third discipline after prompt engineering and context engineering. At single-document scope, it owns four primitives: retry-with-backoff for transient API failures, generate-and-filter for schema-fail retries, loop-until-done for completeness checks, classify-and-act for dispatcher branching. The single rule that separates a useful loop from a spinning one is every retry changes something. The library this series provides implements the first primitive directly; the other three are in the dispatcher (Article 13) and the listing schema (Article 12). The 2026 vocabulary puts a name on a discipline the series has been practising since 2024.

Newspaper

Loop Engineering for RAG: The Small Loops Inside Each Step, the Big Loops Across the Pipeline

the library's LLM wrapper - retry-with-backoff, built into every parse call