Beyond Static Pipelines: Architecting LLM Multi-Agent Systems for Autonomous Workflow Orchestration

The expectation gap

When a team says "we're building agents," two very different pictures are usually in the room.

The engineering picture is a workflow with LLM calls in it. There is a DAG. Node 3 extracts entities, node 4 classifies them, node 5 writes a summary. The LLM is a smarter map(). It is reliable, testable, and shippable — and it is not what anyone else in the room is imagining.

The stakeholder picture is an employee. You hand it a goal — "reconcile last month's vendor invoices and flag anything that looks wrong" — and it figures out the rest. It discovers that three invoices are in a different currency, decides on its own to pull the FX rates, notices the PDF parser choked on a scanned document, and routes that one to a human instead of guessing.

The gap between those two pictures is the whole problem. The first is orchestration of LLM calls. The second is orchestration by an LLM. Moving from one to the other is not a matter of adding a while loop and a tool list. It's an architectural shift in where decisions live.

The rest of this post covers what that architecture looks like, the failure modes that show up the moment you leave the demo environment, and the design patterns that make autonomous behavior survivable in production.

DAG = Directed Acyclic Graph - a structural concept commonly used in mathematics, computer science, and data engineering to represent workflows or dependencies.
 

Part 1: What actually changes

In a static workflow, the control flow is written by a human, at design time, in code. The LLM fills in values.

In an autonomous system, the control flow is generated at runtime, by the model, as a function of what it has observed so far. That single change cascades:

The last row is where most teams get hurt. In a static pipeline, "why did it do that" is answered by reading code. In an autonomous system, the answer only exists if you designed for it to exist.

Part 2: A reference architecture

Here's the shape I'd recommend as a starting point. The critical structural idea: the plan is data, not control flow. The model emits a plan; a deterministic engine executes it. The model never is the scheduler.

The layers, briefly

1) Intent layer. A goal is not a prompt. It's a structured object: the objective, hard constraints, success criteria, a resource budget, and a risk tier. If the success criteria can't be written down, the system has no way to know when to stop — and neither will you.

2) Cognitive layer (stochastic). Planning, routing, and criticism. Everything here is allowed to be non-deterministic because nothing here does anything. It only proposes.

3) Execution layer (deterministic). The scheduler, the step governor, retry logic, checkpointing, compensation. Plain code. This is what makes the system debuggable.

4) Capability layer. The agents themselves — each one a role with a scoped toolset and a scoped permission set. Not "an agent per department," but an agent per distinct capability that needs its own context and its own tools.

5) State layer. A blackboard for working state, an artifact store for anything large, and memory for what should survive the run.

6) Governance layer. Permissions, human gates, and traces. Bolted on at the end, this layer never works. Designed in from the start, it's what lets you raise the autonomy dial later.

The three loops

A useful mental model is that a mature system runs three nested loops at different frequencies:

• Inner loop (seconds): a single agent's reason → act → observe cycle. Bounded, local, cheap to retry.
• Middle loop (minutes): plan → execute → verify → re-plan. This is where autonomy actually lives. A failed verification is not an exception; it's an input to the next plan.
• Outer loop (days/weeks): traces → evaluation → prompt/policy/tool updates. This is the loop most teams never build, and it's the one that determines whether the system gets better or just gets bigger.

Part 3: The challenges

This is where demos and production part ways.

1. Error compounding

The arithmetic is brutal and unavoidable. A step that is 95% reliable, chained twenty times, gives you a 36% chance of an end-to-end success. At 98% per step you're still only at 67%.

Autonomy makes this worse, because autonomous systems choose to take more steps. Longer horizons multiply more terms.

The naive response is "use a better model," which buys you a per-step improvement and loses it again the moment the model decides on a thirty-step plan.

2. Context degradation across handoffs

Agent A does careful work and hands a summary to Agent B. The summary is lossy. Agent B makes a decision that would have been different with the full context, hands its own summary to Agent C, and by the time you reach the end the system is confidently reasoning about a distorted version of reality. It's the telephone game with an unlimited budget.

Long contexts don't solve this either — relevant details in the middle of a large context get attended to less reliably than details at the edges, and stuffing every agent's full history into every other agent's window is how you get a $40 workflow run.

3. Planning is the weakest link

LLMs are strong at local reasoning and notably weaker at long-horizon planning with interdependencies. Plans tend to fail in characteristic ways: steps that assume information that won't exist yet, missing failure branches, sequential plans for work that's naturally parallel, and — most commonly — plans that are actually just a restatement of the goal in five bullet points.

4. No natural termination condition

A static pipeline ends when it reaches the last node. An autonomous system ends when it decides it's done, which means it can also decide it's not done, forever. The characteristic pathologies:

• Ping-pong: two agents delegating to each other.
• Polishing: the critic finds something to improve every single time, because critics asked to find problems will find problems.
• Thrash: the plan gets revised, the revision fails, the revision-of-the-revision recreates the original plan.

5. Cost and latency become unbounded

Cost is now a runtime variable that the model itself controls. A single misjudged decomposition — "I'll analyze each of these 400 records individually" — turns a $2 run into a $600 run. Latency behaves the same way, and multi-agent systems that fan out sequentially can be an order of magnitude slower than the single-agent version that would have been good enough.

6. Side effects have no transaction boundary

Step 7 sent the email. Step 9 failed. There is no rollback for a sent email. Any agent that touches the outside world creates state that outlives the failure, and a re-plan after a partial failure can happily re-execute an action that already succeeded.

7. Debugging is archaeology

"It gave a wrong answer" is now a question about a decision tree that existed for 90 seconds and was never written down. Without deliberate instrumentation, root-causing means re-reading a 200KB blob of interleaved messages and guessing.

8. Evaluation has no ground truth

For a classifier you have labels. For an autonomous workflow, the same goal has many valid trajectories and the "right" answer is often a judgment call. Worse, output quality and process quality are different things: an agent can reach a correct answer through a broken process that will fail next week on a slightly different input.

9. Multi-agent is often the wrong answer

Worth saying plainly: a well-scoped single agent with good tools beats a five-agent committee on most tasks. Every additional agent adds a handoff, a context boundary, a coordination cost, and a new failure mode. Multi-agent earns its keep when subtasks genuinely need isolated context, different tools/permissions, or parallel execution — not because the org chart has five teams.

Part 4: The architecture that survives contact with production

Each pattern below maps to challenges above. The unifying principle: push non-determinism into proposals, keep determinism in execution, and close every loop with verification.

Pattern 1 — Plan as a typed artifact

The planner does not execute anything. It emits a structured plan that a deterministic scheduler validates and runs. If the plan doesn't validate, the planner is asked again — no code has run yet, so the failure is free.

{
  "goal_id": "recon-2026-07",
  "success_criteria": [
    "every invoice has a matched PO or an explicit exception reason",
    "all currency conversions cite an FX rate with a timestamp"
  ],
  "budget": { "max_steps": 40, "max_usd": 5.0, "max_wall_clock_s": 900 },
  "steps": [
    {
      "id": "s1",
      "capability": "invoice.extract",
      "inputs": { "source": "artifact://batch-july" },
      "produces": "artifact://invoices.normalized",
      "depends_on": [],
      "verification": "schema:invoice_v2 AND count == source.count",
      "side_effects": "none",
      "on_failure": "replan"
    },
    {
      "id": "s4",
      "capability": "email.send",
      "depends_on": ["s3"],
      "side_effects": "external",
      "risk_tier": "high",
      "requires_approval": true,
      "idempotency_key": "recon-2026-07:vendor-notify:{vendor_id}"
    }
  ]
}

Notice what this buys you: the dependency graph is now inspectable before execution, side effects are declared, budgets are attached to the plan rather than to a global config, and every step carries its own verification predicate.

Pattern 2 — Verification gates that reset the error rate

This is the single highest-leverage fix for error compounding. Compounding assumes errors propagate silently. A verification gate that catches errors turns the chain from 0.95^n into something closer to (0.95 + 0.05 × recovery_rate)^n.

Verification should be layered by cost:

1. Schema/type check (free) — did it produce the shape we asked for?
2. Deterministic assertions (cheap) — do the numbers reconcile? does the code compile? does the test suite pass?
3. LLM critic (moderate) — is this actually responsive to the goal?
4. Human gate (expensive) — only for high-risk or low-confidence steps.

Run them in that order and short-circuit. Most failures are caught by layers 1–2, which cost nothing.

Two rules that matter more than they look:

• The critic must be a different context than the producer. Self-review inside the same context is mostly self-agreement.
• The critic must be able to say "pass." Give it explicit pass criteria and a rubric, not "find problems," or you've built a perpetual-polishing machine.

Pattern 3 — Blackboard + artifact store, not message-passing

Instead of agents forwarding summaries to each other, agents read from and write to shared state. Handoffs pass references, not content.

{
  "run_id": "recon-2026-07",
  "facts": {
    "invoice_count": 412,
    "currencies": ["USD", "EUR", "KRW"],
    "fx_source": "artifact://fx-rates-2026-07-31"
  },
  "artifacts": {
    "invoices.normalized": { "uri": "s3://...", "schema": "invoice_v2", "rows": 412 },
    "parse_failures":      { "uri": "s3://...", "rows": 3 }
  },
  "open_questions": ["3 invoices are scanned images; OCR confidence < 0.7"],
  "decisions": [
    { "step": "s2", "chose": "spot rate at invoice date", "because": "policy FIN-114" }
  ]
}

This fixes context degradation two ways. Downstream agents pull the source rather than a summary of a summary, and each agent gets a curated view of state sized to its job instead of the full transcript. It also makes the run inspectable at any moment by a human or a monitor.

Pattern 4 — The step governor

A deterministic component that sits between the plan and execution and refuses to let the system hurt itself:

class StepGovernor:
    def check(self, run, step):
        if run.steps_taken >= run.budget.max_steps:
            return Halt("step budget exhausted", escalate=True)
        if run.usd_spent >= run.budget.max_usd:
            return Halt("cost budget exhausted", escalate=True)
        if run.no_progress_streak >= 3:
            return Halt("no measurable progress in 3 steps", escalate=True)
        if self._is_cycle(run.recent_state_hashes, step):
            return Replan("cycle detected", exclude=step.capability)
        if step.side_effects == "external" and step.risk_tier == "high":
            return AwaitApproval(step)
        return Proceed()

The no_progress_streak check deserves emphasis. Define progress concretely — an artifact produced, an open question closed, a success criterion satisfied — and count steps that produce none of it. Loops nearly always show up as progress-free steps before they show up as anything else.

Pattern 5 — Saga semantics for side effects

Every step that touches the world declares:

• an idempotency key, so a re-plan after partial failure doesn't double-send;
• a compensating action where one exists (issue refund, post retraction, revoke access);
• a point of no return flag where one doesn't.

Then order the plan so irreversible actions happen last and together, after everything reversible has succeeded. This is standard distributed-systems hygiene, and it's routinely skipped in agent systems because the LLM framing makes people forget they're building a distributed system. They are.

Pattern 6 — Capability-scoped permissions

Agents don't get tools; they get capabilities bound to a scope. The research agent gets read-only web access and no database. The data agent gets a read replica, not production. The writer agent gets no external network at all.

The permission broker enforces this at call time, independent of what the model asked for. The point is blast radius: a prompt injection in a fetched web page can only exercise the permissions the calling agent holds, not the union of everything the system can do.

Related and non-negotiable: content retrieved by a tool is data, not instructions. Anything that arrives from a web page, document, or email should be structurally separated from the instruction channel, and no plan step should be created purely on the authority of retrieved content.

Pattern 7 — Trace-first observability

Emit a span per agent turn with: inputs (with references, not blobs), the model and parameters, the raw completion, the tool calls attempted and their results, tokens and cost, the verification outcome, and the state hash before and after.

Two things this unlocks:

• Replay. Re-run a trajectory with a modified prompt, model, or tool and diff the outcome. This is how you actually iterate.
• Decision attribution. "Why did it email that vendor" resolves to a specific step, a specific plan revision, and the observation that triggered it.

If you can't replay a run, you don't have a system you can improve — you have a system you can only restart.

Pattern 8 — Evaluate trajectories, not just outputs

Build the eval harness before you build the fifth agent. Minimum viable version:

• A frozen scenario set (20–50 real cases, including the ugly ones).
• Outcome metrics: task success against the declared success criteria, cost per successful run, p50/p95 latency.
• Process metrics: steps per run, re-plan count, verification failure rate by capability, human escalation rate.
• Regression gating: a prompt or model change ships only if outcome metrics hold and process metrics don't degrade.

Process metrics are the leading indicator. Re-plan count creeping up is what a quality problem looks like two weeks before it looks like a quality problem.

Pattern 9 — The autonomy dial

Don't ship full autonomy. Ship a dial, and turn it based on evidence.

The dial is per capability, not per system. invoice.extract can be at L4 while email.send is pinned at L2 forever. That's not a compromise; that's the correct end state.

Part 5: How the pieces fit together

Putting the loops and the guardrails together, one iteration of the middle loop looks like this:

The important detail: a verification failure routes to the planner, with a diagnosis — not to a blind retry. Blind retry with the same context reproduces the same failure at additional cost. Re-planning with the failure as an observation is the mechanism that makes the system look like it's solving problems, because it is.

Part 6: A pragmatic build order

If I were starting Monday:

1. Instrument first. Tracing and replay before the second agent exists.
2. Build a single agent with excellent tools. Get it to a real baseline. Many teams discover here that they never needed multi-agent.
3. Add the verification layer. Schema and deterministic assertions before any LLM critic.
4. Add the governor. Budgets, cycle detection, progress checks.
5. Split into multiple agents only where context isolation, permission isolation, or parallelism demands it. Each split should have a written reason.
6. Add the planner and the re-plan loop. This is where autonomy arrives — and it only works if 1–4 exist.
7. Build the eval harness and start turning the dial.

Teams that do this in reverse — five agents on day one, observability in month three — spend month four unable to tell whether anything they change makes it better.

Closing

The instinct is to treat autonomy and control as opposites: more guardrails, less autonomy. In practice it runs the other way. Every constraint in this architecture — typed plans, verification gates, budgets, scoped permissions, replayable traces — is what makes it safe to let the model decide more.

An agent with no guardrails gets exactly one chance to be trusted, and it will spend it. An agent with strong scaffolding can fail visibly, recover, and be granted more scope next quarter.

Autonomy isn't the absence of structure. It's structure that operates on goals instead of steps.

Autonomous multi-agent systems requires an architectural shift from code-authored control flows to dynamic, goal-driven runtime planning where non-deterministic models propose actions that a deterministic engine executes.

While dynamic orchestration addresses real-world complexity by allowing models to adapt to unexpected context and errors, it introduces severe failure modes including error compounding, context loss during agent handoffs, non-terminating loops, unbounded costs, and un-rollbackable external side effects.

Mitigating these challenges demands robust production patterns, such as expressing plans as typed artifacts, implementing cost-layered verification gates that reset error propagation, utilizing shared blackboard state rather than message passing, enforcing runtime state governors with strict budget and progress checks, applying saga semantics to side effects, scoping capability permissions, maintaining trace-first observability for replaying runs, evaluating full process trajectories, and deploying an incremental autonomy dial.

Ultimately, real-world agentic autonomy is achieved not by removing guardrails, but by surrounding stochastic cognitive proposals with deterministic scaffolding that enables systems to inspect, verify, and safely refine their own progress toward complex goals.

Further reading

• Anthropic — Building Effective Agents (on workflow vs. agent, and when not to use agents)
• The blackboard architecture literature from classical AI — it aged remarkably well
• Saga pattern / compensating transactions, for anything with external side effects
• Your own traces. Nothing in this post beats reading fifty real runs end to end.