Orchestration pattern
Overview
An orchestration pattern is an architectural template for managing the lifecycle and interaction of distributed agents within a larger system. Rather than requiring each agent to manage its own routing logic, state transitions, and error recovery, orchestration patterns centralize these concerns in a dedicated control layer. This approach enables consistent behavior across heterogeneous agents and simplifies the reasoning about system-level guarantees.
Orchestration patterns are distinguished from choreography patterns, where agents independently react to events without central coordination. In orchestration, a central orchestrator maintains awareness of workflow state, decides which agent to invoke next, handles timeouts and silent failures, and manages context propagation—including agent memory and context window allocation. The pattern is foundational to multi-agent orchestration systems and is commonly implemented in production LLM-based agents using frameworks that enforce deterministic control flow.
Orchestration patterns are particularly valuable when managing context between agent steps, implementing planning workflows, or coordinating critic agents with generator agents. They reduce the cognitive load on individual agents by abstracting away communication protocols and state management, allowing each agent to focus on its designated task.
How it works
Orchestration patterns operate through the following structural elements:
- Central Orchestrator: A controller that maintains the execution state machine and decides which agent receives the current context and request. The orchestrator does not perform domain work itself but routes and monitors.
- Context Passing: The orchestrator manages the propagation of context between agents, including accumulated agent memory, intermediate results, and metadata about previous steps. This is critical for maintaining coherence across sequential agent calls.
- Routing Logic: The orchestrator implements decision rules for agent selection, which may be static (predefined workflows), dynamic (based on output classification), or learned (via planning models). Common patterns include sequential routing, conditional branching, and query fan-out (see Query Fan-Out).
- Failure Handling: The orchestrator defines retry strategies, silent failure detection, timeout policies, and fallback routing. For example, if an agent exceeds its time budget, the orchestrator may redirect to a faster agent or decompose the request.
- Evaluation and Logging: Orchestration patterns typically include instrumentation for automated evaluation of intermediate outputs and logging of routing decisions to enable debugging and human evaluation of system behavior.
A concrete workflow: (1) User query enters orchestrator; (2) Orchestrator invokes retrieval agent with hybrid search configuration; (3) Retrieval agent returns ranked results; (4) Orchestrator evaluates retrieval quality; (5) If quality is low, orchestrator invokes reranker agent; (6) Orchestrator passes reranked results to generation agent along with original query; (7) Generation agent produces response; (8) Orchestrator routes response to critic agent for groundedness check; (9) If check fails, orchestrator retries with different retrieval parameters; (10) Otherwise, response is returned to user.
| Term | Distinction |
|---|---|
| Multi-agent orchestration | Multi-agent orchestration is the application of orchestration patterns to systems with three or more agents; orchestration patterns are the design templates that enable such systems. Orchestration patterns are the reusable architectural solutions; multi-agent orchestration is the deployment scenario. |
| Agentic workflow | An agentic workflow describes the procedural logic and steps an agent executes; an orchestration pattern describes how to coordinate multiple agents and manage their interactions. A workflow may be internal to a single agent, while orchestration is inherently multi-entity. |
| Planning | Planning is the reasoning process an agent uses to decide its next action; orchestration is the mechanism for enforcing those decisions and managing state across agent boundaries. Planning may occur within agents; orchestration occurs at the system level. |
| Model Context Protocol | The Model Context Protocol is a specification for agent-to-tool communication; orchestration patterns define how multiple agents coordinate overall execution flow. MCP enables the communication layer; orchestration patterns define the control flow layer. |
| ReAct | ReAct is a prompting technique that encourages reasoning and action within a single agent; orchestration patterns coordinate multiple agents across a workflow. ReAct optimizes individual agent reasoning; orchestration coordinates agent composition. |
Examples
LangChain/LangGraph Orchestration: LangGraph implements orchestration patterns as directed graph structures where nodes represent agents or tools and edges represent routing rules. A typical pattern chains a retrieval agent → reranking agent → generation agent, with the orchestrator (graph runtime) managing failure recovery and context window sizing at each transition.
OpenAI Assistants with Function Calling: The assistants API implements a simple orchestration pattern where the central orchestrator (the API runtime) manages the interaction loop between the LLM agent and external tools. The orchestrator routes function calls to appropriate handlers, manages response formatting, and decides whether the agent has reached a terminal state. This pattern simplifies error recovery because the orchestrator, not the agent, is responsible for retry logic.
Anthropic's Tool Use Pattern: Anthropic documents orchestration of agents performing self-reflection and error correction. An orchestrator invokes an agent, evaluates the output against criteria (using an LLM-as-judge or critic agent), and routes to either a cleanup agent for remediation or a next-stage agent for continuation. The pattern enables principled composition of heterogeneous models with different context windows and fine-tuning strategies.
See also
- Multi-agent orchestration — Application of orchestration patterns to systems with multiple specialized agents
- Planning — Reasoning techniques that orchestration patterns enforce at the system level
- Agent memory vs Context window — How orchestration patterns manage state propagation between agents
- Agentic workflow — The procedural logic orchestration patterns coordinate
- Model Context Protocol — A specification for the communication layer beneath orchestration patterns
