Module 6: Orchestrators

This is Module 6 of a 12-part curriculum: Build Software Products with AI — From First Principles to Production Pipeline.

A single capable agent can handle a lot. But as tasks become more complex, more parallel, or more specialised, you need a way to coordinate multiple agents working together. That’s the orchestrator’s job.

This module covers what orchestrators do, how to build them, and — critically — where the sharp edges are.

What is an Orchestrator?

An orchestrator is an agent whose primary job is to direct other agents.

Where a regular agent focuses on a domain — writing code, processing emails, analysing data — an orchestrator focuses on the work itself: decomposing tasks, assigning them to the right sub-agents, managing the flow of information between them, and ensuring the overall objective is achieved.

Think of it as the difference between a software engineer and a tech lead. The engineer writes code. The tech lead decides who writes what, in what order, and integrates the pieces.

An orchestrator has three core responsibilities:

1. Decomposition — breaking a complex task into sub-tasks that can be handled independently
2. Routing — deciding which agent (or tool, or capability) handles which sub-task
3. Integration — taking the outputs of sub-agents and combining them into a coherent result

The Orchestration Pattern

A typical orchestration flow looks like this:

User → Orchestrator → Sub-Agent A (research)
                    → Sub-Agent B (writing)
                    → Sub-Agent C (review)
                    ← combines outputs
     ← final result

The orchestrator receives the user’s request, decomposes it, dispatches to sub-agents, collects results, and returns the integrated output.

But orchestration doesn’t have to be parallel. Often it’s sequential — where the output of one sub-agent feeds into the next:

Orchestrator:
  1. Dispatch to Research Agent → get raw findings
  2. Feed findings to Analysis Agent → get structured insights
  3. Feed insights to Writing Agent → get draft
  4. Feed draft to Review Agent → get edited version
  5. Return to user

Both patterns are valid. Which you use depends on whether sub-tasks are independent (parallelise) or dependent (sequence).

PARALLEL                                  SEQUENTIAL
─────────                                 ──────────
input                                     input
  ↓                                         ↓
Orchestrator                              Orchestrator
  ├→ Agent A  (research)                    → Agent A  (research)
  ├→ Agent B  (analysis)    vs                  ↓  raw findings
  └→ Agent C  (writing)                     → Agent B  (analysis)
       ↓                                        ↓  structured insights
    merge outputs                           → Agent C  (writing)
       ↓                                        ↓  draft
  final result                             → Agent D  (review)
                                                ↓
                                           final result
Use when: sub-tasks are independent     Use when: each output feeds the next
Faster. Order doesn’t matter.             Slower. B cannot start until A finishes.

Routing Logic

Routing is the orchestrator’s core intelligence: given this task, which sub-agent handles it?

Routing can be implemented in several ways:

LLM-based routing. The orchestrator model reads the task and decides which agent to invoke. Flexible and handles novel cases well — but adds latency and a model call per decision.

Rule-based routing. If the message contains #whorang-dev, route to the Whorang agent. If the message is in the #aim-dev channel, route to the AIM agent. Fast and predictable — but rigid.

Hybrid. Rule-based for clear cases; LLM-based for ambiguous ones. This is the practical choice for most production systems.

In practice, routing rules for channel-based systems (like Slack bots) are often straightforward: the channel is the routing signal. For more general orchestrators handling diverse input types, you’ll want LLM-based routing with a clear taxonomy of agent capabilities.

Task Decomposition

Breaking a task into sub-tasks sounds simple. In practice, it’s the hardest part of building a good orchestrator.

Bad decomposition:

• Sub-tasks that are too granular (spawning an agent for every line of work)
• Sub-tasks that are too coarse (the sub-agent gets an impossible brief)
• Sub-tasks with implicit dependencies the orchestrator hasn’t surfaced
• Sub-tasks that aren’t actually independent (forcing sequential work into parallel creates conflicts)

Good decomposition:

• Sub-tasks are clearly scoped with defined inputs and expected outputs
• Dependencies between sub-tasks are explicit
• Each sub-task is small enough to be handled reliably in a single agent context
• The orchestrator has a clear integration plan before dispatching

A useful forcing question: “If I handed this sub-task brief to a human who had no other context, would they know exactly what to do and what to return?” If yes, the decomposition is good. If they’d have questions, refine the brief.

Session Management

Orchestrators need to manage sessions — the contexts in which sub-agents operate.

Key design decisions:

Isolated sessions vs forked sessions. An isolated session gives the sub-agent a clean context — it knows only what you tell it. A forked session inherits the parent’s conversation history. Use isolated for focused tasks (less noise, lower cost). Use forked when the sub-agent genuinely needs the conversation history to do its work.

Session lifecycle. When does a sub-agent session start and end? Who is responsible for cleaning up? For long-running tasks, sessions may need to persist across multiple interactions.

Output routing. Where does the sub-agent’s output go? Back to the orchestrator for integration? Directly to a channel? Stored in a file for later pickup? Design the output path explicitly.

Context passing. What does the sub-agent need to know? Pass it explicitly. Don’t assume the sub-agent can infer context it wasn’t given.

The Orchestrator Trap

Here’s a failure mode that’s easy to fall into: building an orchestrator that’s actually just a very complicated single agent.

Signs you’ve fallen into this trap:

• The orchestrator is doing all the reasoning, and sub-agents are just execution
• Sub-tasks are so small and specialised that the overhead of spawning agents outweighs the benefit
• Everything flows through the orchestrator as a bottleneck
• Sub-agents rarely need to reason — they’re just calling one tool and returning

If your “orchestrator” is doing all the interesting work and sub-agents are just thin wrappers over tool calls, collapse them. A well-designed single agent with good tools is better than a fragile multi-agent system that added complexity without adding capability.

Orchestration is the right choice when:

• Sub-tasks genuinely benefit from independent reasoning contexts
• Parallel execution provides meaningful throughput gains
• Different sub-tasks require different specialisations (context, persona, tool sets)
• The orchestrator genuinely doesn’t need to know the details of how sub-tasks are executed

A Real Example

My main agent — Ultron — acts as an orchestrator across a roster of specialised agents:

• Alex handles content strategy for my new product builder community (WIP): writing, community management, content calendars
• Coach handles productivity: daily work cycles, weekly & quarterly planning
• Mailbox handles email triage: reading, categorising, flagging urgencies

Ultron doesn’t try to do any of these things itself. When I ask about my content pipeline, it spawns Alex. When I need productivity planning, it routes to Coach. When I ask about my inbox, it checks with Mailbox.

The routing is mostly channel-based (each agent has its own Slack channel) with Ultron as the cross-cutting intelligence that can involve multiple agents when needed.

This is a clean orchestration design because each agent has a clear domain, a clear capability set, and operates independently without needing to know about the others.

Orchestration vs Multi-Agent Architecture

There’s a semantic distinction worth drawing: orchestration and multi-agent architecture are related but not identical.

Orchestration refers to the coordination pattern — how one agent directs others. Multi-agent architecture refers to the overall system design — how multiple agents are structured, specialised, and connected.

You can have multi-agent architecture without a central orchestrator (peer-to-peer agent communication). You can have orchestration without a dedicated orchestrator (each agent can spawn sub-agents for specific sub-tasks). In practice, most systems have a primary orchestrator with a few specialist agents — that’s the pattern we’ll discuss more in Module 7.

What’s Next

You understand orchestration: decomposing tasks, routing to agents, managing sessions, integrating outputs. In Module 7, we zoom out to the full system design: how to architect a roster of specialised agents, how they communicate, and how to make the whole system more reliable than its parts.

Further Reading

• [Paper] A Survey on LLM-based Autonomous Agents — Wang et al., 2023 — Covers orchestration patterns and multi-agent coordination in detail. Section 3 is most relevant.
• [Blog] LLM Powered Autonomous Agents — Lilian Weng — The planning and task decomposition section maps directly to orchestration design.
• [Docs] LangGraph Documentation — The leading framework for building stateful, graph-based agent orchestration. Good for understanding patterns even if you don’t use LangGraph.
• [Docs] Anthropic Claude — Building Effective Agents — Anthropic’s own guidance on agent design patterns. Authoritative and practical.

Referenced from @nikovijay

“This is my Mission Control: A Squad of 10 autonomous @openclaw agents… They create work on their own. They claim tasks on their own.” — @pbteja1998