Multi-Agent Orchestration

One Agent is a Tool. Multiple Agents are a System.

A single AI agent can answer questions, generate content, or execute a task. But real operational complexity — the kind that exists in healthcare staffing, financial compliance, supply chain logistics — requires agents that work together.

Multi-agent orchestration is the discipline of designing systems where specialized agents collaborate, share context, and coordinate their actions to solve problems that no single agent could handle alone.

Why Not Just Use One Big Model?

The instinct is to throw everything at the most powerful model available. Describe the entire problem, give it all the context, and let it figure it out. This works for simple tasks. It breaks down at scale for several reasons.

Context window limits. Even with million-token windows, a single model struggles to hold an entire operational domain in working memory and reason about it coherently.

Specialization beats generalism. An agent tuned for document verification will outperform a generalist agent on document verification — every time. The same applies to scheduling optimization, compliance monitoring, and every other operational function.

Failure isolation. When a single monolithic agent fails, everything fails. In a multi-agent system, one agent's failure is contained while the rest continue operating.

Parallelism. Specialized agents can work simultaneously. A credential verification agent and a shift matching agent can run in parallel rather than sequentially — cutting total processing time significantly.

Core Orchestration Patterns

Through our work on Kapwa and the Strategy Analyzer, we've identified several patterns that appear consistently in well-designed multi-agent systems.

The Orchestrator Pattern

A central orchestrator agent receives incoming requests and delegates to specialized workers. The orchestrator doesn't do the work itself — it routes, sequences, and synthesizes.

In Kapwa's Symphony Mode, the orchestrator receives a user's question, determines which advisors should respond and in what order, manages the conversation context each advisor sees, and synthesizes the multi-perspective output into a coherent response.

The orchestrator pattern works well when you need centralized control over sequencing, when different tasks require fundamentally different agent capabilities, and when the final output needs to combine results from multiple agents.

The Pipeline Pattern

Agents are chained in sequence — the output of one becomes the input of the next. Each agent transforms, enriches, or validates the data before passing it along.

A credential verification pipeline might look like: Document Parser Agent extracts data from uploaded files, then a Validation Agent checks extracted data against known formats and rules, then a Verification Agent queries external databases to confirm authenticity, then a Compliance Agent evaluates whether the verified credentials meet facility-specific requirements.

Pipelines are ideal when the task has clear sequential stages and when each stage requires different expertise or tool access.

The Ensemble Pattern

Multiple agents work on the same input independently, then their outputs are combined. This is the pattern behind Kapwa's multi-advisor conversations — three advisors analyze the same question from different perspectives, and the combined output is richer than any single response.

Ensembles work best for tasks where diverse perspectives improve quality: risk assessment, strategic analysis, creative generation, and decision support.

The Supervisor Pattern

A supervisor agent monitors other agents' work and intervenes when quality drops or errors occur. It doesn't do the primary work — it evaluates, corrects, and escalates.

In practice, we've found supervisors most valuable for compliance-heavy domains where mistakes are costly and for long-running tasks where drift from the original objective is a risk.

Shared Memory and Context

The hardest problem in multi-agent orchestration isn't routing or sequencing — it's memory. How do agents share what they know?

Short-term Context

Within a single task execution, agents need to see relevant outputs from other agents. This is typically handled through a shared context object that accumulates as the task progresses. In Kapwa, each advisor in a Symphony conversation can see what previous advisors said, allowing them to build on or respectfully disagree with earlier perspectives.

Long-term Memory

Across sessions, agents benefit from persistent memory. Kapwa uses vector embeddings stored in pgvector to give advisors access to relevant past conversations. When a user asks about a topic they've discussed before, the semantic search retrieves that context and includes it in the advisor's prompt.

The key design decision is what to remember and what to forget. Storing everything creates noise. Storing too little means agents repeat work and miss connections.

Failure Handling

Multi-agent systems introduce failure modes that don't exist in single-agent setups.

Cascading failures. If agent A's output feeds agent B, and A produces garbage, B amplifies the error. Defense: validate outputs at each handoff point.

Coordination deadlocks. Two agents waiting on each other's output. Defense: set timeouts and fallback behaviors.

Inconsistent state. Agent A updates a record while agent B is reading the old version. Defense: use event-driven architectures with clear state ownership.

We've learned that the most robust multi-agent systems assume every inter-agent communication can fail and design accordingly. Retries, fallbacks, and graceful degradation aren't optional — they're the core architecture.

What We're Still Learning

Multi-agent orchestration is a young discipline. We're actively researching several open questions:

• Dynamic team composition: Can the system itself decide which agents to activate for a given task, rather than using fixed configurations?
• Inter-agent negotiation: When agents disagree, how should conflicts be resolved beyond simple majority voting?
• Cost optimization: Running multiple agents is expensive. How do you balance quality against inference cost?

These are the problems we're working through in our products and will continue publishing findings as we learn.