Quick takeaways
- Use multiple agents only when tasks are independent, require different tools, or need conflicting constraints handled separately.
- Define narrow roles with clear inputs, outputs, and ownership boundaries. Vague roles produce ambiguous handoffs.
- Pick a communication pattern before you scale: direct message passing works for small teams; shared state and pub/sub work for larger flows.
- Plan for partial failure. A multi-agent system is only as reliable as its recovery, retry, and fallback logic.
- Evaluate the system end-to-end, not just each model in isolation. Success metrics should reflect the user's goal.
Building multi-agent systems
Google Cloud Tech demonstrates how to orchestrate multiple AI agents using the Agent Development Kit.
When to use multiple agents
A single model call is often the right place to start. It is simpler, cheaper, and easier to debug. Add agents when the work naturally splits into distinct responsibilities that benefit from separate context windows, tool sets, or optimization targets.
Good signals for multi-agent design: one part of the task needs deep research while another needs strict validation; different steps require different models or cost trade-offs; a human must review intermediate outputs before the next step runs; or the workflow is long enough that a single context window loses coherence.
Do not split a task just because it sounds elegant. Every agent boundary adds latency, serialization cost, and failure surface. Start monolithic and extract agents when you can name a concrete responsibility that improves quality or observability.
Designing agent roles
A role is not just a system prompt. It is a contract: what this agent knows, what it can do, what it returns, and what it must not decide. Good roles reduce coupling because each agent's output is shaped for the next agent's input.
Role definition template: name, goal, input schema, output schema, allowed tools, forbidden decisions, retry policy, and upstream/downstream agents. Writing this down surfaces hidden assumptions before you write code.
Planner
Breaks a user request into ordered subtasks and assigns them to worker agents.
Worker
Executes one task type using specialized tools and returns structured output.
Critic
Reviews output against constraints, flags errors, and rejects work back to the worker.
Keep roles small enough to test independently. If you cannot unit-test an agent's behavior with a few examples, its scope is too broad.
Communication patterns
Agents need a shared understanding of state and a protocol for passing messages. The pattern you choose determines latency, debuggability, and how easily the system can grow.
Direct message passing. Agent A calls Agent B and waits for a response. Simple, synchronous, and easy to trace. It becomes brittle as the graph grows because failures cascade and timeouts multiply.
Shared state with locks. Agents read and write a common state object. Good for workflows where order matters and you need durability. Requires clear ownership of each state key and conflict resolution rules.
Pub/sub events. Agents emit events and listen for relevant ones. Decouples producers and consumers, but observability is harder. Use this when multiple agents may react to the same outcome.
Whatever pattern you choose, log every message and state change. Multi-agent bugs usually show up as ordering or serialization issues, not model errors.
Coordination and control
Coordination decides which agent runs when and what happens when they disagree. Without explicit control, agents can loop, overwrite each other's work, or produce inconsistent outputs.
Orchestrator pattern. A central controller maintains the plan, invokes agents, and decides when the workflow is complete. Easiest to reason about, but the orchestrator can become a bottleneck.
Peer-to-peer pattern. Agents negotiate or delegate among themselves. More flexible, but termination and conflict resolution are harder to guarantee. Use explicit voting or ranking rules.
Human-in-the-loop. Insert approval gates before irreversible actions: sending an email, deploying code, making a purchase, or updating a database. Keep these gates at agent boundaries so a human sees the full context, not a half-finished state.
Failure modes and recovery
Failures in multi-agent systems are rarely about a single bad response. They come from miscommunication: an agent receives a malformed message, a state update is lost, a retry storm overloads a tool, or two agents make conflicting changes.
Design for partial success. If three agents run and one fails, decide in advance whether to pause, rollback, continue with degraded output, or escalate to a human.
Evaluating multi-agent systems
Evaluating each agent with its own test set is necessary but not sufficient. The system can pass every unit test and still fail the user's goal because of handoff errors or compounding mistakes.
End-to-end metrics: task completion rate, latency, cost per task, human correction rate, and user satisfaction. Measure these on realistic traces that span the full workflow.
Per-agent metrics: output validity, tool-use accuracy, hallucination rate, and retry frequency. Use these to locate regressions when end-to-end scores drop.
Edge-case suites: ambiguous inputs, missing tool results, conflicting instructions, and adversarial states. Multi-agent systems fail most often at boundaries, not in the happy path.
Keep a golden set of traces. When you change a prompt, role, or routing rule, replay the traces and diff the outputs before shipping.
Implementation patterns
LangGraph or CrewAI
Define agents, edges, and state graphs explicitly. Good when you need control over loops, branching, and retries.
MCP or A2A
Standardize how agents discover tools and talk to each other. Reduces custom glue between components.
Redis or SQLite
Store shared state with TTL, versioning, and persistence. Useful for long-running or recoverable workflows.
OpenTelemetry or LangSmith
Trace messages, state changes, and model calls across agents so failures are easy to reconstruct.
Build the simplest version first: one orchestrator, two workers, and a shared state object. Add agents only after the two-agent flow is reliable and evaluated.
Without AI vs. with AI
| Task | Without AI | With AI |
|---|---|---|
| Task splitting | One model tries to do research, writing, and validation in a single context. | Multiple agents split responsibilities with separate context windows and clearer boundaries. |
| Tool access | A single agent holds every tool, increasing the chance of wrong tool use. | Each agent owns a focused tool set and hands off to other agents when needed. |
| Conflict handling | Conflicting instructions compete for attention in one prompt. | Separate agents encode conflicting constraints in their own roles and negotiate handoffs. |
| Failure recovery | One error kills the entire workflow because there is no retry boundary. | Agents retry, fallback, or escalate independently based on their local scope. |
| Debugging | It is hard to tell which part of a long prompt caused a bad output. | Agent logs show which role produced which output and where the handoff happened. |
FAQ
When should I choose multi-agent over a single prompt?
Start with a single prompt. Move to multi-agent when the task has separable responsibilities, conflicting constraints, review gates, or tool sets that do not fit one context window.
How many agents is too many?
There is no fixed number, but every new agent adds latency and failure modes. If you cannot explain what each agent owns and why it cannot be merged, you have too many.
Do agents need to be different models?
No. Multiple instances of the same model with different system prompts and tools can be effective. Use different models only when cost, latency, or capability requirements vary by role.
How do I prevent agents from looping?
Set step and token limits, track visited states, and require the orchestrator to produce a termination condition. Add explicit human escalation for ambiguous cases.
What should I log in production?
Log every message, state change, tool call, retry, and final output. Use trace IDs that span the entire workflow so you can reconstruct the full run.
How do I evaluate a multi-agent system?
Use end-to-end task success metrics plus per-agent diagnostics. Include edge cases that stress handoffs, failures, and conflicting instructions. See the how to evaluate LLMs guide for evaluation fundamentals.
When should I use multiple agents?
When tasks are independent, need different tools, or have conflicting constraints that are easier to separate than combine.
How do agents communicate?
Direct message passing, shared state, or pub/sub patterns depending on system size and reliability needs.
What makes a good agent role?
A narrow scope, clear inputs and outputs, explicit ownership, and a defined handoff to the next agent.
What are common multi-agent failure modes?
Ambiguous handoffs, duplicate work, infinite loops, and cascading errors when one agent fails.