Multi-Agent Orchestration

Module: Implement Generative AI and Agentic Solutions

Lesson: Multi-Agent Orchestration in Foundry

Introduction: The Evolution Beyond Single-Prompt AI

In the early days of generative AI, most implementations focused on simple request-response loops. You sent a prompt to a model, and it returned a completion. While effective for basic tasks like summarizing text or drafting emails, this approach hits a wall when faced with complex, multi-step workflows. Real-world business processes—such as supply chain reconciliation, software development lifecycles, or financial auditing—require more than just a single model inference. They require reasoning, planning, tool usage, and, most importantly, the ability to collaborate across different specialized domains.

Multi-agent orchestration represents the shift from "AI as a tool" to "AI as a workforce." By creating a system of specialized agents that can communicate, delegate tasks, and verify each other's work, we move toward solving problems that were previously thought to be too ambiguous or complex for automated systems. In the context of Foundry, multi-agent orchestration allows you to decompose a massive objective into smaller, manageable sub-tasks handled by agents designed with specific personas, knowledge bases, and tool permissions.

Understanding how to build these systems is critical for any developer or architect working in enterprise AI. It is not enough to simply chain prompts together; you must understand state management, inter-agent communication protocols, and the guardrails necessary to keep these autonomous systems aligned with business objectives.

Understanding the Multi-Agent Architecture

At its core, a multi-agent system is a collection of autonomous entities working toward a shared goal. In Foundry, these agents are not just instances of a language model; they are defined by their specific system prompts, their access to external data (like Ontologies or external APIs), and their ability to trigger specific functions.

The Components of an Agent

To build an effective orchestration layer, you need to define three primary components for each agent:

1. The Persona/System Prompt: This defines the agent's identity, its scope of knowledge, and its constraints. An agent tasked with "Data Analysis" will have a vastly different system prompt than an agent tasked with "Quality Assurance."
2. Tools and Capabilities: These are the functions the agent can invoke. In Foundry, this often means linking the agent to specific Object Types, query functions, or external services like email or messaging platforms.
3. Communication Protocol: This determines how the agent speaks to others. Does it use a shared message board? Does it receive direct instructions from a "Manager" agent? Or does it operate in a peer-to-peer network?

Callout: The "Manager-Worker" Pattern vs. The "Swarm" Pattern In a Manager-Worker architecture, a central orchestrator assigns tasks to specialized agents and reviews their outputs. This is highly predictable and easier to debug. In a Swarm architecture, agents interact dynamically based on the current state of the task, allowing for more creative problem-solving but significantly higher risk of loop-based errors or hallucinations. Most enterprise use cases benefit from a hybrid approach where a manager agent controls the workflow, but workers can negotiate tasks between themselves.

Designing the Orchestration Layer

When building in Foundry, the orchestration layer acts as the "brain" of the operation. It must maintain the state of the overall project, track which agents have completed which tasks, and handle error propagation. If an agent fails to retrieve the necessary data to perform an analysis, the orchestration layer must be smart enough to either prompt the user for clarification or assign the task to a different, more specialized agent.

Step-by-Step: Defining the Workflow

1. Goal Decomposition: Start by breaking the high-level task into a Directed Acyclic Graph (DAG). If the goal is "Generate a quarterly financial report," your nodes might include "Data Extraction," "Trend Analysis," "Drafting," and "Compliance Review."
2. Agent Assignment: Map each node in the DAG to an agent. Ensure that each agent has only the permissions necessary to perform its specific task (Principle of Least Privilege).
3. State Management: Implement a central store to track the "Task Object." This object should be passed between agents, containing the original request, intermediate findings, and the final output.
4. Validation Loops: Include agents whose sole purpose is to audit the work of other agents. This is the "Human-in-the-loop" or "AI-in-the-loop" check that ensures accuracy.

Practical Implementation: Building a Research Agent

Let's look at a scenario where we need to research a competitor's recent product launch. We will define two agents: a Search Agent that queries the web and internal knowledge bases, and a Synthesizer Agent that turns the raw data into a report.

The Search Agent Configuration

The Search Agent needs access to a tool called web_search and internal_document_query. Its system prompt should be focused on accuracy and citation.

# Conceptual Agent Definition in Foundry
search_agent = Agent(
    name="CompetitorResearchAgent",
    system_prompt="""You are a research assistant. Your task is to find specific 
    details about product launches. Always prioritize internal documents over 
    public web results. Cite your sources using the [source_id] format.""",
    tools=[web_search_tool, internal_doc_tool]
)

The Synthesizer Agent Configuration

The Synthesizer Agent does not need search tools. It only needs the output of the Search Agent. Its prompt should focus on narrative, tone, and formatting.

synthesizer_agent = Agent(
    name="ReportWriterAgent",
    system_prompt="""You are an expert business analyst. You receive research 
    findings and convert them into a professional executive summary. Use a 
    formal tone and highlight key market impacts."""
)

Orchestration Logic

The orchestrator manages the hand-off. It sends the prompt to the search_agent, waits for the response, checks for citations, and then passes the result to the synthesizer_agent.

Note: Always include a "retry" mechanism in your orchestrator. If an agent returns a "null" or "error" state, the orchestrator should be configured to attempt the task again with a slightly modified prompt or a different agent instance before failing.

Advanced Orchestration: Handling Feedback Loops

One of the most common pitfalls in multi-agent systems is the "infinite loop," where two agents continue to refine each other's work indefinitely. To prevent this, you must implement a strict termination condition.

In Foundry, you can enforce this by adding an iteration_limit to your orchestration logic. Each time an agent passes work to another, the counter increments. If the counter reaches the limit (e.g., 5 rounds), the system is forced to present the current best version to a human for approval.

Best Practices for Managing Agent Communication

• Standardized Message Schemas: Ensure that all agents communicate using a predefined JSON structure. For example, every message should include sender_id, task_id, payload, and status.
• Context Truncation: Do not pass the entire history of the conversation to every agent. As the conversation grows, the context window will fill up, leading to degraded performance or high latency. Instead, pass only the "summary of findings" and the "current task requirements."
• Human-in-the-Loop (HITL) Gates: For high-stakes decisions (e.g., financial transactions or legal filings), insert a mandatory gate where a human must review the output before the next agent can proceed.

Callout: Why Context Management Matters Large Language Models have a limited context window. In a multi-agent system, if you pass the entire history of every agent back and forth, you will quickly hit the limits of the model. By summarizing the state of the project after each agent finishes their task, you keep the input size manageable and the model's focus sharp.

Common Pitfalls and How to Avoid Them

Even with a well-designed architecture, multi-agent systems can behave unpredictably. Below are common mistakes and strategies to mitigate them:

1. Prompt Injection between Agents: If Agent A receives input from Agent B, and Agent B is compromised (e.g., by malicious user data), Agent A could be tricked. Solution: Sanitize all inputs at the orchestration layer before passing them to the next agent.
2. Tool Over-Reliance: Agents sometimes try to use a tool for every problem, even when a direct answer is possible. Solution: Use clear, descriptive docstrings for your tools. If an agent uses a tool incorrectly, update the system prompt to explicitly define the "when and why" of tool usage.
3. Ghosting/Silence: An agent might hang or stop responding. Solution: Implement a heartbeat monitor or a timeout mechanism. If an agent doesn't respond within 30 seconds, the orchestrator should terminate the process and log an error.

Debugging and Observability

When something goes wrong in a multi-agent system, it is rarely obvious which agent caused the failure. Did the Search Agent find bad data? Did the Orchestrator pass the wrong context? Or did the Synthesizer Agent hallucinate?

To address this, you must implement Distributed Tracing. In Foundry, this means logging the state of the Task Object at every transition. You should be able to look at a log and see:

• The input prompt sent to Agent A.
• The tool call made by Agent A.
• The raw output received from the tool.
• The refined output sent to Agent B.

Without this level of visibility, you are effectively flying blind. Use the platform's native logging capabilities to store these interaction snapshots as objects in the Ontology. This allows you to perform retroactive analysis on failed workflows and improve your agents over time.

Industry Standards: Security and Governance

When deploying agents in a corporate environment, security is paramount. You are essentially giving models the ability to execute code or interact with sensitive databases.

Security Checklist

• Role-Based Access Control (RBAC): Ensure that the service account used by the agents has the absolute minimum permissions. If an agent only needs to read from a specific folder, do not give it broad access to the entire file system.
• Input Validation: Use a validation layer to check for PII (Personally Identifiable Information) before it hits the model. If an agent is handling customer data, ensure the data is masked.
• Audit Trails: Every action taken by an agent—especially if it modifies data—must be logged in an immutable audit trail. This is non-negotiable for compliance in regulated industries like finance or healthcare.

Step-by-Step: Implementing an Agentic Workflow in Foundry

This section provides a structured approach to deploying your first orchestration layer.

Phase 1: Environment Setup

1. Define your Agent objects in the Foundry Agent Builder.
2. Register your tools as Functions (TypeScript/Python).
3. Ensure all agents have the necessary access tokens for the APIs they need to call.

Phase 2: The Orchestrator Logic

1. Create a "Main" function that acts as the entry point.
2. Define the workflow state machine. A simple switch statement or a state dictionary works well for most use cases.
3. Implement the call_agent(agent_id, context) function, which handles the communication with the model.

Phase 3: Testing and Refinement

1. Unit Test Individual Agents: Test each agent in isolation to ensure they follow their system prompts and use tools correctly.
2. Integration Testing: Run the full workflow with a "test" dataset.
3. Edge Case Testing: Introduce "noisy" data (e.g., empty files, malformed JSON) to see how the orchestrator handles errors.

# Simplified Orchestration Loop Example
def run_orchestrator(task_input):
    state = {"task": task_input, "history": []}
    
    # Step 1: Research
    research_result = call_agent("research_agent", state["task"])
    state["history"].append(research_result)
    
    # Step 2: Validate
    if not validate_research(research_result):
        return "Task failed: Research quality insufficient."
        
    # Step 3: Synthesize
    final_report = call_agent("writer_agent", research_result)
    return final_report

Avoiding "Agent Fatigue"

Agent fatigue occurs when developers try to build too many agents for a single task. If you have 20 agents interacting for a process that could be handled by 3, you are creating unnecessary overhead, latency, and points of failure.

• Keep it Simple: Start with a single-agent system. Only split it into a multi-agent system when the complexity of the task requires specialized knowledge that a single model cannot maintain effectively.
• Modular Design: If you find that one agent is becoming too complex (e.g., a prompt that is 2000 words long), that is a sign it is time to split the agent into two.
• Documentation: Document the purpose of each agent. If a new developer joins the project, they should be able to understand why the "Researcher" agent exists and what its specific responsibilities are.

Key Takeaways

Building multi-agent systems is a paradigm shift in how we approach software development. By leveraging specialized agents, we can tackle problems that were previously impossible to automate. Remember these core principles:

1. Purpose-Driven Design: Every agent must have a clearly defined role and scope. Avoid "generalist" agents that try to do everything.
2. Orchestration is Key: The logic that connects agents is just as important as the agents themselves. Invest time in building a robust state machine to manage tasks.
3. Observability is Non-Negotiable: You must be able to trace every interaction. If you cannot see how an agent reached a conclusion, you cannot improve the system.
4. Safety First: Always use the Principle of Least Privilege. Agents should only have access to the data and tools they absolutely need to do their jobs.
5. Human-in-the-Loop: For critical workflows, never allow an autonomous chain to execute without a human gatekeeper. AI should assist human decision-making, not replace it entirely in high-stakes environments.
6. Iterate and Refine: Start small. Build a two-agent system, get it working perfectly, and then expand. Do not attempt to build a complex swarm of agents on your first attempt.
7. Context is Finite: Manage the information flow carefully. Summarize and prune data to ensure your models remain focused and performant.

By following these practices, you can build systems in Foundry that are not only powerful but also reliable, secure, and maintainable. The future of enterprise AI lies in the ability to orchestrate these digital workers effectively, and mastering this skill will be a defining capability for the modern software engineer.

Common Questions (FAQ)

Q: How do I know when to use a multi-agent system vs. a single-agent system? A: Use a single agent when the task is linear and requires a single "brain" to process. Use a multi-agent system when the task requires different "perspectives" or specialized tool access that would exceed the capacity or system prompt limits of a single model instance.

Q: Is it better to use many small models or one large model for all agents? A: This depends on your cost and latency requirements. Smaller models are faster and cheaper, making them ideal for simple agents. Large models are better for "manager" agents that need high reasoning capabilities to oversee the process.

Q: How do I handle agents that start arguing or looping? A: This is usually a sign of a poorly defined termination condition or ambiguous goals. Tighten your system prompts and ensure the orchestrator has a hard-coded limit on how many times it will allow a specific hand-off before it stops the process and alerts a human.

Q: Can agents in Foundry interact with external APIs? A: Yes, through the use of functions. You can define a function in your code that calls an external API, and then expose that function as a tool to your agent. Ensure you handle authentication securely using the platform's secret management tools.

Q: What is the biggest mistake beginners make with agents? A: Over-prompting. Beginners often write massive system prompts that try to cover every possible scenario. It is much more effective to write concise, clear prompts and provide the agent with a small set of highly specific tools.

Final Thoughts

Multi-agent orchestration is a powerful tool in your development arsenal. As you begin building these systems, focus on the flow of information. Think of yourself as a manager of a team; your job is not to do the work, but to ensure that everyone on the team has the right instructions, the right tools, and the right oversight to succeed. When you approach AI development with this mindset, you will find that the possibilities for automation and innovation are virtually limitless. Start with a clear goal, keep your architecture simple, and always keep a human in the loop to ensure the final output meets your standards.