Building a parallel AI agent system using LangGraph's Send API means setting up multiple agents that execute simultaneously rather than waiting for each other to finish. You'll use Groq for fast, free LLM inference and Tavily for search, creating a routing function that fans out tasks to independent agents, then a reducer node that merges their results. This approach can cut processing time by 5-10x compared to sequential execution, turning a 50-second workflow into a 10-second one when running five agents at once.
What Is LangGraph's Send API and How Does It Enable Parallel Execution
LangGraph's Send API is a mechanism that lets you dispatch multiple tasks to different nodes simultaneously instead of processing them one after another. Unlike sequential loops or queues where agents wait their turn, the Send API implements a fan-out pattern where your router node creates multiple Send objects, each triggering a separate agent instance that runs concurrently.
Think of it like this: a sequential loop processes items [1, 2, 3, 4, 5] one at a time, taking 10 seconds each for a total of 50 seconds. The Send API processes all five simultaneously, completing in roughly 10 seconds total. The architecture has three parts: a routing node that distributes work, multiple worker nodes that execute in parallel, and a reducer node that collects results.
The Send object itself is simple. It takes two arguments: the name of the target node and the state data to send. When your routing function returns a list of Send objects, LangGraph executes all target nodes at the same time, merging their outputs before moving to the next step in your graph.
Why Parallel Agent Execution Matters for Production AI Systems
Sequential agent execution creates bottlenecks that make production AI systems impractical. If you're building a research automation tool that needs to generate five industry reports, waiting 8 seconds per report means 40 seconds of idle time where four agents sit unused. Parallel execution drops that to 8 seconds total. A 5x improvement.
This matters most when you're processing independent tasks that don't depend on each other's results. Research queries about different topics, data analysis on separate datasets, content generation for multiple products, or document processing workflows are all perfect candidates. The performance gain scales linearly: 10 parallel agents complete in the same time as one agent, 20 agents in the same time again.
For businesses implementing AI automation, this difference determines whether your system handles 10 requests per minute or 100. That's the gap between a prototype and a production tool. If you're exploring how AI fits into your operations, understanding parallel execution patterns is foundational to preparing your business for AI automation.
How to Build a Parallel AI Agent System with LangGraph Send API
You'll build a system that takes multiple research topics, fans them out to independent search-and-summarize agents running simultaneously, then merges the results. This implementation uses Groq's free tier (14,400 requests per day on Llama models) and Tavily's free search API (1,000 searches per month).
Set Up Your Environment and Dependencies
First, install the required packages. You'll need LangGraph for orchestration, LangChain for agent components, and the API clients for Groq and Tavily.
pip install langgraph langchain langchain-groq tavily-python python-dotenvGet your free API keys from Groq (groq.com) and Tavily (tavily.com). Both offer generous free tiers with no credit card required. Create a .env file in your project directory:
GROQ_API_KEY=your_groq_key_here
TAVILY_API_KEY=your_tavily_key_hereDefine Your State Schema and Agent Nodes
Your state needs to track the input topics, individual agent results, and the final merged output. LangGraph uses TypedDict to define state structure with reducers that specify how to merge results from parallel nodes.
from typing import TypedDict, List, Annotated
from operator import add
from langchain_groq import ChatGroq
from tavily import TavilyClient
import os
from dotenv import load_dotenv
load_dotenv()
# Initialize clients
llm = ChatGroq(
model="llama-3.1-8b-instant",
temperature=0,
groq_api_key=os.getenv("GROQ_API_KEY")
)
tavily = TavilyClient(api_key=os.getenv("TAVILY_API_KEY"))
class AgentState(TypedDict):
topics: List[str]
results: Annotated[List[dict], add] # Reducer automatically merges results
final_output: strThe Annotated[List[dict], add] syntax tells LangGraph to concatenate results from parallel executions. Without this reducer, only the last result would be saved, losing data from other agents.
Create the Worker Agent Function
Each parallel agent needs to search for information and generate a summary. This function receives a single topic, performs a Tavily search, and uses Groq to create a brief.
def research_agent(state: dict) -> dict:
"""Independent agent that searches and summarizes one topic."""
topic = state["topic"]
# Search for information
search_results = tavily.search(
query=topic,
max_results=3
)
# Extract content from search results
context = "\n\n".join([
f"Source: {r['url']}\n{r['content']}"
for r in search_results["results"]
])
# Generate summary
prompt = f"""Based on this research about {topic}, create a concise 3-sentence summary:
{context}
Summary:"""
response = llm.invoke(prompt)
return {
"results": [{
"topic": topic,
"summary": response.content,
"sources": [r["url"] for r in search_results["results"]]
}]
}Notice this function doesn't know about other agents or the overall system. It receives a topic, does its work, and returns a result. That isolation is what makes parallel execution possible.
Build the Router Function That Fans Out Tasks
The router is where the Send API comes in. It takes your list of topics and creates a Send object for each one, dispatching them all to the research_agent node simultaneously.
from langgraph.types import Send
def route_topics(state: AgentState) -> List[Send]:
"""Fan out topics to parallel research agents."""
return [
Send("research_agent", {"topic": topic})
for topic in state["topics"]
]That's it. When LangGraph executes this function and sees a list of Send objects, it runs all target nodes at once. If you've got five topics, five research_agent instances execute in parallel, each with its own topic data.
Create the Reducer Node to Merge Results
After all parallel agents complete, you'll need to combine their outputs into a final report. The reducer receives the accumulated results (automatically merged by the add reducer in your state definition) and formats them.
def merge_results(state: AgentState) -> dict:
"""Combine parallel agent outputs into final report."""
results = state["results"]
# Format all summaries into a single report
report_sections = []
for r in results:
section = f"""### {r['topic']}
{r['summary']}
Sources:
{chr(10).join(f"- {url}" for url in r['sources'])}
"""
report_sections.append(section)
final_report = "\n\n".join(report_sections)
return {"final_output": final_report}Assemble the Graph with Parallel Execution Flow
Now you'll connect everything into a LangGraph StateGraph. The key is using conditional_edges with your router function to trigger the fan-out pattern.
from langgraph.graph import StateGraph, END
# Create graph
workflow = StateGraph(AgentState)
# Add nodes
workflow.add_node("research_agent", research_agent)
workflow.add_node("merge_results", merge_results)
# Set entry point with conditional edges for fan-out
workflow.set_conditional_entry_point(
route_topics,
["research_agent"]
)
# After all parallel agents complete, go to merger
workflow.add_edge("research_agent", "merge_results")
workflow.add_edge("merge_results", END)
# Compile the graph
app = workflow.compile()The conditional_entry_point with route_topics creates the parallel execution. Each Send object spawns an independent research_agent execution, and LangGraph waits for all to complete before moving to merge_results.
Run Your Parallel Agent System
Execute the graph with multiple topics and watch them process simultaneously. Time it to see the performance difference.
import time
# Define research topics
topics = [
"latest developments in quantum computing 2024",
"impact of AI on healthcare diagnostics",
"renewable energy storage breakthroughs",
"cybersecurity trends for small businesses",
"remote work productivity tools comparison"
]
# Run with timing
start = time.time()
result = app.invoke({
"topics": topics,
"results": []
})
end = time.time()
print(f"Processed {len(topics)} topics in {end - start:.2f} seconds")
print("\n" + "="*50)
print(result["final_output"])On Groq's infrastructure, this typically completes in 8-12 seconds for five topics. A sequential implementation would take 40-50 seconds. The difference becomes more dramatic as you scale to 10, 20, or 50 parallel agents.
How the Fan-Out Fan-In Pattern Differs from Sequential Agent Loops
Understanding the architectural difference helps you decide when to use parallel execution. Sequential loops process items one at a time using a single agent instance. You iterate through a list, call the agent, wait for results, then move to the next item. Total time equals (number of items) × (time per item).
Queue-based systems improve on this by allowing multiple agent instances to pull from a shared queue, but they still process sequentially from the queue's perspective. You get some parallelism through worker pools, coordination overhead and queue management add complexity though.
The fan-out fan-in pattern using Send API is different. The router distributes work in one step, all agents execute truly simultaneously with no coordination between them, and the reducer merges results in one step. Total time equals (time per item) regardless of how many items you have, up to your infrastructure limits. For tasks that need to build AI agents for independent operations, this pattern is optimal.
The tradeoff is that parallel execution only works when tasks are independent. If agent B needs agent A's output, you can't parallelize them. But for research, data processing, content generation, or analysis across separate domains, independence is natural and parallel execution gives you massive speed improvements.
Optimizing Parallel AI Agents for Cost and Performance
Running multiple agents simultaneously means multiple API calls happening at once. With Groq's free tier, you get 14,400 requests per day on Llama 3.1 8B, which translates to roughly 600 parallel agent executions per hour if each agent makes one LLM call. That's more than enough for most small business and prototyping needs.
Tavily's free tier gives you 1,000 searches per month, or about 33 per day. If each agent does one search, you can run 33 parallel research workflows daily. For higher volume, Tavily's paid tier starts at $100/month for 50,000 searches, dropping the per-search cost to $0.002.
To optimize performance, batch your topics strategically. Instead of processing items one at a time as they arrive, collect them into groups of 10-20 and process each batch in parallel. This amortizes the graph initialization overhead and makes better use of concurrent API capacity. Batches of 15-25 items hit a sweet spot between latency and throughput for most use cases, honestly.
Monitor your execution times to identify bottlenecks. If all agents complete in 3 seconds except one that takes 15 seconds, that slow agent determines your total time. Consider splitting slow tasks into smaller parallel subtasks, or using faster models for time-critical paths. Tools like LangSmith for debugging and monitoring help you trace exactly where time is spent.
You now have a working parallel AI agent system that processes multiple independent tasks simultaneously using LangGraph's Send API, cutting execution time by 80% or more compared to sequential approaches. The pattern scales from five agents to fifty with minimal code changes, and the free-tier setup means you can prototype and test without infrastructure costs. As your AI workflows grow more complex, this fan-out architecture becomes essential for keeping processing times reasonable and building systems that feel responsive rather than sluggish.
Get a free AI-powered SEO audit of your site
We'll crawl your site, benchmark your local pack, and hand you a prioritized fix list in minutes. No call required.
Run my free audit