AI Agent Architecture Patterns for Production Systems

Let's start with a reframe, because the word "agent" is getting thrown around like it means something consistent. It doesn't.

An agent in production is not a smart prompt. It's a distributed system where the LLM happens to be the planner and executor. That distinction matters more than most teams realize until something breaks. Distributed systems have failure modes that prompts don't. Network partitions, state corruption, race conditions, partial failures that look like successes. All real risks. All distinct from anything you'd encounter just tuning a prompt.

Every agent pattern, regardless of complexity, shares three core components.

Perception is the front door. It transforms raw inputs (text, API calls, voice, sensor data) into structured formats the reasoning engine can use. It also handles context window management and input validation. Garbage in still means garbage out, even with a frontier model.

Reasoning is where the agent decides what to do next. Planning happens here. Tool selection happens here. The patterns we'll discuss are different strategies for how that reasoning engine makes those decisions.

Action is the agent touching the world. API calls, database writes, external service calls. Every action is a potential failure point, and most failures in production happen here, not in the reasoning layer.

There's a fourth component teams routinely underspecify: memory. The CoALA framework from Princeton breaks it into four types.

• In-context. Working memory. What's in the prompt right now.
• Episodic. Past interactions. What happened before this session.
• Semantic. Factual knowledge. What the agent "knows" about the world.
• Procedural. Rules and skills. How the agent is supposed to behave.

Now, about those 200K to 1M token context windows everyone keeps citing as the fix for memory problems. They're not. You cannot dump an entire user's history into every prompt without cost and latency becoming untenable at scale. Which memory tier you actually need depends heavily on which pattern you choose. A stateless single-agent task only needs in-context memory. A long-running multi-agent workflow almost certainly needs episodic and semantic stores on top of that.

Each architecture pattern exposes different components to failure under load. You need to know what's at risk before you can honestly evaluate any of them.

The four deployment shapes teams converge on, and what each assumes

Before you pick a pattern, pick a shape. These are the physical and operational contexts your agent will actually live in. Pattern selection and deployment shape are tightly coupled, and ignoring that coupling is one of the more reliable ways to end up with something that works in a demo and quietly falls apart three weeks into production.

Stateless HTTP endpoint. One process per request. All state lives in the trace and an external store. Easiest to scale. Hardest to use for anything long-running. If your agent needs to survive a 30-minute workflow, this shape will fight you the whole way.

Agent runtime on a queue. A worker pool consuming tasks, with checkpointing to a state store. This is the native model for frameworks like LangGraph and Microsoft's Agent Framework. It suits Plan-and-Execute and supervisor/worker patterns because the queue provides durability. A failed step can be retried without restarting the whole workflow from scratch.

Serverless platform. Each agent step is a separate function invocation, with state in a managed store. Good for spiky or unpredictable workloads. The cost model starts to penalize you on deep multi-agent graphs, though, because you're paying for cold starts and state retrieval across potentially many invocations. Short-lived ReAct loops work fine here. A 20-step orchestration gets expensive fast.

Self-hosted on Kubernetes. Full control, full operational cost. Common in regulated industries where you need to own the audit trail and enforce your own security boundaries. Nobody picks this because it's easy.

The coupling that actually matters: supervisor/worker and multi-agent patterns almost always require a queue-backed or self-hosted shape. A stateless HTTP endpoint cannot reliably checkpoint a 20-step orchestration. If you're gravitating toward one of the more complex patterns, be honest with yourself about whether your team can actually operate the infrastructure that pattern requires. That's not a rhetorical question. It's the question.

One thing that applies regardless of shape: observability is not optional. In production, you need to know what the agent decided and why. Audit trails, authentication, authorization boundaries. These aren't nice-to-haves. They're the difference between a system you can operate and one you can only hope works.

Single-agent patterns: ReAct and Plan-and-Execute as the two baseline choices

Most teams don't need a multi-agent system. They need a single-agent system that actually works in production. ReAct and Plan-and-Execute cover the majority of real use cases, and the difference between them comes down to one thing: what kind of environment your agent will be operating in.

ReAct: reasoning and acting in a loop

The mechanic is an iterative think-act-observe loop. The agent decides its next action based on its most recent observation, not a pre-committed plan. It's adaptive by design.

Where it fits:

• Tasks requiring real-time adaptation when conditions change mid-task
• Customer support routing where the user's actual problem shifts mid-conversation
• Live data queries where the answer to one question determines the next one
• Exploratory tool use where the agent needs to discover what's available

The production cost problem is real. A single customer support interaction can require several LLM calls just to resolve one request. At scale, that makes cost modeling genuinely difficult, and per-request latency becomes variable in ways that are hard to set SLAs against.

The failure mode to watch for is loops. Without explicit loop detection and step limits, a ReAct agent can thrash on an unresolvable subproblem indefinitely. This is not theoretical. It happens. Implement limits.

Plan-and-Execute: separate reasoning from action

The mechanic separates planning from doing. The agent builds a full plan upfront, then executes against it. This is the preferred approach for complex, multi-step tasks where planning ahead produces better outcomes than incremental decision-making.

Where it fits:

• Long-running tasks (30 minutes or more)
• Deliverables with a defined structure you know upfront
• Tasks where replanning mid-execution would be prohibitively expensive

The failure mode is plan staleness. In dynamic environments, the world can change between the planning phase and the execution phase. When that happens, the plan becomes incorrect before it's complete. Replanning mid-execution adds exactly the kind of complexity that undercuts the pattern's main advantage.

Choosing between them

Ask one question: can the task state change between steps? If yes, ReAct handles that better. If the deliverable structure is known upfront and the environment is stable, Plan-and-Execute is the cleaner choice. Neither pattern requires coordination infrastructure. Both can run on a stateless endpoint for short tasks.

The Reflection pattern: where quality gates add latency cost

Reflection is a distinct quality-control decision with a specific cost structure you need to understand before you deploy it.

The mechanic: the agent critiques and revises its own output before presenting results. A second (or third) LLM pass acts as an internal reviewer. The critic takes the first pass's output as its input.

What it actually improves: accuracy on structured outputs, code generation, and multi-step reasoning tasks where the first pass reliably produces a draft with correctable errors. The key word is "reliably." If first-pass errors are random and uncorrelated, reflection helps. If the model is systematically wrong about something, a reflection pass often makes the same mistake twice.

Every reflection cycle adds LLM calls and latency. This is not a free improvement. It is a deliberate quality-latency exchange. So the question you have to answer is whether that exchange is worth it for your specific use case.

When it earns its cost:

• Output quality failures are expensive downstream. Wrong code deployed. Incorrect reports filed. Errors that propagate.
• The task is batch or async. Latency is tolerable because no user is waiting.
• Human review is the alternative. At scale, reflection is often cheaper than manual QA.

When it doesn't earn its cost:

• Real-time interactions where a user is waiting for a response
• High-volume, low-stakes outputs where first-pass accuracy is good enough

One architectural note: reflection can be implemented as a self-loop within a single agent, or as a separate critic agent in a supervisor/worker topology. That choice affects how much you can parallelize critique across multiple outputs simultaneously. If you have a batch workload and need throughput, the separate critic model lets you fan out.

Supervisor/worker patterns: what coordination buys and what it costs

This is where a lot of teams get into trouble. Not because the pattern is wrong, but because they adopt it without fully accounting for what it costs to operate.

The mechanic: a supervisor agent decomposes tasks and delegates subtasks to specialized worker agents, then aggregates results. The supervisor orchestrates. Workers specialize.

Within that mechanic, there are two distinct ownership models that produce different failure surfaces.

Manager-as-tool (OpenAI's agent.as_tool() approach): the manager retains ownership of the reply. The worker is a callable that returns a result. The manager stays in the control loop. If a worker fails, the manager knows, because it's waiting for the return value.

Handoff model (OpenAI's handoffs): ownership genuinely transfers to the specialist. The originating agent is no longer responsible for the outcome. Simpler routing, but harder to recover if the specialist fails. When something goes wrong in a handoff-based system, the question of who is responsible for recovery is not always obvious.

What coordination buys:

• Specialization. Workers can be tuned, prompted, or model-selected for narrow tasks. A coding worker can use a different model than a summarization worker.
• Parallelism. Independent subtasks can run concurrently, cutting wall-clock time for long workflows.
• Isolation. A failing worker doesn't necessarily bring down the whole workflow if the supervisor handles errors gracefully.

What coordination costs:

The supervisor itself is a new failure point. If it misroutes a task or misaggregates results, every downstream worker's output is wasted compute. State management complexity goes up significantly. The supervisor must track which workers have completed, what their outputs were, and how to merge partial results in a meaningful way.

And the queue-backed or self-hosted deployment shape becomes effectively mandatory for workflows with more than a few steps.

For teams implementing supervisor/worker patterns with LangGraph, its centralized TypedDict state object with reducer functions controlling merge semantics is the simplest shared state model available right now. Use it as your reference point before you build something custom.

Multi-agent and event-driven patterns: where complexity scales non-linearly

Adding agents doesn't add cost linearly. It compounds. Each new agent is less like adding a lane to a highway and more like adding a new intersection to an already-congested grid. That's not a metaphor meant to scare you off the pattern. It's just an accurate description of what you're signing up for.

Multi-agent with shared context

The AutoGen group chat model is the canonical example here. Agents publish to a shared thread and react from it. No single supervisor. Each agent sees the accumulated conversation history.

The production cost problem is mathematical and unavoidable. Every agent turn involves a full LLM call with the accumulated conversation history. A 4-agent debate running 5 rounds requires a minimum of 20 LLM calls, and the context window grows with each one. That makes this pattern prohibitively expensive for high-volume, real-time use cases.

Where it actually works: low-volume deliberative workflows. Policy review. Design critique. Research synthesis. Situations where quality per cycle matters far more than cost per cycle.

Event-driven multi-agent

The mechanic: agents subscribe to event streams and act on relevant events asynchronously. No central orchestrator holds the execution graph. The system reacts to what happens.

Where it fits: long-running workflows that span hours or days, workflows that need to survive process restarts, workflows triggered by changes in external systems.

The production failure modes are specific.

• Event ordering. Agents acting on stale or out-of-order events can produce nonsensical results.
• Idempotency. The same event triggering duplicate actions is a real risk in distributed systems. Your event handlers need to handle this.
• Observability. Tracing causality across an asynchronous event graph is significantly harder than tracing a synchronous call stack. When something goes wrong, finding out why takes much longer.

A governance note that applies to all multi-agent patterns: in multi-agent systems, governance failures compound because every agent boundary is a potential enforcement gap. Each agent that can call another agent is a trust boundary that needs to be defined and enforced. This is less visible than model capability problems, which is exactly why it catches teams off guard.

It's also worth considering memory architecture here, not just agent topology. How your agents remember things matters as much as how they communicate. Graph-structured memory with a tiered hierarchy has shown meaningful improvements in multi-step task completion across several published benchmarks. The point isn't the specific number. The point is that memory design is a first-class variable, not an afterthought.

How interoperability protocols change the architecture decision in 2025–2026

Until recently, teams solved the "how do agents talk to tools and to each other" problem by building bespoke integrations every time. That's changing, and it shifts some architecture decisions in ways that are genuinely new.

Two protocols now address different layers of the same problem.

MCP (from Anthropic) standardizes how agents connect to tools, APIs, and data sources. Think of it as the agent-to-external-world layer. Every major AI platform now supports it. If your agent needs to call an API, query a database, or read a file, MCP gives you a standard way to do that instead of writing a custom connector every time.

A2A (from Google, announced April 2025) standardizes secure, structured communication and task delegation between autonomous agents. This is the agent-to-agent layer. It reached v1.0 with gRPC support, signed Agent Cards for identity verification, and multi-tenancy support.

Why does this matter for pattern selection?

For single-agent patterns using well-defined tools, MCP alone is sufficient and dramatically reduces integration time. A2A is only necessary when your agent is delegating tasks to other autonomous agents.

For supervisor/worker and multi-agent patterns delegating tasks across agent boundaries, A2A becomes necessary for secure, auditable delegation. Without it, inter-agent communication is bespoke and nearly impossible to maintain at scale.

The timeline reality matters as a planning input. A basic MCP server connecting one agent to two or three internal tools takes roughly 4 to 8 weeks, including security hardening. A full multi-agent system using both MCP and A2A takes 12 to 20 weeks, depending on the number of agents, data sources, and integration complexity.

One caveat: the protocol standardizes structure, but it doesn't enforce meaning. Your agents can call the right endpoints in exactly the right format and still produce nonsensical results if the semantic layer isn't there. Gartner's projections point to a real pattern of failure that teams building on MCP alone tend to underestimate.

Both A2A and MCP moved to Linux Foundation governance in late 2025, with a joint interoperability specification expected in Q3 2026. The standard is converging. Teams building today are still ahead of it, which means you will need to revisit some of what you build now when that spec lands. That's not a reason to wait. It's a reason to build with seams.

Framework choices and what they actually constrain

Pattern selection drives framework selection. Pick the pattern first, then pick the framework that implements it best. Reversing that order is how you end up with workarounds that slowly accumulate into real technical debt.

Based on production deployments across Alice Labs' work, here's where things actually stand.

LangGraph 1.0 (went GA in October 2025) is the strongest overall choice for complex stateful workflows. Its graph-based control gives you fine-grained control over execution paths in ways other frameworks don't match. It's the reference implementation for queue-backed supervisor/worker patterns, and developer adoption reflects this.

Microsoft Agent Framework 1.0 (released April 2026) is the unified successor to AutoGen and Semantic Kernel. Best for enterprise.NET and Microsoft stacks. If your team is already deep in the Microsoft ecosystem, this is the natural fit. Trying to use it outside that ecosystem adds friction that compounds over time.

CrewAI 1.14 is the fastest path to role-based multi-agent workflows. If you need to get something working quickly with clear agent roles and you don't need the fine-grained control of LangGraph, CrewAI gets you there faster.

Every framework simplifies some things by hiding others. The things a framework hides are exactly the things you'll need to understand when something breaks in production. That's not an argument against using frameworks. It's an argument for knowing what yours is hiding before you actually need to know it.

The gap between a working demo and a stable production system isn't mostly a model problem. It's an architecture problem. The patterns covered here each make specific assumptions about reliability, scalability, and operational complexity. The teams that close the gap aren't using better models. They're matching patterns to actual system requirements before they commit to one.

The real question isn't "which pattern is best." It's "which pattern's failure modes can we actually manage."