Agent Architecture Component Breakdown

An AI agent is a stack: perception, planning, tools, execution, memory, and orchestration, each with its own job, its own way of breaking, and its own infrastructure bill.

That distinction matters more than it sounds like it should. "Agent" gets used casually now as shorthand for "chatbot with tools" or "LLM wrapped in a while loop," and that framing hides more than it explains. The corrective is simple to state and harder to build: an agent perceives inputs, reasons over them, plans a sequence of actions, carries those actions out through tools, and holds onto what it learns, all without a human approving every step along the way. Each of those five verbs, perceive, reason, plan, act, remember, maps to a distinct layer of the system, and each layer fails in its own particular way. A retrieval bug looks nothing like a broken tool call, which looks nothing like an agent that forgets its own state halfway through a task. Treating all of these as one undifferentiated "the agent isn't working" problem is how teams spend weeks debugging the wrong thing.

The stakes are not small. MarketsandMarkets puts the AI agents market at roughly $7.84 billion in 2025, projected to reach $52.62 billion by 2030, a 46.3% compound annual growth rate. Whatever gets built underneath that curve, the perception pipelines, the planning loops, the sandboxes, the memory stores, is what this piece is actually about. What follows is a walk through each layer, starting with the one that runs before the model ever "thinks" anything at all: how it perceives.

How agents perceive: turning raw inputs into a context the model can reason over

Perception is the work of taking whatever comes in, in whatever shape it arrives, and turning it into something the model can actually operate on. That sounds almost too basic to name as its own layer, until you notice how much happens before a single token of reasoning occurs.

The inputs themselves come in a handful of flavors. There's plain text: user instructions, the back-and-forth of prior conversation, documents pulled in by a retrieval step. There's structured data, database rows, JSON from an API response, a spreadsheet's worth of numbers. Computer-use agents that click around inside a GUI need to perceive images and screenshots too. And every agent that's taken more than one action needs to feed the outputs of its previous tool calls back in as new input, closing the loop.

None of that arrives model-ready. Someone, or something, has to chunk the documents, embed them, filter out what's irrelevant, and format the rest so it fits the shape the model expects. That work is unglamorous, and it is also where a surprising share of agent failures originate, because underneath all of it sits a hard constraint: the context window. Everything the agent knows at the moment it decides what to do next has to fit inside that window, which means every choice about what to include and what to leave out is a design decision with consequences several steps downstream.

The arxiv survey literature on LLM-based agents frames this cleanly: the model maps the assembled context, instructions, retrieved documents, tool outputs, whatever memory got pulled in, to a decision, whether that's a plan, a tool call, or a plain-language response. The mapping is only as good as what got assembled beforehand. Noisy retrieval, a truncated context window, a raw JSON blob dumped into the prompt with no parsing: each of these degrades reasoning before reasoning has even had a chance to start. Fix perception, and the reasoning layer inherits a cleaner problem to solve. That's the layer we turn to next.

Planning and reasoning: how the agent decides what to do next

Once the context is assembled, the model has to do something with it: this is where it functions as a policy core, taking everything perception handed it and producing a decision, a plan, a tool call, a decomposition into smaller sub-goals, or sometimes just a direct answer.

Agents run in a cycle, plan, act, observe, and repeat, where the output of each action becomes new input for the next round of planning. That loop is the engine room of the whole system, and how it's implemented varies a fair amount across production agents. ReAct interleaves reasoning and action in a single pass, so the model narrates its thinking and calls a tool in the same breath. Tree-of-thought style approaches branch out, exploring several candidate plans before settling on one. Hierarchical planning splits the difference: a top-level planner breaks a big goal into smaller tasks and hands each one off to a specialized sub-agent or module.

Where does this go wrong in practice? Goal drift is one of the more insidious failure modes: the agent takes a locally sensible action at each step, but, over enough steps, those locally sensible choices add up to something that's drifted far from the original ask. Hallucinated tool calls are another, where the model invokes a real tool but fabricates the parameters it passes in, and then there's the plain infinite loop, no clear termination condition, so the agent just keeps cycling without making progress toward anything.

Multi-agent systems raise the difficulty further. When a planner agent delegates work to specialist agents, the planning layer no longer lives inside one model; it spans several, and errors compound across the handoffs between them. There's no single statistic that captures this section well, and it doesn't need one; the point here is structural. Once a plan exists, though, it has to actually happen in the world, and that's the job of the next layer down.

Tool use: the layer that connects reasoning to the real world

A tool, in this context, is anything the agent can invoke that isn't pure text generation: web search, code execution, reading or writing a file, querying a database, hitting a REST API, controlling a browser, running a shell command. The mechanics are fairly consistent across implementations. The model outputs a structured function call, a name and a set of parameters, the orchestration layer routes that call to the right tool server, and whatever comes back gets fed in as a fresh perception input for the next planning step.

This is also where an old integration headache used to live. Before a shared standard existed, every agent application needed a custom connector for every tool or data source it touched, N agents times M tools, meaning N-times-M bespoke integrations that somebody had to write and maintain. MCP, the Model Context Protocol, built by Anthropic and open-sourced in November 2024, gives agents and tools a common language, collapsing that N-times-M problem down to roughly N-plus-M. In December 2025, MCP was handed over to the Linux Foundation's Agentic AI Foundation, a governance move worth noting: vendor-neutral stewardship is what separates real infrastructure from one company's clever abstraction.

Tool-use correctness is measurable, and production teams are already measuring it. Tool selection accuracy and parameter accuracy have emerged as explicit metrics in production agent evaluations, which tells you this isn't a hand-wavy concern anymore; it's something teams monitor the way they'd monitor latency or error rate. The failure modes are concrete too: the model can call the right tool with the wrong arguments, it can misread a structured response that comes back from an API, or a write operation can fire when the intent was only ever to read.

That last one deserves a pause. A tool call that reaches an external system, or that executes code, is the actual attack surface of an agent. Prompt injection and sandbox escapes originate at this boundary, where reasoning becomes action. Which is exactly why the next layer, where code execution physically happens, carries the most security weight in the entire stack.

The execution runtime: where agent decisions become running code

Executing code is a different animal from calling a search API or querying a database. A search query is bounded; you know roughly what can happen. Code execution is open-ended by definition: the agent can write and run more or less anything, including things that damage the host machine, quietly exfiltrate data, or spin up resources nobody asked for.

General-purpose serverless platforms weren't built with this pattern in mind, and it shows. AWS Lambda caps execution at 15 minutes, offers no local storage that's reliably persistent across invocations (the /tmp directory might survive within a single warm container, but there's no guarantee it survives between invocations), and has no built-in mechanism for safely running arbitrary code. Agents need a persistent filesystem that holds state across a multi-step chain of tool calls, cold starts fast enough that a workflow doesn't visibly stall, and the ability to scale out to large numbers of isolated environments running at once.

The isolation technology that's actually deployed in production splits a few ways. Firecracker-style microVMs give each execution environment its own kernel, a hardware-enforced boundary where a compromised guest can't reach the host or a neighboring sandbox. gVisor takes a different approach, intercepting system calls in user space through an application kernel, which cuts down the host kernel's exposed surface without the overhead of a full VM. Kata Containers wraps OCI-compatible containers around lightweight VMs; Northflank, for instance, runs more than 2 million isolated workloads a month using a combination of Kata and gVisor.

Here's a wrinkle worth sitting with: frontier models got a lot better at cybersecurity tasks in a short span, apprentice-level task success climbing from under 10% in late 2023 and early 2024 to somewhere around 50% in 2025. Sandbox designs built around what models could do a year or two ago may simply not hold up against what models can do now. This isn't hypothetical: PromptArmor's early-2026 disclosure documented indirect prompt injection combined with weak command validation in Snowflake's Cortex Code CLI, a combination that bypassed human-in-the-loop approval entirely and opened the door to arbitrary code execution and cached credential access. That's a real, disclosed escape, not a thought experiment.

Meanwhile, the gap between adoption and readiness is stark: 79% of organizations report using AI agents today, but only 5% say they've solved sandboxing to a standard they'd call production-ready. That's not a small gap; that's most of the industry running ahead of its own safety net. What separates a runtime actually built for agents from infrastructure retrofitted to look like one comes down to a handful of concrete features: provisioning fast enough to matter, since agents may spin up a fresh environment per tool call, hardware-enforced isolation rather than a shared kernel and a promise, state that persists across steps, Docker-native compatibility so teams aren't rewriting tooling they already trust, and compliance certifications like SOC 2, HIPAA, and GDPR built in from the start rather than bolted on later. Security is only half the job, though, because the other half is durability, and real agent work doesn't wrap up in a single function call.

Memory: what the agent retains, where it stores it, and why the distinction matters

Engineers generally split agent memory into four scopes, and the distinctions aren't academic; each scope has a different cost, a different failure mode, and a different place in the pipeline. In-context memory, sometimes called working memory, is whatever currently sits in the context window: fast to reach, gone the moment the window clears. Episodic memory holds records of past interactions or completed tasks, pulled back in at the start of a new session. Semantic memory is factual knowledge parked in a vector database and retrieved through similarity search whenever the agent needs domain-specific facts. Procedural memory is the trickiest of the four: learned patterns about how to handle recurring task types, sometimes captured as a handful of few-shot examples, sometimes baked directly into fine-tuned weights.

Retrieval, it's worth noticing, is really just perception happening again, later. Whatever gets pulled from semantic or episodic memory flows straight back into the perception layer covered earlier, which means poor retrieval doesn't stay contained to memory; it compounds straight into worse planning. Memory architecture and retrieval quality aren't two separate concerns you can fix independently. They're the same concern, wearing two names.

None of this is free. Vector database hosting, embedding inference, retrieval latency, these are real line items on an infrastructure bill, not abstractions. Analysis of high-volume agents, the kind making many LLM calls per task, has found that infrastructure costs — memory stores, orchestration, monitoring — can rival or even exceed the cost of model inference itself. That's a number worth sitting with the next time someone assumes the model call is the expensive part.

Production memory design breaks in a few recognizable ways. Stale retrieval is one: the agent confidently acts on information that was accurate months ago and isn't anymore. Context stuffing is another, where too much retrieved material crowds out the actual task at hand, burying the signal the model needed under material it didn't. Lack of cross-session persistence means the agent forgets a user's preferences, or a sub-task it already finished, and quietly redoes work. Memory only pays off if the agent sticks around long enough to use it, which brings up a question that's easy to skip past: what does it actually take for an agent to keep running, reliably, over a long stretch of time?

Long-running and stateful execution: why agents need durable workflows, not ephemeral containers

Most infrastructure primitives were built for short, stateless requests that finish in seconds and vanish. Real agent work rarely fits that shape. A data pipeline run, a research task that unfolds over several hours, an iterative code-generation cycle where the agent writes, tests, and rewrites, none of these are single-invocation problems, and treating them that way is a category error.

Durable execution is the programming model that fixes this: it guarantees a piece of work finishes despite failures, network interruptions, or a host restarting mid-task. That model reached early-majority adoption in 2025, with AWS, Cloudflare, and Vercel all shipping their own offerings, which tells you the demand moved from niche to mainstream fairly quickly. What statefulness actually requires, at the infrastructure level, comes down to three things: a persistent filesystem, so the agent's working directory, downloaded files, and half-finished outputs survive between tool calls rather than evaporating; snapshot and resume, the ability to checkpoint a running environment, filesystem, memory, active processes and all, and restore it later without starting over from zero; and indefinite runtime, meaning no arbitrary timeout that kills a task simply because it ran longer than some default limit anticipated.

Get this wrong and the consequences aren't subtle. An agent that loses its state mid-task either fails without telling anyone, or it reruns work it already completed, burning inference budget on output it already produced, or it stalls out and waits for a human to come restart it. Every one of those outcomes chips away at the autonomy that was supposed to be the whole point of building an agent in the first place.

This is where orchestration frameworks earn their keep. LangGraph, for one, models agent workflows as directed graphs that support cycles, conditionals, and parallel branches, running in production at roughly 400 companies with about 90 million monthly downloads. That graph structure maps naturally onto long-running, resumable work: the graph itself persists even when one individual node inside it fails, and it connects directly back to the execution layer discussed earlier. A sandbox that provisions in under 90 milliseconds and can resume from a snapshot removes the cold-start tax that would otherwise make stateful, multi-step work slow and expensive. That combination, fast provisioning plus real persistence, is what actually separates a runtime built for agents from a generic container that happens to run agent code. All these pieces, perception, planning, tools, execution, memory, still need something to hold them together, which is the last layer to walk through.

Orchestration: how the layers are composed into a working agent

Orchestration is the layer that doesn't do any one thing dramatically, but without it, none of the other layers know how to talk to each other. It routes inputs to the right place, runs the plan-act-observe loop, handles branching logic and error recovery, keeps track of state as the agent moves through a task, and enforces the policies a team decides matter, retry limits, human-in-the-loop checkpoints before a risky action fires.

The pattern that's won out, at least for now, is graph-based execution: modeling a workflow as a directed graph with cycles, conditionals, and branches that can run in parallel. That maps well onto how agent tasks actually unfold in the real world, non-linear, sometimes needing to resume from an earlier point, occasionally doing two things at once.

The framework landscape reflects a few different bets on how much structure to impose. LangGraph 1.0, released in October 2025, leans fully into the graph model and currently leads in production deployments, with something like 400 companies running it and roughly 90 million downloads a month, showing up at ride-sharing platforms, financial institutions, and professional networking sites. CrewAI takes a role-based approach instead, assigning agents distinct roles within a crew; it works well for smaller setups, though teams commonly report hitting scaling friction somewhere between six and twelve months in. The OpenAI Agents SDK stays deliberately minimal with just four primitives, agents, handoffs, guardrails, and tracing, betting that simplicity itself is the feature.

None of these frameworks replace the layers underneath them; they compose them. Which is really the argument this whole piece has been making, one layer at a time: perception feeds planning, planning issues tool calls, tool calls land in a runtime that has to be both secure and durable, all of it wrapped in memory that has to stay fresh, and all of that held together by an orchestration layer that decides, moment to moment, what happens next. Knowing which layer you're standing in when something breaks is most of the battle.