Open Source AI Agent Frameworks and Their Infrastructure Requirements

Open source AI agent frameworks define more than how an agent reasons or calls tools. Their architectures determine what infrastructure must preserve, isolate, and recover.

Why framework choice and infrastructure choice are the same decision

An execution model is already an infrastructure assumption. A framework that branches, pauses, or delegates work assumes that something underneath it can preserve the state those operations create. Treating that requirement as a later configuration detail confuses application design with the conditions needed to execute it.

The mismatch appears when the workload stops behaving predictably. Generic containers and developer sandboxes suit predictable, short-lived, human-authored workloads, while agents introduce long horizons, machine-generated code, and tool calls into external systems. That mismatch causes lost context and runaway loops at runtime.

Orchestration makes the failure surface legible before deployment. Graph-based systems need transitions they can recover from, role-based systems put boundaries between agents, and chain-based systems need to move reliably from one step to the next. Auditing those demands belongs inside framework selection, before implementation makes a change expensive.

What the 2026 framework field actually looks like and how to read it

The 2026 field spans substantially different architectural approaches. LangGraph, CrewAI, OpenAI Agents SDK, Google ADK, and Dify sit within a landscape that also includes document pipelines, TypeScript-first tooling, and lightweight single-agent loops. LlamaIndex Workflows, Haystack, Mastra, Vercel AI SDK, and Smolagents broaden the comparison beyond conversational orchestration. A feature count would flatten those differences.

Names also need care because project lineages have diverged. Microsoft Agent Framework shipped Python and.NET runtimes to general availability on April 3, 2026, with declarative YAML configuration and migration assistants from Semantic Kernel and AutoGen. In November 2024, original AutoGen creators Chi Wang and Qingyun Wu forked it as AG2, retaining the conversational API. Use AutoGen” therefore leaves a consequential choice unresolved between the research-oriented AG2 path and the cloud-platform-supported Microsoft path.

Three questions make this field easier to read. Does the framework require durable state? Does it execute untrusted code? Does its work last long enough to need crash recovery? These questions show what the runtime must do, without ranking developer experience.

Graph-based frameworks: what explicit state control costs at runtime

LangGraph models agents as explicit nodes and edges, with checkpointing, durable state, and human-in-the-loop pauses built into its control model. The runtime must be able to recover every checkpoint and branch for execution to follow the control the graph defines. A stateless sandbox that discards context between calls breaks the model.

Pauses make this dependency especially concrete. An approval can leave a run idle for hours or overnight, while parallel branches require concurrent execution without shared-state bleed. Container restarts must not erase the state needed to continue. LangGraph Platform supplies managed durable execution, checkpointing, approvals, and LangSmith integration for tracing and evaluation, but those capabilities are distinct from the open-source library.

Deep Agents absorbs some complexity at the framework layer. It adds planning, subagents with isolated context windows, filesystem abstraction, context offload, shell access middleware, and reusable skills. These additions do not remove the need for a dependable runtime. Explicit control makes graph-based frameworks well suited to audit-heavy, regulated production systems precisely because execution can be examined, provided the runtime preserves what the graph defines.

Role-based and multi-agent frameworks: isolation demands that scale with the number of agents

Role-based orchestration moves complexity into delegation and handoffs. CrewAI expresses agents as roles such as researcher, writer, and reviewer, while OpenAI Agents SDK offers agents, tools, handoffs, guardrails, and sessions. Microsoft’s declarative configuration places agent topology at the orchestration layer. Despite these different interfaces, every handoff introduces a trust boundary the runtime must respect.

That boundary needs more than separate conversational contexts. Each agent should execute in its own isolated environment and have its own identity for controlled access and traceability. When agents share a container, malicious tool execution or a runaway loop can affect the other agents in it. Provisioning speed then becomes a throughput constraint because each additional concurrent agent needs the same protection.

Could short-lived workflows safely remain stateless? Only if their actual execution needs permit it. When one agent delegates a long-running task, the recipient may need the full tool-call history to continue correctly. The runtime must preserve that continuity across the handoff rather than assume that a friendly role abstraction makes the underlying execution independent.

Document-centric and pipeline frameworks: where the data layer becomes the runtime bottleneck

For agents that work with many documents, more of the critical path rests on data retrieval. LlamaIndex began as a data framework for LLM applications, and production teams often use its retrieval layer beneath another orchestrator. LlamaIndex Workflows adds event-driven orchestration for data-intensive pipelines. Here, reliability depends not only on running an agent but on sustaining the retrieval work that feeds it.

The timing of that work is uneven. Some pipeline steps take milliseconds, but others wait seconds or minutes for data from outside the pipeline. The runtime must handle those asynchronous bursts and provide persistent filesystem access for agents that read, chunk, embed, and write across document corpora. A temporary scratch directory cannot serve as the durable working environment.

Haystack makes pipeline structure explicit and serializable. That shows what should run and in what order, but the execution environment must keep pipeline state that teams can inspect even after a restart. The distinction is important for auditability. A defined pipeline shows what should happen, while a runtime record shows what actually ran and which data it used.

TypeScript-native frameworks and the ecosystem gap in runtime support

Mastra and Vercel AI SDK make TypeScript a serious production option, but their surrounding runtime ecosystem is thinner than Python’s in durable execution, sandbox libraries, and observability integrations. Language choice therefore changes the integration burden. Where the framework does not bundle durability, the team must supply state persistence below it. Otherwise, a transient failure can erase progress.

Vercel AI SDK makes that distinction unusually visible. As of July 2026, ToolLoopAgent supports the classic tool-calling loop, HarnessAgent supports established harnesses such as Claude Code, Codex, and Pi, and WorkflowAgent provides durable, resumable execution. WorkflowAgent automatically retries each tool execution as a durable step and supports recovery across process restarts, deployments, and delayed approvals. Its abstraction includes runtime behavior that other deployments must assemble separately.

Two contrasting approaches clarify the trade-off. Google ADK offers teams native to its vendor’s cloud platform an opinionated runtime with built-in debugging UIs, absorbing infrastructure work at the cost of portability. Smolagents provides a fast route to a single-agent loop with minimal orchestration and a correspondingly small runtime ask. That simplicity also leaves limited production durability.

Why code-executing agents demand a different isolation model regardless of framework

Running AI-generated code changes the whole trust model. Coding-focused frameworks such as OpenHands, Cline, and Aider make this demand especially clear, but it applies to any framework that runs generated code. Standard containers share the host kernel, so a flaw in the runtime can put the host at risk. AG2’s built-in Docker sandboxing does not itself provide kernel-level separation.

The isolation approaches include microVMs such as Firecracker and Kata Containers, the gVisor user-space kernel, and hardened containers. MicroVMs are the default recommendation for untrusted code. Prompt injection also calls for input validation, prompt filtering, output monitoring, and sandboxed tool execution. Generated-code exploits require isolated execution with no network access and minimal system privileges. Zero trust requires explicit permission for actions.

Strong separation need not mean reinstalling tools for every run. Snapshots that hold an OS, packages, dependencies, and configuration let coding agents start with environments that are already set up. This connects security to provisioning performance. Purpose-built sandboxes start quickly, keep state, and have separate execution boundaries, giving untrusted code protection beyond what a developer container alone can provide.

Stateful long-running execution: the infrastructure problem most framework tutorials skip

“Durable execution” in framework documentation is a promise about behavior. The runtime must provide persistence, pause and resume, and crash recovery. LangGraph Platform supplies those services for its managed deployments. Self-hosting the library leaves the durable execution layer with the team.

Redis can hold current conversation context and active tasks for minutes to a few hours, while a vector database stores past interactions and extracted knowledge over the long term. SQL or object storage keeps audit logs and the full history of a run. An agent needs its full execution context after a crash, and no one of these stores supplies all of it.

Durable engines address a distinct part of this problem. OpenAI’s Codex web agent and Cursor run on Temporal, while Replit’s agent migrated from Temporal to Restate. Their use illustrates why orchestration and recovery cannot be treated as interchangeable. A short workflow still needs durability if it waits on an asynchronous API, a human approval, or a compilation job beyond an idle timeout. Persistent sandboxes that can remain idle without billing and resume with intact history are a baseline for such workloads, whether the framework is LangGraph, CrewAI, or Mastra.

Enterprise compliance requirements that the framework cannot satisfy and the runtime must

A deployment needs more than framework behavior alone to meet compliance rules. HIPAA obligations include a BAA with the model provider and an audit trail of actions touching protected health information. The runtime must capture those actions. SOC 2 also requires agents to be within the system boundary, with audit evidence of their actions in logs that remain after a sandbox exits.

GDPR adds lawful processing and data minimization requirements. At the runtime layer, these translate into controls over context limits and persistent PII in agent memory. Separate agent identities make actions attributable and access controllable. These controls belong in the execution architecture, rather than being inferred from a framework’s support for guardrails or serializable workflows.

Platform capability is not the same as deployment security. An Agentforce deployment without Shield Platform Encryption, correctly scoped permissions, and a signed BAA is not HIPAA-compliant simply because the platform can support those controls. Purpose-built runtimes should treat these controls as baseline capabilities, with customer-managed compute keeping workloads in the customer’s own cloud as the architectural answer to data sovereignty.

How to match a framework's runtime profile to infrastructure that can actually support it

LangGraph and Deep Agents call for durable checkpoints, long-idle persistence, and isolated parallel branches. CrewAI adds per-agent environments, rapid provisioning, and tool-call history that survives handoffs. For self-hosted graph deployments, the durable execution service must be supplied explicitly.

Microsoft’s topology and AG2’s execution model require different attention. The former needs infrastructure that follows its declarative agent arrangements and works with its Python and.NET environments, while the latter needs stronger separation for untrusted code than Docker alone provides. OpenAI Agents SDK needs durable sessions. Teams leaving its hosted stack must also supply tracing and state persistence externally.

Data and TypeScript frameworks shift the emphasis again. LlamaIndex Workflows and Haystack need asynchronous capacity, persistent document filesystems, and inspectable pipeline state. Mastra needs the runtime capabilities its framework does not bundle, while WorkflowAgent includes durable step execution in its abstraction. Docker-native compatibility remains useful for teams bringing existing workloads, but compatibility is not an isolation guarantee.

Coding agents make performance and protection inseparable. MicroVM or user-space-kernel isolation sets the execution boundary, while snapshot provisioning and sub-90-millisecond startup keep prepared environments responsive at scale. These requirements support purpose-built agent infrastructure with capabilities beyond generic containers alone. Its final obligation is transparency, so the team can inspect the runtime beneath the workflow with the same rigor applied to the framework itself.