AI Red Teaming for Code Execution Pipelines

A code execution pipeline doesn't just expand the LLM attack surface. It changes its nature entirely. When an AI agent can write and run code in a live environment, the relevant question stops being whether the model can be manipulated into saying something harmful and becomes whether adversarial content reaching the model can turn into executable code that damages the host system, exfiltrates credentials, or persists across sessions to compromise future agent behavior.

Generic LLM adversarial testing, which targets training data, model weights, inference APIs, and system prompts, was never designed to reach most of this. Systems that began as text-completion interfaces now operate as autonomous agents with tool use, code execution, database access, and internet connectivity. Each of those capabilities is a potential entry point that standard red teaming typically ignores.

The pipeline has several discrete layers, each with its own exposure profile: prompt ingestion and instruction parsing, code generation, the sandbox boundary, persistent state and session storage, and tool and network interfaces. A failure at any one of them can propagate damage to those downstream. What makes this surface distinct, not merely larger, is that the model's output is directly executable. In a standard chatbot, a harmful model output is text a human reads and may or may not act on. In a code-executing agent, that human intermediary disappears. The runtime doesn't evaluate intent; it runs what it receives.

The consequence is concrete. Claude Code wiped a user's entire Mac home directory via a trailing ~/ in an rm -rf command. Cursor IDE deleted 70 files despite an explicit instruction not to run anything. These were not adversarial attacks. They were ordinary agent behavior in the absence of a hard enforcement boundary. Adversarial inputs amplify this risk considerably rather than creating it.

Three primary attack vector classes follow from this structure: prompt-to-code injection, sandbox escape, and stateful session exploitation. Each operates at a different layer, requires different red team techniques, and is governed by different design properties in the runtime. The relationship between them matters as much as the individual vectors.

Prompt-to-Code Injection: How Adversarial Instructions Become Executable

The mechanics here are worth understanding carefully, because they're easy to underestimate. Prompt-to-code injection is the class of attack where adversarial content in the model's input causes the model to generate code that performs actions the operator did not intend. The input can be a user prompt, a retrieved document, a tool output, or any external content the agent consumes. The output isn't text the user reads; it's code the runtime executes. That distinction makes the harm immediate and potentially irreversible.

Direct injection is the most legible path: a user steers the agent toward destructive or exfiltrating code. Indirect injection is subtler and, in practice, harder to defend against. The agent retrieves content from the web, a file, or an external API that contains embedded instructions; the model, lacking reliable mechanisms to distinguish data from directives, treats those instructions as legitimate. Tool-output poisoning extends this further. A prior tool call returns a payload that shapes subsequent code generation. Chains of agent steps amplify the opportunity because each step's output can contaminate the next step's input, and the contamination may not become visible until several steps later.

The Snowflake Cortex Code CLI disclosure is the clearest documented example of this reaching production. Indirect prompt injection combined with insufficient validation of generated commands produced arbitrary code execution and credential access. The attack did not require compromising the model itself; it required only that the model behave as designed when presented with content crafted to resemble legitimate instructions. The fix shipped in version 1.0.25. Standard LLM adversarial testing would not have surfaced the vector, because the vulnerability lived in the gap between what the model output and what the host system subsequently executed.

Red team techniques for this vector need to cover every external input channel the agent consumes: retrieved documents, web pages, database query results, prior tool outputs. Injecting adversarial payloads into each of these tests whether the agent draws any functional distinction between data and instructions. Multi-step attack chains, where a benign-seeming first step contaminates the context for a later code-generation step, are particularly important because single-turn red teaming will miss them entirely.

A question worth examining carefully before testing begins: does the code-generation step inherit any of the model's instruction-following constraints, or do those constraints effectively dissolve once output is framed as code rather than prose? In my experience, teams assume the former and are surprised when testing reveals the latter.

Define what a successful attack looks like before running any tests. Exfiltration of environment variables or credentials, file writes outside the expected scope, outbound network calls to attacker-controlled endpoints: these are the primary indicators. A test that doesn't instrument for those outcomes may pass because the attack was invisible, not because it failed.

Sandbox Escape Attempts: Testing the Boundary Between Agent and Host

If prompt-to-code injection succeeds and the sandbox holds, damage is contained. If the sandbox fails, the host system, other tenants, and production infrastructure are all at risk. The sandbox boundary is, in that sense, the last meaningful line of defense, and its strength is entirely a function of the isolation technology underlying it.

The major isolation approaches differ meaningfully in what they expose. Firecracker microVMs provide strong VM-level isolation with a dedicated kernel per sandbox, substantially reducing the shared-kernel attack surface that container-based approaches carry. gVisor interposes a user-space application kernel between workload syscalls and the host kernel, so generated code never reaches the host kernel directly; the interposition layer is the attack surface, not the full host kernel. Standard Docker containers are the most widely deployed option, often for the practical reason that Docker-native compatibility is a real operational requirement, but the shared kernel means a kernel exploit inside a container can reach the host. That gap between convenience and containment is where red teams most often find traction. Whether that tradeoff is acceptable depends on the specific deployment context, but it should be a deliberate choice rather than a default.

There's a compounding problem here. Frontier models' success on cybersecurity tasks rose from under 10% in late 2023 to roughly 50% by 2025, with expert-level task completion appearing during 2025. Sandbox architectures calibrated to 2023 model capability may already be insufficient for what those models can now attempt. The target has moved.

Red team techniques for sandbox escape fall into several categories, none of which can be safely skipped: syscall boundary testing to observe whether privileged syscalls reach the host or are intercepted at the isolation layer; filesystem mount probing for access to paths outside the project directory; network namespace testing for outbound connections to external hosts, lateral connections to other sandboxes, and access to host-network services; resource exhaustion tests to determine whether resource limits are enforced at the sandbox layer or propagate to the host; CVE-class container breakout attempts via misconfigured capabilities, volume mounts, or shared PID namespaces.

When a new agent framework called OpenClaw was released in early 2026, over 100 vulnerabilities appeared in it relatively quickly, including a one-click remote code execution via auth-token theft rated CVSS 8.8. The runtime stack itself is an adversarial surface, not just the model running on top of it.

Several design properties either close or widen escape vectors. Each sandbox should have its own filesystem, network namespace, and resource allocation. Sandboxes should mount only the project directory. OS-level primitives should enforce limits so no sandbox can reach another. Sandboxes should be destroyed after use rather than reused across untrusted workloads. A sandbox that passes all of these checks under normal load may still fail under concurrent execution at scale, so red teaming should include multi-tenant load scenarios, not just isolated single-session tests.

Stateful Session Exploitation: The Attack Surface That Persists Across Turns

Long-running agents are a production reality, not an edge case. Cursor documented an agent operating autonomously for close to a week, writing over a million lines of code across a thousand files. That kind of workflow requires persistent state. LangGraph, Microsoft Agent Framework, and AWS Step Functions all provide explicit checkpointing for exactly this reason. State persistence is a deliberate architectural requirement.

It is also a persistent attack surface, and one that red teams regularly underweight because it requires multi-turn attack scenarios that play out over time rather than the single-turn tests most tooling is built for.

State poisoning injects a malicious value into the agent's persisted context early in a session; it propagates into later steps where it influences code generation or tool calls. The injection may be entirely benign-looking at the point of entry, which is precisely what makes it difficult to catch. Snapshot manipulation targets checkpoint storage directly: if an attacker who has compromised the agent can write to snapshot storage, a tampered snapshot can alter the agent's effective memory of what it has been told or done. Session replay attacks capture a valid session token or checkpoint and replay it to hijack the agent's identity or resume its execution under attacker control. Cross-session leakage, particularly important in multi-tenant deployments, tests whether one session's persisted state is accessible to another, especially when sandboxes are reused rather than destroyed. Long-horizon manipulation, perhaps the hardest variant to test for, crafts an injection that appears harmless in the first few turns but reaches a harmful conclusion only after many steps.

What makes this vector methodologically difficult is less the attack design and more the infrastructure required to test it. Most automated red teaming tools are built for single-turn adversarial inputs. Multi-turn, multi-step attack scenarios that play out over hours or days require a fundamentally different testing harness. That gap between what automated tools can exercise and what long-running agents actually do is, at present, one of the more significant blind spots in the field. Most teams I've seen either skip this vector entirely or test it superficially and count it as covered.

How Runtime Design Determines Which of These Vectors Are Exploitable

The exploitability of each vector is not fixed. It is a function of how the runtime is designed, and red teams should understand those design decisions before they test against them.

Provisioning speed is a concrete example. A runtime that spins up a fresh sandbox in under 90 milliseconds can isolate each agent invocation in its own environment, eliminating cross-session contamination through architecture instead of access control. A runtime that amortizes startup cost by reusing environments trades that security property for performance. The tradeoff is real, worth understanding precisely, and easy to assume away without examining.

The tension between ephemeral and stateful design is similarly concrete. Ephemeral sandboxes that destroy state after each session close the stateful session exploitation vector but break long-running agent workflows. Stateful sandboxes that persist state keep workflows intact but make the state store an explicit attack surface requiring isolation, integrity checking, and access control. A carefully designed runtime can support both properties simultaneously. Indefinite runtime duration with hard session isolation between tenants is architecturally achievable, but it requires building both properties simultaneously rather than defaulting to one.

Network policy governs a vector that sits partially outside the sandbox boundary itself. An agent that can make arbitrary outbound calls from inside the sandbox can exfiltrate data or beacon to attacker infrastructure even if the sandbox boundary is otherwise intact. Egress filtering and network namespacing are runtime-level controls. The model cannot enforce them, and no amount of system-prompt hardening substitutes for them.

Observability deserves more attention than it typically receives in security discussions. Teams need to inspect what code actually ran, which syscalls were attempted, which network connections were made, and what was written to state. A runtime that doesn't expose this instrumentation makes adversarial testing superficial; a passing test may simply mean the attack was invisible. Open-source runtimes offer an advantage here because red teams can inspect the isolation implementation directly instead of relying on vendor claims, which matters considerably when the runtime itself is part of the attack surface.

For teams operating under data-residency requirements, bring-your-own-compute deployment models introduce a specific verification obligation. It isn't sufficient that a runtime claims to keep sensitive code and data within a specific cloud account; testing should verify that the runtime actually enforces that boundary, particularly under multi-tenant concurrent load.

Structuring a Red Team Exercise for a Code Execution Pipeline

Threat modeling should precede adversarial testing. Before designing test cases, map every external input the agent consumes: user prompts, retrieved documents, tool outputs, web content, prior session state. Then map every action the runtime can take: file writes, network calls, process spawns, state reads and writes. That map defines the scope of the exercise. Without it, test coverage is largely a matter of chance.

Organize test cases around the three vector classes. For prompt-to-code injection, design one test suite per external input channel; include both direct and indirect injection payloads; include multi-step chains that span several agent turns. For sandbox escape, cover syscall boundary testing, filesystem traversal, network namespace probing, resource exhaustion, and CVE-class container breakout, and run all of these under both single-session and concurrent-load conditions, because behavior diverges between them. For stateful session exploitation, design multi-turn state poisoning scenarios, test snapshot integrity directly, verify session replay resistance, and confirm cross-session isolation under concurrent load. This last suite requires test harnesses capable of driving agents through full lifecycles. Build or acquire them before the exercise begins, not after.

The runtime stack itself belongs in scope. Version-pin all runtime dependencies and run CVE scanning against the sandbox infrastructure as part of the exercise. Treating the stack as trusted while red teaming the model is a category error that the OpenClaw case makes difficult to defend.

Automated tools and human red teamers serve different purposes here, and each fills a role the other cannot. Automated tools handle high-volume single-turn injection tests efficiently and consistently. Multi-turn stateful attacks and creative sandbox escape attempts require human red teamers who can reason about agent behavior over time, adapt to unexpected outputs, and construct attack chains that no automated tool was specifically designed to discover.

Define failure criteria explicitly before testing begins. What constitutes a sandbox escape? What constitutes a state poisoning success? What level of exfiltration triggers a finding? Without explicit criteria, red team results are difficult to prioritize and nearly impossible to act on. Retest after every significant change to the runtime, the model, or the agent's tool access. The attack surface changes continuously across its lifecycle, and a passing exercise from three months ago says little about the system as it operates today.