How Agents Persist State Across Sandbox Restarts

Most production agents do not survive a sandbox restart intact. They survive it as a partial reconstruction, rebuilt from logs and replay at a cost measured in tokens, latency, and engineering hours. The difference between a crash and an eviction is the difference between losing a step and losing a day. This piece examines how state persistence across sandbox restarts actually works, why the problem is layered rather than singular, and which mechanisms address which failure modes.

Why sandbox restarts are a continuity problem, not just a reliability problem

A sandbox restart that loses a single tool call is recoverable in seconds. A sandbox eviction that loses hours of accumulated filesystem state is a different category of failure entirely. The distinction matters because most infrastructure discussion treats both events as plain reliability, but the operational damage they cause differs by orders of magnitude.

Agents that execute code accumulate state over time: crawled datasets, installed packages, conversation history, warm caches. Every sandbox expiration destroys that accumulated work in one moment, forcing a reconstruction that consumes the same resources a second time. The second is a structural tax on every long-running deployment.

The gap between a sandbox that runs code and a sandbox that remembers what it did last week stalls production deployments in predictable ways. Standby caps, idle billing policies, and ephemeral filesystems push teams into workarounds that add latency, cost, and fragility. The question is not whether continuity problems will arise but how much engineering effort will be spent compensating for them. That is the distinction between an operational nuisance and an architectural design problem.

Why ephemeral infrastructure was never designed to hold an agent's place

Standard serverless platforms terminate containers immediately after processing. That design is intentional and correct for stateless request-response workloads, where any two invocations are interchangeable and holding state between them would be waste. Agents demand the opposite: environments that persist between invocations and restore state without rebuilding. The mismatch is fundamental, not incidental. Infrastructure built for one execution model is being asked to support another, and the assumption that persistence can be added as an afterthought is the root of most continuity failures.

The isolation requirement makes that gap worse. Teams using them for untrusted AI code inherit an ephemeral default they then must engineer around. A reported incident in which a Replit AI agent deleted a production database illustrates why isolation is not optional: agent-generated code must be treated as untrusted, which pushes teams toward the very isolation technologies that are least accommodating of long-lived state.

Teams that hit this wall typically respond with one of three workarounds. Or they accept that the agent starts cold on every run. None of these paths solves the underlying problem. They compensate for it.

The implication is that no single fix exists because the problem has multiple independent failure modes. Filesystem loss, process-memory loss, and workflow-position loss are three distinct things that ephemeral infrastructure destroys, and they require different mechanisms to recover. Understanding those distinctions is the beginning of a coherent persistence strategy. The rest of this piece works through four layers: filesystem snapshots, pause-and-resume with memory, checkpoint-and-restore at the workflow level, and external state stores, each addressing a specific failure mode, all dependent on a sandbox platform designed with statefulness as a first principle.

Filesystem snapshots: preserving what the agent built on disk

Filesystem snapshots address the most visible continuity problem: repositories cloned, packages installed, datasets written, and artifacts generated survive a restart because the disk image is preserved and re-attached rather than discarded. This is the baseline layer on which every other mechanism builds. Without it, no amount of workflow checkpointing or external state management matters, because the agent's concrete work product vanishes on eviction. Snapshots are the first line of defense against the accumulation tax described earlier.

Two distinct persistence primitives exist at the filesystem layer, and they are not interchangeable. Standby or suspend state is the sandbox's own filesystem snapshot, kept alive between pauses. It is fast to resume but tied to the sandbox instance: if the sandbox is deleted, this state is gone. Persistent volumes are durable block storage that survives sandbox deletion and can be re-attached to a new sandbox instance. They are the correct primitive for long-term, cross-session persistence. Sandbox state persists during standby but not after deletion. Volumes provide guaranteed long-term persistence.

The two primitives answer different questions. Volumes answer the question of what survives the complete destruction of an instance. Fly.io Sprites offers filesystem checkpoints for rollback alongside persistent Linux workspaces that idle between tasks, a concrete example of filesystem-layer persistence deployed in a production platform.

But filesystem snapshots have a hard boundary. The disk state survives. The computational state does not. That is the gap the next two mechanisms fill.

Pause-and-resume with memory: suspending a running agent mid-execution

Suspend-and-resume at the microVM level preserves not just the filesystem but the full memory state: running processes, open file descriptors, in-progress computation. An agent can be paused and wake up exactly where it stopped, with no re-execution of work already done. Blaxel's model demonstrates what this means in practice: sandboxes restore filesystem and memory state from standby with exact previous state restored, running processes pick up where they stopped, and resume completes in under 25 milliseconds.

The architectural mechanism is a persistent paused sandbox that keeps the filesystem and a memory snapshot alive between runs. The next resume is sub-second rather than a full boot, meaningfully different from a cold start that must re-execute the agent's initialization sequence. The cost model shapes whether pause-and-resume is financially viable for a given workload.

The strongest objection to relying on memory snapshots alone is straightforward: if the sandbox is not merely paused but actually deleted, by a garbage-collection policy, a billing event, or a platform timeout, the memory snapshot is gone along with it. Cloudflare Sandboxes illustrate the constraint: a container stops after the default 10-minute idle timeout and all files are deleted, a hard limitation for agents needing long-running state. Memory-snapshot persistence is only as durable as the platform's standby policy allows.

There is also a definitional problem: the term "pause" covers meaningfully different capabilities across platforms. Pausing stops running processes and terminal sessions on platforms that implement only filesystem-level pause, and it does not save RAM. Northflank's documentation is explicit on this point. Even with full memory preservation, however, a deleted sandbox destroys everything. Recovering workflow position from deletion requires a mechanism that lives outside the sandbox entirely.

Checkpoint-and-restore at the workflow level: surviving deletion, not just pause

When a sandbox is deleted rather than paused, neither filesystem snapshots nor memory capture help. Recovering the agent's position requires a durable record of the workflow's progress, one that can reconstruct where the agent was without re-executing everything from the start. That record must live outside the sandbox, because the sandbox no longer exists. Two distinct approaches exist at this layer, and they allocate responsibility differently between the developer and the runtime.

Graph checkpointers, the LangGraph model, checkpoint the agent graph at defined boundaries called super-steps. The developer still implements the retry, dedup, and resume logic. The checkpointer is infrastructure for a programming model, not a guarantee. This approach fits short-running agents and human-in-the-loop pauses inside a longer workflow.

Durable execution runtimes such as Temporal and Azure Durable Task invert that allocation. The developer writes ordinary code, and the runtime reconstructs workflow state by replaying the event log to the point of failure. Andreessen Horowitz led a substantial investment into Temporal explicitly on the AI agent thesis, a signal that durable execution is a primary infrastructure layer for agent workloads. The two approaches are not competitors. They solve different scopes of the problem, with graph checkpointers for bounded workflows and durable execution for long-running stateful processes.

A formal correctness concern complicates both approaches. Research on safe-to-resume semantics, documented in arXiv:2608.03836, defines a machine-checked conformance contract for checkpoint, interrupt, and resume semantics in workflow persistence layers, and finds that widely deployed frameworks violate even the fragments of that contract they claim to satisfy. The gap deserves explicit testing in production systems.

A related behavioral concern runs parallel to the formal one. Plans represented only in natural language remain advisory unless the runtime maintains progress and checks at resume time that required conditions have been met. Checkpointing preserves where the agent was. It does not by itself guarantee that the agent finished what it was asked to do.

External state stores: what lives outside the sandbox by design

Some state should be externalized by design rather than as a workaround. Agent memory, including conversation history and learned context, shared artifacts, and cross-session indexes must survive not just sandbox restarts but sandbox replacement. External stores are the correct home for this state because they decouple accumulated knowledge from any single sandbox instance's lifecycle. The mechanism is straightforward: the agent reads relevant context from an external store at session start, operates inside the sandbox, and writes durable results back before any potential eviction.

This is distinct from volumes in an important way. Blaxel's Agent Drive, a distributed filesystem in private preview, mounts to multiple sandboxes with concurrent read-write access, allowing context and artifacts to pass between sandboxes without intermediary storage. That capability matters when an agent's work spans sessions or when multiple agents share a common working context.

The tradeoff is latency. Co-located agent hosting, deploying agent logic alongside sandboxes on the same infrastructure, eliminates the network roundtrip latency from repeated tool calls during an interaction. Design decisions about external state therefore cannot ignore where that state physically lives relative to the sandbox.

External stores are necessary but not sufficient. A strategy that keeps everything local fails the moment the instance is replaced. The layers are complementary by design.

Stateful sandbox design as the layer that makes the others work

The mechanisms covered so far each work better and require less custom engineering when the sandbox platform is built to support long-running, stateful workloads rather than short-lived script execution. External state integration is cheaper when the sandbox sits near the store. Stateful sandbox design is not another mechanism alongside the others. It is the substrate that determines how well the others function.

Three platform design properties govern that substrate. Sub-second resume is the target, and platforms such as Blaxel achieve under 25 milliseconds from standby. Standby policy determines how long a sandbox remains alive without compute charges, and whether that limit is fixed or indefinite. Blaxel keeps sandboxes in standby indefinitely on Tier 2 and above with no compute charges during standby, though storage charges apply and starter tiers enforce time-based expiration. Northflank imposes no fixed session time limit, using microVM isolation for CPU workloads and gVisor for GPU workloads, with a self-serve BYOC model in which Northflank operates as the control plane while customer workloads run inside the customer's VPC.

The isolation model is the third property, and it carries a consequence that is frequently misunderstood. Platforms using microVMs with dedicated kernels per workload provide the security boundary that makes it safe to let a sandbox run indefinitely, because the blast radius of untrusted code is contained regardless of how long the sandbox lives. Five dimensions separate platforms built for ephemeral script execution from those built for production agent workloads: state persistence, resume latency, standby duration, isolation model, and cost model during idle periods. Evaluating a platform on any one of these dimensions while ignoring the others produces a false sense of fit.

Audit trail continuity is a concrete benefit of stateful sandbox design that matters for enterprise deployments. Persistent sandbox identity means every action the agent takes can be attributed to a consistent identity across restarts, producing a tamper-evident log. For organizations subject to compliance or security review, this alone may justify the shift to stateful infrastructure.

How the layers fit together in a production agent and where each one fails alone

No single mechanism solves the continuity problem. Filesystem snapshots lose process memory. Memory snapshots lose workflow position on deletion. The layers are complementary, not redundant, and the failure of each one becomes visible precisely when the others are missing.

Which layers an agent actually needs depends on the shape of its work. Long-horizon autonomous agents that run for hours or days need all four layers: volumes for disk, memory-preserving pause for sub-second resume, durable execution or graph checkpointing for workflow position, and external stores for cross-session knowledge that must outlive any single sandbox instance. This is not a menu of optional enhancements. It is a mapping from failure modes to mechanisms.

The correctness gap remains the open problem. Teams should test resume paths explicitly and resist treating "checkpoint written" as equivalent to "resume verified." The research on safe-to-resume semantics shows that widely deployed frameworks violate even the fragments of their own claimed contracts. That finding counsels humility about what a persistence layer actually guarantees.

The broader research framing returns to the Workspace and Skill paradigm, which argues that the leap from ephemeral tool calls to persistent state is a decisive leap from ephemeral interactions to persistent stateful work. State continuity is not a performance optimization. Stacking persistence layers is not over-engineering. It is the minimum credible answer to the question of whether agents can be trusted with work that spans more than a single session.