Run the single-node daemon¶

Use OpenSandbox 1.1.0 or E2B Python SDK 2.59.0 to drive local Linux VM sandboxes. The daemon owns resource admission, persistent ownership, lifecycle and expiration cleanup. Each sandbox has an independent supervisor that clients can reconnect to after an API restart.

Use native Jobs through pvisor spawn when you need workspace review and merging. The daemon provides a tested core SDK subset without Job stage/apply or checkpoint/fork APIs. Callers own cross-node placement, workflows and business retries.

Prerequisites¶

You need Linux x86_64, a usable /dev/kvm, trusted absolute paths, and a writable, delegated cgroup v2 hierarchy with the CPU/memory/PID controllers and cgroup.kill enabled.

Preflight checks perform real controller writes and KVM API calls. There is no host, OCI-command or registry-pull fallback.

Create requests use a local template key in images_dir/<key>.json; creation performs no registry pull or complete image extraction. First use prepare-image to prepare a trusted image and pin its revision; prefer digests in deployments. The base must contain /bin/sh, /bin/sleep and the workload tools; E2B shell commands also require /bin/bash.

New templates use the Rust process/file engine in pvisor-guest by default. The host supervisor translates SDK protocols into host-only guest control requests; you do not need to write a vsock bootstrap. Explicit --execd/--envd selects upstream services. Every template requires a pinned V2 pool binding; standalone-rootfs manifests are rejected; see the image contract.

Install and start¶

Linux x86_64 and macOS ARM64 wheels install pvisor-daemon alongside pvisor. Cache serving works on both platforms; the sandbox service below requires Linux x86_64:

pip install pvisor-sandbox
pvisor-daemon --help
pvisor-daemon protocol

Use the nightly wheel installer for recent builds, or mise run build -- performance daemon from source. Pin the daemon revision, SDK version and image contents together. Prepare separate protocol templates:

pvisor-daemon prepare-image --name ubuntu --image ubuntu:24.04 --images-dir /srv/pvi
pvisor-daemon prepare-image --name e2b-base --image ubuntu:24.04 --e2b --images-dir /srv/pvi

serve constructs the NativeRuntime from the implemented and required --images-dir and --cgroup-root:

export OPEN_SANDBOX_API_KEY="$(openssl rand -hex 32)"
pvisor-daemon serve --sandbox \
  --images-dir /srv/pvi \
  --cgroup-root /sys/fs/cgroup/pvd \
  --listen 127.0.0.1:8080 \
  --state /run/user/1000/pvd \
  --max-sandboxes 32 \
  --cpu-millis 4000 \
  --memory-bytes 8589934592 \
  --max-timeout-seconds 86400

Runtime paths are absolute and must stay unchanged across daemon restarts. Keep the state path short, for example /run/user/1000/pvd: each sandbox's control.sock must be shorter than 104 bytes, and vsock Unix sockets also have path-length limits. Retain the state; this /run example does not guarantee persistence across logout/reboot, and VMs do not survive a host reboot. /sys/fs/cgroup/pvd must be a real delegated hierarchy, not a plain directory.

Choose a private state directory outside the checkout. Generate the API key once, keep it in protected service secret storage, and reuse the same value across restarts. The example admission budget is 32 records, four CPU units and an 8 GiB total hard memory limit. Whole-node physical occupancy is unmeasured; leave headroom for daemon/cache/host and configure host supervision separately.

Query the lifecycle API from a second shell with the same protected API key:

curl --fail-with-body --config - <<EOF
url = "http://127.0.0.1:8080/v1/sandboxes"
header = "OPEN-SANDBOX-API-KEY: ${OPEN_SANDBOX_API_KEY}"
EOF

Expect a JSON sandbox list, empty on a fresh state. This query validates API access; sandbox creation, native resource enforcement and SDK data-plane readiness need separate validation. Do not publish the key or paste credentialed logs into issues.

OpenSandbox interface scope¶

Compatibility is pinned to OpenSandbox 1.1.0 (release-1.1.0). The unmodified official Python SDK has validated core flows below. Full API conformance, notebook/session kernels and all SDK features are outside the supported scope; see the protocol and tests.

Install opensandbox==1.1.0 and run from a process with the same API key:

import asyncio
import os
from opensandbox.config import ConnectionConfig
from opensandbox.sandbox import Sandbox

async def main():
    config = ConnectionConfig(
        domain="127.0.0.1:8080", protocol="http",
        api_key=os.environ["OPEN_SANDBOX_API_KEY"], use_server_proxy=True,
    )
    sandbox = await Sandbox.create("ubuntu", connection_config=config,
                                   resource={"cpu": "1", "memory": "512Mi"})
    try:
        result = await sandbox.commands.run("printf 'hello from pVisor'")
        print(result.logs.stdout[0].text)
    finally:
        await sandbox.kill()
        await sandbox.close()

asyncio.run(main())

E2B interface¶

Use the pinned official SDK for the e2b-base template, with a data endpoint URL that includes /e2b:

pip install 'e2b==2.59.0'
export E2B_API_KEY="$OPEN_SANDBOX_API_KEY"
export E2B_API_URL='http://127.0.0.1:8080'
export E2B_SANDBOX_URL='http://127.0.0.1:8080/e2b'

from e2b import Sandbox

sandbox = Sandbox.create("e2b-base", timeout=300)
try:
    print(sandbox.commands.run("printf 'hello from pVisor'").stdout)
    sandbox.files.write("/tmp/hello.txt", "private VM file")
    print(sandbox.files.read("/tmp/hello.txt"))
finally:
    sandbox.kill()

E2B supports sync/async lifecycle, commands, stdin, background processes, files and PTYs. The native adapter accepts JSON/Connect, without binary protobuf, cloud template builds, notebook kernels, arbitrary app ports or automatic pause/resume. Pause retains the live VM and its reservation while TTL continues; it creates no durable snapshot. See the partial protocol contract.

Query, delete and expiration¶

Replace {id} with the sandbox ID returned by create/list. An HTTP disconnect does not cancel an accepted operation. Each sandbox has its own lifecycle lock; control and connection establishment for one sandbox are serialized.

After losing a create response, do not blindly retry creation: this API is not the old task-ID idempotency protocol. Check retained state first. When create reports an uncertain native result, keep the sandbox ID from the error message and query/delete it. Admission is released only when creation failed and cleanup is confirmed complete; uncertain cleanup retains the record and reservation.

Create timeout is optional; omit/null means manual cleanup. A configured TTL is at least 60 seconds and at most --max-timeout-seconds (default one day, configurable up to one year). Renewal must extend an existing deadline. Expiration cleanup is best effort, not a hard execution deadline; new proxy requests are rejected after expiration. Stopping the daemon normally preserves native sandboxes and persistent records for restart; cleanup is not guaranteed while the daemon is down. For hard deadlines or downtime cleanup, configure separate native host supervision.

Endpoint authentication¶

Resolve execd through GET /v1/sandboxes/{id}/endpoints/44772 and egress through port 18080. The returned authority carries no URL scheme; traffic goes through the daemon, not directly to the supervisor's loopback publication.

• With use_server_proxy=true, routed requests authenticate with the lifecycle OPEN-SANDBOX-API-KEY header.
• The default mode provides a random, sandbox-scoped X-PVISOR-SANDBOX-TOKEN in the endpoint headers. Clients must preserve these headers; the token authorizes neither lifecycle operations nor other sandboxes.
• Control credentials are removed before forwarding to the guest. A caller-provided guest execd access token is not replaced by the lifecycle key.

OpenSandbox signed endpoint expiration, arbitrary app ports, WebSocket and CONNECT are unsupported. The default native backend bounds response streams, process counts, retained output and upload size per sandbox, with explicit failures on overflow. Explicit upstream backends retain their service quotas. See the native engine contract for limits.

Non-loopback listeners are rejected by default. Prefer TLS termination at a trusted reverse proxy with the daemon backend on loopback, and set --public-endpoint HOST:PORT to the external authority. Explicit --allow-insecure-listen permits remote plaintext binding without installing TLS. Wildcard listeners require a public endpoint; loopback 127.0.0.1:0 publishes its actual allocated port.

Optional cache and memory pool¶

Add --cache for a separate cache endpoint, configured with --cache-listen and --cache-image-store, or run pvisor-daemon serve --cache alone. The daemon supervises its worker; unexpected worker exit fails the daemon, and daemon shutdown/crash stops the worker. Failed in-flight requests require reconciliation. HTTP shutdown waits at most ten seconds.

Add --memory-pool to the serve command to connect new sandboxes to the daemon-held physical shared-page pool; it is off by default. The runtime scans resident 4 KiB pages, preferring cross-instance duplicate content, and the daemon retains one physical page. VM reads stay shared; writes get private pages through kernel COW. This Linux path needs no userfaultfd and does not compress unique pages. Actual occupancy depends on duplication, scanning and writes; see the memory benchmark.

The pool component is independent of the API process and retains pages across API restarts. After a disconnect, mapping references are released only once a pidfd confirms the VM process exited. Losing the pool process fails dependent VMs and does not replace the live pool automatically. Default limits are 512 MiB of page data, 32,768 objects, 32 connections and 32,768 references per connection. The 4 KiB object cap bounds the default pool to 128 MiB of distinct pages and each VM to 128 MiB of references. Indexes, mappings, threads and the allocator add overhead; the pool sits outside per-sandbox cgroups and admission quotas, so reserve host headroom. When upgrading an old compressed pool, drain old VMs first and use a new state directory.

For the experimental compressed mode, add --memory-pool --memory-pool-mode compressed instead. Identical decoded 64 KiB blocks share sealed fill/zstd objects, and unique compressible content can also enter the pool. VM startup requires kernel-fault userfaultfd permission. After pool loss, already-held handles remain usable while new publication stops. The mode binds to persistent configuration; switching requires draining old instances and a new state directory. Process-level sharing and service-exit tests pass, and a two-VM probe verified guest UFFD eviction/refault and file I/O after pool termination. See the sealed compressed-pool contract. These VM/pool tests do not equal full OpenSandbox SDK end-to-end validation.

Cache and pool components can also run independently without the sandbox API. Both support Linux and Apple Silicon macOS. macOS defaults to compression, or explicitly selects --memory-pool-mode physical-pages in fresh state. See memory optimization for connection commands, budgets and pool-loss behavior.

Restart and state ownership¶

After an API-only restart, reconnect to surviving supervisors with the original private state and owner identity. Only one daemon may use a state directory. Sandbox IDs are never reused or restarted. If a connection is lost, retain the record and capacity until you reconcile the state and confirm deletion; the lost connection does not prove the VM is gone. See lifecycle control for identity checks and deletion order, and storage recovery for restart reconciliation.

Pending delete intent survives restarts and is not overridden by native observations. Unknown deletion state retains the reservation. A natively lost sandbox still shows Failed until explicitly deleted. Do not delete state to fix errors, and never share it between two daemons. The registry is bounded by the configured capacity and a 16 MiB total logical v1-equivalent budget; it is neither an infinitely retained distributed-task archive nor a Run Bundle/artifact store.

A version-1 sandboxes.json with existing records but no native owner.json marker is rejected under the existing store's exclusive lock: it may still hold live Podman containers. Use fresh native state and retain/clean the old deployment, or delete all sandboxes through the old Podman daemon, confirm cleanup, and switch backends with the emptied registry. Do not delete registry entries, reservations or ownership state, and do not forge a native marker to bypass the check. The native daemon neither adopts those containers as Missing nor silently releases their reservations.

The current v2 state uses a small sandboxes.json owner/header with an empty map plus private per-sandbox records/. A missing native owner.json still rejects v2 startup; an empty header does not prove no sandbox exists. Ownership and record state must be retained together. Native v1 storage migrates only after the runtime factory accepts the owner; see storage migration.

Deletion/reconciliation can continue tombstone-based cleanup when the original rootfs or firmware is missing, reclaiming empty owned cgroups and private run/RAM/temp/socket/secret records while keeping minimal ID barriers. Reclamation errors retain the reservation for retry; do not delete tombstones or registry state yourself. See storage.

Isolation and trust boundaries¶

Budget resources for the whole supervisor/VMM/helper process tree. Its cgroup enforces total CPU rate 10–8000 millicores, hard memory, zero swap, pids.max=512 and group OOM. vCPU count rounds the quota up (at most 8); guest RAM rounds down to MiB. The hard memory limit includes host-side overhead. Paused, Failed and uncertain records retain their reservations.

See actual controls and physical memory for cgroup entry before startup and supervisor isolation.

Native OverlayNet provides VM egress networking. OpenSandbox network-policy requests are unsupported and rejected; a rejected request does not switch egress to deny-all.

The host account, daemon/firmware and prepared images must be trusted. Private state and same-UID IPC do not stop hostile same-UID or root code. Other local users may reach loopback publications; configure service authentication and host access controls. Secrets stay out of supervisor argv and the host environment but are kept in private records. No security-audit results exist; hostile multi-user isolation is out of scope.

Relationship to native Jobs, the cache and the retired Cluster¶

A daemon sandbox ID is not a Job ID; do not pass it to pvisor checkpoint, fork, apply or drop. A daemon pause is a confirmed vCPU pause on the same live Attempt, not a sealed RAM/CPU/device/filesystem snapshot. Native Job file review, checkpoints and model capture continue through the pvisor spawn path: execution steps are in the VM guide, node-local Job file checkpoints in checkpoints and forks, and model-request routing/capture in the Gateway guide. A compatible standalone-rootfs, network-free VM Job can use execution checkpoints; check capability and blockers with pvisor status --json.

pvisor cache prepare/publish/list/stat/read remain independent commands, not daemon subcommands; see the shared image cache reference for publication, backends, authentication and access. Daemon state is stored separately from native Jobs and cache stores; the Daemon API key authorizes neither the standalone cache nor node-runtime IPC.

The retired Cluster task/client SDK, Controller/Worker registration, placement, DAG, lease renewal, completion outbox and artifact retirement commands are not daemon interfaces. Do not feed old task JSON, worker.toml, Controller credentials or journals to the daemon; there is no automatic path from distributed task history to sandbox state. Retain old results before retiring a deployment, and use fresh state for the OpenSandbox profile.

Troubleshooting¶

OpenSandbox API errors carry {code, message} and responses carry X-Request-ID. Retain the HTTP status, request ID and redacted diagnostics; API success is not the command exit status. See exit codes and errors.