Skip to content

feat(blaze): rebind snapshot host resources - #2477

Draft
WeissonHan wants to merge 47 commits into
alibaba:mainfrom
WeissonHan:feature/blaze/snapshot-resource-rebinding-draft-v1
Draft

feat(blaze): rebind snapshot host resources#2477
WeissonHan wants to merge 47 commits into
alibaba:mainfrom
WeissonHan:feature/blaze/snapshot-resource-rebinding-draft-v1

Conversation

@WeissonHan

Copy link
Copy Markdown
Collaborator

Description

This draft makes portable snapshot state reusable by another sandbox by allocating fresh host resources and rewriting restore inputs instead of reusing stale tap devices, descriptors, sockets, paths, or process-local bindings.

Draft status: the current head is a historical integration candidate. It must be rebuilt after #2475 and #2476 so its production consumers are present while the pull request remains limited to new-sandbox host-resource rebinding. Current historical validation is not evidence for the final rebased commit.

Related Issue

closes #2465

Type of Change

  • Bug fix (non-breaking change that fixes an issue)
  • New feature (non-breaking change that adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to not work as expected)
  • Documentation update
  • Refactoring (no functional change)
  • Performance improvement
  • CI/CD or build changes

Scope

  • blaze (blaze)

Checklist

  • I have read the Contributing Guide
  • My code follows the project's code style
  • I have added tests that prove my fix is effective or that my feature works
  • I have updated the documentation accordingly
  • Lock files are up to date (package-lock.json / Cargo.lock)

Testing

Not run for the final rebased commit. Required evidence includes default and all-feature Linux checks, fresh network and descriptor allocation tests, stale-resource rejection, cancellation and restart compensation, same-sandbox restore compatibility, and real Firecracker restore smoke tests.

Additional Notes

Depends on #2475 and #2476. It does not materialize template storage and does not add the template-backed create request; those remain separate peer and consumer drafts.

This enables explicit destroy and restart cleanup to distinguish a backend
that never started from one whose process may still exist.

Each spawner creates and syncs an empty PID handoff before Starting ownership
is published. During spawn, the child writes and syncs its PID before exec
while an inherited file lock prevents cleanup from treating an active handoff
as absent.

Missing metadata remains an unknown state for compatibility. An unlocked empty
handoff proves that no process crossed the spawn boundary. Bubblewrap and
Firecracker use the same protocol.

Fixes: c33b495 ("feat(blaze): own sandbox runtime resources")
Signed-off-by: Jing Zhang <renyu.zj@linux.alibaba.com>
Signed-off-by: Weisson <Weisson@linux.alibaba.com>
This enables explicit destroy and restart cleanup to distinguish a backend
that never started from one whose process may still exist.

Each spawner creates and syncs an empty PID handoff before Starting ownership
is published. During spawn, the child writes and syncs its PID before exec
while an inherited file lock prevents cleanup from treating an active handoff
as absent.

Missing metadata remains an unknown state for compatibility. An unlocked empty
handoff proves that no process crossed the spawn boundary. Bubblewrap and
Firecracker use the same protocol.

Fixes: c33b495 ("feat(blaze): own sandbox runtime resources")
Signed-off-by: Jing Zhang <renyu.zj@linux.alibaba.com>
Signed-off-by: Weisson <Weisson@linux.alibaba.com>
This enables POST /v1/instances/{id}/reset to reject requests until a
backend can reset both runtime and storage state.

The endpoint now returns 501 without changing lifecycle state, runtime
ownership, or pool membership. Warm-claim tests use a test-only pool
fixture so candidate validation does not rely on a production reset.

A later runtime-reset capability can replace this guard after it resets
both backend and storage state.

Fixes: 1f0cfac
Signed-off-by: Jing Zhang <renyu.zj@linux.alibaba.com>
Signed-off-by: Weisson <Weisson@linux.alibaba.com>
This prevents POST /v1/instances/{id}/checkpoint from reporting a
checkpoint before backend and storage capture are wired.

The handler validates instance existence under the operation lock and
returns HTTP 501 without changing lifecycle state, rewriting state.json,
releasing runtime ownership, or generating a checkpoint ID. The focused
test proves the in-memory and persisted state remain unchanged.

The checkpoint and restore feature will replace this guard after it can
pause the backend, capture runtime and storage data, and compensate
partial failures.

Fixes: 1f0cfac
Signed-off-by: Jing Zhang <renyu.zj@linux.alibaba.com>
Signed-off-by: Weisson <Weisson@linux.alibaba.com>
Firecracker sandboxes previously had no daemon-owned network interface path. A
policy could start a VM, but it could not request an isolated namespace whose
resources followed the backend lifecycle.

This adds an opt-in network manager that allocates deterministic namespace,
veth, tap, address, forwarding, and translation resources. A host-wide lock
serializes allocation across daemon processes, and startup probing checks the
required host commands only when a loaded Firecracker policy enables network.

The backend persists slot ownership before launch and removes resources after
process termination. Restart cleanup uses the PID handoff and stopped marker;
missing launch evidence remains retained instead of guessing that deletion is
safe. Exit cleanup failures keep the observed result and retry on the next
wait.

This enables isolated VM network slots through ordinary sandbox creation. It
remains disabled by default, and upstream routing and DNS stay operator-owned.

Signed-off-by: Jing Zhang <renyu.zj@linux.alibaba.com>
Signed-off-by: Weisson <Weisson@linux.alibaba.com>
This documents the Firecracker policy switch that enables a dedicated network
slot, the host-wide allocation coordination, and the startup checks performed
when a loaded policy requests the capability.

It also records the ownership boundary: namespace, link, and tap cleanup follow
the backend process, while upstream routing and DNS remain host-operator
responsibilities and networking stays disabled by default.

This lets reviewers and operators see what sandbox creation gains and what the
daemon deliberately does not configure.

Signed-off-by: Jing Zhang <renyu.zj@linux.alibaba.com>
Signed-off-by: Weisson <Weisson@linux.alibaba.com>
This enables daemon startup to stop when an owned state directory cannot
be loaded or its record names a different sandbox. Persisted runtime
ownership is never hidden under an invalid or misleading key.

UUID-named directories must contain readable state whose embedded ID matches
the directory name, and ServerState::build propagates any scan failure. A
failed first state publication removes only the empty directory it created;
a directory containing another owned artifact remains in place and fails
closed.

One damaged owned record now prevents API startup until it is repaired.
This avoids serving an incomplete inventory while also preventing an
ordinary first-write failure from leaving an empty directory that blocks
every later startup.

Fixes: 1f0cfac

Signed-off-by: Jing Zhang <renyu.zj@linux.alibaba.com>
Signed-off-by: Weisson <Weisson@linux.alibaba.com>
This enables a long-running daemon to serialize per-instance mutations
without retaining one lock entry for every UUID it has seen.

The registry stores weak references and prunes entries with no holders or
waiters while holding the registry mutex. Concurrent callers for a live
UUID still share one async lock.

Completed instances therefore leave no permanent strong reference in the
registry.

Fixes: c33b495
Signed-off-by: Jing Zhang <renyu.zj@linux.alibaba.com>
Signed-off-by: Weisson <Weisson@linux.alibaba.com>
This keeps accepted HTTP connections under daemon ownership so shutdown no
longer returns while request handlers are still using runtime resources.

The accept loop tracks and reaps connection tasks. On termination it closes
listeners, asks HTTP/1 connections to finish, waits for a bounded drain
window, and aborts and joins any remaining tasks.

This enables shutdown to establish a deterministic boundary before process
exit while still preventing an idle or stuck client from blocking the daemon
indefinitely.

Signed-off-by: Jing Zhang <renyu.zj@linux.alibaba.com>
Signed-off-by: Weisson <Weisson@linux.alibaba.com>
The daemon previously stopped after connection handling ended while tracked
backend processes and provider slots could remain owned by the exiting
process.

After connection tasks drain, shutdown starts the existing destroy path for
every non-terminal sandbox and retained backend owner. Cleanup attempts run
concurrently under one shared work deadline, so a stalled owner cannot
multiply stop latency or prevent independent owners from being released.

Both shutdown stages cancel and join unfinished tasks before returning. The
service manager provides the final stop deadline with cancellation headroom,
so no cleanup task continues to access daemon state after an in-process stage
finishes.

This enables SIGTERM and SIGINT shutdown to release tracked runtime resources
while keeping incomplete ownership recoverable and reporting observed cleanup
failures.

Signed-off-by: Jing Zhang <renyu.zj@linux.alibaba.com>
Signed-off-by: Weisson <Weisson@linux.alibaba.com>
All daemon routes previously buffered request bodies to completion. Add a
validated, configurable 1 MiB default and a shared collector that rejects
oversized declared lengths before polling and stops streamed bodies at the
first frame above the limit.

This enables every existing daemon endpoint to reject oversized input before
buffering the complete body while preserving requests at or below the
configured limit. Body read failures map to HTTP 400, and size violations map
to HTTP 413.

Use one global boundary so new routes inherit it automatically. Per-route
payload limits remain outside this change.

Signed-off-by: Jing Zhang <renyu.zj@linux.alibaba.com>
Signed-off-by: Weisson <Weisson@linux.alibaba.com>
This enables blazed to reject a competing daemon before runtime
initialization or Unix socket replacement. Daemon startup now holds an
exclusive lock derived from the configured API socket for the entire run.

The lock file stays on disk so processes always contend on one inode. It is
opened without following symbolic links and must be a single-link regular
file owned by the effective user with mode 0600. Unexpected lock or endpoint
state aborts startup without deleting the configured socket.

An existing socket receives a bounded ownership probe. A reachable endpoint
is preserved, including one created by an older daemon without the lock; only
an endpoint that reports no listener is treated as stale and replaced.

The guarantee is scoped to one configured API socket. Independent daemons
using different sockets do not contend on this lock.

Signed-off-by: Jing Zhang <renyu.zj@linux.alibaba.com>
Signed-off-by: Weisson <Weisson@linux.alibaba.com>
Prepare a new lifecycle record in a non-owner staging directory and sync
its file and directory before publishing the UUID owner directory.

The previous sequence exposed the final UUID directory before state.json
existed and did not sync the state root. A crash could therefore leave an
owner that startup could neither load nor classify.

Rename the complete staging directory into place and sync the state root.
Existing owner updates keep their atomic file replacement and now sync both
the file and owner directory. This lets startup observe either no owner or a
complete lifecycle record.

Signed-off-by: Jing Zhang <renyu.zj@linux.alibaba.com>
Signed-off-by: Weisson <Weisson@linux.alibaba.com>
Reconstruct provider-owned slots only when the slot root and every required
artifact are direct filesystem objects.

The previous metadata checks followed links, so a restored sandbox could
silently read or write files outside the provider-owned slot after local state
changed.

Use link-aware metadata and classify linked roots or artifacts as incomplete.
This keeps restart and dirty flush bound to the provider's own directory tree.

Signed-off-by: Jing Zhang <renyu.zj@linux.alibaba.com>
Signed-off-by: Weisson <Weisson@linux.alibaba.com>
Sandbox metadata previously recorded only lifecycle state. A crash between
provider allocation, backend start, and final publication could not distinguish
completed work from an operation that might still own resources.

Add a durable create or destroy journal and the RecoveryRequired state. State
publication now syncs the temporary file, atomically renames it, and syncs the
owner directory. A failed first publication removes only the empty directory it
created, and a new operation cannot overwrite an unfinished journal.

This enables later lifecycle management to discover interrupted work after a
restart without mistaking it for a clean sandbox. Older records remain readable
through conservative ownership defaults.

This commit defines and persists the model only; it does not start or release
runtime resources.

Signed-off-by: Jing Zhang <renyu.zj@linux.alibaba.com>
Signed-off-by: Weisson <Weisson@linux.alibaba.com>
Instance handlers previously coordinated metadata, provider slots, backend
processes, and warm claims themselves. Startup and shutdown had no single
owner that could enumerate incomplete work and retry every retained resource.

Add SandboxManager as the owner of create, warm activation, destroy,
startup reconciliation, and graceful shutdown cleanup. Every mutation uses
one per-sandbox operation lock, persists intent before changing resources,
and retains incomplete ownership as RecoveryRequired.

This enables Blaze to create, list, inspect, and destroy sandboxes while
keeping partial failures recoverable. Startup reconciliation bounds each
record and continues after a failure. Graceful shutdown starts every owned
sandbox concurrently under one shared deadline, cancels and joins tasks that
overrun it, and reports all remaining owners.

Reserved checkpoint and reset actions use the same quiescent-state gate.
They still return HTTP 501 without changing runtime or persisted state until
their backend and storage operations are implemented.

This commit does not add sandbox-named routes, guest commands, VM
networking, checkpoint capture, hibernation, or background retries.

Signed-off-by: Jing Zhang <renyu.zj@linux.alibaba.com>
Signed-off-by: Weisson <Weisson@linux.alibaba.com>
Lifecycle management was available only through the instance route family, and
destroy required the older POST action form. Callers could not treat a sandbox
as a conventional collection and item resource.

Add sandbox collection and item aliases plus DELETE forms for sandbox and
instance resources. Every route delegates to the same managed create, list,
inspect, or destroy handler.

This enables callers to create, enumerate, inspect, and delete sandboxes through
`/v1/sandboxes` without introducing a second lifecycle implementation. Existing
instance routes, checkpoint and reset actions, and response formats remain
available.

Signed-off-by: Jing Zhang <renyu.zj@linux.alibaba.com>
Signed-off-by: Weisson <Weisson@linux.alibaba.com>
The API reference listed only instance-named routes and did not explain what
happens when create, destroy, startup recovery, or daemon shutdown stops partway
through resource cleanup.

Document sandbox and compatible instance routes, the operation journal,
RecoveryRequired state, per-record startup reconciliation, and ordered graceful
shutdown cleanup. The English and Chinese examples now use the sandbox
collection while retaining all compatible route forms.

This enables operators and reviewers to distinguish implemented lifecycle
behavior from reserved interfaces. Checkpoint and reset are documented as
validation-only routes that return `501` without changing runtime or persisted
state.

The documented limits remain explicit: interrupted create is cleaned up rather
than resumed, existing backend processes are not adopted after restart, and
failed recovery has no background retry loop.

Signed-off-by: Jing Zhang <renyu.zj@linux.alibaba.com>
Signed-off-by: Weisson <Weisson@linux.alibaba.com>
Managed sandboxes had no shared request and response format for communicating
with a compatible guest agent.

Define ping, exec, read, and write operations with request IDs, command inputs,
file data, exit status, and standard-base64 outputs. Response IDs and outcome
flags are required so a missing field cannot be mistaken for a confirmed
rejection.

This gives later transport and API commits one bounded wire contract to
validate. It does not open a guest connection or expose a daemon route.

Signed-off-by: Jing Zhang <renyu.zj@linux.alibaba.com>
Signed-off-by: Weisson <Weisson@linux.alibaba.com>
Managed sandboxes had no production path from the daemon API to a compatible
guest endpoint, so callers could not execute commands or transfer files.

Add a JSON-line client for the Firecracker vsock proxy with bounded inputs,
deadlines, response frames, decoded outputs, and readiness polling. Exec and
write failures become outcome-unknown only after delivery begins; pre-delivery
failures and reads remain safe to retry.

Let backend owners report an optional guest endpoint. SandboxManager holds the
existing per-sandbox operation lock while it validates Running state and
performs guest work. Cold create waits for readiness only when the selected
owner reports an endpoint, and cancelled readiness runs the normal create
compensation path.

Expose exec, read, and write actions through both sandbox and compatible
instance routes. Handlers enforce command, HTTP body, base64, decoded file, and
response limits before delegating to the manager, and return stable error codes
for known failures and unknown outcomes.

This enables API callers to use bounded guest operations without bypassing
runtime ownership. Guestless backends keep their create behavior and reject
guest work. Warm activation does not repeat readiness, and the default 1 MiB
HTTP limit must be raised to use the full 16 MiB decoded file limit.

Signed-off-by: Jing Zhang <renyu.zj@linux.alibaba.com>
Signed-off-by: Weisson <Weisson@linux.alibaba.com>
The daemon documentation did not explain how callers use the new guest
operations or which runtime and size conditions apply.

Document the exec, read, and write routes in English and Chinese. Explain cold
readiness, lifecycle serialization, warm-pool behavior, retry boundaries,
stable error codes, per-request buffering, and the relationship between the
default 1 MiB HTTP limit and the 16 MiB decoded file limit.

Update Firecracker policy comments to state when guest transport is enabled,
and make the quick-start example use a policy that does not require a guest
agent.

This gives operators enough information to configure and call the feature
without assuming unsupported streaming, session reuse, or automatic retries.
It does not change runtime behavior.

Signed-off-by: Jing Zhang <renyu.zj@linux.alibaba.com>
Signed-off-by: Weisson <Weisson@linux.alibaba.com>
Runtime cleanup after restart needs a complete view of provider-owned slots,
including directories left before all artifacts were created. The storage
contract previously exposed release by ID but did not declare whether a
provider could enumerate every owned ID.

Add an explicit capability for complete inventory under the currently
configured provider root. The file provider returns validated IDs in stable
order and refuses entries that are not direct slot directories.

Keep release-by-ID idempotent for absent or partial directories and reject
non-directory aliases. This gives later runtime cleanup a provider-level
foundation without recording or selecting a provider or root.

Signed-off-by: Jing Zhang <renyu.zj@linux.alibaba.com>
Signed-off-by: Weisson <Weisson@linux.alibaba.com>
POST /v1/sandboxes can now claim a compatible runtime slot after the first
eligible create configures bounded background construction. Each slot owns
independent storage; prefork mode also starts and checks the selected backend
before the slot becomes ready.

Persist pool ownership before allocation and transfer it to lifecycle state
with a token so create, destroy, startup reconciliation, and daemon shutdown
agree on one owner. Retain incomplete builds, handoffs, and cleanup work for
retry instead of returning them to the ready queue.

The target bounds pool-owned and in-flight work, not sandboxes after handoff.
The first accepted build shape remains fixed for this daemon run; incompatible
requests use the normal create path. Restart cleans unclaimed slots instead of
restoring the in-memory ready queue.

Signed-off-by: Jing Zhang <renyu.zj@linux.alibaba.com>
Signed-off-by: Weisson <Weisson@linux.alibaba.com>
Operators need to configure background runtime capacity without confusing it
with the lifecycle pool contract or storage health status.

Add paired user guides and README summaries for eligibility, asynchronous
construction, prefork behavior, TTL, capacity accounting, restart, and
shutdown. Correct example policy comments so reserved sizing and reset fields
are not presented as active runtime-slot controls.

Document the ownership handoff and failure decisions with request, state, and
startup flows. This enables operators and reviewers to distinguish implemented
runtime-slot behavior from unconnected lifecycle return and management paths.

Signed-off-by: Jing Zhang <renyu.zj@linux.alibaba.com>
Signed-off-by: Weisson <Weisson@linux.alibaba.com>
The daemon had no stable record for checkpoint identity, parentage, runtime
provenance, or artifact integrity. Backend snapshot requests also had no shared
full-snapshot vocabulary.

Add canonical checkpoint IDs, manifests, API views, artifact-set validation,
and full snapshot request/result DTOs. This enables capture code in this PR to
reject malformed catalog data before publication or listing; it does not
expose a runtime operation by itself.

Signed-off-by: Jing Zhang <renyu.zj@linux.alibaba.com>
Signed-off-by: Weisson <Weisson@linux.alibaba.com>
Lifecycle journals previously recorded only an operation kind and start time.
After an interrupted capture, persisted state could not distinguish an
unpublished stage from a committed checkpoint or a changed HEAD.

Record checkpoint identity, monotonic durable phases, and the last completed
capture, and allow a checkpointed sandbox to return to running. This exposes
the latest boundary confirmed in persisted state while catalog listing reports
what is actually visible; it does not add a provider or API entry point by
itself.

Signed-off-by: Jing Zhang <renyu.zj@linux.alibaba.com>
Signed-off-by: Weisson <Weisson@linux.alibaba.com>
Checkpoint requests previously stopped after lifecycle validation with 501.
The daemon had no provider contract, durable catalog, or list operation for
captured state.

Add opt-in backend and storage capture contracts, self-contained file-provider
copies, a mock backend implementation, verified catalog publication with HEAD
tracking, and serialized capture/list API paths. Compensation resumes the
runtime and removes unpublished work, while uncertain publication or resume
outcomes retain ownership for recovery.

This enables callers to capture and list independent checkpoints on supporting
backends. Unsupported backends or storage providers fail before journal or
catalog mutation. The contracts, first implementations, catalog coordinator,
and API stay together because they form one production call path and none is
independently usable here.

Signed-off-by: Jing Zhang <renyu.zj@linux.alibaba.com>
Signed-off-by: Weisson <Weisson@linux.alibaba.com>
This enables operators to remove committed checkpoint branches that are not
retained by HEAD or durable sandbox state through the checkpoint prune API.

The catalog atomically renames each candidate to a tombstone before deletion.
Retries, destroy, and startup reconciliation finish tombstones, trading
immediate all-or-nothing deletion for durable incremental cleanup. The
per-sandbox operation lock and unfinished-journal gate keep pruning from racing
lifecycle operations.

Checkpoint capture and backend-specific snapshot support are unchanged;
checkpoint restore remains unavailable.

Signed-off-by: Jing Zhang <renyu.zj@linux.alibaba.com>
Signed-off-by: Weisson <Weisson@linux.alibaba.com>
This enables Firecracker-backed sandboxes to use the existing checkpoint API
by pausing the VM, writing full VM and memory snapshots through the Firecracker
UDS API, and resuming the owned runtime.

The spawner resolves the configured binary version on every start and freezes
it into the runtime owner, so concurrent probes and binary replacement cannot
reuse stale global provenance. Firecracker records require that version before
capture begins, and the UDS client bounds response size and request duration.

Only full capture is enabled; restore and incremental snapshots remain
unavailable.

Signed-off-by: Jing Zhang <renyu.zj@linux.alibaba.com>
Signed-off-by: Weisson <Weisson@linux.alibaba.com>
This provides blaze-core with a durable restore operation and monotonic
ownership phases for replacing a running sandbox from a selected checkpoint.
A later sandbox-manager commit consumes these records; this commit does not
expose restore through a production entry point.

Previously the lifecycle model could journal capture progress but could not
distinguish a staged rootfs, a stopped backend, an activated replacement, or
a committed storage transaction during restore. Treating every interruption
as the same state would hide whether compensation is still possible.

The Restoring state is entered only after the old backend is stopped and may
return to Running only after storage commit. The journal keeps the selected
target separate from last_checkpoint. That field records only the last
completed capture; restore never changes it, while HEAD records the selection.

Storage, backend, manager, and API implementations remain out of scope.

Signed-off-by: Jing Zhang <renyu.zj@linux.alibaba.com>
Signed-off-by: Weisson <Weisson@linux.alibaba.com>
This enables FileStorageProvider to prepare a checkpoint rootfs through opt-in
StorageProvider restore transactions and later select that staged copy.

Staging copies the checkpoint without changing the live rootfs. Activation uses
same-directory renames to retain the original inode. Durable states let commit,
abort, or restart reconciliation finish each interrupted boundary. This trades
temporary disk capacity for deterministic rollback.

The provider contract is fail-closed by default. Lifecycle orchestration,
backend restore, daemon routes, and user-facing documentation remain unwired.

Signed-off-by: Jing Zhang <renyu.zj@linux.alibaba.com>
Signed-off-by: Weisson <Weisson@linux.alibaba.com>
Blaze could capture and list full checkpoints, but the daemon had no route
that selected one as the live runtime state. Callers could inspect history
but could not replace a running sandbox from that history.

Add opt-in storage and backend restore contracts, Firecracker and mock
adapters, a verified checkpoint target, and the manager-owned rollback API.
This enables callers to replace a running sandbox from a selected checkpoint.

The file provider stages an independent rootfs while the old backend runs,
retains the predecessor through activation, and commits only after the new
owner is ready and HEAD has changed. Firecracker requires the exact captured
binary version before starting a replacement process.

Failures before backend shutdown restore the original running state. Later
failures retain the resources that actually exist under RecoveryRequired,
including partial owners and uncertain HEAD outcomes, so destroy can finish
cleanup without losing ownership.

The contracts, adapters, manager, API, tests, and documentation stay together
because they form the first production restore path; none is independently
usable without the others.

Signed-off-by: Jing Zhang <renyu.zj@linux.alibaba.com>
Signed-off-by: Weisson <Weisson@linux.alibaba.com>
This enables API users to stop a running sandbox after publishing a verified
hibernation image, then resume it after daemon restart without losing
guest-visible state.

The manager checks capture and restore compatibility before journaling, hashes
artifacts before publication, retains the storage slot while stopped, and owns
a restored backend before readiness checks. Durable phases distinguish
retryable failures from ownership that requires explicit cleanup.

Tests cover API aliases, restart continuity, unsupported adapters, corrupt
artifacts, and failures around snapshot, backend stop, publication, restore,
readiness, and cleanup.

Signed-off-by: Jing Zhang <renyu.zj@linux.alibaba.com>
Signed-off-by: Weisson <Weisson@linux.alibaba.com>
This explains when hibernation is available, which API routes release and
resume a sandbox backend, and which durable checks users can expect before
either state change becomes visible.

The English and Chinese guides describe artifact verification, restart
behavior, failure ownership, explicit cleanup, and the disk-space tradeoff of
retaining the latest hibernation image.

Signed-off-by: Jing Zhang <renyu.zj@linux.alibaba.com>
Signed-off-by: Weisson <Weisson@linux.alibaba.com>
This enables operators to schedule bounded provider synchronization for
running sandboxes while leaving the default schedule disabled.

Each sweep enters the sandbox operation lock, rechecks lifecycle and journal
state, confirms backend and storage ownership, and isolates one provider
failure from later sandboxes. Attempt deadlines retain the slot for retry.

The daemon supervises the worker, stops and joins it before connection drain
and runtime cleanup, and preserves both worker and shutdown failures.

Signed-off-by: Jing Zhang <renyu.zj@linux.alibaba.com>
Signed-off-by: Weisson <Weisson@linux.alibaba.com>
This explains how operators opt into periodic provider synchronization, which
running sandboxes are eligible, and how one failed or timed-out slot remains
owned and retryable without blocking the rest of a sweep.

The English and Chinese guides also document missed-tick behavior, the
provider cancellation contract, and the worker, connection, and runtime
shutdown order.

Signed-off-by: Jing Zhang <renyu.zj@linux.alibaba.com>
Signed-off-by: Weisson <Weisson@linux.alibaba.com>
This enables operators to import, list, and inspect durable runtime artifact
sets through /v1/runtime-templates.

Imports are constrained to a configured source root, copied into private
staging directories, and published atomically. Capacity reservations, source
revalidation, and shutdown coordination prevent incomplete data from becoming
visible. If durability or cleanup becomes uncertain, later imports stop until
the catalog is repaired and the daemon restarts.

Sandbox creation does not consume catalog entries in this commit.

Signed-off-by: Jing Zhang <renyu.zj@linux.alibaba.com>
Signed-off-by: Weisson <Weisson@linux.alibaba.com>
This explains how operators configure and use the runtime artifact catalog
through its import, list, and lookup endpoints.

The guide documents source requirements, capacity limits, private output
permissions, atomic publication, startup checks, and shutdown behavior. It also
distinguishes this durable catalog from the existing in-memory template
registry.

Sandbox creation, reference counting, and catalog deletion remain out of scope.

Signed-off-by: Jing Zhang <renyu.zj@linux.alibaba.com>
Signed-off-by: Weisson <Weisson@linux.alibaba.com>
…aze/p3-final-local-v2

# Conflicts:
#	src/blaze/crates/blazed/src/api.rs
#	src/blaze/crates/blazed/src/spawner/firecracker.rs
…n/blaze/p3-final-local-v2

# Conflicts:
#	src/blaze/crates/blaze-core/src/policy.rs
#	src/blaze/crates/blazed/src/spawner/firecracker.rs
#	src/blaze/examples/policies/agent-rl.toml
#	src/blaze/examples/policies/agent-tool.toml
…blaze/p3-final-local-v2

# Conflicts:
#	src/blaze/README.md
#	src/blaze/README_zh.md
#	src/blaze/crates/blaze-core/src/lifecycle.rs
#	src/blaze/crates/blazed/src/api.rs
#	src/blaze/crates/blazed/src/sandbox/manager.rs
#	src/blaze/crates/blazed/src/spawner/firecracker.rs
…ze/p3-final-local-v2

# Conflicts:
#	src/blaze/AGENTS.md
#	src/blaze/README.md
#	src/blaze/README_zh.md
#	src/blaze/crates/blazed/src/api.rs
#	src/blaze/crates/blazed/src/sandbox/manager.rs
…ion/blaze/p3-final-local-v2

# Conflicts:
#	src/blaze/README.md
#	src/blaze/README_zh.md
#	src/blaze/crates/blaze-core/src/config.rs
#	src/blaze/crates/blazed/src/daemon.rs
#	src/blaze/crates/blazed/src/sandbox.rs
#	src/blaze/crates/blazed/src/sandbox/manager.rs
…aze/p3-final-local-v2

# Conflicts:
#	src/blaze/README.md
#	src/blaze/README_zh.md
#	src/blaze/crates/blaze-core/src/config.rs
#	src/blaze/crates/blazed/src/daemon.rs
#	src/blaze/crates/blazed/src/sandbox.rs
#	src/blaze/crates/blazed/src/sandbox/manager.rs
#	src/blaze/crates/blazed/src/state.rs
This enables checkpoint rollback and hibernation resume to recreate the
host-side device names embedded in a VM snapshot before loading it.

Checkpoint and hibernation manifests now record guest transport and the exact
network slot. The Firecracker adapter reserves that slot and fails without
selecting a different one when the name is unavailable. Either restore path
therefore keeps its recoverable failure state instead of loading a snapshot
against mismatched devices.

This restore invariant spans the shared adapter and both production consumers.
It does not change ordinary cold or warm sandbox network allocation.

Signed-off-by: Jing Zhang <renyu.zj@linux.alibaba.com>
Signed-off-by: Weisson <Weisson@linux.alibaba.com>
Snapshots previously retained sandbox-specific rootfs, runtime, network, and guest transport paths. That made loading valid artifacts into a different owned sandbox unreliable.

Run Firecracker behind a private mount view with stable rootfs and runtime paths, then override the network and guest transport resources during snapshot load. Same-sandbox checkpoint and hibernate restores keep their fixed network allocation, while new-sandbox restores request a fresh allocation. Existing ownership paths retain cleanup responsibility when startup fails.

This enables a full snapshot to restore into another sandbox without reusing the source sandbox's host resources.

Signed-off-by: Jing Zhang <renyu.zj@linux.alibaba.com>

Signed-off-by: Weisson Han <wenshu.hx@linux.alibaba.com>
@github-actions github-actions Bot added component:blaze src/blaze scope:documentation ./docs/|./*.md|./NOTICE labels Aug 13, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

component:blaze src/blaze scope:documentation ./docs/|./*.md|./NOTICE

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[blaze] feat: rebind snapshot host resources

1 participant