feat(blaze): capture sandbox checkpoints - #2472
Conversation
cc02562 to
c7aa317
Compare
|
@codex review Please review the generic full-checkpoint capture and listing workflow, with particular attention to durable publication boundaries, cancellation and restart recovery, and retained filesystem ownership. |
c7aa317 to
56bc391
Compare
|
@codex review Please review the updated exact head. The only change since the previous request is a test-only feature gate required by the default-feature Rust 1.88 Clippy job; please focus on the generic checkpoint capture/list workflow and its durable publication, cancellation, restart, and retained-ownership behavior. |
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 56bc391ed7
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
56bc391 to
995bab7
Compare
|
@codex review Please review exact head |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 995bab7ebb
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
995bab7 to
93cbdb9
Compare
|
Addressed the exact-head Codex permission finding in Checkpoint catalog, sandbox, staging, and committed directories are now created with owner-only Focused coverage verifies the complete tree under a permissive umask, rejects external hard links without permission side effects, and preserves cleanup of an interrupted internal rootfs link. The two storage artifact synchronization documents remain unchanged. Exact-head hosted and native Linux validation are running now. |
|
@codex review Verify that checkpoint directories and published artifacts remain private under the packaged service defaults, including hard-link failure behavior. |
|
Codex Review: Didn't find any major issues. Keep it up! Reviewed commit: ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
If Codex has suggestions, it will comment; otherwise it will react with 👍. Codex can also answer questions or update the PR. Try commenting "@codex address that feedback". |
![qoderai[bot]](https://avatars.githubusercontent.com/in/1396158?s=60&v=4){kind=small}
There was a problem hiding this comment.
[P1] 沙箱检查点生命周期和 journal 设计整体合理，当前审查重点是并发保护与补偿路径是否始终与持久化边界一致。下面三点是本次审查中相对更重要的细节建议，均已通过行内评论标注。
- 检查点操作在并发下对
operationjournal 的保护需要严格约束所有调用路径使用同一锁，并在 API 层明确拒绝重复 checkpoint 请求。 - 失败补偿与恢复路径目前主要根据状态和 staging id 区分，建议进一步利用
OperationPhase的 durable 边界信息，以保证 journal 与目录真实状态持续同步。 - 文件存储检查点目标目录在 Linux 与其他平台的 retained adapter 语义存在差异，建议在能力探测和错误处理上做显式约束，并在设计文档中标明跨平台行为差异。
🤖 Generated by Qoder • View workflow run
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 93cbdb994d
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
93cbdb9 to
39b9446
Compare
|
@codex review Add generic full-checkpoint capture and listing with durable publication boundaries and retained staging ownership. |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 39b9446d7b
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
39b9446 to
dcc2284
Compare
|
@codex review Add generic full-checkpoint capture and listing while preserving sparse storage and bounded capture-time lineage validation. |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: dcc2284322
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
|
Codex Review: Didn't find any major issues. 👍 Reviewed commit: ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
If Codex has suggestions, it will comment; otherwise it will react with 👍. Codex can also answer questions or update the PR. Try commenting "@codex address that feedback". |
c951f78 to
caa05a1
Compare
|
@codex review Please review exact head |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: caa05a1a07
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
c115f2c to
6f2234d
Compare
|
@codex review Please review exact head |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 6f2234d998
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
5264a1d to
922af12
Compare
|
@codex review Please review exact head |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 922af12f97
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
b2e403a to
d3295ee
Compare
|
@codex review Please review exact head |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: d3295eee7a
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
f16f667 to
80db550
Compare
|
@codex review Please review exact head |
|
Codex Review: Didn't find any major issues. Nice work! Reviewed commit: ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
If Codex has suggestions, it will comment; otherwise it will react with 👍. Codex can also answer questions or update the PR. Try commenting "@codex address that feedback". |
|
@codex review |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 80db550822
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| let staging_name = format!(".{id}{STAGING_SUFFIX}"); | ||
| match create_child_directory(&sandbox, &staging_name) { | ||
| Ok(directory) => { | ||
| sync_directory(&sandbox)?; |
There was a problem hiding this comment.
When sync_directory(&sandbox) fails after create_child_directory succeeds—for example, due to a transient filesystem fsync error—the ? returns without aborting the newly created .ckpt-*.tmp directory or recording an operation journal. Each retry therefore creates another orphaned stage, and nothing removes these entries until the entire sandbox is destroyed; clean up the owned stage on this error or retain a recovery record rather than silently leaking catalog entries.
AGENTS.md reference: src/blaze/AGENTS.md:L32-L36
Useful? React with 👍 / 👎.
This commit exposes checkpoint creation and history queries under the sandbox API. It captures backend state, guest memory, and the provider-owned writable root in a self-contained, integrity-checked catalog. Each durable boundary is journaled so cancellation and restart retain ownership until publication, HEAD movement, state persistence, and backend resume have converged. Unsupported providers fail before mutation. Failed stage synchronization removes the owned scratch directory before returning. Restore, checkpoint deletion, pruning, and Firecracker capture remain outside this change. Signed-off-by: Jing Zhang <renyu.zj@linux.alibaba.com> Signed-off-by: Weisson <Weisson@linux.alibaba.com>
80db550 to
345dc42
Compare
1 participant
Why
Blaze can persist sandbox lifecycle state, but it does not provide an API that
captures a complete checkpoint and exposes it only after every required
artifact is durable. A failed or cancelled capture must also retain truthful
ownership information so restart recovery does not report success or delete an
uncertain result.
What changed
POST /v1/sandboxes/{id}/checkpointto capture a running sandbox.GET /v1/sandboxes/{id}/checkpointsto list committed history and identifythe current checkpoint head.
in a self-contained, integrity-checked checkpoint.
and backend resume so interrupted work can be reconciled after restart.
provider; unsupported combinations return
501before sandbox mutation.publish with no-replace renames, and clean an owned staging directory when
its parent synchronization fails.
remains recoverable when cleanup or its durability cannot be proven.
and restart behavior in English and Chinese.
Checkpoint restore, checkpoint deletion or pruning, and capture support for
Firecracker, Bubblewrap, and process backends are outside this change.
Related issue
Closes #2444
User / Agent impact
Users and agents can create and inspect full checkpoints through the two sandbox
endpoints above. Capture currently succeeds with the built-in mock backend and
file storage provider. Other built-in backends fail with
501before pausing orchanging the sandbox until their capture adapters are implemented.
Successful capture pauses the backend, writes and verifies all artifacts,
publishes the manifest and checkpoint head, resumes the backend, and returns the
committed checkpoint record. Listing returns validated committed records with
parentage, logical size, head status, and head reachability.
Risk and compatibility
This adds two HTTP endpoints and a daemon-owned checkpoint namespace below the
configured state directory. Existing backend and storage implementations remain
fail-closed because checkpoint capability defaults to unsupported. New
lifecycle journal and last-checkpoint fields have deserialization defaults for
records written by earlier versions.
Filesystem risk is limited by retained directory descriptors, owner-only modes,
no-replace publication, identity revalidation, and explicit parent-directory
synchronization. Uncertain publication or cleanup retains recovery ownership
instead of deleting by pathname.
Validation
Exact revisions:
345dc429bee66b42e0611dabd8b332dc86d4da7e0fa38b3b283efbf37ebb311fa67a8d8a08e34581385a098b136b8fb3e2ddc414ae83e71258ab50bd9636c4bd660aafb27eff396b9e8beee18665c39d385a098b136b8fb3e2ddc414ae83e71258ab50bd,345dc429bee66b42e0611dabd8b332dc86d4da7e0fa38b3b283efbf37ebb311fa67a8d8a08e34581The head and GitHub merge candidate have the same tree. Validation used a fresh
Linux x86_64 source tree, Rust and Cargo 1.88.0, locked offline dependencies, a
fresh Cargo home, and a separate empty target directory for every major stage.
Passed:
-D warningsand no unused production checkpoint entrypoints, with default and all features;
storage, backend, destruction, and durability-boundary regression tests. Each
named test was required to exist and pass exactly once.
Evidence archive SHA-256:
088145de7e0f5504c7d6814de78e2564cd39e74d5cd781cebf25ee9322e5df6f.The exact-head Blaze, documentation, pull-request metadata, commit-message,
website-build, and CLA checks all passed.
Documentation and rollback
The English and Chinese Blaze README, runtime user guide, lifecycle consistency
design, and storage synchronization design now describe checkpoint capture,
listing, persistence, cleanup, and recovery.
To roll back, stop the Blaze daemon and verify that no sandbox has an unfinished
checkpoint operation or
RecoveryRequiredstate before reverting this commit.Checkpoint directories created by this version are not consumed by an older
binary; preserve or archive them before starting that binary. Do not remove
checkpoint state while the daemon is running.