Route high-churn local artifacts to Developer-Artifacts

[shiny-code-bot]

Goal

Route high-churn local build targets, caches, temp output, and runner scratch for Codex Lab development to /Volumes/Developer-Artifacts, without moving existing artifacts today.

This plan makes local development, spawned worktrees, and Every Code agent sessions use predictable artifact locations while preserving safe fallback behavior for contributors who do not have the volume mounted.

Related context: #22 covers CI/local speed observations. This issue owned local high-churn artifact routing.

Non-Goals

• No move to /Volumes/Docker-External.
• No migration of existing artifact trees.
• No requirement that ordinary contributors have /Volumes/Developer-Artifacts mounted.
• No committed machine-specific absolute paths in shared repo config.
• No tracked .npmrc or global package-manager config mutation.

Completed Policy

Use /Volumes/Developer-Artifacts/local/codex-lab/ as the local root for rebuildable high-churn data owned by this repo when CODEX_LAB_DEVELOPER_ARTIFACTS_ROOT is configured and writable.

Shared or reusable cache candidates:

• Cargo shared local target for human/main checkout workflows by default.
• Bazel disk, repo contents, and repository caches via generated local user.bazelrc.
• pnpm store, npm cache, temp output, and optional sccache through opt-in sourceable local env.

Isolated mutable output candidates:

• Worktree/agent Cargo targets via CODEX_LAB_CARGO_TARGET_SCOPE=worktree|agent and optional CODEX_LAB_CARGO_TARGET_KEY.
• External agent process Cargo targets keyed by agent thread id.
• Exec harness Cargo target and output/report roots.

Safety posture:

• Missing artifact volume remains a visible fallback, not a hard requirement for normal contributors.
• Cleanup remains dry-run by default.
• Cleanup only deletes known rebuildable paths and uses physical path bounds for custom exec-harness output and artifact temp paths.

Merged Implementation

• Route local Cargo recipes through artifact root #200 merged as 8769813b8f03a0a887c845c0810c6ca4355b6205.

  • Bootstrapped selected Unix Cargo-heavy just recipes through scripts/local/cargo-build-env.sh.
  • Preserved separate exec-harness target routing.
  • Kept app-server-test-client cross-platform with Unix/Windows recipe bodies.
  • Expanded just local-speed-status to show Cargo, worktree, exec-harness, Bazel, Node, optional sccache, temp, runner, and remote compile host status.
  • Made remote exec server startup respect CARGO_TARGET_DIR when locating the built codex binary.

• Add scoped local cargo targets for worktrees #201 merged as 2673316bdf889c2e5b534941154fcdd5e05a70d1.

  • Added CODEX_LAB_CARGO_TARGET_SCOPE=shared|worktree|agent and optional CODEX_LAB_CARGO_TARGET_KEY to scripts/local/cargo-build-env.sh.
  • Preserved shared repo+host default for humans.
  • Added artifact-volume worktree/agent target selection for spawned sessions.
  • Hardened explicit target keys so path-significant keys such as .. cannot escape the worktrees namespace.
  • Reported active target scope and worktree-scoped target candidates in status output.

• Route external agent cargo targets by thread #202 merged as 54e587778e86664eff17a05539667d2fb075a47b.

  • Set CODEX_LAB_CARGO_TARGET_SCOPE=agent and CODEX_LAB_CARGO_TARGET_KEY=<thread_id> for external agent child processes.
  • Removed inherited CARGO_TARGET_DIR / CODEX_LAB_CARGO_TARGET_DIR from external agent child env so scope/key routing cannot be shadowed.
  • Added unit and process-level coverage for external agent artifact env injection.

• Route exec harness output to local artifacts #203 merged as 935873a735906763f560123e133565e77f5c1d13.

  • Exported exec-harness output/report paths from scripts/local/exec-harness-env.sh.
  • Updated just exec-harness-test to consume those exports in one shell invocation.
  • Added scripts/local/setup-artifacts.sh to dry-run or generate ignored local user.bazelrc routing.
  • Taught status and cleanup about configured exec-harness output paths.
  • Added canonical cleanup bounds and regression tests for custom/traversal output roots.

• Add opt-in local artifact environment #204 merged as f980ff9429f19e6d78799aad56f73825a4795fdb.

  • Added sourceable, opt-in scripts/local/artifact-env.sh for pnpm store, npm cache, temp output, and optional sccache routing under the artifact root.
  • Added just local-artifact-env.
  • Updated setup-artifacts.sh dry-run/apply output to advertise/provision the opt-in local env paths.
  • Reported artifact candidate pnpm/npm/temp/sccache paths in just local-speed-status without mutating active env.
  • Let cleanup remove artifact temp output with canonical physical bounds and --keep-artifact-temp.
  • Added CI-visible tests for local artifact cleanup scripts.

Validation Evidence

• Add scoped local cargo targets for worktrees #201 PR checks: ci 1m01s, exec-harness 43s, Codespell, cargo-deny. Post-merge main: ci 1m00s, cargo-deny 1m35s, Codespell.
• Route external agent cargo targets by thread #202 PR checks: ci 1m08s, exec-harness 15m37s, codex-lab-app 11m42s, Codespell, cargo-deny. Post-merge main: ci 1m02s, cargo-deny 1m29s, Codespell 22s.
• Route exec harness output to local artifacts #203 PR checks: ci 1m02s, exec-harness 43s, Codespell, cargo-deny. Post-merge main: ci 1m04s, cargo-deny 1m36s, Codespell 23s.
• Add opt-in local artifact environment #204 PR checks: ci 58s, exec-harness 44s, codex-lab-app 26s, Codespell, cargo-deny. Post-merge main: ci 1m13s, cargo-deny 1m29s, Codespell 26s.
• Local focused validation for the final state included tools/codex-exec-harness/test_local_cleanup.py, scripts/local unit tests, bash -n, shfmt -d, git diff --check, and a live artifact-env.sh probe for pnpm/npm/temp/sccache env behavior.
• Focused read-only review agents reviewed each implementation slice. Findings on target key escaping, inherited Cargo target overrides, Unix test portability, cleanup traversal, and artifact temp deletion bounds were fixed before merge.
• Background auto-review did not produce findings for the later slices because the broad review scope exceeded configured background size limits; there was no detached finding to apply.

Final State

• just local-speed-status now reports all planned high-churn categories and candidate artifact paths.
• just local-cleanup-space remains dry-run by default and covers known rebuildable local paths with keep flags for shared caches/temp.
• just local-artifact-setup --apply can generate the ignored local user.bazelrc and prepare package/temp artifact directories when the artifact root is mounted.
• eval "$(just local-artifact-env)" is the opt-in shell path for pnpm/npm/temp and optional sccache routing. This intentionally avoids tracked .npmrc changes or global config mutation.
• Spawned external Every Code/Codex agent processes receive per-agent Cargo target scope/key routing.
• Main checkout codex-rs/target symlink can remain a local guardrail, not the worktree strategy.

Closed

All acceptance criteria for this local artifact-routing plan are satisfied in repo-owned tooling. Future changes, if desired, should be new issues rather than reopening this plan:

• Observing real-world agent/worktree cache behavior after several PRs.
• Enabling sccache automatically for a specific workflow or host class.
• Moving any caches/runners to /Volumes/Docker-External, which was explicitly out of scope here.

[cbusillo]

Completed via #200, #201, #202, #203, and #204. The issue body has been updated with merged implementation details, validation evidence, and final state. Closing this plan; future optional observation or Docker-External/sccache automation should be tracked separately.