Skip to content

feat(cache): compact selected inactive indexes - #642

Merged
N0zoM1z0 merged 11 commits into
mainfrom
feat/cache-selective-compaction
Oct 7, 2026
Merged

N0zoM1z0 merged 11 commits into
mainfrom
feat/cache-selective-compaction

Conversation

@N0zoM1z0

@N0zoM1z0 N0zoM1z0 commented Oct 7, 2026 •

Copy link
Copy Markdown
Collaborator

Summary

Retained SQLite indexes can contain reusable pages, while dense or active caches make blanket VACUUM waste CPU and temporary space. Add cache compact for 1–8 explicit managed cache IDs: preview by default, apply with --yes, and require both reusable-byte and percentage thresholds. Process serially with default 1-GiB work bounds, an 8-MiB page-cache target, disabled mmap, and a cooperative 120-second budget. Combine database/temp-volume headroom when they share a volume.

Maintenance holds the exclusive lifetime lease, requires existing WAL storage, and preserves generations, schema, FTS results, metadata, and lease identity without initializing Services or migrating indexes. Unix rejects a selected database already open in the calling process before pinning, preserving existing POSIX locks; apply verifies actual opened descriptor identities with bounded fstat observations. Windows pins without delete sharing and respects SQLite temp-directory precedence. Native CLI preview remains available with outside-process readers; blocked apply reports failure without forcing their snapshots to release. Reports distinguish estimated benefit, actual reduction, and VACUUM committed before a later failure.

Keep legacy retention stable by ignoring empty inspection-created WAL timestamps and skipping mtime-based compaction. In-place maintenance requires an owner-controlled root, lease-cooperating clients, and quiescent filesystem tools; observed link checks cannot isolate arbitrary same-owner file mutations. Stronger detached publication is tracked in #644. Retrieval never runs automatic VACUUM, and no MCP schema or host runtime configuration changes.

Align concurrent MCP startup and terminal-failover leader fixture setup with the existing 30-second indexing-liveness budget. Initialize/cold-context/terminal-failover deadlines, exact generation assertions, retry policy, and stress repetition counts stay unchanged; this is a fixture readiness correction, with no product latency claim.

Closes #641. Closes #643.

Validation

  • Final focused nextest: 51 passed, including the complete cache module, independent-process public lock oracle, native CLI outside-reader preview/apply/failure, and terminal-failover lifecycle witness.
  • Storage regressions cover links/path replacement, swap/open/restore, descriptor file types, existing/concurrent descriptors, rollback refusal, legacy age, lease/generation/FTS retention, shared-volume space boundaries, interruption, fingerprint bounds, and post-commit reporting. Actual metadata query-plan assertion verifies primary-key lookup. Windows cases cover rename prevention and invalid TMP without TEMP fallback.
  • All-target/all-feature Clippy with warnings denied, formatting, and Git whitespace passed. CLI adapter/help/documentation selection previously passed 61 checks; only inspected CLI help snapshots changed, with no MCP snapshot change.
  • Authoritative source coverage: compact 88.39/100/90.95, compact_models 100/100/100, compact_sqlite 87.15/90.38/83.31 (line/function/region). New-owner floors remain 84/95/86, 95/95/95, and increase to 82/85/78; existing other-file floors/exclusions remain unchanged. The stricter policy assessor passed against every original file summary with only filename-prefix normalization.
  • Full source CI 37590032325 runs on f23f8f2; All lanes passed, including Linux/macOS/Windows product behavior, coverage, and 100/25/25 lifecycle repetitions. Policy-only head c013500 has an empty product/test/Cargo diff; current-head PR CI 37590998937 passed and current-head review completed.

Local work used disposable fixtures in a network-disabled container capped at one CPU, 3 GiB memory, and one build job. The container is stopped after validation. No retained production cache was compacted and no host CLI/MCP installation or configuration was upgraded. Copy-only feasibility measurements and separately authorized old-cache cleanup remain in the private feedback log.

@N0zoM1z0
N0zoM1z0 requested a review from morluto as a code owner October 7, 2026 03:48
@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Oct 7, 2026 •

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-10-07T08:10:28.696902Z c013500 Manual request
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: bcdc63c761

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread src/cache/compact.rs
@N0zoM1z0

N0zoM1z0 commented Oct 7, 2026

Copy link
Copy Markdown
Collaborator Author

@codex review

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 679302efee

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread src/cache/compact_sqlite.rs Outdated

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: f868886d99

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread src/cache/compact.rs Outdated
@N0zoM1z0

N0zoM1z0 commented Oct 7, 2026

Copy link
Copy Markdown
Collaborator Author

@codex review

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 0774250d7a

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread src/cache/compact_sqlite.rs Outdated

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 072650fbeb

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread src/cache/compact.rs

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 00aefe0c51

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread src/cache/compact.rs
@N0zoM1z0

N0zoM1z0 commented Oct 7, 2026

Copy link
Copy Markdown
Collaborator Author

@codex review

Please review latest policy-only head c013500. Product/test/Cargo sources are unchanged from reviewed and measured f23f8f2; the only diff raises compact_sqlite coverage floors to 82/85/78 and records final source measurements. The stricter policy assessor and current-head PR CI passed. Full source CI 37590032325 has passed all three product platforms and both 25-repetition macOS/Windows stress lanes; Linux 100-repetition stress is still running. The same-owner filesystem boundary remains documented and tracked in #644.

@chatgpt-codex-connector

Copy link
Copy Markdown

Codex Review: Didn't find any major issues. Keep it up!

Reviewed commit: c013500331

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

@N0zoM1z0
N0zoM1z0 merged commit 1f3738c into main Oct 7, 2026
9 checks passed
@N0zoM1z0
N0zoM1z0 deleted the feat/cache-selective-compaction branch October 7, 2026 08:12
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Read-only cache inspection refreshes legacy access age through empty WAL sidecars Add opt-in selective compaction for inactive managed caches

1 participant