Skip to content

review11 security close-out (Phases 150–152) + federation withdrawal & new direction plans#133

Merged
jsilvanus merged 11 commits into
mainfrom
claude/semantic-objects-federation-uh685o
Jul 10, 2026
Merged

review11 security close-out (Phases 150–152) + federation withdrawal & new direction plans#133
jsilvanus merged 11 commits into
mainfrom
claude/semantic-objects-federation-uh685o

Conversation

@jsilvanus

@jsilvanus jsilvanus commented Jul 10, 2026 •
edited
Loading

Copy link
Copy Markdown
Owner

Summary

This PR carries two related strands: the review11 security close-out (Phases 150–152, implemented and tested) and the withdrawal of the Semantic Federation plan (Phases 154–158) together with finalized design documents for the three ideas salvaged from it.

Security fixes (Phases 150–152) — implemented

  • Phase 150 — Git argument-injection close-out (d06182f): new shared src/core/git/runGit.ts helper that rejects unsafe refs (isSafeGitRange() → UnsafeGitRefError) before spawning git and always passes --end-of-options, closing the PoC-confirmed arbitrary-file-write reachable via semantic_bisect/triage. Routed resolveRefToTimestamp, parseDateArg, and getMergeBase/getBranchExclusiveBlobs through it. 8 new integration tests prove a --output=<path> ref is rejected and writes no file.
  • Phase 151 — Read-route repo authorization gate (842be12): new src/server/middleware/repoAuth.ts mounted on all 10 data-route groups; in multi-tenant mode (explicit GITSEMA_MULTI_TENANT, else GITSEMA_SERVE_KEY presence) a request must hold a read grant unless the repo is public, else 403. Global admin key and legacy repo-scoped tokens bypass. 15 new middleware tests.
  • Phase 152 — BYOK SSRF guard + list-tool bounds (6bf15d8): verified and extended tests for validateByokUrl (loopback/link-local/metadata/RFC-1918 blocked by default, GITSEMA_BYOK_ALLOW_HOSTS allowlist); audited all Phase 147/148 list tools and capped the only unbounded input — graph traversal depth — at a shared MAX_GRAPH_DEPTH_REQUEST=64 on both HTTP and MCP.

Verification: pnpm build clean; full test suite 1616 passed / 22 skipped / 0 failing (all tests executed, including native better-sqlite3). Three patch changesets included. PLAN.md/features.md/README.md/parity.md/CLAUDE.md updated.

Federation withdrawal + new direction (docs)

  • Withdrawn Phases 154–158 (Semantic Federation) from PLAN.md before any implementation — the design centered on a speculative P2P/gossip network and wasn't reconciled with the shipped auth/storage/model-profile layers. The design doc is retained, marked ⛔ withdrawn; phase numbers 154–158 are retired. (Removing the block also repaired PLAN.md's Phase 153 entry, which the federation insertion had split in two.)
  • docs/prebuilt-index-distribution-plan.md (new, complete): "index once, serve many" — publishable, content-addressed index bundles keyed by (repo, commit, embed profile), index attach/index publish, sqlite-delta incremental transfer, layered static resolution (no registry service, no P2P). Five-phase implementation plan.
  • docs/semantic-enrichment-plan.md (new, complete): opt-in LLM-generated summary/keywords/entities metadata for whole-file blobs and chunks, stored in a new enrichments table referencing existing embedding rows (no vector duplication), mandatory inbound redaction via redact.ts before storage, narrator-model reuse, additive surfacing through SearchResult across CLI/REPL/MCP/HTTP/guide. Three implementation phases (E1–E3); lazy enrich-on-search explicitly rejected.
  • docs/design/concept-vocabulary.md (new, complete): SKOS-subset, model-independent concept vocabulary (schemes/concepts/labels/relations + recomputable per-method assignments) mapped onto blobs/chunks/symbols/clusters — the cross-model/cross-repo interoperability layer raw vectors can't provide. Hybrid curation (manual primary, LLM proposals human-gated), rename-proof content-addressed assignment targets, SKOS-idiomatic deprecation/forwarding. Five implementation phases (C1–C5).
  • docs/feature-ideas.md: all three salvaged entries converted to "Refined into:" pointers at their design docs.

Each design doc contains a "Decisions taken autonomously (pending user review)" section recording every design-gap resolution made during non-interactive refinement — review those sections before phase-planning. None of the three designs is scheduled in PLAN.md yet.

🤖 Generated with Claude Code

https://claude.ai/code/session_01JscPmRxVjcHHJzmxUyk3Sh

claude added 11 commits July 9, 2026 19:15
@claude
Withdraw Semantic Federation plan (Phases 154-158)
0c881cd 
Remove Phases 154-158 and the federation summary from PLAN.md (restitching
Phase 153's entry, which the insertion had split in two), mark
docs/design/semantic-federation.md as withdrawn, and re-capture the
salvageable kernels in docs/feature-ideas.md as three independent ideas:
Prebuilt Index Distribution, Chunk-Level Semantic Enrichment, and SKOS-Style
Concept Vocabulary. Phase numbers 154-158 are retired; next phase starts
at 159.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JscPmRxVjcHHJzmxUyk3Sh
@claude
Design: Prebuilt Index Distribution (index once, serve many)
37fbb12 
Refine the docs/feature-ideas.md entry salvaged from the withdrawn
semantic-federation design into a full standalone design document,
docs/prebuilt-index-distribution-plan.md. The design keeps the artifact
framing and drops all P2P/gossip/DHT machinery: bundle manifest v2 on the
Phase 54 tar.gz format with embed_config-derived provenance (configHash +
per-config quantization, no new version string), same-schema sqlite delta
bundles applied via ATTACH + INSERT OR IGNORE, `index attach`/`index
publish` subcommands with a layered HTTPS resolution ladder (explicit URL
-> config template -> gitsema server -> GitHub rolling release +
gitsema-bundles.json), server-side /bundles routes gated by the existing
Phase 122-126 grant model (bundle access >= repo read access), sqlite-only
v1 scope with `storage migrate` as the non-sqlite path, and a zero-server
CI loop. Five-phase implementation plan (not yet scheduled in PLAN.md).

Run non-interactively: all Design Gaps resolved from codebase research and
recorded in a "Decisions taken autonomously (pending user review)" section,
including the finding that Phase 54's `index export --after/--since` never
actually filtered (provenance-only) and should be deprecated when Phase A
ships. feature-ideas.md entry replaced with a pointer to the design doc.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JscPmRxVjcHHJzmxUyk3Sh
@claude
WIP: preserve semantic enrichment design draft (subagent interrupted)
7961f91 
The /refine-idea subagent for Chunk-Level Semantic Enrichment hit the
session usage limit before finalizing; its 498-line draft is preserved
here so it isn't lost if the environment is reclaimed. To be replaced by
the finalized version (including the feature-ideas.md pointer update)
when the agent resumes.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JscPmRxVjcHHJzmxUyk3Sh
@claude
Design draft: SKOS-style concept vocabulary (preserve interrupted sub…
8b9a283 
…agent work)

The /refine-idea subagent for the SKOS-Style Concept Vocabulary idea was
interrupted by the session usage limit after writing this 718-line design
doc but before updating the feature-ideas.md pointer and committing. The
doc is preserved here; the pointer update and final review happen when the
agent resumes.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JscPmRxVjcHHJzmxUyk3Sh
@claude
Phase 150: close git argument-injection sink (review11 §2.1 + §3.2)
d06182f 
Add a shared runGit() helper that rejects leading-`-` refs before spawning
git and always inserts git's --end-of-options separator, so a caller-supplied
"ref" like --output=<path> can never be reparsed as a git flag. Route the
network-reachable sink (resolveRefToTimestamp, reached via semantic_bisect/
triage) and the lower-impact §3.2 sites (parseDateArg, getMergeBase,
getBranchExclusiveBlobs) through it. Add tests/integration/gitArgInjection
regression suite proving the arbitrary-file-write PoC no longer reproduces.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JscPmRxVjcHHJzmxUyk3Sh
@claude
Phase 151: enforce repo authorization on read routes (review11 §2.2)
842be12 
Add repoAuthMiddleware, mounted after repoSessionMiddleware on all data
routes (search/analysis/evolution/graph/insights/protocol/watch/projections/
narrate/guide). In multi-tenant mode it requires the caller to hold a `read`
grant on the addressed repoId (via resolveUserRepoAccess) unless the repo is
public, else 403 — closing "read any private repo by ID". Multi-tenant mode is
opt-in via GITSEMA_MULTI_TENANT (defaults to GITSEMA_SERVE_KEY presence); the
global serve key and legacy per-repo scoped tokens bypass the check, and a
default open server is unchanged. Extract resolveRequestedRepoId() from
repoSessionMiddleware; mark req.globalKeyAuth / req.repoTokenScoped in
authMiddleware. 15 new middleware unit tests; full route suite unchanged.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JscPmRxVjcHHJzmxUyk3Sh
@claude
Phase 152: BYOK SSRF guard verification + list-tool depth bounds (rev…
6bf15d8 
…iew11 §3.1 + §3.3)

§3.1: verify + strengthen the BYOK SSRF guard (validateByokUrl already blocks
loopback/link-local/metadata/RFC-1918 by default with a GITSEMA_BYOK_ALLOW_HOSTS
allowlist); extend byokCredentials tests (metadata IP, RFC-1918, non-http
scheme, public pass, CIDR re-permit) and document in README/features/CLAUDE.

§3.3: upper-bound the last unbounded newly-exposed list input — deps
(depth ?? Infinity, no server-side clamp) and blast_radius depth now carry
.max(MAX_GRAPH_DEPTH_REQUEST=64) on the HTTP schema and MCP tool. New shared
constant in storage/types.ts. serverRoutes tests assert over-large depth → 400.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JscPmRxVjcHHJzmxUyk3Sh
@claude
Merge review11 security close-out (Phases 150-152)
9faae3a 
Phases 150 (git argument-injection), 151 (read-route repo authorization
gate), 152 (BYOK SSRF guard + list-tool depth bounds) implemented with
tests (full suite 1616 passed / 0 failing), docs, and changesets.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JscPmRxVjcHHJzmxUyk3Sh
@claude
docs: refine Chunk-Level Semantic Enrichment idea into full design plan
d00ac23 
Promote the feature-ideas.md entry (salvaged Layer-1 kernel of the
withdrawn semantic-federation design) into docs/semantic-enrichment-plan.md
via the /refine-idea flow:

- Opt-in LLM enrichment (summary/keywords/entities) of retrieval units:
  whole-file blobs AND chunks (default chunker produces no chunk rows,
  so chunk-only would no-op)
- New 'enrichments' table (schema v33) referencing existing rows — no
  vector duplication; chunk units keyed by portable (blob_hash,
  start_line, end_line), not local chunk_id
- Storage-seam coverage via new MetadataStore methods (sqlite + postgres;
  qdrant covered by its companion Postgres metadata store)
- Mandatory INBOUND redaction (redact.ts on LLM output before storage;
  bundles/server responses ship stored text verbatim), audited via
  withAudit('enrich')
- Reuses narrator model configs (no new embed_config kind); trigger model
  is 'index start --semantic-enrich' + 'index enrich' backfill; lazy
  enrich-on-search rejected (read paths stay network-free)
- Additive SearchResult.summary/keywords surfacing across
  CLI/REPL/MCP/HTTP/guide; interpretations.ts + gen:skill + parity.md
  obligations spelled out
- 3 implementation phases (E1 core/storage/backfill, E2 index-time,
  E3 surfacing/parity), not yet scheduled in PLAN.md

Session was non-interactive: all four original Design Gaps resolved
autonomously with rationale in the plan's §6 'Decisions taken
autonomously (pending user review)' (13 decisions, D1–D13).

feature-ideas.md: entry body replaced with a 'Refined into:' pointer +
one-paragraph summary; 'Last updated' bumped to 2026-07-10. (Worktree
base predates the salvage entries, so the pointer-form entry is added
fresh here; coordinator merge resolves the overlap.)

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JscPmRxVjcHHJzmxUyk3Sh
@claude
Merge finalized semantic enrichment design (replaces WIP draft)
4e6b248 
Chunk-Level Semantic Enrichment refined into docs/semantic-enrichment-plan.md
(3-phase design E1-E3, all design gaps resolved); feature-ideas.md entry
reduced to a pointer. Conflict resolution keeps the Prebuilt Index
Distribution pointer and SKOS entry alongside the new pointer.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JscPmRxVjcHHJzmxUyk3Sh
@claude
docs: refine SKOS-Style Concept Vocabulary idea into full design doc
69a6957 
Promote the feature-ideas.md entry into docs/design/concept-vocabulary.md
(refine-idea flow): SKOS-subset vocabulary in durable relational tables
(v33) + recomputable content-addressed assignments (v34) behind a
fail-loud ConceptStore; lexical assignment as the model-independent
backbone with centroid/LLM/cluster methods layered on; concepts CLI group
+ --concept faceting at CLI/MCP/HTTP parity; deprecation/replaced_by
scheme evolution; 5-phase plan (C1-C5). All five original Design Gaps
resolved autonomously with rationale in §11 (pending user review).

Final review fixes over the preserved draft: corrected redact.ts export
names, fixed a §4.3 cross-reference, renumbered §6.3→§6.1, noted the v33
schema-number collision with semantic-enrichment-plan.md. feature-ideas.md
entry replaced with a pointer + summary per sibling-entry precedent;
header dates bumped.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JscPmRxVjcHHJzmxUyk3Sh
@jsilvanus
jsilvanus merged commit 33d4121 into main Jul 10, 2026
4 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@jsilvanus @claude