Skip to content

feat(sdk): Jev-gated hybrid tree navigation (find_pages tool + search()) - #534

Open
akushonkamen wants to merge 1 commit into
VectifyAI:mainfrom
akushonkamen:feat/jev-router
Open

akushonkamen wants to merge 1 commit into
VectifyAI:mainfrom
akushonkamen:feat/jev-router

Conversation

@akushonkamen

Copy link
Copy Markdown

What

Adds a Jev-gated hybrid navigation layer for retrieval-time tree search:

  • pageindex/jev_router.py (new): a minimal TypeSafe System One client (requests-only, one endpoint, choice questions; transport retries mirror mcp_bridge's policy) and an iterative routing kernel that walks a document's section tree, asking Jev at each fan-out.
  • find_pages agent tool: local-only (registered in its own _LOCAL_ONLY_TOOLS registry — the cloud MCP contract snapshot is untouched), so agentic chat can navigate a known document directly and feed the returned pages spec into get_page_content().
  • client.search(doc_id, query): programmatic access to the same kernel, local-only (raises PageIndexAPIError on a cloud client).

Gate policy (from the replay experiment below): P(top1) ≥ 0.7 → descend the single top child; 0.5 ≤ P < 0.7 → descend top-2; P < 0.5 or fan-out > 64 → escalate to the SDK's own LLM; LLM escalation failure → expand all children (counted in navigated.fallbacks).

Jev is a hard dependency, never silently bypassed: a missing TYPESAFE_API_KEY or a Jev call that failed past its retries raises PageIndexAPIError (search) / returns a JEV_UNAVAILABLE error envelope (find_pages). Only the SDK's own model may fall back.

Why

Every retrieval step previously cost a full LLM call at each tree level. A replay experiment over 120 real routing decisions (2 docs × 10 queries × every internal node, experiments/jev_retrieval harness, not part of this PR) measured the same decision points through both models:

reference LLM Jev
latency p50 / p95 9.6 s / 22.2 s 0.33 s / 0.84 s
top-1 agreement — 79% overall, 83% when the reference is confident
calibration — Jev P(top1) tracks the reference's probability for the same choice
failure mode — conservative (over-prunes): hence the 0.5–0.7 band + LLM escalation

End-to-end smoke on an indexed earnings PDF: a 3-level tree navigates in 1.2 s with Jev vs ~10 s per level with the LLM path.

Tests

  • 19 new tests (tests/test_jev_router.py): gate bands, NONE pruning, LLM escalation + fallback semantics, Jev-failure hard-error semantics (search raises / tool envelope), fan-out/budget/depth/visited guards, page-spec formatting, tool registration + cloud-contract snapshot untouched, agent-loop e2e.
  • Full suite: 758 passed, 11 skipped (skips are pre-existing live-key gates).
  • Live smoke: real Jev routing through search() and call_tool("find_pages"); missing-key error path verified.

Notes for reviewers

  • Thresholds/budgets are module constants for now (ACCEPT, EXPAND, MAX_FANOUT, …) — config.yaml is a loud-fail validated surface and we didn't want to grow it without tuning evidence.
  • JevUnavailable subclasses PageIndexAPIError; search() lets it propagate unwrapped.
  • The endpoint/model (api.typesafe.ai/v1/systemone, jev-latest) are hardcoded constants; usage/cost auditing left as TODO.

🤖 Generated with Claude Code

Retrieval-time tree search walked the section tree with the chat LLM at
every fan-out (measured at seconds per level). A replay experiment over
120 routing decisions showed the TypeSafe System One decision API
("Jev") making the same calls in ~0.3s p50 (29x faster) with 83%
agreement when the reference model is confident — so the walk now asks
Jev first and keeps the SDK's own LLM for what Jev should not decide
alone:

- P(top1) >= 0.7: descend the single top child
- 0.5 <= P < 0.7: descend the top-2 children
- P < 0.5, or fan-out > 64: escalate to the LLM
- LLM escalation failure: expand all children (fallbacks counted)

Jev is a hard dependency, never silently bypassed: a missing
TYPESAFE_API_KEY or a Jev call that failed past its retries raises
PageIndexAPIError (search) or a JEV_UNAVAILABLE error envelope
(find_pages); only the SDK's own model may fall back.

New pageindex/jev_router.py holds a minimal System One client
(requests.Session + Retry, mirroring the MCP bridge's policy) and the
iterative routing kernel (visited/budget/depth guards, over-inclusive
on truncation). find_pages registers as a local-only tool (the cloud
MCP contract snapshot is untouched) and client.search() exposes the
same kernel programmatically; both share the hit -> page-range shaping.

19 new tests (kernel gates, guards, failure semantics, tool contract,
agent-loop e2e); full suite 758 passed, 11 skipped.

Co-Authored-By: Claude <noreply@anthropic.com>

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant