feat(sdk): Jev-gated hybrid tree navigation (find_pages tool + search()) - #534
Open
akushonkamen wants to merge 1 commit into
Open
akushonkamen wants to merge 1 commit into
akushonkamen wants to merge 1 commit into
Conversation
Retrieval-time tree search walked the section tree with the chat LLM at
every fan-out (measured at seconds per level). A replay experiment over
120 routing decisions showed the TypeSafe System One decision API
("Jev") making the same calls in ~0.3s p50 (29x faster) with 83%
agreement when the reference model is confident — so the walk now asks
Jev first and keeps the SDK's own LLM for what Jev should not decide
alone:
- P(top1) >= 0.7: descend the single top child
- 0.5 <= P < 0.7: descend the top-2 children
- P < 0.5, or fan-out > 64: escalate to the LLM
- LLM escalation failure: expand all children (fallbacks counted)
Jev is a hard dependency, never silently bypassed: a missing
TYPESAFE_API_KEY or a Jev call that failed past its retries raises
PageIndexAPIError (search) or a JEV_UNAVAILABLE error envelope
(find_pages); only the SDK's own model may fall back.
New pageindex/jev_router.py holds a minimal System One client
(requests.Session + Retry, mirroring the MCP bridge's policy) and the
iterative routing kernel (visited/budget/depth guards, over-inclusive
on truncation). find_pages registers as a local-only tool (the cloud
MCP contract snapshot is untouched) and client.search() exposes the
same kernel programmatically; both share the hit -> page-range shaping.
19 new tests (kernel gates, guards, failure semantics, tool contract,
agent-loop e2e); full suite 758 passed, 11 skipped.
Co-Authored-By: Claude <noreply@anthropic.com>
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Adds a Jev-gated hybrid navigation layer for retrieval-time tree search:
pageindex/jev_router.py(new): a minimal TypeSafe System One client (requests-only, one endpoint, choice questions; transport retries mirrormcp_bridge's policy) and an iterative routing kernel that walks a document's section tree, asking Jev at each fan-out.find_pagesagent tool: local-only (registered in its own_LOCAL_ONLY_TOOLSregistry — the cloud MCP contract snapshot is untouched), so agentic chat can navigate a known document directly and feed the returnedpagesspec intoget_page_content().client.search(doc_id, query): programmatic access to the same kernel, local-only (raisesPageIndexAPIErroron a cloud client).Gate policy (from the replay experiment below):
P(top1) ≥ 0.7→ descend the single top child;0.5 ≤ P < 0.7→ descend top-2;P < 0.5or fan-out > 64 → escalate to the SDK's own LLM; LLM escalation failure → expand all children (counted innavigated.fallbacks).Jev is a hard dependency, never silently bypassed: a missing
TYPESAFE_API_KEYor a Jev call that failed past its retries raisesPageIndexAPIError(search) / returns aJEV_UNAVAILABLEerror envelope (find_pages). Only the SDK's own model may fall back.Why
Every retrieval step previously cost a full LLM call at each tree level. A replay experiment over 120 real routing decisions (2 docs × 10 queries × every internal node,
experiments/jev_retrievalharness, not part of this PR) measured the same decision points through both models:End-to-end smoke on an indexed earnings PDF: a 3-level tree navigates in 1.2 s with Jev vs ~10 s per level with the LLM path.
Tests
tests/test_jev_router.py): gate bands, NONE pruning, LLM escalation + fallback semantics, Jev-failure hard-error semantics (search raises / tool envelope), fan-out/budget/depth/visited guards, page-spec formatting, tool registration + cloud-contract snapshot untouched, agent-loop e2e.search()andcall_tool("find_pages"); missing-key error path verified.Notes for reviewers
ACCEPT,EXPAND,MAX_FANOUT, …) — config.yaml is a loud-fail validated surface and we didn't want to grow it without tuning evidence.JevUnavailablesubclassesPageIndexAPIError;search()lets it propagate unwrapped.api.typesafe.ai/v1/systemone,jev-latest) are hardcoded constants; usage/cost auditing left as TODO.🤖 Generated with Claude Code