pipeline-correctness-fixes: cache-or-LLM audit + 250+ commits backlog by yashb98 · Pull Request #2 · yashb98/multi-agent-patterns

yashb98 · 2026-05-09T11:03:18Z

Summary

pipeline-correctness-fixes is 254 commits ahead of main and represents
months of work spanning the cache-or-LLM audit (closed at S8), the
pipeline-bugs audit (closed at S18 prior to this branch's HEAD), and a
broad set of feature work and bug fixes. This PR is draft because the
branch carries 40 modified files of working-tree WIP that are NOT in
this push, plus you (the author) need to decide which slices belong in
the integration vs. should be peeled off into separate PRs.

What's on the branch (by commit prefix)

Prefix	Count	Examples
`fix(...)`	93	cache-llm S1–S8 + S7-EXT, pipeline-bugs S1–S18, screening / form / nav fixes
`feat(...)`	77	adaptive form pipeline, page-reasoner semantic cache, AUQ confidence escalation, learned form patterns
`docs(...)`	50	audits + design specs + plan docs (incl. journal v1)
`refactor(...)`	10	application orchestrator split, navigator decomposition
`chore(...)`	4	setup tooling (timeout, parallelism, test-mode guards)
`test(...)`	3	regression suites for new caches

cache-or-LLM audit summary (most recent work, S1 → S8 + S7-EXT)

Closed in 16 cache-llm commits + 1 chore + 2 follow-ups. Full report:
docs/audits/cache-llm-completion-report.md.

Session	Cluster	Commit	Outcome
S1	Catalog every LLM call site	`93c1987`	70-row catalog + reproducibility tool
S2	Field-mapping (DETERMINISTIC)	`3de61d3`	Verified — §2.2 #3 framing was stale; map_fields already dict-first
S3	Screening alignment	`5c5841e`	Cache add: `hiring_message_cache`. §2.2 #4 framing stale (no LLM in `_align_to_options`)
S4	CV tailoring	`4509f6d`	Cache add: `tailored_cv_cache` (saves 4 LLM calls per JD repeat)
S5	Cover letter	`7aba244`	Cache add: `cover_letter_cache` (saves 1 LLM call per JD repeat)
S6	Page reasoner	`8a9bcc8`	Verified — §2.2 #5 framing stale; comprehensive cache exists at HEAD
S7	Cognitive bypasses	`0a3cfda`	Migrated `strategy_reflector` from `smart_llm_call` to `cognitive_llm_call`
S7-EXT	gmail_agent migration	`f6ced0d`	Migrated `_classify_email` through cognitive (mirrors S7)
S8	Final reconciliation	`f5de8b5`	Catalog 70/70 covered, completion report written

Audit findings: §2.2 of the audit doc had a 50/50 real-vs-stale rate.
Real claims (#1, #2, #6) correctly identified missing caches; stale
claims (#3, #4, #5) had location/framing errors against current HEAD.

Net cache savings: ~6 LLM calls eliminated per JD repeat
(4× cv_tailor + 1× cover-letter polish + 1× hiring message), plus
strategy_reflector L0 hits if cognitive engine has templates for the
domain.

Schema migrations (3 new tables in applications.db, all
idempotent): hiring_message_cache, tailored_cv_cache,
cover_letter_cache.

Test plan

cache-llm-S2: live URL on Anthropic Greenhouse (commit message of 3de61d3 quotes DIRECT ID FILL: 2/6 fields set evidence)
cache-llm-S3 / S4 / S5 / S6 / S7 / S7-EXT: unit tests + stash drills (38+ tests across 5 new test files; all fail under stash-drill)
Adjacent existing tests still pass after each migration (cv_tailor, screening_v2, gmail_agent_real, strategy_reflector — verified per session)
Live URL §6 acceptance ("0 cache misses on second run") — DEFERRED. Local Ollama qwen3:32b times out on 1 of 4 cv_tailor calls even with the 180s timeout from chore 31317ad; OpenAI key returned insufficient_quota during S8 verification. Documented in completion report §5.
Pre-existing tests in test_cv_tailor.py::TestValidateSummary depend on working-tree WIP (_send_validation_alert); will return when the WIP lands.

Caveats / known risks

40 modified files of WIP in the local working tree are NOT in this push. Some test failures (e.g. _send_validation_alert import) will resolve once those land.
Deferred backlog in cache-llm-completion-report.md §7 lists 6 cluster-EXT items + 2 setup-tooling items still open.
Branch is long-lived — consider whether to merge in slices (cache-llm only, then pipeline-bugs separately, then features) or all-at-once.
The two new chore(setup) tweaks (timeout 180s, cv_tailor max_workers=1 on local) reduce fast-path performance for cloud-OpenAI users running cv_tailor — they explicitly take the slower-but-correct path on local. Both are guarded by is_local_llm().

🤖 Generated with Claude Code

The finalize_session(push_to_learning=True) bridge was silently no-op'ing for the typical Claude/Kimi emergency-assist flow because: 1. Emergency assists rarely pass original_mapping at start_session or final_mapping at finalize_session, so the session row stores empty JSON literals "{}" for both. 2. _push_fixes_to_corrections only fell back to reconstructing mappings from individual fixes on json.JSONDecodeError. Empty-but-parseable "{}" parses fine to {}, so the fallback never triggered. 3. record_corrections({}, {}) saw no diff and wrote zero rows — but _push_fixes_to_corrections returned len(value_fixes) regardless, masking the failure as success in finalize_session's log line. 4. AgentRulesDB.auto_generate_from_correction was never called from the ai_assist path — only from confirm_application — so even when fixes reached field_corrections, no override rules were generated. Net effect: every emergency Claude/Kimi field fix this year landed in ai_assist_sessions.db and went nowhere else. The same Country / City / Eligibility React-Select bugs kept recurring on Greenhouse forms because the agent had no field-level rules to consult on the next run. Fix: - Reconstruct agent_mapping/final_mapping from individual fixes whenever EITHER mapping is empty after parsing (not only on JSONDecodeError). - Call AgentRulesDB.auto_generate_from_correction per fix from inside the bridge, mirroring applicator.confirm_application:545-557. Wiring test (tests/jobpulse/test_ai_assist_learning_bridge.py): - test_emergency_assist_without_original_mapping_propagates_to_all_dbs reproduces the original failure (asserts 0 == 2 before the fix) and verifies field_corrections.db AND agent_rules.db get rows after finalize. - test_assist_with_original_mapping_still_works guards the existing diff-based path. Verified live on the Contentful run from earlier today: backfilling the 3 React-Select corrections through this fixed path populated all four target stores (field_corrections, agent_rules, screening_semantic_cache, optimization signals + ScreeningFeedbackLoop prototypes). 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

…nostics Live bug 2026-05-04 on Contentful Greenhouse: _click_navigation only matched buttons via page.get_by_role("button", name=...). Greenhouse's <button type="submit">Submit application</button> failed to match (likely extra aria-describedby content or accessible-name quirk), so the function returned "" → run_submit_and_confirm took the "Submit button / navigation" manual-help branch → infinite loop, forcing Claude to step in. Fix: - After role-based and Workday paths, try CSS-selector fallback: button[type='submit'], input[type='submit'], button.submit-application, button[data-qa*='submit']. These match by HTML attribute, no name-string fragility. - Skip disabled buttons in the CSS path (avoid clicking grayed-out submits while form validation is pending). - When ALL matchers exhaust, log a warning with a snapshot of every visible button on the page so future failures don't require a CDP inspection trip. - Add info-level logs on every successful click path noting which matcher won — makes log review actionable. 35 native_form_filler tests pass. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

Live bug 2026-05-04 on Contentful submit: drive_uploader logged "file not found" because application_materials wrote CV/CL to data/applications/<job_id>/ (sha-named) while drive_uploader and post_apply_hook read from data/applications/<safe_company>/, the default for cv_templates.generate_cv and generate_cover_letter. Two writers, two readers, three different path conventions: - application_materials.ensure_tailored_cv_for_job → <job_id>/ - application_materials.build_lazy_cover_letter_generator → <job_id>/ - cv_templates.generate_cv default → <safe_company>/ - cv_templates.generate_cover_letter default → <safe_company>/ Net effect: Contentful had Cover_Letter at <sha>/ but Yash_Bishnoi_*.pdf at <Company>/ (older dry-run path) — drive_uploader read DB cv_path (which was the latest write to <sha>/), found CL but not CV, silently failed the Drive upload. Recruiters get a Notion link with no resume. Fix: - Extract _application_dir(company) — single source of truth, canonical path is data/applications/<safe_company>/. - Both application_materials call sites now resolve through it, matching cv_templates' default + drive_uploader's read. - Multi-job-per-company collisions (rare for single applicant) get resolved by cv_templates' filename pattern Yash_Bishnoi_<Company>.pdf overwriting the prior file. 14 application_orchestrator tests pass. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

Live log on Contentful submit: strategy_reflector: LLM reflection failed: get_llm() got an unexpected keyword argument 'tier' shared.agents.get_llm signature is (temperature, model, timeout, max_tokens, agent_name) — no `tier` parameter. The reflector was silently failing Pass 2 (LLM-based heuristic extraction), so only Pass 1 (deterministic) heuristics ever made it into ExperienceMemory. Pass 2 yielded zero rows on every recent application. Fix: pass model="gpt-5-mini" directly. Verified 28 strategy_reflector tests pass. Notion "Applied Time" rejection (also from #85 task scope): already handled by job_notion_sync.update_application_page's self-healing retry loop (line 669-) which strips rejected properties and retries — live run confirmed it succeeds on the second attempt without the property. No code change needed; user can add an "Applied Time" rich_text column to the Notion DB schema if they want the timestamp captured. Drive token expiry (also from #85 scope): operational fix — user must re-authenticate via `python scripts/setup_integrations.py`. Not a code change. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

Live bug 2026-05-04 on Contentful Greenhouse: profile city is "Dundee", Greenhouse autocomplete returned five "Dundee, ..." options across UK, US states, and South Africa. The agent's React-Select picker called form_scanner.best_option_match (alias ax_best_match) which only does exact / alias / substring matching — when multiple options contain the desired value, it returns the first by dict insertion order, i.e. whatever Greenhouse rendered first. On Contentful that turned out to be a US "Dundee" city, not the UK one the user actually lives in. Fix: - form_scanner.best_option_match: add prefer_substrings tiebreaker. When multiple options contain the desired value as substring, prefer one that ALSO contains any of the prefer_substrings (typically the user's country). - form_engine.semantic_matcher.semantic_option_match: same parameter added to the 6-tier matcher used by _resolve_with_options. Affects token-overlap (tier 5) via +0.5 score boost and substring (tier 6) via tiebreak. - field_mapper._resolve_with_options: read user's country from ProfileStore.sensitive("country") (or identity().location's last comma-separated segment) and pass as prefer_substrings. - native_form_filler React-Select picker (3 ax_best_match call sites): same ProfileStore-derived country preference passed through. Smoke-tested with the actual Contentful option list: ["Dundee, Dundee City, United Kingdom", "Dundee, Florida, United States", "Dundee, Michigan, United States"] best_option_match("Dundee", opts) → UK option (was first by luck) best_option_match("Dundee", opts, prefer_substrings=("United Kingdom",)) → UK option (deterministic) Reordered with US options first → still UK (preference wins). 38 form_scanner tests pass. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

Extends the screening HITL plan to cover the application-level submit approval gate using the same primitive (asyncio.Future registry + pending_questions row + multi-sink notification_router emit). Resolves the live regression observed on Contentful 2026-05-04 where: 1. scan_pipeline DRY-RUN STOP closed the Playwright session, 2. LiveReviewSession's resume re-ran the full ApplicationOrchestrator (re-navigate, re-scan, re-fill 14+ fields, ~3 min wasted), 3. _remaining_manual_help_labels diverged between dry-run and submit verifier states, falsely flagging filled fields as failed and triggering a manual-help loop that forced Claude to step in and submit directly via Playwright (rule violation). Task 16 is ~80 LOC plus a wiring test that asserts the page object isn't closed during the pause. After this lands, dry-run+approve = single Submit click, no re-fill, no fresh session, no rule violation. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

Live regression on 2026-05-04 apply loop: every iteration picked the same expired Indeed job. Trace: 1. PageReasoner correctly identifies job as expired (confidence=1.00) 2. _mark_expired calls update_application_page(status="Expired", ...) 3. Notion 400: "Status" rejected — "Expired" is not in the Status schema (Found / Analyzing / Ready / Pending Approval / Applied / Interview / Offer / Rejected / Withdrawn). 4. update_application_page's self-healing retry strips Status and submits without it — Notion update returns OK. 5. Notion row stays Status=Found. 6. Next loop iteration re-fetches Found queue → same expired job → same abort → same retry-strip → same loop. Forever. Fix: map _mark_expired's Notion status to "Withdrawn" (closest semantic to "we're removing this job from the queue"). Note string preserves the real reason. After this lands, every dead/expired Indeed posting in the queue self-clears in ~45 seconds and leaves the Found queue. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

Live regression on 2026-05-04 JPMC apply: each `job-apply-next 1` invocation re-ran Indeed → "Apply on company site" → new-tab. Three problems compounded: 1. JPMC's Oracle Cloud opens fresh tabs from each Apply click, stacking 3-4 duplicate JPMC tabs in Chrome. 2. The new tab points at the candidate-experience LANDING URL, not the deep /apply/section/N URL — which means manual user logins on prior tabs are bypassed (auth context doesn't follow the new tab). 3. Wasted 2-3 min per iteration re-navigating the redirect chain. Fix: at LiveReviewSession.__init__, scan Chrome tabs (CDP /json) for an existing in-progress ATS tab matching this company. If found, swap self.url so PlaywrightDriver attaches there directly, skipping the Indeed → Apply-click chain entirely. Matcher heuristic (intentionally loose): - exclude same-host as the listing source (we want the ATS, not Indeed/LinkedIn/Reed) - require apply-path pattern in URL (/apply/, /section/, /application/, etc. — structural URL validation, not classification) - match by company name in host/title (strict path) OR ≥2 distinctive job-title tokens overlapping the page title (loose path — handles cases like "JPMorganChase" company vs "jpmc.fa.oraclecloud.com" abbreviated host) - pick deepest-path candidate (most progressed) Smoke-tested live: returns the actual /apply/section/1 tab for JPMC-from-Indeed; correctly returns None for same-host and random unrelated companies. 37 live_review/applicator/job_autopilot tests pass. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

Live regression on 2026-05-04 JPMC apply: agent's _find_in_progress_apply_tab correctly identified the user's logged-in /apply/section/1 tab and updated self.url, but PlaywrightDriver.connect ignored the URL hint — picked pages[-1] (whatever Chrome opened most recently, typically LinkedIn Cadillac F1) and then navigated THAT tab to about:blank, destroying any in-progress session including the user's manual auth. Two-part fix: 1. PlaywrightDriver.connect(prefer_url=...) — when supplied, scans pages for matching URL (exact → prefix → host+path-prefix) before falling back to pages[-1]. Sets self._attached_existing_url=True on success. LiveReviewSession._fill_async passes self.url (post _find_in_progress_apply_tab swap) and skips the about:blank reset when the attach matched. 2. FormNavigator.navigate_to_form skips the redundant driver.navigate(url) when current page URL already matches the target. Without this, the orchestrator's first call would page.goto-reload the user's logged-in tab, destroying the SPA state — confirmed live: pre-fix, the JPMC apply form was scanned as 4 fields (post-reload landing) instead of 31 fields (pre-reload form). Live verification on JPMC after both fixes: - "PlaywrightDriver attached to existing tab .../apply/section/1 (prefer_url match)" - "navigate_to_form: skipping initial navigate — already on ..." - FormScanner: 31 fields (was 4 before fix) - 6 fill ✓ on page 1: Address Line 3, City or Town, County, Preferred Location, etc. - Agent advanced through pages 1→2→3 before stuck-detector fired 32 navigator/driver/applicator tests pass. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

Live regression on 2026-05-04 pls-solicitors.co.uk: agent's _phase_plan took the DOM-fast-path to fill_form because the DOM classifier confidently labelled the page (job-description with embedded search/contact form elements) as APPLICATION_FORM at >=80%. PageReasoner was skipped, fill attempted on a non-form page, scroll_into_view_if_needed timed out 30s, graceful Pending Approval exit. Page reasoning was working — it just never ran. Symptom: zero PageReasoner log lines between the host-change "OBSERVE: host change uk.indeed.com→ pls-solicitors.co.uk" and the eventual "fill failed". Fix: only trust the DOM-only fast path when FormExperienceDB has a record with success=1 for this domain. Untrusted/new domains fall through to PageReasoner regardless of DOM confidence — the slower path is correct for unknown territory. Fully dynamic per .claude/rules/jobpulse.md "Dynamic Over Hardcoded": - No hardcoded domain allow-list - Trust accrues automatically from successful applications via FormExperienceDB.lookup(domain).success - Threshold (>=0.8) unchanged - Behavior generalizes to ALL new domains After this fix, on pls-solicitors: 1. DOM classifier still says APPLICATION_FORM at >=0.8 2. domain_trusted = False (no prior success) 3. Fast path SKIPPED — falls through to PageReasoner 4. Reasoner sees `/jobs/graduate-...` URL + page text → likely classifies as job_description → recommends click_element on "Apply Now" / similar 5. Navigator clicks Apply → real form renders → fills 4 wiring tests added in tests/jobpulse/test_navigator_domain_trust.py covering: unknown domain, failed-only record, successful record, malformed URL. All pass. 25 broader navigator/applicator tests still pass. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

Live regression on 2026-05-04/05 pls-solicitors.co.uk: agent's fill chain blocked for 30s on a single element it couldn't scroll to (page-header search bar, hidden honeypot, or off-viewport apply form field). Playwright default for scroll_into_view_if_needed is 30000ms — too long when the element is technically visible (offsetParent != null) but obscured by a sticky header / dismissed modal / off-viewport position. Result: text_filler caught the timeout via its outer try/except, returned FillResult(success=False), but the upstream live_review_applicator recorded the entire run as a navigation_failure even though most fields were fillable. Fix: - text_filler.fill_text: pass the function's `timeout` kwarg (default 5000ms) to scroll_into_view_if_needed; on timeout return early with scroll_timeout error instead of falling through to .fill() and getting a confusing downstream failure. - text_filler.fill_autocomplete: same treatment. - native_form_filler._smart_scroll: 5s timeout + log and proceed (the caller may still be able to interact via JS even without scroll). Net effect: bad fields cost 5s instead of 30s. A page with 2 unscrollable fields (e.g. Search bar + honeypot) wastes 10s instead of 60s; the rest of the fill chain proceeds. 43 text_filler/native_form_filler tests pass. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

Live regression on pls-solicitors.co.uk: PlaywrightDriver._smart_scroll called scroll_into_view_if_needed() with no timeout (Playwright default 30s). When NativeFormFiller delegated to driver._smart_scroll for a field that was technically visible (offsetParent != null) but obscured (page-header Search bar after cookie banner dismissal), it blocked 30s before failing. Earlier today I patched native_form_filler._smart_scroll's fallback path with 5s timeout — but that path only runs when driver lacks _smart_scroll. Driver _had_ the method, so delegation hit the 30s default. This patch fixes the actual call site. 5s timeout + log-and-return on failure (caller may still interact via JS even without the scroll completing). 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

Live regression on pls-solicitors.co.uk: the page has 4 separate <form> elements (header search, cookie consent modal, hidden honeypot, actual apply form). _detect_form_container's common-ancestor walk goes up to <body> in this case (because the form elements span multiple unrelated forms), then returns null when commonAncestor === document.body fires. Result: no container resolved → field scanner falls back to whole-page scan → picks up Search bar, cookie checkboxes, honeypot fields → fill chain trips on a non-apply-form field's 30s Playwright auto-wait. Fix: when common-ancestor approach fails, enumerate every visible <form> that has a submit-like button. Score each by count of meaningful inputs (text/email/tel/textarea/role=textbox/role=combobox — excluding honeypot patterns name=*honeypot*, name ends with -hp/_hp, plus type=hidden/disabled/readonly). Pick the form with the highest score (min 2 to qualify — single-input newsletters don't count). Live verification on pls-solicitors: detected: '#nf-form-2-cont > div > div:nth-of-type(4) > form' inputs inside: 12 (vs ~4 in header search + cookie modal combined) Fully dynamic per .claude/rules/jobpulse.md Dynamic-Over-Hardcoded: - No hardcoded site selectors - Honeypot detection uses generic name patterns (honeypot/-hp/_hp) - Form scoring is text-input weighted (apply forms vs cookie checkbox modals) 224 form_engine tests pass. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

Live regression on 2026-05-05: Ollama returned HTTP 500 with "model requires more system memory (21.9 GiB) than is available (19.6 GiB)" on every LLM call. The retry layer treated it as transient (it's a 5xx) and burned 4 retries × 5-15s = 30-60s before the cloud-fallback layer in shared/agents.py finally saw the error. Across the agent's many LLM calls per application, that's 5-10+ minutes lost waiting for retries that could never succeed within this session. Fix: PERMANENT_FAILURE_PATTERNS short-circuits is_retryable_error. When the error message contains a known permanent marker (memory exhaustion, model-not-found, bad API key, context-length-exceeded, auth/permission failures), return False immediately. Caller's fallback chain kicks in within milliseconds instead of after a full retry burn. Patterns covered: - "requires more system memory" (Ollama OOM) - "model not found" (Ollama model not pulled) - "context length" (token limit; needs truncation) - "context_length_exceeded" - "invalid_request_error" (malformed request) - "permission_denied" - "authentication" / "invalid api key" - "model_not_found" - "billing_hard_limit_reached" 7 wiring tests in tests/shared/test_llm_retry_permanent_failures.py covering OOM, generic 500, 429 rate-limit, model-not-found, bad API key, context length, network timeout. All pass. 23 broader retry tests still pass. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

CorrectionCapture only saw the read-only review page and missed mid-flow user edits. Forge regression on 2026-05-05: user corrected City and "Worked at Forge?" on a screening page, agent advanced past it, the review-module captured 0 inputs, AgentRulesDB never learned. NativeFormFiller now snapshots every visible field's value right before clicking Next/Continue (textbox/combobox/radiogroup/checkbox/ textarea via role queries). live_review_applicator merges those snapshots into final_mapping (oldest → newest, later wins) so confirm_application can diff against agent_mapping and route to CorrectionCapture / AgentRulesDB. Also fixes the CV identity-bleed bug from the same run: - cv_templates: build_applicant_identity now treats empty ProfileStore values as missing and falls back to APPLICANT_PROFILE (phone was '' in user_profile.db.identity). - application_materials: _sanitize_location rejects JD-prose bleed (provided/required/wifi/setup/etc) and falls back to "Remote (UK)" or "United Kingdom". The Forge JD scrape gave location='remote, provided', which rendered into the CV contact line. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

Live regression on 2026-05-05 (JPMC Strategic Testing): three tabs in Chrome — Indeed JD, JPMC HCM JD, JPMC HCM apply form/section/1. The session correctly picked the apply tab via _find_in_progress_apply_tab and attached via prefer_url match. But _phase_observe blindly switched to pages[-1] on the first observe step, clobbering the deliberate attachment and grounding the page reasoner on the Indeed JD tab. Agent then looped on wait_human/bypass because it was reading the wrong page. Two locks added: 1. _should_auto_switch_tab returns False when driver._attached_existing_url=True (the prefer_url match was intentional) OR when the current page is already on an apply-path URL. 2. _pick_target_tab — when a switch is warranted (current tab is on a JD/listing page, e.g. SSO redirect from "Apply on company site"), prefer pages on apply-path URLs over plain pages[-1]. Both helpers reuse the path regex from live_review_applicator._find_in_progress_apply_tab so the navigator's heuristic agrees with the session's explicit pick. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

Live regression on JPMC 2026-05-05: agent missed all 4 Yes/No questions on the JPMC "Job Application Questions" page (18+, work auth, visa sponsorship, 5-day office). They render as <ul role="list"> + <li role="listitem"> + <button> with selected state encoded via CSS class (not aria-checked, not role=radio). Every existing scan strategy in field_scanner queried input/select/textarea/role=radio and returned 0 fields for the section, so the agent silently advanced past them with defaults the user then had to correct manually each time. Two changes: 1. _scan_dom_query now picks up the widget as type='list_button_radio'. Selected-state detection covers (a) aria-pressed/aria-current/ data-selected, (b) CSS class match (selected/active/is-selected/ chosen/current), (c) computed background-color non-transparent as visual fallback. Label-gated on aria-label / aria-labelledby and widget-shape-gated (every <li> must contain a <button>) to avoid capturing nav menus and breadcrumbs. 2. NativeFormFiller routes input_type='list_button_radio' to a JS click on the option button whose text matches the fill value. Locator points to the <ul>; a single page.evaluate handles discovery + scrollIntoView + click in one round-trip. The live correction values for jpmc.fa.oraclecloud.com are already in qdrant (screening_cache) from the ai_assist push earlier today, so the next JPMC Oracle HCM application will scan + match + fill autonomously across all 25 user-confirmed values. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

upload_files() was skipping any <input type='file'> whose label/id/name contained the substring "drag and drop". The skip was originally added to avoid LinkedIn's autofill-profile widget, but it over-matched: the canonical drag-and-drop CV upload pattern wraps a hidden file input in a <label> reading "Upload CV / or drag and drop your cv here". The agent saw the label, matched the skip, and silently advanced without uploading. Live regression on Revolut welovealfa.com 2026-05-05: the only CV upload field on /apply/upload-cv was bypassed for this exact reason. set_input_files() works on hidden file inputs by design — Playwright explicitly accepts them regardless of CSS visibility — so there's no need to skip them. The autofill skip is preserved (LinkedIn's profile- JSON import widget is a real false-positive case). Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

Live regression on Revolut welovealfa.com 2026-05-05: the apply page is just <Upload CV / drop zone> + Continue with no scanned text fields. The reasoner correctly classified it as application_form with action fill_and_advance, but fill_and_advance was NOT in TERMINAL_ACTIONS, so the navigator handled it inline via NavigationActionExecutor — which only executes the reasoner's pre-planned `field_fills` (text-only). The plan was empty (no text fields), the executor did nothing, the verifier saw no change → reflection looped to dismiss_overlay → wait_human → abort. The CV never reached upload_files() because that lives inside NativeFormFiller.fill_form() which only runs when the navigator hands back a TERMINAL action with page_type=APPLICATION_FORM. Treating fill_and_advance as terminal hands control to NativeFormFiller which always calls upload_files() regardless of scanned-field count, so pages with only file inputs are now handled correctly. _make_result also updated so fill_and_advance returns page_type=APPLICATION_FORM (was falling through to UNKNOWN). Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

Live regression on Revolut welovealfa.com 2026-05-05: Playwright's set_input_files attached the CV file (98165 bytes, logged success), but the page's react-dropzone component never picked it up — the input's files list stayed empty, the disabled "Upload CV" submit button stayed disabled, and no page transition fired. Most React drop-zone widgets (react-dropzone, custom dropzones) bind their handler to the WRAPPER element with the dropzone styling, not the hidden file input. Playwright's automatic event dispatch from set_input_files only fires on the input element itself, so wrapper listeners never see the change. Fix: after set_input_files, walk up the DOM up to 6 levels looking for ancestors whose className matches /drop|upload|cv|resume|file/ and dispatch bubbling input + change events on them. Also verify by reading el.files.length — if zero, log a warning so OPRAL learn / stuck recovery has signal that the upload silently dropped. Plus diagnostic logging in upload_files entry/scan path so the next regression is obvious from logs. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

Live regression on Revolut welovealfa.com 2026-05-05: • 5 screening questions rendered as <button role="switch" aria-checked="false">. Field scanner queried only inputs/selects/ radios → 0 fields, all 5 Qs skipped silently. • Min salary (GBP) / Max salary (GBP) inputs had empty <label> elements. The agent's LLM saw the JD's listed range £85,500-£118,000 on the page text and wrote those into both inputs as the user's salary expectation. Three structural fixes: 1. field_scanner._scan_dom_query now picks up [role="switch"] toggles with question label discovered via previous-sibling/ancestor walk (the Q lives in a sibling heading, not aria-label). Returns type='switch' with aria-checked/aria-pressed state. 2. field_scanner._scan_dom_query now picks up <input type='number'> whose ancestor text matches /salary|compensation|gbp|usd|gross| annual|per year|per annum/. Tags type='salary_number' and a salary_role hint (min_salary / max_salary / salary) inferred from text preceding the input within its parent. 3. screening_answers.lookup_user_salary is a new public helper that does substring match first, then token-Jaccard fallback so titles like "Software Engineer (Data)" match "software engineer" tier (£35k) instead of falling to default. Existing _resolve_role_salary delegates to it. NativeFormFiller dispatches: • input_type='switch' → click-to-toggle handler that reads aria-checked, no-ops when already in target state, fires single click otherwise. • input_type='salary_number' → lookup_user_salary(job_title) and fill the number input directly. Adds 5k buffer for max_salary. Job context propagated to per-input handlers via self._job_context (set at fill() entry from custom_answers["_job_context"]). Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

Add should_force_vision() — true when the DOM scanner returns <=10 fields on a page the reasoner classified as application_form with >=0.7 confidence. Trigger only; the augment call lands next. Live regression on Revolut welovealfa.com 2026-05-05: scanner found 9 fields, real form had 13+. Existing vision_gate skipped because reasoner confidence was 0.90 — predicate now flips that on sparse scans. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

Vision LLM gets a page screenshot + the existing scanner field list, returns any visible questions the DOM scanner missed. Tagged vision_only=True so callers can route them through the LLM-recovery fill path (label-string match won't reach them — these have no DOM anchor). Adapted from the plan: uses the existing get_openai_client + client.responses.create pattern (vision_tier.py) instead of smart_llm_call, since smart_llm_call expects a LangChain LLM and LangChain message format. Cost tracked via record_openai_usage. Cached by (url, screenshot-hash). Returns [] on any failure. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

When the multi-strategy scan returns <=10 fields on a page the reasoner classified as application_form with >=0.7 confidence, auto-augment via vision_augment_scan. Augmented fields are tagged vision_only=True; downstream filler routes them through the LLM-recovery fill path (label-string match cannot reach them since they have no DOM anchor). Reasoner hints stamped on page object in _phase_act so scan_fields can consult them without an import cycle. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

Schema for capturing DOM signatures of fields the scanner missed but a human / ai_assist correction filled. Keyed (domain, label, selector) with fix_count incremented on conflict. Patterns retrieved ordered by fix_count desc so the most-confirmed wins. Lives in jobpulse/form_engine/gotchas.py (the actual GotchasDB location — the plan referenced jobpulse/gotchas_db.py which doesn't exist). Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

Optional dom_signature dict on record_fix lets callers (Claude, the correction-capture pipeline, future agent recovery) record the DOM selector + widget type + ancestor classes whenever they fill a field the scanner missed. Results land in GotchasDB.widget_patterns keyed by domain so future visits learn from past fixes. Backwards compat: existing callers that don't pass dom_signature work unchanged. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

New first strategy: query GotchasDB.widget_patterns for the current domain, walk each stored selector, emit fields with the locator pre-attached. Pre-attached locators mean the dispatcher uses the learned widget directly — no label-string re-resolution that fails on synthetic labels. _merge_fields now prefers learned_pattern=True fields when the same (label, type) key is emitted by a generic strategy too. Wired into STRATEGIES tuple + _run_strategy switch (the actual dispatch shape — the plan assumed a list-of-callables that doesn't exist in this codebase). Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

When confirm_application diffs agent_mapping vs final_mapping and records a correction, also capture the field's DOM signature (selector + widget_type + ancestor_classes + aria_label) into GotchasDB.widget_patterns. Pulled from final_mapping[label + "__dom"] which is now populated by NativeFormFiller._snapshot_live_form_state on every per-page snapshot. Closes the feedback loop: every human/agent correction becomes a permanent agent capability for that domain. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

Walks the page's visible text in document order, returns question-shaped fragments with bounding-box y + CSS path. Heuristic filter: ends with '?' OR starts with 'Are you / Do you / What is / ...'. Excludes button text, nav verbs, and bare field labels (those have their own <label> already captured by shape-based scanners). Piece 1 of the semantic-first scanner — proximity match and widget classification land in follow-up commits. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

Two-tier search: (1) walk question's ancestors for fieldset/section/ [role=group] containing an interactive element; (2) fall back to pixel proximity <=400px below the question's y. Returns a dict with selector + match metadata. Ancestor match is strongest — widgets in the same fieldset are intentionally grouped with their label. Proximity match handles the common 'flat div soup' layout used by React form libraries. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

…S26) Treats the rendered form as the source of truth. After each form-page fill round, the verifier screenshots the form, asks Kimi K2.6 vision what is actually shown for each filled field, compares against the filler's claim, and (when correction is enabled) re-fills mismatches with cache invalidation routed through ai_assist_logger. Why: prior sessions surfaced TP-31/32/33/34/35 — five distinct propagation gaps in the metadata pipeline (cache cross-question, options dropped, help-text missing from prompt, third-person leaks, *-suffix label normalisation). Architecturally, the answer is to stop reconstructing what the form shows and just look at it. Module (`jobpulse/form_engine/vision_verifier.py`) - One Moonshot/Kimi-K2.6 vision call per chunk; lossless WebP encoding + vertical chunking with overlap so detail is preserved without cropping or resize. Kimi 4K rec respected. - Verdicts in a closed tier vocabulary: passed | mismatch_detected | correction_succeeded | correction_failed | vision_unavailable | skipped_no_expected_value. Recorded in data/semantic_decisions.db with decision_type='vision_verification'. - Phase B correction proposes new values ONLY when help-text gives a directive; refuses to guess on ambiguous mismatches (visa-style questions whose right answer depends on profile + JD context). - Successful corrections route through ai_assist_logger → CorrectionCapture + AgentRulesDB + screening_semantic_cache so upstream caches are invalidated (OPRAL rule 5). - Parallel chunk verification (asyncio.gather) and parallel correction proposals to keep latency bounded. - Artifact persistence (WebP chunks + verdicts JSON) under data/audits/vision_verifier/ for replayable human spot-check. - Kill switches: VISION_VERIFICATION_ENABLED (off by default) and VISION_VERIFICATION_CORRECT (off by default). Observe-only first. Hook (`jobpulse/native_form_filler.py`) - Inserted as step 8b in the per-page fill loop, outside the _is_submit_page() conditional so it runs every page. Best-effort — verifier failures never break the apply. Other changes - `shared/semantic_decisions.py`: DECISION_TYPES += 'vision_verification'. - `jobpulse/ai_assist_logger.py`: VALID_AGENTS += 'vision_verifier'. - `jobpulse/form_engine/field_mapper.py`: default vision model swapped to kimi-k2.6 (env-overridable). Live evidence in S26 showed the preview model line under sustained 429s while k2.6 responded. Tests (16/16 pass) - Unit: kill-switch, empty mapping, blank-value skip, tier mapping, unparseable JSON, client raises, correction success + learning routing, correction fail, decision-row write, retry on transient, retry exhausted, non-transient no-retry, WebP compression, oversized resize, MIME detection, chunked aggregation. Live evidence (recorded in audit doc) - RUN 4 Anthropic Greenhouse observe-only: verified=13 mismatches=5 cost=$0.0089 elapsed=189s. Manual spot-check 11/11 fields correct (100% read accuracy, well above 95% gate). - RUN 5 Anthropic Greenhouse Phase B: corrections_succeeded=2 (both AI Policy combobox variants flipped Select... → Yes), corrections_failed=3 (visa ×2 + pronunciation — vision correctly returned null for ambiguous cases). Learning chain confirmed end-to-end via field_corrections.db + screening_semantic_cache.db + agent_rules.db. - RUN 6 Graphcore Greenhouse cross-ATS: verified=16 mismatches=2 cost=$0.0032 elapsed=104s. Same code path, no per-ATS branches. Outcomes 1, 2, 3, 5, 6 ✅ verified. Outcome 4: cost ✅ ($0.0032–$0.0098 vs $0.05 ceiling), latency ⚠️ (post-parallelization projection ~103s vs 60s budget; tracked as follow-up). 🤖 Generated with Claude Code Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

… layers upstream of cache; F+G filed RULE A trace + live diagnostic showed the cache layer is not the fix point for the visa-sponsorship mismatch S26 RUN 4/5 surfaced on Anthropic Greenhouse. Production lookups query with non-empty hashes (location alone populates profile_state_hash); the 288 empty-hash legacy rows are zombies — written by 5 unhashed cache.cache() paths but never served. The wrong "No" originates upstream in _llm_answer, with two distinct root causes: (1) screening_answers.py:1066 hardcodes a UK-only visa_sponsorship_required="No" into the merged profile, dominating the LLM's reasoning on non-UK JDs; (2) jd_analyzer.extract_location defaults to "United Kingdom" when no UK city matches, mis-locating non-UK JDs at the JD-context layer. Per RULE C decision tree (cache fix is a no-op without the upstream fix → STOP + record BLOCKED-WITH-PLAN), this slice produces no code commit. The cache-keying slice itself is reclassified to P3 (hygiene) since the zombie rows are inert. S26-follow-up-F (P0, confirmed-firing) lands first; G (P1, conditional on F's outcome) gated by re-running the vision verifier. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

…ip_required hardcoding; H/I/J filed Removed the line in screening_answers._get_v2_pipeline() that baked WORK_AUTH.requires_sponsorship (a UK-only env flag) into the merged ScreeningPipeline profile as visa_sponsorship_required. That hardcoded constant dominated the LLM tier's reasoning on non-UK JDs, producing the wrong "No" the S26 vision verifier flagged on Anthropic Greenhouse. Synthetic-probe acceptance MET: with the fix, the same UK Graduate-Visa profile produces "Yes" on a US JD and "No" on a UK JD via pipeline.answer(visa_q, job_context=ctx) — both via the LLM tier reasoning from visa_status + job_context.country. Live-verifier acceptance NOT met: Anthropic apply still filled "No" for the visa question through a path that bypasses _llm_answer (no DIAG line emitted for visa during the live run). Confirmed not the visa_sponsorship_required hardcoding (the fix is in place and the synthetic probe proves it works) — the live "No" originates upstream, likely via decomposer-sub-question cache match or a sparser job_context than the probe used. Traced to follow-up-J. Also surfaced and filed: - H (P0): right_to_work hardcoding has the same bug shape; DE probe failed ('No' instead of 'Yes' on Germany JD), strongly indicating the LLM treats UK right_to_work as covering EU territory. - I (P1): 3 remaining regex-pattern rules in agent_rules.db violate the project's "No Regex for Classification (MANDATORY)" rule; rule_id=4 deleted in-slice (factually wrong: claimed UK permanent residency the candidate does not have). - J (P1): trace where the live apply's visa "No" actually originates given _llm_answer was not called. All 114 adjacent unit tests pass. Impact analysis clean (no external caller breakage). Diff is one line of production code + one comment block citing this audit entry. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

…osite verifier Replace the vision verifier's "screenshot → 3 vertical chunks → 3 parallel kimi calls → aggregate" pipeline with "DOM-coords per filled field → crop per field → tile composite → ONE kimi call". chunks_used=1 on all live runs, ≤30 KB WebP-lossless composites with per-field ordinal caption strips. What ships: - _FIELD_BBOX_JS extracts document-relative bbox per input (label + input + help-text union, with wrapper-label / far-label / tall-label rejection) - _resolve_label_locator cascade mirrors _fill_by_label primitives - _build_composite tiles per-field crops into ONE WebP-lossless with [NN] ordinal captions + red borders, height-cap auto-shrink - _build_prompt ordinal-indexed claim list; vision verdicts keyed by ordinal - _save_artifact records chunks_used + composite_path + composite_layout + per-panel bboxes (machine-checkable Outcome-1 evidence) - _call_vision tightened: 90s per-attempt timeout, max_retries=0 on the OpenAI client, 1 verifier-level backoff (worst case 184s vs pre-K 36 min) - native_form_filler passes _fields_by_label so verifier consults scanner selectors before falling back to get_by_label Live cross-URL evidence (Anthropic 19/19, Graphcore 18/18, Drweng 27/27): - O1 single-shot ✅ chunks_used=1 on all artifacts - O2 field-area only ⚠️ PARTIAL — Anthropic free-text + react-select fields bleed JD section headings; filed as K-followup-M - O3 ≤30s ⚠️ PARTIAL — Moonshot vision floor confirmed via probes (7-11s tiny prompt, 124s timeout on verifier-shaped 2k-char prompt against SAME composite); filed as K-followup-L - O4 zero timeouts ❌ — Moonshot returns APITimeoutError on every Greenhouse URL; failure now fails-fast (~184s) instead of 36-min compound storm - O5 zero missed fields ✅ — every claim has a resolved bbox panel - O6 spot-check / O7 visa-symptom ❌ BLOCKED on L (no content verdicts) K-L filed P0 to fix Moonshot reliability for verifier-shaped prompts. K-M filed P1 to clean bbox extraction for react-select / free-text layouts. Tests: 17/17 pass on test_vision_verifier (chunking test replaced with single-shot composite test); retry tests updated to new 2-attempt policy. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

Replace the soft "PARTIAL" verdict with explicit failure framing: "FAIL on free-text + react-select, PASS on tightly-labeled inputs" with the numerical 5/19 (≈26%) clean-panel count so future readers sizing K-followup-M get the right scope signal. No code change. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

…+ iframes Why --- S26-follow-up-L set out to get vision_verifier returning content verdicts across every active ATS adapter within G3 ≤ 60 s. The slice ships the architecture (vendor fallback wiring + adapter-agnostic iframe handling) and surfaces the latency floor empirically — Moonshot ~290 s on verifier prompts, OpenAI gpt-4o-vision exhausted (429 insufficient_quota), qwen3-vl ~160-300 s with parseable content (timing out at 300 s on 19-field forms). G3 ≤ 60 s NOT MET today — gated on a faster vision endpoint, not architecture. Code changes (`jobpulse/form_engine/vision_verifier.py`) -------------------------------------------------------- - _call_vision refactored into a two-provider pipeline. Primary stays Moonshot via get_openai_client() (backwards-compat per spec). New _call_provider helper does ONE attempt per provider (drops the pre-L 4 s backoff + retry — Moonshot's stalls are non-transient, retry doubles wall-clock without changing outcome). Fallback _get_fallback_client builds an OpenAI-SDK-compatible client (default Ollama qwen3-vl:4b at localhost:11434/v1) — triggers only on primary timeout. New env vars: VISION_VERIFIER_FALLBACK_MODEL, VISION_VERIFIER_FALLBACK_BASE_URL, VISION_VERIFIER_FALLBACK_TIMEOUT_S (defaults: qwen3-vl:4b / Ollama / 90 s; set MODEL=none to disable). - _resolve_label_locator now returns (locator, owner_frame). After the main-page cascade fails, iterates page.frames and re-runs the cascade in each child frame — unblocks iCIMS-class adapters with the same primitive (no `if platform == "icims"`). - _extract_field_bboxes routes frame-resolved locators through locator.bounding_box() (page-relative regardless of context) and skips the JS label-union — cleaner than translating frame coords per element. Main-page locators continue to use _FIELD_BBOX_JS unchanged. - _VISION_CALL_TIMEOUT_S default tightened 90 s → 25 s (single attempt). Probe evidence (rejected options stay rejected) ----------------------------------------------- - Option 1 (shrink schema): full schema returns at 290 s, minimal at 529 s on the same composite. REJECTED. - Option 2 (stream=True): TTFB doesn't fix total wall-clock for our workload. SKIPPED. - Option 3 (vendor fallback): SHIPPED. Architecture verified. - L4 (shadow DOM): no live regression evidence yet. DEFERRED to L-3. - L5 (iframes): SHIPPED. Tests (`tests/jobpulse/form_engine/test_vision_verifier.py`) ------------------------------------------------------------ - Replaced retry tests with single-attempt tests: test_retry_on_transient_429_then_success → test_primary_success_returns_content test_retry_exhausted_returns_unavailable → test_primary_failure_no_fallback_returns_unavailable - Autouse fixture sets VISION_VERIFIER_FALLBACK_MODEL=none so tests isolate the primary path (no reach for real Ollama). - 17/17 vision_verifier tests pass. Full tests/jobpulse/form_engine/ 244/245 (pre-existing test_diversity_keyword_fallback untouched). Audit doc (`docs/audits/2026-05-10-semantic-audit-verified.md`) --------------------------------------------------------------- - New S26-follow-up-L SHIPPED-PARTIAL section with diff summary, probe evidence, live-run evidence, G1-G10 verdict table, SG distance delta (+5pp mechanism, +3pp cross-ATS, +2pp per-real-run, +1pp OPRAL), and two follow-ups: L-2 (P0) to provision a faster vision endpoint, L-3 (P1) to run Phase A+B once L-2 lands. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

…for clean per-field crops The pre-M bbox-math + full-page-crop path bled JD body text into vision crops (M-1 artifact: 11/11 panels showed "Build automation Go..." instead of form field values on Graphcore live run). Root cause was `_FIELD_BBOX_JS`'s `el.labels[0]` + `aria-labelledby` union resolving to wrong DOM elements on Greenhouse demographic-survey widgets; PIL.crop dutifully cropped the reported coordinates which pointed at JD positions in the full-page screenshot. This replaces the bbox-math path with Playwright's native `ElementHandle.screenshot()` on a dynamically-resolved form-row container. New `_field_crop.py` carries a 5-tier resolver cascade (visible-target → closest fieldset → closest role=group → ancestor walk with label-containment → relaxed walk → element-fallback) — all adapter-agnostic, no `if platform == "X"` branches. Document-relative bbox keying collapses Greenhouse-style duplicate-labeled widgets into a single panel; the verifier still emits one FieldVerdict per original claim row pointing at the shared panel's observed value. Live evidence (Greenhouse Graphcore, the same URL that produced the M-1 smoking-gun): 11/11 verdicts went from `vision_unavailable` to `passed`. Duplicate-claim variant: 14 claim rows → 9 panels → 14 verdicts. Two more adapters (Lever Mistral, Ashby OpenAI) validated via dev-phase probe against open Chrome tabs. Remaining 8 adapters filed as M-2 follow-up (blocked by unrelated upstream LLM-stack bug in process_single_url's page reasoner, not the verifier). Tests: 18/18 vision_verifier (17 updated + 1 new dedup test); 245/246 form_engine baseline preserved (the 1 pre-existing failure is unrelated to this diff). Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

…fields, not JD bleed Adds the S26-follow-up-M section to the semantic-audit-verified doc with diff summary, before/after composite WebP filenames, per-adapter evidence table (Greenhouse + Lever + Ashby validated, 8 remaining adapters filed as M-2), G1-G9 verdict table, and SG2 distance delta. Also commits the scoping prompt (2026-05-11-vision-bbox-fix-prompt.md) that drove this slice. The dev-phase Telegram-reconfirmation loop referenced in Requirement 3 was never wired (user picked "Batch review at end" — replaced by Claude-side visual inspection); the audit doc records this so future readers don't re-introduce a human gate that doesn't belong in the autonomous loop. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

… after dedup Code-review pass on the M live evidence (1778592280_*.json) revealed a real bug: when dedup collapsed two claim rows into one panel, the FieldCrop's preserved `ordinal` was non-contiguous (e.g. [1,2,3,5,7,9]) but `_build_prompt` enumerated the panel list 1..N contiguously. The vision model keyed its verdicts to the captions it saw in the image ([01]..[05],[07]..) while the verifier's `by_ordinal` lookup keyed them to the prompt index (1..N). After the first dedup, every claim row shifted by one: visa-status verdict read "Man" (the gender panel's value), gender read "No" (the disability panel's value). Fix: use contiguous panel-positions (1..N) for BOTH the caption stamp AND the prompt enumeration. Add `original_ordinal` + `dedup_with` to panels_meta so the verdict-mapping (`panel_pos_by_claim_ordinal`) correctly routes each original claim row to its panel. Regression test: `test_non_contiguous_ordinals_after_dedup_align_caption_with_prompt` exercises [1,3,5] original ordinals + [2]/[4] dedup_with — fails on the pre-fix code, passes on this fix. Live re-verify on the duplicate-claim Graphcore artifact: visa-status now reads "Graduate Visa" (the actual form selection — `mismatch_detected` correctly fires because the claimed "Tier 4 (General) Student Visa" doesn't match). 14 verdicts, 2 correctly-detected mismatches, zero off-by-one shifts. 19/19 vision_verifier tests; 245/246 form_engine baseline preserved. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

…arer composite_layout naming The M-5 fix added `original_ordinal` to in-memory `panels_meta` but `_save_artifact` wasn't serializing it. Forensic audit of the sidecar JSON couldn't reconstruct claim-row → panel mapping after the fact (every entry showed `original_ordinal=None`). Also `composite_layout.panels_unresolved` was misleading after the M-era dedup landed — it counted dedup-collapsed claim rows together with genuinely-unresolved ones. Split into two metrics: - `claims_collapsed_via_dedup` — duplicate-bbox widgets sharing a panel (intentional; not a failure) - `claims_unresolved` — claims with no panel AND no dedup_with parent (TRUE coverage gap; should be 0 in healthy runs) Cross-check on live Greenhouse evidence after this fix: - 14 claim rows → 9 panels → 5 dedup-collapsed → 0 unresolved - reachable_claim_ordinals = [1..14], missing = [] (100%) Verified by /tmp/m_crosscheck.py: every scanner-fillable field with a non-empty DOM value maps to a captured panel (or an explicit dedup_with entry); zero genuine coverage gaps on the Greenhouse form. Lever / Ashby tabs were unfilled during the audit so they have no fillable-with-value claims to cross-check; their Phase-0 probe coverage is structural-resolver evidence only. 19/19 vision_verifier tests pass. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

…nAI-only kwargs for Anthropic/Gemini The page-reasoner binds `response_format={"type":"json_object"}` on get_llm() to force JSON output from Moonshot. When the multi-provider fallback chain falls back from Moonshot → claude-3-5-haiku-20241022, the bound `response_format` propagated unchanged to the Anthropic SDK, which raises: TypeError: Messages.create() got an unexpected keyword argument 'response_format' That made every adapter where Moonshot's vision returned a length-limit error fail at the page-reasoner with action=abort, which in turn made the M-1 URL matrix sweep impossible (process_single_url couldn't reach the verifier on a fresh tab). Fix: `_MultiProviderLLM.bind()` now identifies the provider family per slot and drops `response_format` for non-OpenAI providers. The provider's own structured-output mechanism (Anthropic tool use, Gemini schema) is the right channel for JSON enforcement on those vendors; markdown- strip retry remains the fallback when this isn't available. Also adds: - S26-follow-up-M-3 — new `option_label` tier in `_field_crop._resolve_form_row`'s cascade for Ashby-style per-option radios where each option is `<label><input type=radio> option text</label>`. Lifts Ashby ethnicity rows from `element_fallback` to `option_label` (clean label + radio). - docs/audits/2026-05-12-vision-verifier-coverage-prompt.md — M-2 slice prompt scoping the cross-adapter coverage sweep. 19/19 vision_verifier tests pass. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

…required fields User report (2026-05-12): the legal name field (\"Have you added your full legal name and surname...\") is visible to the form scanner but silently dropped by the production filler (logged as `fill ⊘ reason=no_mapping required=True`) because the screening pipeline doesn't have a category for these introspection questions. The verifier never saw it because it only verifies fields the filler CLAIMED to fill. That's a FILLER-coverage gap, not a verifier gap — but the verifier is the natural place to surface it, since it has both `filled_mapping` (filler's claims) and `field_metadata` (scanner's full output) in scope. Adds `scanner_unfilled_required` to VerifierResult + sidecar JSON. For every scanner field marked `required=True` that's NOT in `filled_mapping` (after stripping required markers for the `*` ↔ non-`*` Greenhouse duplicate-label case), surface a `{label, type, reason}` entry. This lets cross-adapter audits flag silent-drop required fields without re-scraping filler logs. Live evidence on filled Graphcore tab: 12 scanner-required fields surfaced as unfilled, including the legal-name combobox + text input the user pointed out. NOT in scope: actually fixing the filler to fill these. That's upstream of M (screening_pipeline / no-mapping fallback). M-4 just makes the gap auditable. 19/19 vision_verifier tests pass. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

…age + DOM pre-check + verified-state cache User-driven follow-up from 2026-05-12 verification session. Three coverage / efficiency gaps remain after M + M-2 + M-4: 1. Scanner-to-filler gap: scanner sees fields (legal name) the filler silently drops. M-4 surfaces this in the sidecar but doesn't fix the filler-side gap. N-1 makes the sidecar's bucketing complete so every scanner field has a verdict (filled-verified / scanner-saw-skipped / noise-excluded). 2. No DOM pre-check: every fill goes through 25-90s vision call even for text inputs where input_value() would resolve in <10ms. N-2 adds read_dom_value() in _field_crop.py and only escalates to vision for comboboxes + custom widgets. 3. No verified-state cache: every dry-run refills every field even when the DOM already has the right value. N-3 adds verified_fills.db keyed (domain, label, value) — the filler short-circuits on cache hit + DOM-still-matches, refills on cache miss or drift. Scoped as one slice because the cache (N-3) is fed by the verifier (N-2's DOM pre-check produces the same passed tier), and the scanner coverage (N-1) defines the universe of fields the verifier examines. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

…erifier sidecar The M-4 surfacing already listed scanner-seen required fields the filler skipped. N-1 extends this to a full ``scanner_coverage`` block in the sidecar JSON enumerating every scanner-discovered field in exactly one of six buckets: - filled_verified_passed (verdict tier == passed or correction_succeeded) - filled_verified_mismatch (mismatch_detected / correction_failed) - filled_vision_unavailable (vision_unavailable / skipped_no_expected_value) - scanner_saw_filler_skipped_required (M-4 surface, now split by required) - scanner_saw_filler_skipped_optional (NEW) - scanner_noise_excluded (button + file inputs) ``scanner_coverage.total`` equals the scanner output count; bucket sum equals total. Sidecar consumers (cross-adapter audits) can machine- check coverage from the JSON alone without re-reading filler logs. When ``field_metadata`` is None (fallback / non-scanner paths) the helper returns None so the sidecar emits ``scanner_coverage: null``. No per-platform branches; bucketing is purely type- and tier-driven. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

…on on deterministic types For deterministic field types the browser already knows the rendered value at fill time — vision is unnecessary. ``read_dom_value`` reads the value off the input directly; ``dom_value_matches_claim`` does the type-aware comparison: text/textarea/email/tel/url/number/password/search → input_value() checkbox → is_checked() (truthy-set) radio → is_checked() + value attr select / native_select → selectedOptions[0].textContent file → files[0]?.name (basename) combobox / custom_dropdown / multiselect → return None (vision) In ``_capture_field_crop`` the DOM pre-check runs after the input locator resolves but before the row resolution + screenshot. On match the crop is flagged ``resolve_method='dom_match'`` with ``crop_bytes=None``; the composite-build step (``crop_bytes is not None`` filter) excludes them naturally. ``verify_form_page`` partitions field_crops into dom_match + non_ dom_match. If every filled field DOM-matched, vision is skipped entirely; emitted verdicts are tier_reached='passed' with reason "DOM input value matched claim (no vision needed)". Otherwise the composite is built from non_dom_match_crops only; the verdict mapping loop short-circuits dom_match claim_idxs to passed without consulting the vision response. Headline win on a 14-field Greenhouse form: ~50% wall-clock cut by avoiding ~6-8 vision-worth-of fields whose values were already in the DOM. Combobox + custom_dropdown still go through vision because their displayed value lives in a sibling element via platform-specific selectors (read_dom_value returns None for those). No per-platform branches; readability is purely type-driven. Tests: test_dom_match_skips_vision_call, test_dom_mismatch_still_calls_vision. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

…er short-circuit New SQLite-backed cache of vision-verified fills. Key shape: ``(domain, label_norm, verified_value)``; columns: ``field_type``, ``ts``, ``method`` (``dom_match`` / ``vision``). Writes happen at the verifier — ``_persist_verified_fills_cache`` is called at every terminal site (all-DOM-matched early return + normal path). Only ``tier_reached == 'passed'`` records a row (strong- evidence tier); ``mismatch_detected`` invalidates the cached row so the next run does not trust a stale hit. ``correction_succeeded`` intentionally skips recording — a correction means the original claim was wrong; caching it under the corrected rendering would mislead. Reads happen at the filler — ``NativeFormFiller._try_verified_fills_ skip()`` runs at the top of ``_fill_by_label``. It looks up the cache; on hit, it re-reads the DOM via the same ``read_dom_value`` primitive (introduced in N-2) to verify the cached value is still rendered. Only if the DOM still matches does the filler short-circuit with ``success=True, skipped='already_verified'``. Types where DOM state is unreliable (combobox / custom_dropdown / multiselect) get ``read_dom_value=None`` and fall through to the normal fill path. Kill switch: ``VERIFIED_FILLS_CACHE_ENABLED=0``. Test isolation: ``VERIFIED_FILLS_DB_PATH`` env var (defaults to data/verified_fills.db). TTL helper ``prune(ttl_days=30)`` exposed for ad-hoc operator runs; not wired into the daemon's hourly tick — row growth is bounded by ``distinct_domains × distinct_labels × distinct_values`` which is small for a single-user pipeline. Cross-page collision on same-domain same-label is acknowledged (TODO ``page_hash``); the filler's DOM re-check catches drift in practice. Test: test_verified_fills_cache_short_circuits_filler exercises both the hit-skip path AND the drift-refill path. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

…llow-ups (O, N-1 prune, N-2 page_hash) Records the N slice as shipped: scanner-complete sidecar coverage, DOM-level pre-check skipping vision on deterministic types, and the verified-fills cache that short-circuits already-verified fills on subsequent runs. Includes: - Acceptance gates N-G1..N-G6 with status - Files touched (one-line each) - Latency / cost impact table (pre-N → first-N → cache-warm second run) - Out-of-scope items (legal-name screening-pipeline gap → S26-follow-up-O) - Follow-ups filed: S26-follow-up-O (screening pipeline introspection questions), N-1 follow-up (daemon prune wiring), N-2 follow-up (page_hash cache key) Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

yashb98 · 2026-05-12T15:22:57Z

S26-follow-up-N — SHIPPED (2026-05-12)

Four new commits on top of M-4 closing three coverage / efficiency gaps in the vision verifier.

What landed

5097a3a feat(form_engine): S26-follow-up-N-1 — sidecar scanner_coverage block enumerates every scanner-discovered field in 6 buckets (filled_verified_passed / mismatch / vision_unavailable / scanner_saw_filler_skipped_required / *_optional / scanner_noise_excluded). total matches scanner output count; bucket sum equals total. Machine-checkable from JSON alone.
af74d25 feat(form_engine): S26-follow-up-N-2 — DOM-level pre-check (read_dom_value + dom_value_matches_claim in _field_crop.py) short-circuits vision on deterministic types (text / textarea / email / tel / url / number / password / checkbox / radio / native select / file). Combobox / custom_dropdown still go through vision. If every filled field DOM-matches, vision is skipped entirely.
7d57c6e feat(form_engine): S26-follow-up-N-3 — jobpulse/form_engine/verified_fills_db.py SQLite cache (domain, label_norm, verified_value). Writes happen at verifier on tier_reached=passed (invalidate on mismatch). Reads happen at filler — NativeFormFiller._try_verified_fills_skip() at top of _fill_by_label consults cache + DOM re-check; short-circuits with skipped="already_verified" only when DOM still shows cached value.
22f277a docs(audits) — S26-follow-up-N — STATUS: ✅ SHIPPED section in docs/audits/2026-05-10-semantic-audit-verified.md with acceptance gates and follow-ups.

Tests

tests/jobpulse/form_engine/test_vision_verifier.py: 22/22 pass (19 existing + 3 new: test_dom_match_skips_vision_call, test_dom_mismatch_still_calls_vision, test_verified_fills_cache_short_circuits_filler). tests/jobpulse/test_native_form_filler*.py: 48/48 pass.

Expected impact (Greenhouse 14-field form)

Run	Vision call	Wall-clock
Pre-N	1 covering 14 fields	70-90 s
Post-N first	1 covering 6-8 residue fields	35-55 s
Post-N second (cache warm)	0 or 1 (tiny residue)	<5 s fill phase

Out of scope (filed)

S26-follow-up-O — extend screening_pipeline.resolve to handle introspection ("did you add X") / consent / agreement question categories so the legal-name combobox+text fields stop landing in scanner_saw_filler_skipped_required. N made the gap auditable; O fixes it.
N-1 follow-up: wire verified_fills.prune() into daemon hourly tick if cache size grows.
N-2 follow-up: add page_hash to verified_fills cache key (currently cross-page collision on same-domain same-label is caught by the filler's DOM re-check).

Verification still pending

Live ATS dry-run on Graphcore (or any Greenhouse form) — unit tests pass but no live sidecar evidence yet. The M-2 page-reasoner LLM-stack blocker is fixed per b3488e6/46eb6f9, so a live run is now possible.

🤖 Generated with Claude Code

…lState + resolver Closes the legal-name fill gap surfaced by N-1's ``scanner_saw_filler_skipped_required`` sidecar bucket. - Add ``ScreeningIntent.INTROSPECTION_CONFIRMATION`` enum member with 10 embedding seed prototypes covering "Have you added X?", "Did you upload X?", "Do you confirm X?" paraphrases. - Module-level ``classify_intent(question) -> str`` wrapper so tests + screening_pipeline don't have to remember the tuple shape. - New ``jobpulse/screening_session_state.py`` ``SessionFillState`` class with stripped-required-marker label normalization, fill recording, and ``references_present(candidate_labels)`` for the introspection resolver to consult. - ``ScreeningPipeline.answer`` accepts ``session_state`` kwarg; the new ``_try_introspection_answer`` runs BEFORE the semantic cache because caching session-derived answers across runs would be wrong. Returns Yes/No matched against field options via ``semantic_option_match``. - ``_extract_referenced_fields`` synonym dict maps "legal name" / "surname" / "contact information" / "resume" to canonical label tokens; the session_state.references_present check is final authority so synonym false-positives collapse to "No". Tests: 6/6 in test_screening_session_state.py, 5/5 in test_screening_pipeline_introspection.py. 98/98 existing screening tests still pass. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

…e retry Moves DOM verification from the post-page vision verifier into the filler itself. Every fill is now validated against the rendered DOM at fill time, not deferred to end-of-page vision. New surface on ``NativeFormFiller``: - ``_verify_fill_immediate(label, value)`` — runs ``read_dom_value`` (N-2 primitive) + ``dom_value_matches_claim`` against the just-filled input. Returns True (DOM match), False (mismatch), or None (combobox / custom_dropdown — vision handles end-of-page). Logs ``verified_via=dom`` on match for live-run grep audit. - ``_llm_regen_alternate(label, original_value)`` — bounded one-shot LLM call for an alternate when DOM keeps rejecting the filler's claim. Costs ~$0.001 per regen, off the hot path. - ``_record_session_fill(label, value, verified)`` — writes the outcome into ``self._session_state`` (no-op when session_state isn't attached, which the O-3 commit fixes). - ``_fill_with_validation(label, value)`` — 3-strike loop: strike 1: ``_fill_raw`` → verify → return on match/None strike 2: re-fill same value → verify (transient hydration race) strike 3: LLM-regen alternate → fill → verify bail: return ``deferred_to_vision`` so the per-page composite verifier still gets a final say - ``_fill_raw(label, value)`` — single-shot primitive. The old ``_fill_by_label`` body verbatim, sans the N-3 cache short-circuit which now lives in the new wrapper. - ``_fill_by_label(label, value)`` — thin wrapper: N-3 cache check then ``_fill_with_validation``. Tests (7 new): - DOM-matches → True - DOM-mismatches → False - combobox → None (deferred) - strike 1 first-try pass - strike 2 retry-same pass - strike 3 LLM-regen pass - three strikes bail to vision 48/48 existing native_form_filler tests still pass. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

… same session Adds an in-memory ``SessionFillState`` to ``NativeFormFiller`` so a single dry-run never re-issues a fill for a (label, value) pair the session has already verified. - ``NativeFormFiller.__init__`` now allocates a fresh ``SessionFillState`` per filler instance (one per dry-run / apply_job call). - ``_try_in_run_skip(label, value)`` runs at the TOP of ``_fill_by_label`` — cheaper than the N-3 cross-run DB lookup because it touches no disk. Two gates: session has filled the label AND verified=True AND the cached value equals the claim. On hit, logs ``fill ⊘ reason=already_verified_in_run`` and returns a success-skipped dict. - ``_record_session_fill`` from O-2 now actually writes (the session_state is no longer None). Lookup order in ``_fill_by_label`` is now: O-3 in-memory → N-3 disk-backed cache → O-2 fill+validate. Cheapest gate first. Tests (4 new): - second fill of verified field is skipped - unfilled field passes through - filled-but-unverified does not skip - different value does not skip (cache-miss on value change) 48/48 existing native_form_filler tests + 7/7 O-2 tests still pass. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

…s deferred-to-vision) Surfaces a per-field outcome report in the verifier sidecar showing WHERE verification happened. With O-2 and O-3 shifting verification into the filler, the verifier's existing ``scanner_coverage`` (N-1) is now an incomplete picture — fields verified at fill time never reach the verifier's tier-mapping loop. New sidecar block ``coverage_realtime`` (only emitted when the caller passes ``session_state``): - filled_verified_at_fill_time : was_verified == True - filled_deferred_to_vision : session has the label but verified=False (combobox etc.) - scanner_saw_filler_skipped_required / *_optional / scanner_noise_excluded: carried forward from M-4 / N-1. Invariant the audit doc machine-checks: ``sum(coverage_realtime buckets) == scanner_coverage.total`` Implementation: - ``verify_form_page`` takes a new ``session_state`` kwarg (default None for legacy callers). - ``_compute_coverage_realtime`` produces the dict or None. - ``_save_artifact`` accepts ``coverage_realtime`` and writes it to the payload. - All three terminal sites (all-DOM-matched early return, unparseable-vision early return, normal-path tail) pass it. Tests (4 new): - coverage_realtime present in sidecar when session_state supplied - bucket-sum invariant holds against scanner_coverage.total - verified field lands in fill-time bucket, NOT deferred bucket - no session_state → coverage_realtime null (legacy path) 22/22 existing test_vision_verifier.py tests still pass. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

…verification pending) Records the four-phase O slice as code-complete with all unit tests passing (194/194 across touched suites). Documents the remaining live URL verification work and the two pieces of wiring needed before it can execute (auto-confirm flag, specific Graphcore job URL). Filed as S26-follow-up-O-live (P0). Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

…46, cap form_recovery LLM at 30s, defer truncated-combobox picks to type-to-search Three pre-existing infrastructure bugs surfaced by the S26-follow-up-O live runs on Graphcore (2026-05-12). All fixes are infrastructure- layer, separate from the O slice itself. 1. **Kill Anthropic from the LLM provider fallback chain.** Per ``.claude/rules/jobs.md`` and ``shared/CLAUDE.md``, the rule is "Kimi mandatory" via Moonshot's OpenAI-compatible endpoint. The ``_PROVIDER_FALLBACK_CHAIN`` in ``shared/agents.py`` had ``["openai", "anthropic", "gemini"]`` — Anthropic was returning 401 in production because ``ANTHROPIC_API_KEY`` isn't set, which then propagated as ``All LLM providers failed`` and aborted apply_job at the page-reasoner stage (live run 5 evidence). New chain: ``["openai", "ollama"]`` — Kimi primary, local Ollama fallback (free, no credentials, already running for the embedder). ``OLLAMA_FALLBACK_MODEL`` env (default ``qwen3:32b``) picks the fallback model. 2. **Raise page_reasoner max_tokens from 500 → 4046 + add concise instruction to the system prompt.** Live run 5 evidence: Moonshot returned ``LengthFinishReasonError`` because the reasoning trace + JSON response together exceeded the 500-token budget on a 53-field Graphcore page. 4046 covers the trace overhead while staying well under Kimi's 8k cap. The prompt now also tells the model to keep every string ≤80 chars and reasoning to one short sentence, so semantic analysis gets the action + targets without prose padding. 3. **Cap ``form_recovery`` (and ``form_field_mapping``) LLM calls at 30 s in ``_direct_llm_call``.** Live runs 1+2 evidence: the orchestrator stalled post-upload at 0 % CPU for >1 min because the form_recovery LLM call (for the failed gender combobox) sits under the 180 s default OpenAI client timeout. New per-domain timeout dict caps these orchestrator-blocking domains at 30 s — slow recovery calls now fail fast and the vision verifier picks up the field instead. 4. **Defer truncated-option combobox picks to type-to-search.** Live run evidence (Graphcore School*): the LLM was given a TRUNCATED 100-option list ("Aalborg, Aalto, Aarhus, Abertay, Aberystwyth, ...") and forced to pick from it, even though "University of Dundee" was alphabetically out of view. The LLM picked "Abertay University" (same city, different uni) — wrong. New rule in ``screening_pipeline._llm_answer``: when ``field_type in {combobox, custom_dropdown}`` and ``len(options) > 50``, switch OFF the option-constraint and let the LLM emit the canonical profile value (free text). The filler's existing ``combobox_type_to_search`` technique then types the value, the widget filters server-side, and the correct option surfaces for selection. Short closed-set comboboxes (Yes/No, 5-option gender) stay on the option-constrained path. Tests: 48/48 pass on touched suites (O-1..O-4 tests + vision_verifier + session_state + native_form_filler validation/dedup). No regression in existing screening pipeline tests. This is a separate-slice fix on top of S26-follow-up-O. Unblocks the live-half verification of clauses (4-live), (5), (8) in a subsequent goal run, conditional on the three structural issues from O's goal also being resolved (single-page vs multi-page, dry-run vs corrections, SHIPPED-PARTIAL vs ✅ SHIPPED). Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

…Ollama only Per the "Kimi mandatory" rule in `.claude/rules/jobs.md` + `shared/CLAUDE.md`, the production LLM stack is Kimi (Moonshot's OpenAI-compatible endpoint) for cloud and Ollama (local qwen-family) as next-tier fallback. Everything else was either dead code after the last chain change, or actively harmful (Anthropic 401 in prod, Gemini unused, OpenAI-direct unused). Removed from `shared/agents.py`: - `_FALLBACK_MODELS` remap dict (gpt-5-mini → gpt-4o-mini etc.) — dead after the chain change; `_kimi_remap_model` covers all OpenAI-name → Moonshot remapping at call time - `_ANTHROPIC_MODEL`, `_GEMINI_MODEL` constants - `_make_anthropic_llm`, `_make_gemini_llm` factories - The `anthropic` / `gemini` branches in the chain builder - `langchain_anthropic` + `langchain_google_genai` imports - `_use_fallback_models` state flag (no callers after dict removal) Rewrote `shared/llm_fallback.py:FallbackLLM`: - Chain is now `["kimi", "ollama"]` (was `["openai", "anthropic", "ollama"]`) - `_call_openai` + `_call_anthropic` deleted - New `_call_kimi` uses KIMI_API_KEY + Moonshot endpoint directly - `_call_ollama` reads `OLLAMA_FALLBACK_MODEL` (default qwen3:32b) Replaced hardcoded vendor model names at three call sites: - `jobpulse/native_form_filler.py:_llm_regen_alternate` — `model="claude-haiku-4-5"` → drop the kwarg, use form_recovery domain default (`kimi-k2.6`) - `jobpulse/page_analysis/page_reasoner.py` cloud-fallback path — `model="gpt-4o-mini"` → drop the kwarg, `get_llm` picks Kimi - `jobpulse/page_analyzer.py:detect_page_type_with_vision` — `vision_model = "gpt-4o-mini"` → `"kimi-k2.6"` 48/48 tests pass on touched suites. No more silent vendor fallbacks to providers we don't have credentials for. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

yashb98 and others added 30 commits May 4, 2026 15:55

yashb98 and others added 17 commits May 11, 2026 09:18

yashb98 force-pushed the pipeline-correctness-fixes branch 2 times, most recently from 18c4bbc to 1b418c1 Compare May 12, 2026 16:31

yashb98 and others added 3 commits May 12, 2026 17:33

yashb98 force-pushed the pipeline-correctness-fixes branch 2 times, most recently from d94d648 to 81371a4 Compare May 12, 2026 16:46

yashb98 force-pushed the pipeline-correctness-fixes branch from 81371a4 to 344ad40 Compare May 12, 2026 16:52

yashb98 and others added 2 commits May 12, 2026 19:48

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

pipeline-correctness-fixes: cache-or-LLM audit + 250+ commits backlog - #2

pipeline-correctness-fixes: cache-or-LLM audit + 250+ commits backlog#2
yashb98 wants to merge 370 commits into
mainfrom
pipeline-correctness-fixes

yashb98 commented May 9, 2026

Uh oh!

yashb98 commented May 12, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Conversation

yashb98 commented May 9, 2026

Summary

What's on the branch (by commit prefix)

cache-or-LLM audit summary (most recent work, S1 → S8 + S7-EXT)

Test plan

Caveats / known risks

Uh oh!

yashb98 commented May 12, 2026

S26-follow-up-N — SHIPPED (2026-05-12)

What landed

Tests

Expected impact (Greenhouse 14-field form)

Out of scope (filed)

Verification still pending

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant