test(hy3): Hy3 Tier-1 5th family in integration matrix + Ultra-only docs (PR 3 of 3)

[raullenchai]

Rebased onto main — PR-2 (#1070, parsers) + the P0 leak fix (#1075) are both merged. This branch was originally cut from feat/011.0-hy3-parsers; it has now been rebased --onto origin/main so only the PR-3 test/docs commits replay. Diff-vs-main is exactly the 4 PR-3 files (docs/reference/models.md + tests/integrations/{README.md,conftest.py,test_hy3_offline.py}) — no parser or engine code (PR-2's parser files are already in main and correctly dropped out of this diff). Ready for review.

Why

Hy3 (Tencent Hunyuan 3) landed as the Tier-1 5th family in #1069 (vendor) + #1070 (parsers), but the always-on integration matrices still only knew four families. Without this PR the family has zero CI coverage: the live matrix cells would either be missing entirely or, worse, try to boot the 166 GB hy3-preview-4bit on non-Ultra CI and OOM. This PR closes that gap the same way the DeepSeek V4-Flash Ultra-only precedent did — register Hy3 as a strict-xfail column (so an XPASS is a deliberate tripwire if the model ever becomes always-on-bootable) and add a real, sub-second, model-free offline parser test that gives the family genuine per-PR coverage. Refs #1069, #1070. Completes the 3-PR Hy3 series.

PR-3 of the Hy3 (Tencent Hunyuan 3) series. PR-1 (#1069, merged) vendored the 295B/21B-active MoE + hy3-preview-4bit alias; PR-2 (#1070, open) adds the tool + reasoning parsers. PR-3 adds Hy3 as the Tier-1 5th family across the agent + framework integration matrices, plus the Ultra-only launch docs. Strictly test/docs layer — no parser or engine code touched (verified: only tests/integrations/** + docs/reference/models.md).

What changed

A/B — matrix family registration + strict-xfail cells (conftest.py)

• hy3-preview-4bit registered as the 5th _FAMILY_ALIASES entry (Ultra-only / 166 GB / weekly-Golden-Path reason).
• All 14 Hy3 cells (11 agents + 3 frameworks) marked xfail(strict=True) via pytest_collection_modifyitems, mirroring the DeepSeek V4-Flash single-node-infeasible precedent: 166 GB weights + the G11 100 GB free-disk floor > 256 GB M3 Ultra. Per G8 this is strict-xfail, not plain skip — an XPASS forces a revisit if a smaller quant or bigger Mac ever makes always-on CI feasible.
• Family map recognises hy3 / hunyuan-3 served ids.

C — offline parser-level integration test (test_hy3_offline.py) — the CI-runnable value-add

Drives captured Hy3 wire strings (from the same 2026-07-09 REAP50/75 spike that seeded PR-2's unit tests) through the hy_v3 tool + reasoning parsers without booting the 166 GB model, asserting the OpenAI-API-shape contract: tool_calls array well-formed, <think> reasoning routed to its own channel, no content leak, and a composed reasoning-then-tool-call turn. 8 tests, sub-second, runs in the normal pytest tests/ sweep.

D — Ultra-only docs

• docs/reference/models.md: new "Ultra-only: Hunyuan 3 (Hy3)" section with the min_memory_gb: 192 pre-download warning example + LM-family table row.
• tests/integrations/README.md: Hy3 column in both status matrices, family-alias row, cell counts (56 → 70), offline-test note.
• Verified the min_memory_gb: 192 warning (PR-1 wiring, cli.py::_check_alias_min_memory) actually fires for hy3-preview-4bit on a sub-192 GB Mac — no gap, purely documented.

Local verification (post-rebase, against the merged PR-2 parser)

• pytest tests/integrations/test_hy3_offline.py → 8 passed — re-run against the now-merged hy_v3 parser (5 codex rounds of literal-close / reasoning-leak / brace-depth fixes); captured fixtures still match the merged contract, no assertion change needed.
• All 14 Hy3 matrix cells confirmed carrying strict=True xfail markers (verified via collection introspection); cells skip cleanly with no matching server — no false red.
• ruff check vllm_mlx/ tests/ + ruff format --check vllm_mlx/ tests/ clean under ruff 0.15.21 (CI-pinned version).

Notes for reviewer

• skip-version-bump label applied — no pyproject.toml change.
• If PR-2 changes parser wire behavior mid-flight, the captured fixtures in test_hy3_offline.py may need a re-sync at rebase time (single fixture block at top of file) — expected, not synced live.

🤖 Generated with Claude Code

[github-actions]

PR #1072 validation scorecard

Title: test(hy3): Hy3 Tier-1 5th family in integration matrix + Ultra-only docs (PR 3 of 3)
Author: raullenchai
Diff: 4 file(s), +583/-48 LOC, blast radius: medium

Verdict: MERGE-SAFE

step	status	summary	time
fetch	PASS	4 files, +583/-48 LOC, blast=medium	1.6s
test_plan_check	PASS	no checkbox-style test plan found	0.0s
cl_description_quality	PASS	title OK + body has rationale (4363 chars)	0.0s
supply_chain	PASS	no hooks touched, no suspicious patterns, deps clean	0.0s
test_env_check	PASS	installed trusted-pins (2 missing plugin(s) recovered)	4.4s
codex_review	skip	codex CLI not found on PATH (install: npm i -g @openai/codex)	0.0s
lint	PASS	clean (2 file(s))	0.0s
targeted_tests	PASS	8 passed in 1.38s (in 2 target file(s))	2.0s

[raullenchai]

codex converge disposition (rebase + finalize round)

Ran pr_validate (real codex, /opt/homebrew/bin/codex) on the rebased branch, then a second adversarial round on the gpt-5.6-generation model per operator directive. Disposition of every finding:

Fixed (genuine):

• Parser-import skip guard → hard import (test_hy3_offline.py). The _tool_parser/_reasoning_parser factories wrapped imports in try/except pytest.skip — a vestigial guard from when this file predated the now-merged PR-2 parser (feat(hy3): tool parser + reasoning parser + reasoning_effort default override (PR 2 of 3) #1070). Post-merge the hy_v3 parsers are permanent, so a parser-import regression or deletion must fail the always-on offline test, not silently skip it green. Dropped the now-unused pytest import.
• Cell-count comment 11 → 14 (11 agents + 3 frameworks) in conftest.py; documented that the matrix parametrizes by a single family arg so every Hy3 nodeid is exactly [hy3].
• PR body rationale — added ## Why + Refs #1069/#1070 (fixes cl_description_quality).

Refuted (arguing against merged, intentional design — not a correctness bug):

• "xfail(strict=True) will XPASS-fail the weekly Golden Path run." This is the intended tripwire, identical to the already-merged DeepSeek (conftest.py:483) and gpt-oss (:496) blocks. The merged gpt-oss block explicitly documents it (:400-403: "this strict-xfail will XPASS and force the marker to be removed"). Making it env-conditional would diverge from two in-file precedents and defeat the documented mechanism. There is also no committed Golden Path workflow to gate against. Left as-is per G8.
• gpt-5.5-round "nodeid [agent-hy3] won't match" — false positive: the matrix parametrizes by a single family value, so ids are always the bare [hy3] (verified via collection: 14/14 cells carry the strict-xfail marker). Same bare-token match the merged blocks use.
• "test builds the OpenAI envelope itself, doesn't exercise server serialization" — the offline test's declared scope (docstring) is the parser-level API-shape contract, deliberately model-free/server-free; the parser's {name, arguments} output IS the contract the server serializes. Out of scope by design.

Known pre-existing red (not this PR): full_unit fails only on test_readme_quickstart_mentions_vision_extra — verified failing on clean main with zero PR-3 changes (touches top-level README, which this PR does not modify).

Local: offline suite 8/8 green; ruff 0.15.21 (CI-pinned) clean; diff-vs-main is exactly the 4 PR-3 files.

[raullenchai]

codex converge — round 2 (gpt-5.6 generation)

Second adversarial round (gpt-5.6-sol; note: literal gpt-5.6 is 400-rejected by codex+ChatGPT auth, gpt-5.6-sol is the valid gpt-5.6-family model and the codex config default). Surfaced 5 findings; disposition:

Fixed (genuine — commit fe1ef477):

• Parsers now resolved via alias config — _tool_parser/_reasoning_parser read the parser name from resolve_profile("hy3-preview-4bit") instead of the literal "hy_v3", so the test also guards the alias → parser wiring (fails if the alias stops declaring/repoints the parsers).
• Reasoning-channel cleanliness asserted — the reasoning tests now assert the extracted reasoning field is itself tag-clean (exact payload + no <think> tag + no :opensource suffix), not only content. Verified against the merged parser (it returns reasoning stripped) — asserting correct behavior, not downgrading (G8).
• Disk/RAM rationale de-conflated — the infeasibility comment now evaluates the RAM constraint (~156 GB peak vs 256 GB unified memory) and the disk constraint (166 GB weights + G11 100 GB free-disk floor) separately, and states plainly that a supported M3 Ultra CAN boot Hy3 (weekly Golden Path) — only always-on per-PR runners cannot.

Refuted (design/scope arguments vs merged precedent, not correctness bugs):

• xfail(strict=True) XPASS-on-Golden-Path (appeared in all 3 rounds) — this is the intended tripwire, byte-identical to the merged DeepSeek (conftest.py:~490) and gpt-oss (~500) strict-xfail blocks. The merged gpt-oss block explicitly documents it: "this strict-xfail will XPASS and force the marker to be removed." No committed Golden Path workflow exists to gate against. Changing it would diverge from two in-file precedents and defeat the documented mechanism.
• "exercise production response serialization" — out of this file's declared scope (parser-level API-shape contract, deliberately server/model-free). The parser's {name, arguments} output IS the contract the server serializes.

All fixes keep the offline suite 8/8 green and ruff 0.15.21 clean.

[raullenchai]

codex converge — round 3 (final) + convergence note

Third gpt-5.6-sol round after the r2 fixes. Findings dropped to 3 BLOCKING + 2 NIT; the r2 fixes (alias-resolved parsers, reasoning-channel cleanliness, disk/RAM de-conflation) are gone from the list.

Fixed the 2 NITs (commit 092f727f):

• docs/reference/models.md: reworded the Ultra callout — 192 GB is the enforced min_memory_gb floor (no chip-gen check), 256 GB M3 Ultra is the validated config. Matches actual runtime behavior.
• tests/integrations/README.md: "Current cell status" heading now states the matrix spans through 0.11.0 while attributing the PASS/XFAIL data to the 2026-07-06 0.10.2 pilot.

3 remaining BLOCKINGs — all refuted (recurring design/scope arguments vs merged precedent, not correctness bugs):

1. strict-xfail XPASS on Golden Path (4th appearance) — the intended tripwire, identical to the merged DeepSeek + gpt-oss blocks; the gpt-oss block documents it verbatim. No committed Golden Path workflow exists.
2. "guard pytest.skip pre-empts the xfail, so cells skip not xfail in always-on CI" — verified this is the exact same behavior for the merged DeepSeek and gpt-oss strict-xfail cells (they also skip on a mismatched/absent server). The strict-xfail only activates when the matching server is booted (Golden Path). Consistent, intended, family-agnostic harness behavior.
3. in-test OpenAI envelope (3rd appearance) — out of this file's declared parser-level API-shape scope.

Convergence call: 3 rounds, findings 4→5→3-BL with the substantive ones fixed each round; the survivors are the same design/scope debates re-raised each round, each provably consistent with merged precedent. Per project SOP (codex diminishing-returns → converge once evidence is met; G8 — don't cave to policy-arguments framed as bugs) I'm holding here. Quality gate met: offline suite 8/8, GH checks green, ruff 0.15.21 clean.

Model note: operator asked for gpt-5.6; that literal id is 400-rejected by codex under ChatGPT auth ("model not supported"). All gpt-5.6-generation rounds ran on gpt-5.6-sol (the valid family model + codex config default). pr_validate's own gate ran its pinned gpt-5.5 and converged to the same single design-argument BLOCKING.