feat(klaud): exempt baseline points retired by a dated MODELS.md deprecation

[adibarra]

Summary

Some Klaud Cold families can never validate: their published baseline still contains points that a later, deliberate retirement removed from the master config (for example the Single-turn 1k1k workload). Since #3764, select defers them before dispatch, but it does so on every wave, so these families can never be refreshed until a newer full public baseline exists.

This lets the frozen roster exempt a point only when the repository explicitly retired it after the baseline date. Every other missing point still fails coverage.

Which signal counts as a retirement

I checked how retirements are recorded today:

• perf-changelog.yaml: retirement PRs (Deprecate 1k1k benchmark configurations #2263, [Klaud Cold] Enact the August 3, 2026 scenario and precision deprecations / 执行 2026 年 8 月 3 日场景与精度下线 #2493, [Klaud Cold] Enact the August 6, 2026 Kimi-K2.5/2.6/2.7-Code retirement / 执行 2026 年 8 月 6 日 Kimi-K2.5/2.6/2.7-Code 完全退役 #2527, [Klaud Cold] Enact the September 8, 2026 DeepSeek-V4-Pro Single-turn 8k1k deprecation / 执行 2026 年 9 月 8 日 DeepSeek-V4-Pro 单轮 8k1k 场景下线 #2921, [Klaud Cold] Retire GLM-5.1 and its B200 TileRT config / [Klaud Cold] 退役 GLM-5.1 及其 B200 TileRT 配置 #3638) add no entries. Entries are sweep triggers whose config keys must still exist, and their descriptions are free text.
• Master-config history: shows when points disappeared, but a removal commit is only absence plus a commit message.
• configs/deprecated/ archives: deleted in [Klaud Cold] Remove pointers to the deleted single-node bash folders; delete configs/deprecated and amd_utils/deprecated #3463. AGENTS.md now says retired entries are deleted, not archived.
• docs/MODELS.md: the documented source of truth for retirements. AGENTS.md requires its retirement statements to match the active configs in the same PR. The Scenarios table gives each deprecated workload a date and a PR link, and the Model support matrix lists the deprecated scenarios per model.

So the planner uses MODELS.md at the candidate's base. It retires a frozen point only when all of these hold:

• the current family no longer generates any point with that point's single-turn ISL/OSL;
• the only Scenarios row for that ISL/OSL starts with Deprecated since YYYY-MM-DD ([#N](PR link)) (or the bold Deprecated for all models form), links a PR of this repository, and its date is later than the baseline date;
• the model's support-matrix row lists that scenario under deprecated scenarios and not under active scenarios.

Absence from the current family, a removal commit or a changelog entry retires nothing. A partial topology or concurrency removal has no such statement, so it still fails coverage.

Recording and reporting

• Baseline gains a strict retirements list. Each entry holds the scenario, date, PR link and the retired point keys. The model rejects retirements that are not after the baseline date, that name unknown or repeated points, or that would retire every point. Older records without the field still load.
• Retired points stay in the roster with their published values, so baseline-preflight.json and the frozen PR-body record keep them.
• The PR-body baseline table and the final report add a note that names the retired points with the scenario, date and PR link.
• missing_baseline_points skips only recorded retirements, so select, check-final, finish and recovery all apply the same exemption.
• The candidate prompt, klaud-reporting.md, klaud.md and MODELS.md (with their Chinese versions) describe the rule. The MODELS.md note tells editors that Klaud reads this wording.

Verification

• New tests in infx/tests/klaud/test_klaud_github.py:
  • select keeps a family whose only missing points were deprecated after the baseline, and the preflight records the retirement. It still defers with baseline-point-mismatch when the deprecation is dated on the baseline day, when MODELS.md has no deprecation, when the model row does not list the scenario, or when the family still runs that workload.
  • Final coverage exempts the retired point but still fails when an unretired point is missing. The body and the final report both name the retired point with its evidence.
  • The Baseline model rejects retirements it cannot justify.
• The positive select case, the final-coverage test and the model test fail without the implementation. The four still-deferred cases pass both before and after, as intended.
• infx/tests/klaud: 31 passed. ruff check infx and ruff format --check infx pass.
• Replay of the real frozen candidates from the latest wave (base 216e6558), with live public API data, producer regeneration and MODELS.md at that base. The re-resolved rosters match the wave's preflights exactly.

Family	Baseline	Missing points	Retired by	Coverage
dsr1-fp8-b200-sglang	2026-05-22	7 (1k/1k TP8 c1-c64)	Single-turn 1k1k, 2026-07-17, #2263	passes (failed before)
dsr1-fp8-mi355x-atom-mtp	2026-06-02	8 (1k/1k TP8 c4-c512)	Single-turn 1k1k, 2026-07-17, #2263	passes (failed before)
qwen3.5-fp4-mi355x-atom	2026-05-01	10 (1k/1k TP2/TP4)	Single-turn 1k1k, 2026-07-17, #2263	passes (failed before)
glm5.2-fp8-mi325x-sglang-agentic-mtp	2026-09-25	0	none	passes (unchanged)
minimaxm3-fp4-gb300-dynamo-vllm-agentic-mtp-disagg	2026-08-21	0	none	passes (unchanged)

Notes

• The MODELS.md wording is now load-bearing. If the Scenarios or support-matrix rows are reworded, retirements stop applying and those families are deferred again. Nothing is waived by accident.
• Like the rest of the frozen roster, the published baseline record comes from the candidate agent. The prompt forbids changing retirements.

Update

• check-final, finish and recovery no longer trust recorded retirements. The candidate agent can edit the PR-body record and the preflight, so these steps re-run the planner's rule on the recorded roster and baseline date against the family and MODELS.md at the candidate base, never the PR head. They fail unless it derives exactly the recorded retirements. This replaces the prompt-only rule in the last note.
• Each retirement now stores its model prefix and ISL/OSL, taken from the frozen point labels, so the re-check needs no public API call.
• Retirements may name only fixed-seq-len points. Retirement and baseline dates are compared as validated YYYY-MM-DD calendar days.
• The MODELS.md parser treats a table with a malformed row as unreadable. It groups ISL/OSL cells regardless of spacing, so a duplicate row stays ambiguous.
• Records without retirements omit the field, so code from before this PR can still read them after a rollback.
• The MODELS.md notes and the reporting guide state the exact conditions and describe the re-check.
• New tests: forged retirements fail check-final and the validation that finish and recovery run, and a correctly recorded retirement passes. Further tests cover each fail-closed MODELS.md case and the record format. On the previous commit, check-final accepts every forged record. Each fail-closed case fails when its guard is removed. infx/tests/klaud: 47 passed. ruff check infx and ruff format --check infx pass.

[claude]

Claude finished @adibarra's task in 57s —— View job

---

Review of PR #3766

• Gather context (diff of the merged commit 23789a4f9)
• Review planner, validation and reporting changes
• Post findings

LGTM - no blocking issues found.

What I checked

• Labels match MODELS.md. label_workload reads labels in the 1024/1024 c… form that point_label writes. The Scenarios table writes the same workload as 1024 / 1024, and the \s*/\s* regex accepts that spacing.
• select and the re-check use the same inputs. Both build current as point key → public_point(entry) from canonical_matrix at candidate.base. Their retirement lists come out in the same order, so the strict != comparison only fails when someone has actually changed the record.
• Every caller is updated. check_baseline_coverage is called from check-final (__main__.py:920), from Session validation (lifecycle.py:58) and from the tests, and all of them pass the new arguments. baseline_table is called only from render_body.
• The MODELS.md checks fail closed. A duplicate or malformed table, a mismatched PR number, a link to another repository, an invalid date, or a scenario that is also listed as active all retire nothing.
• Old records still load and older code can read new ones. Records without retirements default to [], and the serializer drops the empty field when writing.

Not verified

• I couldn't run pytest infx/tests/klaud because this job isn't allowed to run that command.

[claude]

Nothing blocking. The comments below are optional suggestions. There is no need to push a fix for them before merging.

Beyond the inline finding, I also examined two other candidate concerns raised during the bug hunt and ruled them out: the [model] = {...} unpacking crash in retirements() (reporting.py:1151) when a family currently generates zero points, and a baseline-point-reordering worry in check_baseline_coverage (reporting.py:945) — the retirement re-derivation always replays over the baseline's own recorded point order, not the live matrix's order, so reordering doesn't affect it.

Extended reasoning...

This PR adds a strict retirements list to the frozen Klaud baseline model and makes check-final/finish/recovery re-derive it from MODELS.md at the candidate base to fail closed against a tampered record, touching reporting.py, validation.py, lifecycle.py, main.py and several docs. It is security/trust-sensitive anti-forgery validation logic and a large, intricate change; one confirmed reporting-duplication bug (a retired-and-failed point listed under two separate Note lines) was found and filed inline. I additionally traced two other candidate crash/correctness concerns in the retirement derivation code and ruled both out from reading the surrounding logic.

[claude]

🟡 (optional) Readers of the PR body and final report can see one baseline point listed under two separate 'Note:' lines, duplicating and muddling its status. In baseline_table (reporting.py:511-516), a point that is both retired and has result != "passed" (e.g. "unavailable", which resolve_baseline assigns to legacy points it adds from the historical producer family with no matching published row) is appended to issues under both point.result and retired[point.key]. Fix: when point.key in retired, skip or merge the result-based note so each point contributes exactly one explanatory line; the same two-bucket append pattern is the root cause, not just this one site.

Why this was flagged

Trigger: a frozen baseline point whose producer-revision entry was never matched to a published API row (result="unavailable", set in resolve_baseline's Point(... result="unavailable") fallback at reporting.py:1212-1220) and whose workload is later retired via a MODELS.md deprecation after the baseline date. In baseline_table, the for-loop at reporting.py:511-516 appends shown to issues[point.result] (line 514) and separately to issues[retired[point.key]] (line 516) for the same point, since both conditions are independent ifs, not elif. On the base branch retirements did not exist, so a point could only ever get one Note. Nothing in render_body/baseline_table dedupes or merges per-point reasons, so the rendered PR body and klaud.md-documented report show the same label twice with differing, redundant explanations.

Verification: nit. The double-listing is real and reachable, but cosmetic. In baseline_table (reporting.py:511-516) the two appends are independent with no elif/dedup: line 514 adds shown under point.result when point.result != "passed", and line 516 adds the same shown under retired[point.key]. note_lines (reporting.py:359-363) emits one **Note:** line per reason key, so a point meeting both conditions appears in two Note lines.