Both registrations of test_sglang_lease_identity are deletable at rc=0, and the control shows the SUITES array itself is ungated

[localai-bot]

Found by the fresh review of PR #1831 (row SGLANG-ORACLE-LEASE-WHEEL, W2). PRE-EXISTING and NOT fixed there: the control below shows it is a property of the SUITES array itself, not something W2 introduced, and repairing check-test-registration.py is a semantic checker change that needs its own row, spec and red-first evidence.

The claim

tests/scripts/test_sglang_lease_identity.py is registered twice, deliberately:

• scripts/agent-preflight.sh:176 — the SUITES array
• .github/workflows/ci.yml:341 — a named CI step

Two registrations should mean that deleting either one is caught. Neither is.

Mutation 1 — delete the SUITES entry

Run on row/SGLANG-ORACLE-LEASE-WHEEL-W2 at 85c247580:

$ python3 - # assert count==1, then delete '  test_sglang_lease_identity\n'
applied
$ git diff --stat -- scripts/agent-preflight.sh
 scripts/agent-preflight.sh | 1 -
 1 file changed, 1 deletion(-)
$ bash -n scripts/agent-preflight.sh   # still a valid script
$ python3 scripts/check-test-registration.py; echo rc=$?
OK: required regression tests have executable + CTest registration and the guard is wired into preflight/CI.
rc=0

Mutation 2 — delete the whole CI step

$ python3 - # delete the '- name:' block containing the invocation
deleting lines 331 .. 341
$ git diff --stat -- .github/workflows/ci.yml
 .github/workflows/ci.yml | 11 -----------
 1 file changed, 11 deletions(-)
$ python3 -c "import yaml; yaml.safe_load(open('.github/workflows/ci.yml'))"
YAML STILL PARSES
$ grep -c 'sglang_lease_identity' .github/workflows/ci.yml
0
$ python3 scripts/check-test-registration.py; echo rc=$?
rc=0

Both mutations are proven applied (non-empty diffstat; the mutated file still parses, so the checker was reading a valid file rather than failing on a broken one) and the tree was restored byte-for-byte after each.

Control — this is the array, not this suite

$ python3 - # delete '  test_tower_skip_rss_report\n' from SUITES instead
applied
$ git diff --stat -- scripts/agent-preflight.sh
 scripts/agent-preflight.sh | 1 -
 1 file changed, 1 deletion(-)
$ python3 scripts/check-test-registration.py; echo rc=$?
rc=0

An unrelated suite behaves identically. Any entry can be silently removed from the SUITES array and no gate notices, so the array is a list with no guard, and adding a suite to it buys a run today and nothing tomorrow.

Why this is not already covered

• #408 names suites executed by nothing, and diagnoses the root cause correctly: REQUIRED_TESTS in scripts/check-test-registration.py is a fixed map naming a C++ target, so the checker is a self-guard, not a population guard. This suite is executed, twice.
• #1730 names a suite with one registration, in ci.yml, and absent from SUITES. This one is in both places.

What neither states, and what the control above proves, is that deleting an entry from the SUITES array is itself ungated — so the "registered in TWO places, deliberately" pattern that .github/workflows/ci.yml:339-343 documents for the main-baseline suite buys no protection at all in either direction. Triage may prefer to fold this into #408; it is filed separately because the control is executable and because a spec has to name an owner for it.

Repair, when it is taken

Give check-test-registration.py a population rule rather than a fixed map: for every tests/scripts/test_*.py, require it to appear in SUITES and in a CI lane, and red when either falls away. That is a semantic checker change under "Changing the rules or a checker" — it needs a spec, a red-before mutation, and green-after evidence, and it will red on the twelve suites #408 already lists, which is the point.

Owner: this issue. Listed under ## Owed of sglang-wheel-in-lease.md. Related: #408, #1509, #1730, #1265.