ci: run the repository-tooling tests when only agent-evals or a tooling test module changes

[yeongseon]

What this PR does

Makes the repo-tests job run when a pull request changes only agent-evals/** or only a repository-tooling test module. It also adds a guard so that a future module cannot fall out of the filter again.

Why

repo-marked tests (the REPO_TOOLING_MODULES in test/conftest.py) run only in repo-tests, because the offline lanes deselect them with -m "not repo". repo-tests starts only when the tooling path filter in ci.yml matches.

On main (ea9a4b7), test_workflow_path_impact.py's _selected() shows these paths selecting code (or docs) but never tooling:

• agent-evals/**
• test/test_agent_eval.py
• nine other repository-tooling modules

So a pull request that changed only an agent-evals task or fixture, or only one of those modules, ran those tests nowhere. Lane A found the agent-evals case after #961–#963. The guard found the other nine.

Alternative not chosen

• One broad test/** glob. It would send every test change to repo-tests. Listing the modules keeps the lane's selection precise, and the guard keeps the list complete.
• Running repo tests in the offline lanes. That would change the lane split from test: move Makefile and signal-handling tests out of the default offline run #594, which is out of scope.

Changes

• .github/workflows/ci.yml: the tooling filter now also lists agent-evals/**, test/conftest.py (where the registry lives) and the following modules, with a comment explaining why:
  • test/test_agent_eval.py
  • test_bench_artifact.py
  • test_check_docs_translation.py
  • test_docs_provenance_hook.py
  • test_docs_reason.py
  • test_make_integration.py
  • test_release_detect.py
  • test_sqlalchemy_range.py
  • test_support_claims.py
  • test_verify_dist.py
• test/test_workflow_path_impact.py:
  • cases for agent-evals/README.md (docs + tooling), an agent-evals task and fixture, and test/test_agent_eval.py;
  • test_every_repository_tooling_module_selects_the_tooling_lane, which asserts that every REPO_TOOLING_MODULES entry selects tooling.
• docs/CI_POLICY.md and docs/ko/CI_POLICY.md: state that tooling includes these paths. docs/llms-full.txt is regenerated.
• CHANGELOG.md: a ### CI entry.

Known differences / Not changed

• No change to any job, lane, marker, other filter group, ruleset or setting.
• ci.yml is a protected path, so this needs maintainer review.

Validation

• pytest -m repo test/test_workflow_path_impact.py → 48 passed. With the old ci.yml and the new tests, 5 tests fail, the guard among them, which lists the 10 missing modules.
• pytest -m repo test/ → 1064 passed, 12 skipped.
• Docs checks → OK:
  • check_docs_links.py
  • lint_changelog.py
  • check_docs_translation.py
  • check_agents_docs.py
  • check_support_claims.py
• generate_llms_full.py → llms-full.txt regenerated.

Development route: Claude Code (standalone agent); protected path (.github/workflows/ci.yml), needs a maintainer review before merge.

Refs #964

🤖 Generated with Claude Code

[Copilot]

🟡 Changes recommended

Registry changes do not trigger the tooling lane, allowing future omissions to bypass the new guard.

1 open finding

• Include conftest.py in tooling filter for completeness guard · New

What changed in this PR

Closes CI selection gaps for repository-tooling tests while preserving the existing lane split.

Changes:

• Adds agent-evaluation inputs and ten tooling test modules to the tooling filter.
• Adds path-selection cases and a module-completeness guard.
• Updates CI documentation and the changelog.

File	Description
test/​test_workflow_path_impact.py	Adds selection cases and completeness guard.
docs/​llms-full.txt	Refreshes generated CI documentation.
docs/​ko/​CI_POLICY.md	Updates Korean tooling-lane guidance.
docs/​CI_POLICY.md	Documents expanded tooling triggers.
CHANGELOG.md	Records the CI selection fix.
.github/​workflows/​ci.yml	Expands the tooling path filter.

🧠 Review effort: Balanced

---

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

[Copilot]

🔵 Needs a closer look

The protected CI workflow change requires maintainer review and approval before merge.

0 open findings

1 resolved since last review

• Include conftest.py in tooling filter for completeness guard

🧠 Review effort: Balanced

[yeongseon]

Maintainer review (delegated) of head 0654640.

• ci.yml: the tooling filter gains agent-evals/**, test/conftest.py (where REPO_TOOLING_MODULES lives) and the ten repository-tooling modules that previously selected no lane. Path additions only; no permission, trigger, secret, action, lane or marker change.
• The new guard test_every_repository_tooling_module_selects_the_tooling_lane fails against the old ci.yml, listing exactly the ten missing modules, and keeps future registrations honest; registering a module touches conftest.py, which now selects the lane that runs the guard. Path-impact cases cover agent-evals README/task/fixture, test_agent_eval.py and conftest.py.
• CI_POLICY (English and Korean), llms-full and CHANGELOG match.
• Copilot round 2 COMMENTED at this head with 0 open findings (round 1's conftest.py finding resolved); required checks pass. The advisory agent-review failure is the protected-path maintainer approval this review supplies.

Approved for merge.