Skip to content

Commit 9055ade

Browse files
igerberclaude
andcommitted
Add docs-tests.yml workflow; carve test_doc_snippets.py out of rust-test.yml
PR igerber#389 (Batch A) shipped six broken HAD doc snippets in 2026-04 that no CI run caught because rust-test.yml only triggers on rust/, diff_diff/, tests/, pyproject.toml, and the workflow file — none of which include docs/. PR igerber#396 patched the snippets but did not address the structural gap. This PR addresses it. Two changes: 1. New .github/workflows/docs-tests.yml — separate workflow that runs `pytest tests/test_doc_snippets.py -v` on a single ubuntu-latest / py3.14 / pure-Python runner. Triggers on docs/, diff_diff/, tests/test_doc_snippets.py, pyproject.toml, and the workflow file itself; same ready-for-ci label gate as rust-test.yml / notebooks.yml. Mirrors notebooks.yml's shape (the existing precedent for `pytest`-validated docs assets) so the two doc-validation workflows look like siblings. 2. .github/workflows/rust-test.yml: add --ignore=tests/test_doc_snippets.py to all three pytest invocations so doc snippets stop riding the code workflow. The Pure Python Fallback edit (line 193) is the only one that changes CI signal: that job runs from the repo root and was the ONLY place where test_doc_snippets.py actually executed. The two Rust-matrix edits (lines 158, 165) are defensive consistency: the matrix copies tests/ to /tmp/tests (rust-test.yml:138, 142) without docs/, so DOCS_DIR resolves to /tmp/docs/ which doesn't exist; the test collector silently skips every RST file via the guard at tests/test_doc_snippets.py:129. Adding --ignore there prevents the no-op from becoming a real run if anyone later adds `cp -r docs ...` to the copy steps. Each invocation now carries an in-YAML comment documenting which case it's the defensive vs behavior-changing edit. Verification: - python -c "import yaml; yaml.safe_load(open('.github/workflows/ docs-tests.yml')); yaml.safe_load(open('.github/workflows/ rust-test.yml'))" — both files well-formed. - pytest tests/ --ignore=tests/test_doc_snippets.py --ignore=tests/test_rust_backend.py --collect-only — 0 occurrences of test_doc_snippets in the collected set (was 115 cases collected when not ignored), confirming pytest accepts repeated --ignore flags as the existing line-193 pattern with --ignore=tests/ test_rust_backend.py already showed. After this PR opens, the workflow file itself triggers docs-tests.yml on its own change, providing the first end-to-end CI validation. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
1 parent 8e1282b commit 9055ade

2 files changed

Lines changed: 67 additions & 3 deletions

File tree

.github/workflows/docs-tests.yml

Lines changed: 56 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,56 @@
1+
name: Documentation Tests
2+
3+
on:
4+
push:
5+
branches: [main]
6+
paths:
7+
- 'docs/**'
8+
- 'diff_diff/**'
9+
- 'tests/test_doc_snippets.py'
10+
- 'pyproject.toml'
11+
- '.github/workflows/docs-tests.yml'
12+
pull_request:
13+
branches: [main]
14+
types: [opened, synchronize, reopened, labeled, unlabeled]
15+
paths:
16+
- 'docs/**'
17+
- 'diff_diff/**'
18+
- 'tests/test_doc_snippets.py'
19+
- 'pyproject.toml'
20+
- '.github/workflows/docs-tests.yml'
21+
schedule:
22+
# Weekly Sunday 6am UTC - smoke test that snippets still execute
23+
# against current upstream deps (mirrors notebooks.yml schedule).
24+
- cron: '0 6 * * 0'
25+
26+
permissions:
27+
contents: read
28+
29+
jobs:
30+
doc-snippets:
31+
name: Validate RST code snippets
32+
if: >-
33+
github.event_name != 'pull_request'
34+
|| contains(github.event.pull_request.labels.*.name, 'ready-for-ci')
35+
runs-on: ubuntu-latest
36+
37+
steps:
38+
- uses: actions/checkout@v6
39+
40+
- name: Set up Python
41+
uses: actions/setup-python@v6
42+
with:
43+
# 3.14 to mirror Pure Python Fallback (the only existing job
44+
# that actually ran these tests). notebooks.yml uses 3.11 for
45+
# nbmake compat, not relevant here.
46+
python-version: '3.14'
47+
48+
- name: Install dependencies
49+
# Keep in sync with pyproject.toml [project.dependencies] and [project.optional-dependencies.dev]
50+
run: pip install numpy pandas scipy pytest
51+
52+
- name: Run doc snippet tests in pure Python mode
53+
# PYTHONPATH=. lets the test import diff_diff directly from
54+
# source without invoking the maturin/Rust build (mirrors Pure
55+
# Python Fallback at rust-test.yml:189-193).
56+
run: PYTHONPATH=. DIFF_DIFF_BACKEND=python pytest tests/test_doc_snippets.py -v

.github/workflows/rust-test.yml

Lines changed: 11 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -155,14 +155,18 @@ jobs:
155155
- name: Run tests with Rust backend (Unix)
156156
if: runner.os != 'Windows'
157157
working-directory: /tmp
158-
run: DIFF_DIFF_BACKEND=rust pytest tests/ -q -n auto --dist worksteal -m ''
158+
# Doc snippet tests own .github/workflows/docs-tests.yml; ignore
159+
# them here to keep one workflow per surface and avoid double
160+
# execution (the matrix copies tests/ to /tmp/tests without
161+
# docs/, so this ignore is defensive on the Rust path).
162+
run: DIFF_DIFF_BACKEND=rust pytest tests/ --ignore=tests/test_doc_snippets.py -q -n auto --dist worksteal -m ''
159163

160164
- name: Run tests with Rust backend (Windows)
161165
if: runner.os == 'Windows'
162166
working-directory: ${{ runner.temp }}
163167
run: |
164168
$env:DIFF_DIFF_BACKEND="rust"
165-
pytest tests/ -q -n auto --dist worksteal -m ''
169+
pytest tests/ --ignore=tests/test_doc_snippets.py -q -n auto --dist worksteal -m ''
166170
shell: pwsh
167171

168172
# Test pure Python fallback (without Rust extension)
@@ -190,4 +194,8 @@ jobs:
190194
PYTHONPATH=. python -c "from diff_diff import HAS_RUST_BACKEND; print(f'HAS_RUST_BACKEND: {HAS_RUST_BACKEND}'); assert not HAS_RUST_BACKEND"
191195
192196
- name: Run tests in pure Python mode
193-
run: PYTHONPATH=. DIFF_DIFF_BACKEND=python pytest tests/ -q --ignore=tests/test_rust_backend.py -n auto --dist worksteal
197+
# Doc snippet tests own .github/workflows/docs-tests.yml; ignore
198+
# them here to keep one workflow per surface (this is the only
199+
# invocation that actually executes test_doc_snippets.py since
200+
# this job runs from the repo root, not /tmp/tests).
201+
run: PYTHONPATH=. DIFF_DIFF_BACKEND=python pytest tests/ -q --ignore=tests/test_rust_backend.py --ignore=tests/test_doc_snippets.py -n auto --dist worksteal

0 commit comments

Comments
 (0)