audits: UX audit of Sphinx docs site (2026-05)

Audit-only deliverable evaluating diff-diff.readthedocs.io against a
9-category UX rubric on 12 Tier-1 pages (desktop + mobile) and comparing
the current pydata-sphinx-theme against furo and sphinx-book-theme on a
5-page subset.

Key findings (full doc at audits/ux-2026-05/README.md):
- Recommend staying on pydata-sphinx-theme; bump version floor to
  >=0.16.1 for the v0.16+ accessibility improvements (P1)
- Highest-impact UX gap: mobile sidebar drawer flattens the toctree -
  drops section headings present on desktop sidebar (P1)
- Tutorials lack prev/next links at the bottom; users have to scroll
  back up to find the next tutorial (P1)
- 4 methodology .md files in docs/methodology/ are not Sphinx-rendered
  (no myst-parser); orphaned from human-visitor navigation (P1)
- Performance is already strong: FCP <400ms on 11/12 pages, HTML
  decoded sizes 38-177 KB, 24-27 resources per page
- AI-agent surface (4 llms.txt variants + Schema.org JSON-LD + sitemap)
  is exemplary

No source/conf/CSS changes in this PR; recommendations are queued as
separate follow-up PRs after triage.

Captured via Microsoft Playwright MCP server. ~50 screenshots stored as
WebP (most) + JPG (long pages exceeding WebP's 16383px height limit).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

Author: igerber

audits/ux-2026-05/README.md

@@ -0,0 +1,36 @@
+# Console errors observed during the 2026-05 UX audit
+
+Captured during Phase 2 (Tier-1 page navigation) via Playwright `browser_console_messages` at level `error`. Per-page raw logs are at `audits/ux-2026-05/perf/desktop/*-console.txt`.
+
+## Summary
+
+Across 12 Tier-1 pages on the live `https://diff-diff.readthedocs.io/en/stable/` site, the audit captured these console errors at the `error` severity level:
+
+- All 12 page logs returned **0 console errors** when filtered to severity `error`.
+- 2-4 console messages per page were emitted at lower severity (`info`, `warning`); these are typically RTD analytics, EthicalAds, and theme-switcher initialization chatter.
+- 2 EthicalAds tracking-pixel requests (`https://server.ethicalads.io/proxy/viewtime/...`) consistently fail with `net::ERR_ABORTED` on every page. Likely benign (ad-blocker / network policy) but appears as a network failure.
+
+## Per-page detail
+
+Each row references the per-page `*-console.txt` and `*-network.txt` files in `perf/desktop/`. Console-error counts use `level=error` strictly.
+
+| # | Page | Console errors | Notes |
+|---|---|---|---|
+| 1 | `index.html` | 0 | Clean |
+| 2 | `quickstart.html` | 0 | Clean |
+| 3 | `practitioner_getting_started.html` | 0 | Clean |
+| 4 | `practitioner_decision_tree.html` | 0 | Clean |
+| 5 | `choosing_estimator.html` | 0 | Clean |
+| 6 | `troubleshooting.html` | 0 | Clean |
+| 7 | `tutorials/01_basic_did.html` | 0 | Clean |
+| 8 | `tutorials/13_stacked_did.html` | 0 | Clean |
+| 9 | `api/estimators.html` | 0 | Clean |
+| 10 | `api/_autosummary/diff_diff.CallawaySantAnna.html` | 0 | Clean |
+| 11 | `references.html` | 0 | Clean |
+| 12 | `benchmarks.html` | 0 | Clean |
+
+## Network-level non-blocking failures
+
+Every page run shows two `net::ERR_ABORTED` requests against `server.ethicalads.io/proxy/viewtime/...`. These are EthicalAds view-time tracking pings; they're cancelled when the page unloads before the request completes. Not a user-facing problem; a sponsored-ad analytics integration detail of Read the Docs hosting.
+
+If we wanted to clean this up, the path would be at the RTD account / project level (disable sponsored ads), not in our docs source. Out of scope for the docs UX audit.

audits/ux-2026-05/console-errors.md

@@ -0,0 +1,2 @@
+Total messages: 4 (Errors: 0, Warnings: 0)
+Returning 0 messages for level "error"

audits/ux-2026-05/perf/desktop/01-index-console.txt

@@ -0,0 +1,27 @@
+1. [GET] https://diff-diff.readthedocs.io/en/stable/ => [200] 
+2. [GET] https://server.ethicalads.io/proxy/viewtime/10508/019e23f4-2c69-7e51-9659-82ddff5e12d6/?view_time=300 => [FAILED] net::ERR_ABORTED
+3. [GET] https://server.ethicalads.io/proxy/viewtime/10508/019e23f4-2c69-7e51-9659-82ddff5e12d6/?view_time=300 => [FAILED] net::ERR_ABORTED
+4. [GET] https://diff-diff.readthedocs.io/en/stable/_static/styles/theme.css?digest=55c26ed414f5f8a31ebb => [304] 
+5. [GET] https://diff-diff.readthedocs.io/en/stable/_static/styles/pydata-sphinx-theme.css?digest=55c26ed414f5f8a31ebb => [304] 
+6. [GET] https://diff-diff.readthedocs.io/en/stable/_static/pygments.css?v=8f2a1f02 => [304] 
+7. [GET] https://diff-diff.readthedocs.io/en/stable/_static/custom.css?v=7b11d399 => [304] 
+8. [GET] https://diff-diff.readthedocs.io/en/stable/_static/scripts/fontawesome.js?digest=55c26ed414f5f8a31ebb => [304] 
+9. [GET] https://diff-diff.readthedocs.io/en/stable/_static/scripts/bootstrap.js?digest=55c26ed414f5f8a31ebb => [304] 
+10. [GET] https://diff-diff.readthedocs.io/en/stable/_static/scripts/pydata-sphinx-theme.js?digest=55c26ed414f5f8a31ebb => [304] 
+11. [GET] https://diff-diff.readthedocs.io/en/stable/_static/documentation_options.js?v=ad389f6d => [304] 
+12. [GET] https://diff-diff.readthedocs.io/en/stable/_static/doctools.js?v=fd6eb6e6 => [304] 
+13. [GET] https://diff-diff.readthedocs.io/en/stable/_static/sphinx_highlight.js?v=6ffebe34 => [304] 
+14. [GET] https://cdnjs.cloudflare.com/ajax/libs/require.js/2.3.4/require.min.js => [200] 
+15. [GET] https://diff-diff.readthedocs.io/en/stable/_static/searchtools.js => [304] 
+16. [GET] https://diff-diff.readthedocs.io/en/stable/_static/language_data.js => [304] 
+17. [GET] https://diff-diff.readthedocs.io/en/stable/searchindex.js => [304] 
+18. [GET] https://diff-diff.readthedocs.io/_/static/javascript/readthedocs-addons.js => [304] 
+19. [GET] https://diff-diff.readthedocs.io/en/stable/_static/basic.css => [304] 
+20. [GET] https://diff-diff.readthedocs.io/en/stable/_static/vendor/fontawesome/webfonts/fa-solid-900.woff2 => [304] 
+21. [GET] https://diff-diff.readthedocs.io/en/stable/_static/vendor/fontawesome/webfonts/fa-brands-400.woff2 => [304] 
+22. [GET] https://diff-diff.readthedocs.io/_/addons/?client-version=0.49.0&api-version=1&project-slug=diff-diff&version-slug=stable => [200] 
+23. [GET] https://media.ethicalads.io/media/client/ethicalads.min.js => [304] 
+24. [GET] https://media.ethicalads.io/abp/px.gif?ch=1&rn=8.180608301897207 => [200] 
+25. [GET] https://media.ethicalads.io/abp/px.gif?ch=2&rn=8.180608301897207 => [200] 
+26. [GET] https://server.ethicalads.io/api/v1/decision/?publisher=readthedocs&ad_types=readthedocs-sidebar&div_ids=ethical-ad-placement&callback=ad_1778753996630_506465&keywords=only+words%7Creadthedocs-project-1174205%7Creadthedocs-project-diff-diff%7Capi%7Ctests%7Cpip&campaign_types=community%7Chouse%7Cpaid&format=jsonp&client_version=2.4.0&placement_index=0&url=https%3A%2F%2Fdiff-diff.readthedocs.io%2Fen%2Fstable%2F => [200] 
+27. [GET] https://media.ethicalads.io/images/2023/07/mongodb-codedark-240x180_KOok2ZA.png => [200] 

audits/ux-2026-05/perf/desktop/01-index-network.txt

@@ -0,0 +1,9 @@
+{
+  "DOMContentLoaded_ms": 244,
+  "load_ms": 265,
+  "transferSize_KB": 8,
+  "encodedBodySize_KB": 8,
+  "decodedBodySize_KB": 38,
+  "FCP_ms": 252,
+  "FP_ms": 252
+}

audits/ux-2026-05/perf/desktop/01-index-timing.json

@@ -0,0 +1,2 @@
+Total messages: 5 (Errors: 0, Warnings: 0)
+Returning 0 messages for level "error"

audits/ux-2026-05/perf/desktop/02-quickstart-console.txt

@@ -0,0 +1,26 @@
+1. [GET] https://diff-diff.readthedocs.io/en/stable/quickstart.html => [200] 
+2. [GET] https://diff-diff.readthedocs.io/en/stable/_static/styles/theme.css?digest=55c26ed414f5f8a31ebb => [200] 
+3. [GET] https://diff-diff.readthedocs.io/en/stable/_static/styles/pydata-sphinx-theme.css?digest=55c26ed414f5f8a31ebb => [200] 
+4. [GET] https://diff-diff.readthedocs.io/en/stable/_static/pygments.css?v=8f2a1f02 => [200] 
+5. [GET] https://diff-diff.readthedocs.io/en/stable/_static/custom.css?v=7b11d399 => [200] 
+6. [GET] https://diff-diff.readthedocs.io/en/stable/_static/scripts/fontawesome.js?digest=55c26ed414f5f8a31ebb => [304] 
+7. [GET] https://diff-diff.readthedocs.io/en/stable/_static/scripts/bootstrap.js?digest=55c26ed414f5f8a31ebb => [200] 
+8. [GET] https://diff-diff.readthedocs.io/en/stable/_static/scripts/pydata-sphinx-theme.js?digest=55c26ed414f5f8a31ebb => [200] 
+9. [GET] https://diff-diff.readthedocs.io/en/stable/_static/documentation_options.js?v=ad389f6d => [200] 
+10. [GET] https://diff-diff.readthedocs.io/en/stable/_static/doctools.js?v=fd6eb6e6 => [200] 
+11. [GET] https://diff-diff.readthedocs.io/en/stable/_static/sphinx_highlight.js?v=6ffebe34 => [200] 
+12. [GET] https://cdnjs.cloudflare.com/ajax/libs/require.js/2.3.4/require.min.js => [200] 
+13. [GET] https://diff-diff.readthedocs.io/en/stable/_static/searchtools.js => [200] 
+14. [GET] https://diff-diff.readthedocs.io/en/stable/_static/language_data.js => [200] 
+15. [GET] https://diff-diff.readthedocs.io/en/stable/searchindex.js => [200] 
+16. [GET] https://diff-diff.readthedocs.io/_/static/javascript/readthedocs-addons.js => [200] 
+17. [GET] https://diff-diff.readthedocs.io/en/stable/_static/basic.css => [200] 
+18. [GET] https://diff-diff.readthedocs.io/en/stable/_static/vendor/fontawesome/webfonts/fa-solid-900.woff2 => [304] 
+19. [GET] https://diff-diff.readthedocs.io/en/stable/_static/vendor/fontawesome/webfonts/fa-brands-400.woff2 => [304] 
+20. [GET] https://diff-diff.readthedocs.io/_/addons/?client-version=0.49.0&api-version=1&project-slug=diff-diff&version-slug=stable => [200] 
+21. [GET] https://media.ethicalads.io/media/client/ethicalads.min.js => [304] 
+22. [GET] https://media.ethicalads.io/abp/px.gif?ch=1&rn=8.278298046972816 => [200] 
+23. [GET] https://media.ethicalads.io/abp/px.gif?ch=2&rn=8.278298046972816 => [200] 
+24. [GET] https://server.ethicalads.io/api/v1/decision/?publisher=readthedocs&ad_types=readthedocs-sidebar&div_ids=ethical-ad-placement&callback=ad_1778754081810_146921&keywords=only+words%7Creadthedocs-project-1174205%7Creadthedocs-project-diff-diff%7Capi%7Cpip%7Cci&campaign_types=community%7Chouse%7Cpaid&format=jsonp&client_version=2.4.0&placement_index=0&url=https%3A%2F%2Fdiff-diff.readthedocs.io%2Fen%2Fstable%2Fquickstart.html => [200] 
+25. [GET] https://media.ethicalads.io/images/2025/08/Ethical_Ads_everyenterpirsev1_-_240x180.png => [200] 
+26. [GET] https://server.ethicalads.io/proxy/view/10124/019e2601-5485-7f61-b0b4-f8cb885d48cd/ => [200] 

audits/ux-2026-05/perf/desktop/02-quickstart-network.txt

@@ -0,0 +1,9 @@
+{
+  "DOMContentLoaded_ms": 1081,
+  "load_ms": 1333,
+  "FCP_ms": 1080,
+  "htmlTransferSize_KB": 0,
+  "htmlDecodedSize_KB": 45,
+  "resourceCount": 25,
+  "totalResourceTransfer_KB": 1
+}

audits/ux-2026-05/perf/desktop/02-quickstart-timing.json

@@ -0,0 +1,2 @@
+Total messages: 5 (Errors: 0, Warnings: 0)
+Returning 0 messages for level "error"

audits/ux-2026-05/perf/desktop/03-practitioner-getting-started-console.txt

@@ -0,0 +1,28 @@
+1. [GET] https://diff-diff.readthedocs.io/en/stable/practitioner_getting_started.html => [200] 
+2. [GET] https://server.ethicalads.io/proxy/viewtime/10124/019e2601-5485-7f61-b0b4-f8cb885d48cd/?view_time=41 => [FAILED] net::ERR_ABORTED
+3. [GET] https://server.ethicalads.io/proxy/viewtime/10124/019e2601-5485-7f61-b0b4-f8cb885d48cd/?view_time=41 => [FAILED] net::ERR_ABORTED
+4. [GET] https://diff-diff.readthedocs.io/en/stable/_static/styles/theme.css?digest=55c26ed414f5f8a31ebb => [200] 
+5. [GET] https://diff-diff.readthedocs.io/en/stable/_static/styles/pydata-sphinx-theme.css?digest=55c26ed414f5f8a31ebb => [200] 
+6. [GET] https://diff-diff.readthedocs.io/en/stable/_static/pygments.css?v=8f2a1f02 => [200] 
+7. [GET] https://diff-diff.readthedocs.io/en/stable/_static/custom.css?v=7b11d399 => [200] 
+8. [GET] https://diff-diff.readthedocs.io/en/stable/_static/scripts/fontawesome.js?digest=55c26ed414f5f8a31ebb => [304] 
+9. [GET] https://diff-diff.readthedocs.io/en/stable/_static/scripts/bootstrap.js?digest=55c26ed414f5f8a31ebb => [200] 
+10. [GET] https://diff-diff.readthedocs.io/en/stable/_static/scripts/pydata-sphinx-theme.js?digest=55c26ed414f5f8a31ebb => [200] 
+11. [GET] https://diff-diff.readthedocs.io/en/stable/_static/documentation_options.js?v=ad389f6d => [200] 
+12. [GET] https://diff-diff.readthedocs.io/en/stable/_static/doctools.js?v=fd6eb6e6 => [200] 
+13. [GET] https://diff-diff.readthedocs.io/en/stable/_static/sphinx_highlight.js?v=6ffebe34 => [200] 
+14. [GET] https://cdnjs.cloudflare.com/ajax/libs/require.js/2.3.4/require.min.js => [200] 
+15. [GET] https://diff-diff.readthedocs.io/en/stable/_static/searchtools.js => [200] 
+16. [GET] https://diff-diff.readthedocs.io/en/stable/_static/language_data.js => [200] 
+17. [GET] https://diff-diff.readthedocs.io/en/stable/searchindex.js => [200] 
+18. [GET] https://diff-diff.readthedocs.io/_/static/javascript/readthedocs-addons.js => [200] 
+19. [GET] https://diff-diff.readthedocs.io/en/stable/_static/basic.css => [200] 
+20. [GET] https://diff-diff.readthedocs.io/_/addons/?client-version=0.49.0&api-version=1&project-slug=diff-diff&version-slug=stable => [200] 
+21. [GET] https://diff-diff.readthedocs.io/en/stable/_static/vendor/fontawesome/webfonts/fa-solid-900.woff2 => [304] 
+22. [GET] https://diff-diff.readthedocs.io/en/stable/_static/vendor/fontawesome/webfonts/fa-brands-400.woff2 => [304] 
+23. [GET] https://media.ethicalads.io/media/client/ethicalads.min.js => [304] 
+24. [GET] https://media.ethicalads.io/abp/px.gif?ch=1&rn=0.835403376779389 => [200] 
+25. [GET] https://media.ethicalads.io/abp/px.gif?ch=2&rn=0.835403376779389 => [200] 
+26. [GET] https://server.ethicalads.io/api/v1/decision/?publisher=readthedocs&ad_types=readthedocs-sidebar&div_ids=ethical-ad-placement&callback=ad_1778754123597_229730&keywords=only+words%7Creadthedocs-project-1174205%7Creadthedocs-project-diff-diff%7Cpip%7Cmatplotlib%7Cpython&campaign_types=community%7Chouse%7Cpaid&format=jsonp&client_version=2.4.0&placement_index=0&url=https%3A%2F%2Fdiff-diff.readthedocs.io%2Fen%2Fstable%2Fpractitioner_getting_started.html => [200] 
+27. [GET] https://media.ethicalads.io/images/2023/07/mongodb-codedark-240x180.png => [200] 
+28. [GET] https://server.ethicalads.io/proxy/view/10122/019e2601-f7a2-7771-ab5a-fbd0fb847bd2/ => [200]