|
| 1 | +# Website metrics publication evidence — 2026-10-03 |
| 2 | + |
| 3 | +Scope: `REQ-WEB-002`, `AC-WEB-002-1..4`, `TASK-WEB-002`, |
| 4 | +`TEST-WEB-002-1..4`, ADR-010. Use the current checkout; preserve pre-existing |
| 5 | +model/runtime/docs changes. No worktree, stash, dependency or model change. |
| 6 | + |
| 7 | +## Behavior-first verification |
| 8 | + |
| 9 | +- Benchmark JSON export: initial 4/4 failed because `aggregate --json` was |
| 10 | + unsupported (exit 2); final 4/4 passed. Existing hosted-report 2/2 passed. |
| 11 | +- TRX export: initial five real-child tests failed because `test-report` was |
| 12 | + unsupported; the final seven test cases passed. They execute real TUnit |
| 13 | + fixtures and mutate real XML for invalid input cases; no service double. |
| 14 | +- Pipeline/site data: the initial three cases had two failures (no trigger |
| 15 | + and no site-data command). Four additional source/report regressions exposed |
| 16 | + invalid-data exception handling and scalar metadata acceptance before the |
| 17 | + repairs. The final positive case executes real TUnit -> TRX -> test JSON and |
| 18 | + recorded raw benchmark -> numerical JSON -> public site JSON. |
| 19 | +- Final Website suite: 19/19 passed, zero failed/skipped, with the real TRX |
| 20 | + retained under ignored `artifacts/website-publication/final-tests/`. |
| 21 | + |
| 22 | +## Commands and outcomes |
| 23 | + |
| 24 | +| Command | Outcome | |
| 25 | +|---|---| |
| 26 | +| `dotnet restore Synapse.slnx --locked-mode` | Exit 0, all six projects | |
| 27 | +| `dotnet build Synapse.slnx --configuration Release --no-restore --disable-build-servers -m:1` | Exit 0, zero warnings/errors | |
| 28 | +| `dotnet format Synapse.slnx --no-restore --verify-no-changes` | Exit 0 after the final constructor style fix | |
| 29 | +| `dotnet artifacts/website-publication/coverage-snapshot/ManagedCode.Synapse.IntegrationTests.dll --treenode-filter '/*/ManagedCode.Synapse.IntegrationTests.Features.Website/*/*' --minimum-expected-tests 19 --coverage --coverage-output-format cobertura --coverage-output /Users/ksemenenko/Developer/Synapse/artifacts/website-publication/coverage.cobertura.xml --results-directory /Users/ksemenenko/Developer/Synapse/artifacts/website-publication/final-tests --report-trx` | Exit 0, 19 passed, 17.105s | |
| 30 | +| `dotnet experiments/Synapse.ReferenceBenchmarks/bin/Release/net10.0/Synapse.ReferenceBenchmarks.dll test-report --results artifacts/website-publication/final-tests --runner osx-arm64 --output artifacts/website-publication/final-test-results.json` | Exit 0; actual 19 passed, zero failed/skipped | |
| 31 | +| `cargo test --manifest-path native/Cargo.toml --locked` | Exit 0; 32 actual tests, empty unit/doc targets do not count as required suites | |
| 32 | +| `cargo fmt --manifest-path native/Cargo.toml --all --check` | Exit 0 | |
| 33 | +| `cargo clippy --manifest-path native/Cargo.toml --workspace --all-targets -- -D warnings` | Exit 0 | |
| 34 | +| `dotnet list Synapse.slnx package --vulnerable --include-transitive` | Exit 0; no reported vulnerable packages in six projects | |
| 35 | +| `actionlint .github/workflows/pages.yml .github/workflows/verify.yml .github/workflows/performance.yml` | Exit 0 | |
| 36 | +| `bash -n .github/scripts/collect-site-results.sh` | Exit 0 | |
| 37 | +| `git diff --check`, `git diff --cached --check` | Exit 0 | |
| 38 | + |
| 39 | +Rust commands use `PATH=/opt/homebrew/opt/rustup/bin:$PATH`. Sandbox feed/IPC |
| 40 | +access was granted for required checks. Coverage runs from a copied binary |
| 41 | +directory to preserve the normal build. C# XML analysis deduplicates filename |
| 42 | +and line number: Website source coverage is 340/367 (92.64%). Overall benchmark |
| 43 | +assembly coverage is 9.3% for this focused filter; this is not a full-solution |
| 44 | +or changed-line coverage claim. |
| 45 | + |
| 46 | +## Artifact and source qualification |
| 47 | + |
| 48 | +The real collector verified existing GitHub artifact SHA-256 values and |
| 49 | +retained independent metadata for verification run `36547564346` (failure) |
| 50 | +and performance run `36547564428` (success), both source |
| 51 | +`7a219c6cb6fdb851962a3f23a2f141abde20679d`. These older runs have no new JSON, |
| 52 | +so the produced `current-latest-validated.json` truthfully contains absent |
| 53 | +test artifacts and a null benchmark report. Historical Markdown-only |
| 54 | +artifacts are not relabeled as numerical evidence. |
| 55 | + |
| 56 | +An independent source export consists of `git archive HEAD` plus exactly the |
| 57 | +staged publication patch; it contains none of the pre-existing model changes. |
| 58 | +Its first build caught an incorrectly positioned staged switch insertion; |
| 59 | +the staged-only patch coordinates were corrected before publication. The |
| 60 | +corrected export passed locked restore, Release solution build (5.80s, |
| 61 | +zero warnings/errors), 19/19 Website tests (12.268s) and 2/2 hosted-report |
| 62 | +compatibility tests (263ms), all exit 0. Remote publication follows separately. |
| 63 | + |
| 64 | +Architecture review: no new runtime engines/dependencies, generated branches, |
| 65 | +or Node/Python tooling. Files/types stay within repository size limits. Only |
| 66 | +trusted completed `main` push/manual workflow runs may publish; collector |
| 67 | +checks attempt freshness, artifact digests, bounded content and expected root |
| 68 | +members without extracting untrusted paths. The site uses textContent for |
| 69 | +report data and validates GitHub links and numeric/schema fields. |
| 70 | + |
| 71 | +## Current limitations |
| 72 | + |
| 73 | +The C# preview serves source-identical HTML/JS with HTTP 200 and a truthful |
| 74 | +404 before metrics data exists. CUA exposes no enabled browser/app surface, so |
| 75 | +desktop/mobile rendering and browser JavaScript execution are unverified. |
| 76 | +The broader runtime/statistical benchmark plans remain open; these tests do |
| 77 | +not establish a performance winner. `TASK-WEB-002` stays `in_progress` until |
| 78 | +remote/live evidence and the outstanding browser review are recorded. |
| 79 | + |
| 80 | +Automatic approval review initially rejected the scoped commit/push to `main` |
| 81 | +because the dependency-repair authorization does not cover this feature. No |
| 82 | +commit or push occurred during that attempt. On 2026-10-03 the user explicitly |
| 83 | +authorized pushing these prepared changes to `main`; publication proceeds |
| 84 | +under that authorization. |
0 commit comments