Repository navigation
feat(conformance): add client-initiated metadata, artwork and controller scenarios - #144
Merged
chrisuthe merged 6 commits intoOct 7, 2026
Conversation
…ler scenarios RC1 requires every server to support client-initiated connections, but the matrix only opened one for the player role. Declare the three scenarios the adapters already dispatched on without a ScenarioSpec behind them, each a client-initiated repeat of its server-initiated sibling at scenario_revision 1. The sendspin-cpp and sendspin-js client adapters take the new ids on their existing outbound path. The sendspin-go and sendspin-rs client adapters only drive a player session outbound, so they fail fast as a harness gap instead of reporting a missing audio stream.
There was a problem hiding this comment.
🟡 Changes recommended
The newly activated JVM controller path can send commands before support is advertised, producing a harness-induced failure.
2 open findings
What changed in this PR
Expands the conformance matrix with client-initiated metadata, artwork, and controller coverage.
Changes:
- Registers three new scenarios with matching sibling configuration.
- Updates adapters for new scenario dispatch or explicit harness-gap failures.
- Adds documentation and scenario-invariant tests.
| File | Description |
|---|---|
src/conformance/scenarios.py |
Registers scenarios and shares CLI argument constants. |
README.md |
Documents scenarios and adapter coverage. |
adapters/sendspin-cpp/client/main.cpp |
Recognizes new scenario IDs. |
adapters/sendspin-go/client/main.go |
Fails unsupported outbound paths explicitly. |
adapters/sendspin-js/client.mjs |
Adds metadata and controller dispatch. |
adapters/sendspin-rs/client/src/main.rs |
Reports missing outbound role paths. |
tests/test_client_initiated_role_scenarios.py |
Verifies parity with server-initiated siblings. |
tests/test_stream_start_gate.py |
Updates streamed-role expectations. |
🧠 Review effort: Balanced
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
The client-initiated-request-format-* scenarios no longer exist, and the branch behind them sent stream/request-format, a message RC1 does not have.
…he declared scenarios The branch that sends the client/state format preference answered to the retired client-initiated-request-format-* ids, so the scenarios that replaced them never reached it and SendspinKit failed both.
…re sending it client-initiated The server sets repeat and shuffle before it lists supported commands, so the first controller state can arrive without the command the scenario sends.
…the-missing-client-initiated-scenarios-and # Conflicts: # src/conformance/scenarios.py
chrisuthe
marked this pull request as ready for review
October 7, 2026 18:26
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.


Part of #24.
RC1
connection.mdsays "Servers MUST support both methods described below" (checked at the1.0.0-rc1tag). The matrix opened a client-initiated connection for the player role only. This adds the same coverage for metadata, artwork and controller.What changes
client-initiated-metadata,client-initiated-artworkandclient-initiated-controller. Each repeats its server-initiated sibling with the client opening the connection: same verification mode, role family and CLI arguments. Each is a new id atscenario_revision=1. No existing scenario's revision or assertions change, andrunner.pyis untouched.ScenarioSpecbehind them. They are now real scenarios, so those branches are no longer dead.Adapter branches, checked by running them
Every SDK is a fresh clone of its upstream default branch (spec at
1.0.0-rc1).server-initiated-artworkdoes on the live baseline.artworkrole family, soclient-initiated-artworkis harness-stampedunsupportedand never launched, as for the sibling.client-initiated-controllerbranch sent the command on the first controller state, and the aiosendspin server sets repeat and shuffle before it lists supported commands, so the command could go out before it was advertised and be rejected. It now waits forsupported_commandsto list the command, the same loop the server-initiated branch uses a few lines above it. Unverified until CI builds it.Matrix growth
The matrix grows by 48 rows (3 scenarios x 16 pairs), from 192 on the live baseline to 240. Re-measured after merging
mainat55c4d0b, so every figure here includes thetime_exchange,group/update, activation and available-gate verdicts on all 48 new rows; the figures did not change. Measured locally on macOS against the liveindex.json:detect_regressions.pyreports no regressions (the live baseline still carries pre-#143 revisions, so it exempts every existing cell; the statement that follows is a direct status comparison, not that script's verdict), and of the 192 existing rows only the two SendspinKit cells listed further down change status, both from failed to passed.client-initiated-metadataclient-initiated-controllerclient-initiated-artworkExpected status per cell:
unsupported); SendspinKit on artwork (server times out waiting for the client, as in the sibling); sendspin-jvm (see below).client-initiated-pcm. Theclient-initiated-controllercells for sendspin-jvm do not measure sendspin-jvm's conformance in this PR: the adapter branch behind them was changed without being run, so whatever colour they show, read it as the state of an untested adapter until someone runs it with a JDK.Retired
client-initiated-request-format-*idsThese two ids have had no
ScenarioSpecsince theclient-initiated-state-format-*scenarios replaced them, but three adapters still dispatched on them.stream/request-format, a message RC1 does not have, and nothing else used it. Its cells are unchanged.client/stateformat preference and writes therenegotiationblock theformat-preferencecomparison reads, so it now answers toclient-initiated-state-format-pcmand-flac. The change is the two id strings in one file.After this change no adapter dispatches on a scenario id that no
ScenarioSpecdeclares, exceptclient-initiated-request-format-pcmand-flacin the sendspin-jvm adapter. Checked by scanning every tracked adapter file for scenario-id literals against the registry.Cell movement from the SendspinKit change
Two existing cells go failed to passed. Every other SendspinKit cell keeps its live-baseline status (all 30 SendspinKit rows re-run).
client-initiated-state-format-pcm, aiosendspin to SendspinKitclient-initiated-state-format-flac, aiosendspin to SendspinKitNo
scenario_revisionis bumped: only an adapter changed, not any scenario's stimulus or assertions.Both passes rest on what the aiosendspin server recorded, not on the client's account of itself:
format_preference.received, taken from theclient/stateit received, is the target format (16-bit PCM; FLAC);stream, its laststream/start, is in that format;stream/startmessages, each after aclient/statereporting available.The comparison requires the first two before it reads anything the client reported. The client summary's
request_message_type: "client/state"is a fixed string in the adapter and is not what the verdict relies on.Found in review
My first sendspin-rs fail-fast guard let through only the player ids it listed, which also caught
client-initiated-state-format-pcmand-flacand would have turned two real player cases into harness gaps. It is narrowed to the metadata, controller and artwork predicates, and those two cells were re-run to confirm they reach the SDK's outbound path again.Not in this PR
Checks
python -m unittest discover -s tests: 344 tests pass on the merged tree.main; the three new ones are at 1.client-initiated-artwork:test_only_scenarios_that_open_a_stream_are_judgedrequires it to be judged, and the aiosendspin server's summary for that case records itsclient/statethen artworkstream/starttrace.client-initiated-artworkaiosendspin to sendspin-cpp case page and a harness-gap case page.gofmt,go vetandcargo fmt --checkare clean for the two adapters touched.