Advertise runtime peer companions on loaders - #89
Conversation
Expose multi-model path peers (aligner, VAD, codec, ASR, Whisper) via --list-loaders --json so integrators can wire companion UX from the contract instead of hard-coding family graphs.
Point README, usage, CONTRIBUTING, and model pages at the --list-loaders companions contract so integrators discover peers from the documented API instead of hard-coded graphs.
|
Thanks for the PR. I need to think about it. |
|
Take your time. It might be redundant if you’re planning to move to manifest style definitions instead of having them in the code. |
|
@lapy Check https://github.com/0xShug0/audio.cpp/tree/main/examples/model_spec_demo. This is the init design. Let me know your thoughts. |
|
Thanks for the pointer, I went through examples/model_spec_demo and docs/maintainers/model_specs.md. This lines up with what I was worried about on redundancy. Putting companions in the typed model spec (with validated kind / family / option / required / required_for) is a cleaner long-term home than advertising them from C++ loaders via --list-loaders. The toy Qwen3 ASR companions (forced aligner + VAD) are essentially the same contract we were trying to expose in this PR, just as data instead of code. A few notes from comparing the shapes: Same problem (runtime peer models for features like timestamps). |
|
@lapy Feel free to improve the schema. I still need to work out the migration plan. For now, the new spec system is decoupled from the framework, and no models are using it yet. I won't migrate any models before the shape is stable. |
|
@lapy Just a quick udpate. I'm testing the new v1 model specs now. The updated JSONs are currently under model_specs_v1/. There are still a few schema issues to work through, especially on Windows where the large monolithic embedded JSON can hit compiler/string-size limits. The schema likely needs more refinement before we make it the default catalog. I'm also considering folding dependencies into the relevant option definitions. That may simplify the schema and make dependency requirements easier to understand, but I still need to think through the tradeoffs. |
Summary
After the machine-readable loader catalog (#74 / #86), integrators can discover which families exist and which packages install. They still cannot discover which peers a family needs at runtime (aligner, VAD, codec, best-of-N ASR, external Whisper, …) without reading C++ session code or hard-coding a UI graph.
This PR makes those peers part of the same contract: each loader can advertise a
companions[]array onaudiocpp_cli --list-loaders --json. Integrators should consume that export instead of owning a seed table of family→peer edges.schema_versionstays 1 (additive fields only).Why this belongs in audio.cpp
parent_package_idcovers install-time deps; companions cover runtime peers wired via load/session options.Schema (per companion)
idforced_aligner)config_keyscopesession,load, orrequesttarget_familyoptionalrequired_forlabelbundled_defaultassets/)Seeded families
qwen3_asrqwen3_forced_aligner,silero_vad(for timestamps)vibevoice_asrsilero_vad(foraudio_chunk_mode=vad)outettsqwen3_forced_aligner(voice cloning /outetts.aligner_model_path)miottsmiocodec(required),qwen3_asr(optional best-of-N)vevo2vevo2.whisper_model_path; emptytarget_family)Also fills in missing
advertised_capabilities()onoutettsso the list-loaders row has tasks.Docs / CI
docs/maintainers/loader_and_catalog.mdwith companion rules + checklist.tools/check_loader_catalog_sync.pyso non-empty companiontarget_familyvalues must be active registry families (no binary required).Test plan
python3 tools/check_loader_catalog_sync.py --self-testpython3 tools/check_loader_catalog_sync.pyaudiocpp_cliand confirmcompanionsappear for the families above in--list-loaders --jsonconfig_key/required_formatch the intended UI wiring for each family