Skip to content

Advertise runtime peer companions on loaders - #89

Open
lapy wants to merge 2 commits into
0xShug0:mainfrom
lapy:feature/loader-companions-contract
Open

Advertise runtime peer companions on loaders#89
lapy wants to merge 2 commits into
0xShug0:mainfrom
lapy:feature/loader-companions-contract

Conversation

@lapy

@lapy lapy commented Jul 21, 2026

Copy link
Copy Markdown
Contributor

Summary

After the machine-readable loader catalog (#74 / #86), integrators can discover which families exist and which packages install. They still cannot discover which peers a family needs at runtime (aligner, VAD, codec, best-of-N ASR, external Whisper, …) without reading C++ session code or hard-coding a UI graph.

This PR makes those peers part of the same contract: each loader can advertise a companions[] array on audiocpp_cli --list-loaders --json. Integrators should consume that export instead of owning a seed table of family→peer edges.

schema_version stays 1 (additive fields only).

Why this belongs in audio.cpp

  • The path option keys and peer families are owned by the loaders today.
  • An external companion graph would drift whenever a loader adds/renames an option.
  • Catalog parent_package_id covers install-time deps; companions cover runtime peers wired via load/session options.

Schema (per companion)

Field Meaning
id Stable integrator id (e.g. forced_aligner)
config_key Option key that receives the peer path
scope session, load, or request
target_family Registered loader family to install/select; omitted when external
optional Whether the primary can run without the peer
required_for Feature keys that need this peer
label Short UI label
bundled_default Optional in-tree default (e.g. Silero under assets/)

Seeded families

Primary Companions
qwen3_asr qwen3_forced_aligner, silero_vad (for timestamps)
vibevoice_asr silero_vad (for audio_chunk_mode=vad)
outetts qwen3_forced_aligner (voice cloning / outetts.aligner_model_path)
miotts miocodec (required), qwen3_asr (optional best-of-N)
vevo2 external Whisper dir (vevo2.whisper_model_path; empty target_family)

Also fills in missing advertised_capabilities() on outetts so the list-loaders row has tasks.

Docs / CI

  • Extends docs/maintainers/loader_and_catalog.md with companion rules + checklist.
  • Extends tools/check_loader_catalog_sync.py so non-empty companion target_family values must be active registry families (no binary required).

Test plan

  • python3 tools/check_loader_catalog_sync.py --self-test
  • python3 tools/check_loader_catalog_sync.py
  • Build audiocpp_cli and confirm companions appear for the families above in --list-loaders --json
  • Review that config_key / required_for match the intended UI wiring for each family

lapy added 2 commits July 21, 2026 19:54
Expose multi-model path peers (aligner, VAD, codec, ASR, Whisper) via
--list-loaders --json so integrators can wire companion UX from the
contract instead of hard-coding family graphs.
Point README, usage, CONTRIBUTING, and model pages at the
--list-loaders companions contract so integrators discover peers
from the documented API instead of hard-coded graphs.
@0xShug0

0xShug0 commented Jul 21, 2026

Copy link
Copy Markdown
Owner

Thanks for the PR. I need to think about it.

@lapy

lapy commented Jul 21, 2026

Copy link
Copy Markdown
Contributor Author

Take your time. It might be redundant if you’re planning to move to manifest style definitions instead of having them in the code.

@0xShug0

0xShug0 commented Jul 23, 2026

Copy link
Copy Markdown
Owner

@lapy Check https://github.com/0xShug0/audio.cpp/tree/main/examples/model_spec_demo. This is the init design. Let me know your thoughts.

@lapy

lapy commented Jul 23, 2026

Copy link
Copy Markdown
Contributor Author

Thanks for the pointer, I went through examples/model_spec_demo and docs/maintainers/model_specs.md.

This lines up with what I was worried about on redundancy. Putting companions in the typed model spec (with validated kind / family / option / required / required_for) is a cleaner long-term home than advertising them from C++ loaders via --list-loaders. The toy Qwen3 ASR companions (forced aligner + VAD) are essentially the same contract we were trying to expose in this PR, just as data instead of code.

A few notes from comparing the shapes:

Same problem (runtime peer models for features like timestamps).
Field mapping is close but not identical: my config_key / target_family / optional ≈ your option / family / required.
I also had scope, label, and bundled_default; happy to drop those or fold anything useful into the typed schema if you want them later.
Real model_specs/*.json still look layout-only today, only the toy demo uses the full typed shape, so integrators can’t consume companions from production specs yet.
I’m fine closing this PR and following the model-spec path instead. If helpful, I can open a follow-up that seeds companions on the real families that need peers (qwen3_asr, vibevoice_asr, outetts, miotts, vevo2, …) using the toy schema as the template.

@0xShug0

0xShug0 commented Jul 24, 2026

Copy link
Copy Markdown
Owner

@lapy Feel free to improve the schema. I still need to work out the migration plan. For now, the new spec system is decoupled from the framework, and no models are using it yet. I won't migrate any models before the shape is stable.

@0xShug0

0xShug0 commented Jul 27, 2026

Copy link
Copy Markdown
Owner

@lapy Just a quick udpate. I'm testing the new v1 model specs now. The updated JSONs are currently under model_specs_v1/. There are still a few schema issues to work through, especially on Windows where the large monolithic embedded JSON can hit compiler/string-size limits. The schema likely needs more refinement before we make it the default catalog. I'm also considering folding dependencies into the relevant option definitions. That may simplify the schema and make dependency requirements easier to understand, but I still need to think through the tradeoffs.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants