MSE-for-WebCodecs

[wolenetz]

Request for Mozilla Position on an Emerging Web Specification: MSE-for-WebCodecs feature addition to MSE

• Specification Title: MSE-for-WebCodecs (feature addition to the Media Source Extensions Specification)
• Specification or proposal URL: MSE-for-WebCodecs draft feature specification w3c/media-source#302
• Caniuse.com URL (optional):
• Bugzilla URL (optional):
• Mozillians who can provide input (optional):

Other information

This feature has been discussed within the Media WG via w3c/media-source#184.
Chromium has an experimental implementation, and I expect to soon have a shareable demo beyond the existing web-platform-tests and the simple sample in the draft spec PR. The feature is currently in Chrome origin trials (as of M95). Notably, this feature diverges from the legacy event-driven MSE asynchronous append communication, using a Promise-based form for the new asynchronous appendEncodedChunks() method.

[annevk]

cc @padenot

[asineth0]

LGTM.

[wolenetz]

Work on this in Chrome and in spec is currently stalled. We're looking for potential users of this API. If you are aware of users or use cases that could benefit from this work, please share if you can. Otherwise, this spec feature may not progress beyond the current preliminary experimental implementation in Chrome and unmerged spec PR.

[asineth0]

This would be pretty helpful for my use case. I’m using WebCodecs to encode video and send it over a RTCDataChannel and play it back on the other end. Being able to use MSE would be helpful because it would (at least in theory) allow smoother playback as it would provide a jitter buffer and handle decode/presentation timing.

On a higher-level, the application is a chat app and the feature that uses this is screen sharing (We have custom capture code and don’t use WebRTC’s media stack).