Proposal: Have a detachable MediaSource object

[jyavenard]

A MediaSource object is typically attached to a media element.

When this MediaSource is detached from the media element (such as when the src or srcObject attribute is cleared or set to another object), the “3.15.2 Detaching from a media element” algorithm is to be run which will clear all its source buffers and their content and change the readyState of the MediaSource object to “close”

Discussion with users has show interests on having the ability to detach a MediaSource from a media element temporarily to re-attach it again later so that a new MediaSource doesn’t need to be created and having to reload all the original content again which is slow, impact playback quality and result in an interruption.

Proposal:
Provide an optional dictionary to the MediaSource constructor.

it could take an optional dictionary with { detachable: true }
such as const ms = new MediaSource({ detachable: true })

dictionary MediaSourceInit {
    boolean detachable = false;
};

The in the MediaSource IDL, the constructor would be amended as follow:

constructor();

to

constructor(optional MediaSourceInit init = { });

and same for the ManagedMediaSource's constructor.

A new detachable attribute would be added, allowing to detect the functionality and test test if a MediaSource is detachable or not.

readonly attribute boolean detachable;

When a detachable MediaSource is detached from the mediaElement, the Detaching from a media element algorithm https://w3c.github.io/media-source/#mediasource-detach

step 5, 8 and 9 would be skipped.

in the Attaching to a media element algorithm (https://w3c.github.io/media-source/#mediasource-attach)
steps would be modified to allow re-using a MediaSourceHandle (if in worker) or a MediaSource. Such as ignoring the "has ever been attached" internal slot

[dalecurtis]

This came up in the past. I don't recall why we didn't do it, but there's probably some existing issues around it in the tracker.

Off hand I'd be worried folks wouldn't do anything with the detached MediaSource and it ends up sitting around eating memory, so maybe we'd want some sort of timeout on usage if it's detached? Or at least leave room for UA to do so in the spec.

[joeyparrish]

Detaching would seem to improve efficiency in switching between ads and main content on platforms where you only have one media element available.

However, on those platforms, we likely don't have the memory to tolerate having a detached MediaSource object holding onto buffers. It could be precisely because those platforms don't have sufficient memory for more than one set of buffers that we only have one media element available.

In cases where we can have more than one element, you can typically get away with pausing the one with main content and overlaying it with another showing the ad break.

[jyavenard]

This came up in the past. I don't recall why we didn't do it, but there's probably some existing issues around it in the tracker.

I searched and I couldn't find one. We discussed it in the past at FOMS. So maybe that never went further from there.

Off hand I'd be worried folks wouldn't do anything with the detached MediaSource and it ends up sitting around eating memory, so maybe we'd want some sort of timeout on usage if it's detached? Or at least leave room for UA to do so in the spec.

Yes, not being GCed is a concern.
One thing that would improve the situation I believe is only allowing the use of a detachable MediaSource by using the media's element srcObject rather than the URL. MSE's URL aren't automatically revoked (issue #156). We could have the HTMLMediaElement move to the "error" state if given a URL to a MediaSource that's detachable.

If the MediaSource is a ManagedMediaSource it can be cleared of its content if the system is run out of memory.

We could limit the ability of the MediaSource to be detachable only to ManagedMediaSource.

[dalecurtis]

#36 was the closest I could find.

A srcObject restriction seems reasonable to me.

[chrisn]

Notes from TPAC 2024 discussion: https://www.w3.org/2024/09/26-mediawg-minutes.html#t07