Skip to content

Extend EncodedVideoChunkMetadata for Spatial Scalability #756

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 4 commits into
base: main
Choose a base branch
from
Open

Extend EncodedVideoChunkMetadata for Spatial Scalability #756

Changes from 1 commit
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
517f6a0
Extend EncodedVideoChunkMetadata for Spatial Scalability
aboba Jan 4, 2024
c7dcedc
Move frameId and dependencies
aboba Jan 9, 2024
16ba584
Fix temporal/spatial layer assignments
aboba Jan 9, 2024
9efde67
Move frameId and dependencies
aboba Jan 9, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
30 changes: 28 additions & 2 deletions index.src.html
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -133,6 +133,10 @@
:: A grouping of {{EncodedVideoChunk}}s whose timestamp cadence produces a
particular framerate. See {{VideoEncoderConfig/scalabilityMode}}.

: <dfn>Spatial Layer</dfn>
:: A grouping of {{EncodedVideoChunk}}s which produces a particular
resolution. See {{VideoEncoderConfig/scalabilityMode}}.

: <dfn>Progressive Image</dfn>
:: An image that supports decoding to multiple levels of detail, with lower
levels becoming available while the encoded data is not yet fully buffered.
Expand Down Expand Up @@ -1652,11 +1656,20 @@
|svc|.{{SvcOutputMetadata/temporalLayerId}}.
4. Assign |svc| to
|chunkMetadata|.{{EncodedVideoChunkMetadata/svc}}.
8. If |encoderConfig|.{{VideoEncoderConfig/alpha}} is set to `"keep"`:
8. If |encoderConfig|.{{VideoEncoderConfig/scalabilityMode}}
describes multiple [=spatial layers=]:
1. Let |svc| be a new {{SvcOutputMetadata}} instance.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This will override everything produced by the step 8 for SVC modes that have both temporal and spatial layers. for example: L2T2

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think I have fixed this. PTAL.

2. Let |spatial_layer_id| be the zero-based index describing the
spatial layer for |output|.
3. Assign |spatial_layer_id| to
|svc|.{{SvcOutputMetadata/spatialLayerId}}.
4. Assign |svc| to
|chunkMetadata|.{{EncodedVideoChunkMetadata/svc}}.
9. If |encoderConfig|.{{VideoEncoderConfig/alpha}} is set to `"keep"`:
1. Let |alphaSideData| be the encoded alpha data in |output|.
2. Assign |alphaSideData| to
|chunkMetadata|.{{EncodedVideoChunkMetadata/alphaSideData}}.
9. Invoke {{VideoEncoder/[[output callback]]}} with |chunk| and
10. Invoke {{VideoEncoder/[[output callback]]}} with |chunk| and
|chunkMetadata|.
</dd>
<dt><dfn>Reset VideoEncoder</dfn> (with |exception|)</dt>
Expand Down Expand Up @@ -1704,6 +1717,9 @@

dictionary SvcOutputMetadata {
unsigned long temporalLayerId;
unsigned long spatialLayerId;
unsigned long long frameId;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

frameId and dependencies are never set

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@Djuffin @tonyherre dependencies would be set to the sequence of frameId values that the encodedChunk depends on. But the bigger question is the behavior of frameId, which is discussed here: w3c/webrtc-encoded-transform#220

sequence <unsigned long long> dependencies;
};
</xmp>

Expand All @@ -1723,7 +1739,17 @@
:: A number that identifies the [=temporal layer=] for the associated
{{EncodedVideoChunk}}.

: <dfn dict-member for=SvcOutputMetadata>spatialLayerId</dfn>
:: A number that identifies the [=spatial layer=] for the associated
{{EncodedVideoChunk}}.

: <dfn dict-member for=SvcOutputMetadata>frameId</dfn>
:: A number that identifies the associated {{EncodedVideoChunk}}.


: <dfn dict-member for=SvcOutputMetadata>dependencies</dfn>
:: A sequence containing the {{frameId}} values that the associated {{EncodedVideoChunk}} depends on.

Configurations{#configurations}
===============================

Expand Down