Add RFC 2198 RED support for Opus

[gokuljs]

Continuing the discussion from pion/webrtc#3495 here so that the implementation details for each repository can remain in the appropriate repo. The goal is to add support for RTP Payload for Redundant Audio Data (RED), as defined in RFC 2198, for Opus streams.

Changes required in pion/interceptor

I believe we need to add a RED package to pion/interceptor with the following functionality:

• A parser and writer for RFC 2198 RED payloads.
• A sender interceptor that stores the previous two Opus payloads and includes them as redundant blocks in the next RED packet.
• A receiver interceptor that extracts the primary Opus payload and recovers missing Opus packets from the redundant blocks when possible.
• Support for plain Opus and RED packets being used on the same SSRC.
• Correct handling of:
  • packet loss,
  • duplicate packets,
  • packet reordering,
  • RTP sequence-number wraparound, and
  • RTP timestamp wraparound.
• A small receiver-side queue, since a single RED packet may result in multiple Opus packets being delivered after recovery.
• Packet-size handling so that older redundant blocks are removed when the resulting packet would become too large. If no redundant payload can be included within the allowed packet size, the packet should be sent as plain Opus.

The implementation should use the payload types negotiated through SDP. It should not assume fixed payload types such as 111 for Opus or 63 for RED.

Changes required in pion/webrtc

I think a few smaller changes will also be required in pion/webrtc:

1. Register audio/red as an optional Opus-related format.
2. Negotiate the RED and Opus payload types through SDP.
3. Validate that RED's fmtp value refers to the negotiated Opus payload type.
4. Pass the negotiated payload-type information to the RED interceptor.
5. Install the RED sender and receiver at the appropriate positions in the interceptor chain.

Reference implementation

I looked at LiveKit's Opus RED implementation as a reference. Since LiveKit is an SFU, a significant portion of its implementation deals with routing media and translating packets between publishers and subscribers, which is not required for Pion's use case. However, I think we can take inspiration from the parts related to:

• storing previous Opus packets,
• constructing and parsing RFC 2198 RED payloads,
• recovering missing audio packets,
• enforcing packet-size limits, and
• handling RTP sequence-number and timestamp wraparound.

The actual implementation should be designed specifically around Pion's interceptor architecture rather than directly following the SFU-specific parts of LiveKit's implementation.

Expected packet flow

Outbound:

Application Opus → RED sender interceptor → Network

Inbound:

Network RED/Opus → RED receiver/recovery interceptor → Application Opus

This is my current understanding of the required work. @Sean-Der, @JoTurk, I would appreciate your thoughts on the proposed split between pion/interceptor and pion/webrtc, the sender/receiver design, and whether RED-protected Opus should advertise NACK.

[JoTurk]

@gokuljs I think the payload format should go in pion/rtp, interceptor should handle per-stream history, redundancy selection, policy etc both can go into pion/interceptor too.

and whether RED-protected Opus should advertise NACK

We use NACK with Opus + RED at my work, it actually help with audio quality in many cases in really poor conditions (only chrome sends audio NACKs, other browsers can respond to nack). But pion/webrtc shouldn't enable NACK or negotiate NACK by default.