Skip to content

Add support for dynamic server-sent offers via 406 responses #9

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 1 commit into
base: plugins
Choose a base branch
from
Open

Add support for dynamic server-sent offers via 406 responses #9

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -93,7 +93,7 @@ The object to pass when creating a new endpoint must refer to the following stru
}
```

Subscribing to the WHEP endpoint via WebRTC can be done by sending either an SDP offer or (non-standard approach) an empty request to the created `/endpoint/<id>` endpoint via HTTP POST, which will interact with Janus on your behalf. Depending on what you sent, if successful, it will return an SDP answer or offer back in the 200 OK, plus an address for the allocated resource; if you sent an empty request initially, to complete the negotiation you'll have to send the SDP answer to the resource URL via PATCH. If you're using [Simple WHEP Client](https://github.com/meetecho/simple-whep-client) to test, the full HTTP path to the endpoint is all you need to provide as the WHEP url. Notice that, to be able to send an offer to the WHP endpoint you'll need to use at least the `1.1.4` version of Janus.
Subscribing to the WHEP endpoint via WebRTC can be done by sending an SDP offer to the created `/endpoint/<id>` endpoint via HTTP POST, which will interact with Janus on your behalf. Both client- and server-sent offers are supported, which means the WHEP server return an SDP answer (via the 201 error code) or offer (via the 406 error code) back in the response, plus an address for the allocated resource; if the server changed stance to use a server-sent offer, to complete the negotiation you'll have to send the SDP answer to the resource URL via PATCH. If you're using [Simple WHEP Client](https://github.com/meetecho/simple-whep-client) to test, the full HTTP path to the endpoint is all you need to provide as the WHEP url. Notice that not all the Janus plugins supported by the WHEP server can handle client-sent offers, which means the server will automatically switch to server-sent offers when nedded.

As per the specification, the response to the subscribe request will contain a `Location` header which points to the resource to use to refer to the stream. In this prototype implementation, the resource is handled by the same server instance, and is currently generated automatically as a `<basePath>/resource/<uuid>` endpoint (returned as a relative path in the header), where `uuid` is randomly generated and unique for each subscriber. That's the address used for interacting with the session, i.e., for tricking candidates, restarting ICE, and tearing down the session. The server is configured to automatically allow trickle candidates to be sent via HTTP PATCH to the `<basePath>/resource/<uuid>` endpoint: if you'd like the server to not allow trickle candidates instead (e.g., to test if your client handles a failure gracefully), you can disable them when creating the server via `allowTrickle`. ICE restarts are currently not supported. Finally, that's also the address you'll need to send the HTTP DELETE request to, in case you want to signal the intention to tear down the WebRTC PeerConnection.

Expand Down
1 change: 1 addition & 0 deletions examples/web/index.html
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -48,6 +48,7 @@
<div class="panel-heading">
<h3 class="panel-title">
<span class="label label-info" id="wheplabel">WHEP Endpoint</span>
<span class="label label-primary hide" id="whepsso">Server Sent Offer</span>
</h3>
</div>
<div class="panel-body" id="videoremote"></div>
Expand Down
213 changes: 114 additions & 99 deletions examples/web/index.js
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -16,8 +16,6 @@ function getQueryStringValue(name) {
}
// Get the endpoint ID to subscribe to
const id = getQueryStringValue('id');
// Check if we should let the endpoint send the offer
const expectOffer = (getQueryStringValue('offer') === 'false')

$(document).ready(function() {
// Make sure WebRTC is supported by the browser
Expand All @@ -41,29 +39,28 @@ $(document).ready(function() {

// Function to subscribe to the WHEP endpoint
async function subscribeToEndpoint() {
let headers = null, offer = null;
let headers = null;
if(token)
headers = { Authorization: 'Bearer ' + token };
if(!expectOffer) {
// We need to prepare an offer ourselves, do it now
let iceServers = [{urls: "stun:stun.l.google.com:19302"}];
createPeerConnectionIfNeeded(iceServers);
let transceiver = await pc.addTransceiver('audio');
if(transceiver.setDirection)
transceiver.setDirection('recvonly');
else
transceiver.direction = 'recvonly';
transceiver = await pc.addTransceiver('video');
if(transceiver.setDirection)
transceiver.setDirection('recvonly');
else
transceiver.direction = 'recvonly';
offer = await pc.createOffer({});
await pc.setLocalDescription(offer);
// Extract ICE ufrag and pwd (for trickle)
iceUfrag = offer.sdp.match(/a=ice-ufrag:(.*)\r\n/)[1];
icePwd = offer.sdp.match(/a=ice-pwd:(.*)\r\n/)[1];
}
// Prepare an offer: in case the server only supports server-sent
// offers by replying with a 406, we'll switch stance on the fly
let iceServers = [{urls: "stun:stun.l.google.com:19302"}];
createPeerConnectionIfNeeded(iceServers);
let transceiver = await pc.addTransceiver('audio');
if(transceiver.setDirection)
transceiver.setDirection('recvonly');
else
transceiver.direction = 'recvonly';
transceiver = await pc.addTransceiver('video');
if(transceiver.setDirection)
transceiver.setDirection('recvonly');
else
transceiver.direction = 'recvonly';
let offer = await pc.createOffer({});
await pc.setLocalDescription(offer);
// Extract ICE ufrag and pwd (for trickle)
iceUfrag = offer.sdp.match(/a=ice-ufrag:(.*)\r\n/)[1];
icePwd = offer.sdp.match(/a=ice-pwd:(.*)\r\n/)[1];
// Contact the WHEP endpoint
$.ajax({
url: backend + rest + '/endpoint/' + id,
Expand All @@ -72,85 +69,104 @@ async function subscribeToEndpoint() {
contentType: offer ? 'application/sdp' : null,
data: offer ? offer.sdp : {}
}).error(function(xhr, textStatus, errorThrown) {
// Check if this is a switch to a server-sent offer
if(xhr.status === 406 && xhr.getResponseHeader('Content-Type') &&
xhr.responseText && xhr.responseText.indexOf('v=0') === 0) {
// It is, close the previous PeerConnection handle the offer
try { pc.close(); } catch(e) {};
pc = null;
handleWhepResponse(xhr, xhr.responseText, true)
return;
}
// If we got here, it's a failure
bootbox.alert(xhr.status + ": " + xhr.responseText);
}).success(function(sdp, textStatus, request) {
console.log('Got SDP:', sdp);
resource = request.getResponseHeader('Location');
console.log('WHEP resource:', resource);
// TODO Parse ICE servers
// let ice = request.getResponseHeader('Link');
let iceServers = [{urls: "stun:stun.l.google.com:19302"}];
// Create PeerConnection, if needed
createPeerConnectionIfNeeded(iceServers);
// Pass the SDP to the PeerConnection
let jsep = {
type: expectOffer ? 'offer' : 'answer',
sdp: sdp
};
pc.setRemoteDescription(jsep)
.then(function() {
console.log('Remote description accepted');
if(!expectOffer) {
// We're done: just check if we have candidates to send
if(candidates.length > 0) {
// FIXME Trickle candidate
let headers = null;
if(token)
headers = { Authorization: 'Bearer ' + token };
let candidate =
'a=ice-ufrag:' + iceUfrag + '\r\n' +
'a=ice-pwd:' + icePwd + '\r\n' +
'm=audio 9 RTP/AVP 0\r\n';
for(let c of candidates)
candidate += 'a=' + c + '\r\n';
candidates = [];
$.ajax({
url: backend + resource,
type: 'PATCH',
headers: headers,
contentType: 'application/trickle-ice-sdpfrag',
data: candidate
}).error(function(xhr, textStatus, errorThrown) {
bootbox.alert(xhr.status + ": " + xhr.responseText);
}).done(function(response) {
console.log('Candidate sent');
});
}
return;
handleWhepResponse(request, sdp, false)
});
}

// Helper to process the response, whether it's for client- or server-sent offers
function handleWhepResponse(request, sdp, offer) {
console.log('Got SDP ' + (offer ? 'offer' : 'answer') + ':', sdp);
resource = request.getResponseHeader('Location');
console.log('WHEP resource:', resource);
// TODO Parse ICE servers
// let ice = request.getResponseHeader('Link');
let iceServers = [{urls: "stun:stun.l.google.com:19302"}];
// Create PeerConnection, if needed
createPeerConnectionIfNeeded(iceServers);
// Pass the SDP to the PeerConnection
let jsep = {
type: offer ? 'offer' : 'answer',
sdp: sdp
};
pc.setRemoteDescription(jsep)
.then(function() {
console.log('Remote description accepted');
if(!offer) {
// We're done: just check if we have candidates to send
if(candidates.length > 0) {
// FIXME Trickle candidate
let headers = null;
if(token)
headers = { Authorization: 'Bearer ' + token };
let candidate =
'a=ice-ufrag:' + iceUfrag + '\r\n' +
'a=ice-pwd:' + icePwd + '\r\n' +
'm=audio 9 RTP/AVP 0\r\n';
for(let c of candidates)
candidate += 'a=' + c + '\r\n';
candidates = [];
$.ajax({
url: backend + resource,
type: 'PATCH',
headers: headers,
contentType: 'application/trickle-ice-sdpfrag',
data: candidate
}).error(function(xhr, textStatus, errorThrown) {
bootbox.alert(xhr.status + ": " + xhr.responseText);
}).done(function(response) {
console.log('Candidate sent');
});
}
// If we got here, we're in the "WHEP server sends offer" mode,
// so we have to prepare an answer to send back via a PATCH
pc.createAnswer({})
.then(function(answer) {
console.log('Prepared answer:', answer.sdp);
// Extract ICE ufrag and pwd (for trickle)
iceUfrag = answer.sdp.match(/a=ice-ufrag:(.*)\r\n/)[1];
icePwd = answer.sdp.match(/a=ice-pwd:(.*)\r\n/)[1];
pc.setLocalDescription(answer)
.then(function() {
console.log('Sending answer to WHEP server');
// Send the answer to the resource address
$.ajax({
url: backend + resource,
type: 'PATCH',
headers: headers,
contentType: 'application/sdp',
data: answer.sdp
}).error(function(xhr, textStatus, errorThrown) {
bootbox.alert(xhr.status + ": " + xhr.responseText);
}).done(function(response) {
console.log('Negotiation completed');
});
}, function(err) {
bootbox.alert(err.message);
return;
}
// If we got here, we're in the "WHEP server sends offer" mode,
// so we have to prepare an answer to send back via a PATCH
$('#whepsso').removeClass('hide');
pc.createAnswer({})
.then(function(answer) {
console.log('Prepared answer:', answer.sdp);
// Extract ICE ufrag and pwd (for trickle)
let headers = null;
if(token)
headers = { Authorization: 'Bearer ' + token };
iceUfrag = answer.sdp.match(/a=ice-ufrag:(.*)\r\n/)[1];
icePwd = answer.sdp.match(/a=ice-pwd:(.*)\r\n/)[1];
pc.setLocalDescription(answer)
.then(function() {
console.log('Sending answer to WHEP server');
// Send the answer to the resource address
$.ajax({
url: backend + resource,
type: 'PATCH',
headers: headers,
contentType: 'application/sdp',
data: answer.sdp
}).error(function(xhr, textStatus, errorThrown) {
bootbox.alert(xhr.status + ": " + xhr.responseText);
}).done(function(response) {
console.log('Negotiation completed');
});
}, function(err) {
bootbox.alert(err.message);
});
}, function(err) {
bootbox.alert(err.message);
});
});
}, function(err) {
bootbox.alert(err.message);
});
}, function(err) {
bootbox.alert(err.message);
});
}, function(err) {
bootbox.alert(err.message);
});
}

// Helper function to attach a media stream to a video element
Expand Down Expand Up @@ -222,7 +238,6 @@ function createPeerConnectionIfNeeded(iceServers) {
console.log('Handling Remote Track', event);
if(!event.streams)
return;
console.warn(event.streams[0].getTracks());
if($('#whepvideo').length === 0) {
$('#video').removeClass('hide').show();
$('#videoremote').append('<video class="rounded centered" id="whepvideo" width="100%" height="100%" autoplay playsinline/>');
Expand Down
53 changes: 34 additions & 19 deletions src/whep.js
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -59,6 +59,7 @@ class JanusWhepServer extends EventEmitter {

// Resources
this.janus = null;
this.multistream = false;
this.endpoints = new Map();
this.subscribers = new Map();
this.logger = new JanusWhepLogger({ prefix: '[WHEP] ', level: debug ? debugLevels.indexOf(debug) : 2 });
Expand Down Expand Up @@ -216,10 +217,13 @@ class JanusWhepServer extends EventEmitter {
this.emit('janus-disconnected');
// Reconnect
this.janus = null;
this.multistream = false;
setTimeout(this._connectToJanus.bind(this), 1);
});
this.janus = await connection.create();
this.logger.info('Connected to Janus:', this.config.janus.address);
const info = await connection.getInfo();
this.logger.info('Connected to Janus v' + info.version_string + ':', this.config.janus.address);
this.multistream = info.version >= 1000;
if(this.started)
this.emit('janus-reconnected');
}
Expand Down Expand Up @@ -249,22 +253,19 @@ class JanusWhepServer extends EventEmitter {
return;
}
this.logger.verb('/endpoint/:', id);
// If we received a payload, make sure it's an SDP
this.logger.debug(req.body);
let offer = null;
if(req.headers['content-type']) {
if(req.headers['content-type'] !== 'application/sdp' || req.body.indexOf('v=0') < 0) {
res.status(406);
res.send('Unsupported content type');
return;
}
offer = req.body;
// Make sure we received an SDP
if(!req.body || req.body.indexOf('v=0') < 0) {
res.status(415);
res.send('Missing SDP offer');
return;
}
if(offer && endpoint.plugin !== 'streaming') {
res.status(406);
res.send('Client offers unsupported by this endpoint');
this.logger.debug(req.body);
if(req.headers['content-type'] !== 'application/sdp') {
res.status(415);
res.send('Unsupported content type');
return;
}
let offer = req.body;
// Check the Bearer token
let auth = req.headers['authorization'];
if(endpoint.token) {
Expand Down Expand Up @@ -306,15 +307,21 @@ class JanusWhepServer extends EventEmitter {
let details = {};
if(endpoint.plugin === 'streaming') {
subscriber.handle = await this.janus.attach(StreamingPlugin);
// If we're connected to Janus 0.x, use server sent offers
subscriber.serverSentOffer = !this.multistream;
details.id = endpoint.mountpoint;
details.pin = endpoint.pin;
} else if(endpoint.plugin === 'videoroom') {
subscriber.handle = await this.janus.attach(VideoRoomPlugin);
// The VideoRoom plugin only supports server sent offers for subscriptions
subscriber.serverSentOffer = true;
details.room = endpoint.room;
details.feed = endpoint.feed;
details.pin = endpoint.pin;
} else if(endpoint.plugin === 'recordplay') {
subscriber.handle = await this.janus.attach(RecordPlayPlugin);
// The Record&Play plugin only supports server sent offers for subscriptions
subscriber.serverSentOffer = true;
details.id = endpoint.feed;
}
subscriber.handle.on(Janode.EVENT.HANDLE_DETACHED, () => {
Expand Down Expand Up @@ -342,8 +349,8 @@ class JanusWhepServer extends EventEmitter {
this.subscribers.delete(uuid);
}
});
if(offer) {
// Client offer (we still support both modes)
if(offer && !subscriber.serverSentOffer) {
// Client-side offer
details.jsep = {
type: 'offer',
sdp: offer
Expand All @@ -361,7 +368,7 @@ class JanusWhepServer extends EventEmitter {
endpoint.subscribers.set(uuid, true);
subscriber.resource = this.config.rest.basePath + '/resource/' + uuid;
subscriber.latestEtag = this.generateRandomString(16);
if(offer) {
if(offer && !subscriber.serverSentOffer) {
subscriber.sdpOffer = offer;
subscriber.ice = {
ufrag: subscriber.sdpOffer.match(/a=ice-ufrag:(.*)\r\n/)[1],
Expand Down Expand Up @@ -393,7 +400,8 @@ class JanusWhepServer extends EventEmitter {
}
res.setHeader('Link', links);
}
res.writeHeader(201, { 'Content-Type': 'application/sdp' });
// Reply with a 201 for client-side offers, and a 406 for server-sent-offers
res.writeHeader(subscriber.serverSentOffer ? 406 : 201, { 'Content-Type': 'application/sdp' });
res.write(result.jsep.sdp);
res.end();
endpoint.emit('new-subscriber');
Expand All @@ -417,7 +425,7 @@ class JanusWhepServer extends EventEmitter {
res.sendStatus(405);
});

// Patch can be used both for the SDP answer and to trickle a WHEP resource
// Patch can be used both for the SDP answer (for server-sent-offers) and to trickle a WHEP resource
router.patch('/resource/:uuid', async (req, res) => {
let uuid = req.params.uuid;
let subscriber = this.subscribers.get(uuid);
Expand All @@ -441,6 +449,13 @@ class JanusWhepServer extends EventEmitter {
}
if(req.headers['content-type'] === 'application/sdp') {
// We received an SDP answer from the client
if(!subscriber.serverSentOffer) {
// This is not a server-sent offer subscription, reject this
res.status(422);
res.send('Janus unavailable');
return;

}
this.logger.verb('/resource[answer]/:', uuid);
this.logger.debug(req.body);
// Prepare the JSEP object
Expand Down