Move forward and back with the arrow keys
Press F to find text within the slide deck

Watch this presentation on YouTube

Low cost, high quality audio and video communication
...and data!
WebRTC is a new front in the long war for an open and unencumbered web
Brendan Eich
– Mozilla CTO and inventor of JavaScript

WebRTC on Chrome

  • Bringing real-time communications to the web
  • Building a state-of-the-art media stack into Chrome
  • Developing a new communications platform
Voice is just another JS application
Henning Schulzrinne
– CTO, US FCC

WebRTC across platforms

  • Chrome
  • Chrome for Android
  • Firefox
  • Opera
  • Native Java and Objective-C bindings
Firefox/Chrome interoperability Firefox/Chrome interoperability
1,000,000,000+
WebRTC Endpoints
Apps using WebRTC

The WebRTC APIs

Three main tasks

  • Acquiring audio and video
  • Communicating audio and video
  • Communicating arbitrary data

Three main JavaScript APIs

  • MediaStream (aka getUserMedia)
  • RTCPeerConnection
  • RTCDataChannel

MediaStream

Acquiring audio and video

MediaStream

  • Represents a stream of audio and/or video
  • Can contain multiple 'tracks'
  • Obtain a MediaStream with navigator.getUserMedia()
MediaStream diagram

MediaStream

aka getUserMedia

var constraints = {video: true};

function successCallback(stream) {
  var video = document.querySelector("video");
  video.src = window.URL.createObjectURL(stream);
}

function errorCallback(error) {
  console.log("navigator.getUserMedia error: ", error);
}

navigator.getUserMedia(constraints, successCallback, errorCallback);

Constraints

  • Controls the contents of the MediaStream
  • Media type, resolution, frame rate
video: {
  mandatory: {
    minWidth: 640,
    minHeight: 360
  },
  optional [{
    minWidth: 1280,
    minHeight: 720
  }]
}
 

getUserMedia + Web Audio

// Success callback when requesting audio input stream
function gotStream(stream) {
    var audioContext = new webkitAudioContext();

    // Create an AudioNode from the stream
    var mediaStreamSource = audioContext.createMediaStreamSource(stream);

    // Connect it to the destination or any other node for processing!
    mediaStreamSource.connect(audioContext.destination);
}

navigator.webkitGetUserMedia({audio:true}, gotStream);

Make sure to enable Web Audio Input in about:flags!

gUM screencapture

var constraints = {
  video: {
    mandatory: {
      chromeMediaSource: 'screen'
    }
  }
};

navigator.webkitGetUserMedia(constraints, gotStream);

RTCPeerConnection

Audio and video communication between peers

Communicate Media Streams

WebRTC video chat: caller

getUserMedia
+
RTCPeerConnection
WebRTC video chat: callee

RTCPeerConnection does a lot

  • Signal processing
  • Codec handling
  • Peer to peer communication
  • Security
  • Bandwidth management

...

WebRTC architecture

WebRTC architecture diagram

RTCPeerConnection sample

pc = new RTCPeerConnection(null);
pc.onaddstream = gotRemoteStream;
pc.addStream(localStream);
pc.createOffer(gotOffer);

function gotOffer(desc) {
  pc.setLocalDescription(desc);
  sendOffer(desc);
}

function gotAnswer(desc) {
  pc.setRemoteDescription(desc);
}

function gotRemoteStream(e) {
  attachMediaStream(remoteVideo, e.stream);
}

RTCDataChannel

Bidirectional communication of arbitrary data between peers

Communicate arbitrary data

Game: caller
onreceivemessage = handle(data); ... var myData = [ { id: "ship1"; x: 24, y: 11, velocity: 7 }, .... ] send(myData);

RTCDataChannel
+
RTCPeerConnection
onreceivemessage = handle(data); ... var myData = [ { id: "ship7"; x: 19, y: 4, velocity: 18 }, .... ] send(myData);
Game: callee

RTCDataChannel

  • Same API as WebSockets
  • Ultra-low latency
  • Unreliable or reliable
  • Secure

RTCDataChannel API

var pc = new webkitRTCPeerConnection(servers,
  {optional: [{RtpDataChannels: true}]});

pc.ondatachannel = function(event) {
  receiveChannel = event.channel;
  receiveChannel.onmessage = function(event){
    document.querySelector("div#receive").innerHTML = event.data;
  };
};

sendChannel = pc.createDataChannel("sendDataChannel", {reliable: false});

document.querySelector("button#send").onclick = function (){
  var data = document.querySelector("textarea#send").value;
  sendChannel.send(data);
};

Servers and Protocols

Peer to peer — but we need servers :^\

Abstract Signaling

  • Need to exchange 'session description' objects:
    • What formats I support, what I want to send
    • Network information for peer-to-peer setup
  • Can use any messaging mechanism
  • Can use any messaging protocol

Signaling Diagram

JSEP architecture diagram

An RTCSessionDescription

v=0
o=- 7614219274584779017 2 IN IP4 127.0.0.1
s=-
t=0 0
a=group:BUNDLE audio video
a=msid-semantic: WMS
m=audio 1 RTP/SAVPF 111 103 104 0 8 107 106 105 13 126
c=IN IP4 0.0.0.0
a=rtcp:1 IN IP4 0.0.0.0
a=ice-ufrag:W2TGCZw2NZHuwlnf
a=ice-pwd:xdQEccP40E+P0L5qTyzDgfmW
a=extmap:1 urn:ietf:params:rtp-hdrext:ssrc-audio-level
a=mid:audio
a=rtcp-mux
a=crypto:1 AES_CM_128_HMAC_SHA1_80 inline:9c1AHz27dZ9xPI91YNfSlI67/EMkjHHIHORiClQe
a=rtpmap:111 opus/48000/2
...

STUN and TURN

P2P in the age of firewalls and NATs

An ideal world

Data pathways between peers if there were no NATs or firewalls

The real world

Data pathways showing firewalls

STUN

  • Tell me what my public IP address is
  • Simple server, cheap to run
  • Data flows peer-to-peer

STUN

Data pathways between peers using STUN

TURN

  • Provide a cloud fallback if peer-to-peer communication fails
  • Data is sent through server, uses server bandwidth
  • Ensures the call works in almost all environments

TURN

Data pathways between peers using TURN

ICE

  • ICE: a framework for connecting peers
  • Tries to find the best path for each call
  • Vast majority of calls can use STUN (webrtcstats.com):
Data pathways between peers using TURN

Deploying STUN/TURN

  • stun.l.google.com:19302
  • WebRTC stunserver, turnserver
  • rfc5766-turn-server
  • restund

Security

Security throughout WebRTC

Secure pathways

Secure pathways between peers

Architectures

Peer to peer: one-to-one call

Topology diagram: one to one

Mesh: small N-way call

Topology: full mesh

Star: medium N-way call

Topology diagram: one to three

MCU: large N-way call

Topology: MCU

Beyond browsers

Phones and more

  • Easy to interoperate with non-browser devices
    • sipML5 open source JavaScript SIP client
    • Phono open source JavaScript phone API
    • Zingaya embeddable phone widget

Telephony

Tethr

Tethr in action at Google I/O 2012

Building a WebRTC app

chrome://webrtc-internals

chrome://webrtc-internals screenshot

adapter.js

Lets you use the same code in all browsers:

  • Removes vendor prefixes
  • Abstracts Chrome/Firefox differences
  • Minimizes effects of spec churn
This is doing my head in.

JavaScript frameworks

SimpleWebRTC

Easy peer-to-peer video and audio

var webrtc = new WebRTC({
  localVideoEl: 'localVideo',
  remoteVideosEl: 'remoteVideos',
  autoRequestMedia: true
});

webrtc.on('readyToCall', function () {
    webrtc.joinRoom('My room name');
});

PeerJS

Easy peer-to-peer data

var peer = new Peer('someid', {key: 'apikey'});
peer.on('connection', function(conn) {
  conn.on('data', function(data){
    // Will print 'hi!'
    console.log(data);
  });
});

// Connecting peer
var peer = new Peer('anotherid', {key: 'apikey'});
var conn = peer.connect('someid');
conn.on('open', function(){
  conn.send('hi!');
});

Services

  • Complete video services:
Data center locations

More Information

Contact Us

WebRTC and HTML5 could enable the same transformation for real-time communications that the original browser did for information.
Phil Edholm
— NoJitter

<Thank You!>