Category Archives: JavaScript

Peer Connection Tutorial – Part1

Wow! I’ve had a huge break from blogging for a while now especially about WebRTC. I’m sure you all have been tracking how much WebRTC has been evolving over the past few months. The most recent and exciting news which you have got to know is the interop between Chrome and Mozilla has landed and it is undeniably super cool. You could check out the demo by following up this link. So, without any further delay let me start with the peer connection tutorial which I am sure is a component everyone is dying to try.

I have realized that a peer connection tutorial has got to have a minimum of two parts so that I do at least some justice to it. So here in part1, lets start with the basics of what you have got to know to get started with establishing your own peer connection i.e learn how a basic call is established.
Assumptions : I assume that you can understand the basics of HTML and JavaScript so that you can understand the other parts of what is happening and that you have already ready my tutorial on gUM or you know the basic usage of gUM. A little bit of going through the WebRTC blogs (I’ve listed a few in the last) would be really helpful as well.

I will be using the code from Anant’s demo to walk you through the steps of setting up a call. It provides an easy way of testing the peer connection component (especially when you are not hosting your app). This page basically has two video elements, one called Remote and the other called Local. The video captured by your local machine (using gUM) is displayed by your Local video component. The basic idea is to establish a call using WebRTC’s peer connection component between the Remote and Local video components on the page. Thus finally when the call is established the video captured by your local machine is streamed to the Remote Video element on that page through its connection with the Local Video element. I highly recommend you try out this demo yourself on Peer Connection Test Page before getting started with this tutorial so that you have a clear idea of what you are trying to build. (I’m not sure if it works on windows yet :|)

Note: For the demo to work you have to have the latest version of Mozilla Nightly. Now that the peer connection preferences are turned on by default I don’t think you need to tinker with your browser preferences.

Lets start of from the very beginning. When I want to initiate a call what should I do? ( what to do in the code when I click on the start button in the pc test page ) – The first and foremost step is to create the peer connection objects based on the number of connections that are required. For our example, we require only two, so we create pc1 and pc2 as follows:

pc1 = new mozRTCPeerConnection();
pc2 = new mozRTCPeerConnection();

The peer connection objects has several in built methods which can be used quite effectively as you will notice in this tutorial. The next important step is to capture the local video by making use of gUM as follows:

navigator.mozGetUserMedia({video:true}, function(video1) {

In the above function we have set the captured media by the gUM function to video1 i.e. currently video1 is the local machine’s captured video. The next part deals with setting the local video source as video1. Once the video begins to play (thanks to invoking play()) we add this video to the Stream attribute of the peer connection object by making use of the method addStream(video)

// Add stream obtained from gUM to <video> to start media flow.
localvideo.mozSrcObject = video1;
localvideo.play();
pc1.addStream(video1);

So now, we have the local video running. We will also create fake audio streams for both pc1 and pc2 (The reason we have pc1’s audio as a fake stream is due to the echo problem) . These can be used later on in case some captured audio has to be added to the stream.

navigator.mozGetUserMedia({audio:true, fake:true}, function(audio1) {
pc1.addStream(audio1);
pc2.addStream(audio1);

Similarly we also add a fake video stream to the pc2 object, so that once the call is established the video from pc1 can be added to this stream.

navigator.mozGetUserMedia({video:true, fake:true}, function(video2) {
pc2.addStream(video2);

You might be wondering if I’ve forgotten the method onaddstream. Don’t worry I don’t forget that easily πŸ˜› So, this function is invoked everytime the addStream function is called. Let me explain this with respect to pc1.onaddstream. In this method we set the source stream for the pc2 video and we play it using the play method. Since, intially there is only a fake video stream nothing will play, but later on once the call is established – voila! pc2 will be playing the local video from pc1 i.e it plays the remote stream when a video call is established.

pc1.onaddstream = function(obj) {
log("pc1 got remote stream from pc2 " + obj.type);
pc2video.mozSrcObject = obj.stream;
pc2video.play();
}

The pc2.onaddstream does something similar, but this is for the case where pc2 is the local stream and pc1 is the remote stream.

pc2.onaddstream = function(obj) {
log("pc2 got remote stream from pc1 " + obj.type);
pc1video.mozSrcObject = obj.stream;
pc1video.play();
}

Now that we have everything ready, the only thing left is to establish a call. In WebRTC, calls are established through a process called Signalling which can roughly be defined as – “For a caller to be able to make a call he first has to make an offer to the callee (general procedure) and for the call to be established it is required that the callee accepts the offer and return an answer which is in turn accepted by the callee (yeah, yeah its the preliminary handshake). “ Signalling in itself is a huge topic to cover and is achieved through a protocol called Interactive Connectivity Establishment (ICE) and I plan to explain it in the next blogpost. For now, all you need to know about signalling is a higher level abstraction of it which is pretty much clearly stated in the definition. So the first step to signalling takes place here (creating an offer):

pc1.createOffer(step1, failed);

Here, tha basic idea is to make pc1 create an offer to pc2 as mentioned before. The offer will contain the Local Description of pc1 object and this Description is generated in the method setLocalDescription(..) in the function step1 as follows:

// pc1.createOffer finished, call pc1.setLocal
function step1(offer) {
pc1_offer = offer;
pc1.setLocalDescription(offer, step2, failed);
}

Now that pc1 has set its description, we have to set the Remote Description of pc2. This is done through the method setRemoteDescription(..) using the previously generated offer in the function step2 which is invoked while setting the Local Description of pc1 above. The function step2 looks like this:

// pc1.setLocal finished, call pc2.setRemote
function step2() {
pc2.setRemoteDescription(pc1_offer, step3, failed);
};

Now that the Local and Remote Descriptions are set for pc1 and pc2 respectively, it is required for pc2 to create an Answer which it will then send to pc1. This answer is created using the method pc2.createAnswer(..) in the function step3 invoked while creating the remote Description of pc2.

// pc2.setRemote finished, call pc2.createAnswer
function step3() {
pc2.createAnswer(step4, failed);
}

After creating the answer now pc2 also creates a Local Description using this answer. This is done through the method pc2.setLocalDescription(answer, step5, failed) as shown in the function step4 which is invoked previously while creating the answer:

function step4(answer) {
pc2_answer = answer;
pc2.setLocalDescription(answer, step5, failed);
}

Finally, the last step that remains is to set the Remote Description of pc1 using the answer generated previously by pc2. This is done in the method pc1.setRemoteDescription(pc2_answer, step6, failed) in the function step5 invoked while setting the Local Description of pc2:

// pc2.setLocal finished, call pc1.setRemote
function step5() {
pc1.setRemoteDescription(pc2_answer, step6, failed);
}

At long last our call is established – and your eyes will be filled with tears of joy πŸ˜›

// pc1.setRemote finished, media should be running!
function step6() {
log("HIP HIP HOORAY");
}

Now look back at all the steps – not so bad eh? – Doable? – Of course. This is the power of WebRTC – peerConnection established in a few simple steps. Though it does get a little tricky when it comes to real world apps (which I realized very recently), its still manageable right?! For those of you who think its not, there are a few great APIs (one of which is AddLive’s API) available and more coming πŸ˜‰

I almost forgot – still a few final steps left.
In case of an error you can add the required error handlers and handle the errors differently/better depending upon your application/usage.But this is not a part that we shall cover here.

function failed(code) {
log("Failure callback: " + code);
}

Finally when you want to end the call you just close the peer connection using the syntax pc.close()

function stop() {
pc1.close();
pc2.close();

button.innerHTML = "Start!";
button.onclick = start;
}

That’s about it for this PeerConnection Overview.

In the next tutorial, I will try to explain what is going on underneath this neat JavaScript code. So it is enough if you have an abstract idea of how this works for now πŸ˜‰ If possible I will also try to give you guys a brief idea of the underlying ICE protocol in the next blogpost. Hope you find this helpful. If you have any queries, feel free to comment here or ask on Twitter πŸ™‚

Advertisements

WebRTC – Getting started !

I have got some exciting news – The WebRTC project is gaining some real momentum now. How do I know this? For starters the weekly Mozilla WebRTC meeting on Google+ has become crowded. A couple of months back when I first joined the hangout to see what WebRTC was all about there used to be hardly six people. Now, I have to fight for space on the hangout as only ten people can be accommodated at a time, and people have to keep dropping off after their relevant part is done to accommodate others (Yes! we need a better meeting solution – hopefully WebRTC will cater to our needs). The other reason for this gain in momentum is more people are getting involved as the components are getting stabler and better by the day thanks to the super enthusiastic, skilled and hardworking people involved in the project. Presently, there are a few developers working on creating some cool demos (and hopefully tutorials) for WebRTC which is exactly what we want now. Now, gUM is pretty much stable. Peer Connection has made real progress with a few demos where you could make video calls and Datachannels is being used to create a demo – this is a social app with filesharing also included as of its many awesome features. This is what I call real progress. Itching to try out WebRTC? Don’t know where to start? Want to know what can be done? Well, I am hoping this post will be of some use to all those who are deluded by these questions and maybe it will even fuel your interest to start experimenting with WebRTC.

A good place to start exploring WebRTC is gUM. My first reason for suggesting gUM is that it is the most stable and least buggy of all the WebRTC components. The second reason is that it is the easiest to use for a newbie as there is very little to learn about how it works and it can be easily integrated with whatever you are working on πŸ˜‰ I forgot to mention the obvious, the reason I use it – its just so muchΒ  FUN! πŸ™‚ A simple demo on gUM created by Anant is the best place to try out gUM. Here, you can see that you can get audio and video independently or together, how cool is that? And of course for those of you obsessed with yourselves there is an option to take a photo of yourself! Yay! To get this demo to work you will need the Nightly version of Firefox from nightly.mozilla.org Then you will have to change the preferences which can be accessed by typing about:config in the address bar. Then toggle the following two preferences to true – media.navigator.enabled and media.navigator.permission.disabled. This should have you all set for the demo and voila! Your first look into WebRTC! (*Any bugs or problems you have, we would be happy to hear about them and fix them asap – you can report on bugzilla or just add a comment to this post*)

To get a better idea on how to write code check out the source of the demo on gUM that I had previously mentioned. Now, let us walk through the code or rather the part that matters the most and help you get a better idea on how this works! I am sure you’ll manage with the rest of the code. (My explanations might sound naive to a first time user but you are welcome to mail me or catch me on IRC)

    try {

The gUM function starts here. You can specify the parameters you want to pass through the param argument. A valid set of parameters would be the ones which determine the values of video_status, audio_status and picture_status bool values used later on.

window.navigator.mozGetUserMedia(param, function(stream) {
message.innerHTML = "<p>Success!</p>";

Depending on the bool value of video_status the video stream is obtained and played or the code in the if block is ignored.

if (video_status) {
 content.appendChild(video);
 video.mozSrcObject = stream;
 video.play();

Depending on the bool value of audio_status the audio stream is obtained and played or the code in the if block is ignored.

} else if (audio_status) {
content.appendChild(audio);
audio.mozSrcObject = stream;
audio.play();

Depending on the bool value of picture_status the picture is taken and displayed or the code in the if block is ignored.

} else if (picture_status) {
content.appendChild(picture);
picture.src = window.URL.createObjectURL(stream);
picture.onload = function(e) {
window.URL.revokeObjectURL(this.src);
}
}

The function and catch statements are used to display error messages in case gUM fails due to some of the commonly known ways of incorrect usage.

 }, function(err) {
message.innerHTML = "<p class='error'>" + err + "</p>";
stopMedia();
});
} catch(e) {
message.innerHTML = "<p class='error'>" + e + "</p>";
stopMedia();
}

For more info on gUM you should check out the following links –
W3C Draft

First part of WebRTC landed

gUM has just Landed

gUM still image capture for Android has landed

Talk on WebRTC

JSFoo Bangalore

Let me start this off with the story of how my talk got selected….yeah I SPOKE at JSFoo ! I submitted my talk proposal for PyCon and JSFoo at around the same time because they were the conferences I found most apt and tailored to my requirements, taking into consideration this year alone (Sept-Nov). I was lucky that my talk for JSFoo did actually get through, because this was my first chance to deliver an official talk to a tech oriented audience with more or less similar interests. The theme of this year’s JSFoo was “JavaScript everywhere”. You can check out the website (I know its too late – but you can always go next year) at JSFoo 2012

T-shirt

I am presently contributing to this project called – WebRTC at Mozilla, doing some bug fixes and helping out with a few tests. I thought this would be a great topic to talk about because this was a project with immense potential with all the big names like Google, Mozilla, Cisco, Ericsson, Opera etc involved in it and once this is out it will revolutionize the Web. So I talked to those involved in this project on the IRC and they thought it was a great idea too. They also helped me get in touch with the Mozilla Developer Relations team who would help me with selecting the content to present at JSFoo. My talk proposal can be found here – JSFoo WebRTC. I was very sure that the proposal wouldn’t get through because of the number of votes on it, but it did get through and nothing could contain my excitement. In this process, I also applied for the Mozilla Reps program and after a brief interview I became a Mozilla rep before i knew it. Whoa! So much happened in just over three weeks. Funny that nothing exciting like this ever happens at university (I thought universities were meant for this) !

Now, it was time to get started with preparing the slides for my talk as I had to submit them for review before JSFoo. Oh! I almost forgot I also did a sneak preview talk – WebRTC preview during PyCon about my talk at JSFoo thanks to HasGeek. I know the expression in the end is funny (made it quite popular at uni) but that’s not the point πŸ˜› Got the slides prepared with some help from a few blogs, Justin Uberti’s talk at Google i/o , Anant’s talk at MozCamp Europe and of course my own content. The slides can be found at JSFoo WebRTC slides. All the references are added under – “Want to know more?” – So, no copyright issues please πŸ™‚

Hall

Time flies real fast, before I knew it I was NIMHANS Convention center at Bangalore. It was a really cool place for a conference. It was great to be back in this environment, I had been wanting to ever since PyCon. Learnt a lot of new stuff during the first half – kicked off with a brilliant keynote, followed by a great talk by Rakesh Pai about what was being done at his startup called Errorception with node.js which was really informative. Then, there was a talk by Shreyank Gupta (who also became good friend) about Underscore.js and I realized then how fucked up my JavaScript code was in terms of readability and understandability. My advice to you all from the first session – use Java Script, experiment with node.js and score with Underscore.js. Then during the lunch I met a a few guys from a startup called Bang the table. They had interesting ideas and a lot of content to share. Had great food and good discussions. After lunch was the session on using CreateJS and Canavas by Haris Shivaramakrishanan, he was Adobe’s technical Evangelist. He showed us how to create a simple game using CreateJS in around 30 mins and how much fun it was to do it! He is a great speaker and it was a really entertaining talk which showed me that I could finish my Graphics mini project at university in 2 hours. Wasn’t really into the next session but I enjoyed the session by Jon Maim, founder of Minish on AngularJS.Β  Now, it was time for a cup of tea and then I was back again listening to how to internationalize Web Applications with the help of Wikimedia Libraries. The final session by Rakesh Pai just blew my mind – he built a robot with Raspberry Pi, Arduino, an Ultrasonic sensor and a few hardware parts on wheels. He controlled the robot which ran node on its Raspberry Pi with his laptop (obviously there was a wifi module) and this was really amazing to watch. It was one of the best sessions I had been to. Forgot to mention, in the midst of all the excitement during the breaks I was trying to get my WebRTC demos working – yeah they were NOT working :/ During this time I met some really great and helpful people – Kingsley, Sajjad, Yuvi, Harris, Vamsi and anyone whom I forgot to mention do forgive me (writing this blog in a hurry). Kingsley is like the GOD, Sajjad was a fellow Mozilla rep and he does a lot of stuff for HasGeek, Yuvi also does a lot of stuff for HasGeek and is a Wikimedia contributor, Harris is a new intern at HasGeek and Vamsi worked at Bang the Table. In the evening made friends with Shreyank who worked with Red Hat R&D at Pune. So, made great friends again – which is again one of the primary purposes of attending conferences.

Nagarjuna

It was 6pm and that meant it was time to PARTY! We had a party at the Xtreme Sports Bar. Been long since I had been there and it was great time – drinking beer, hogging, meeting new people and socializing. Also, played pool for sometime and in simple terms that night was just AWESOME! Got back and tried to see if I could do something about the demos not working on Windows. Went on IRC asked the guys, who told me they had just landed a patch on this issues and now the demos had to work. Updated my Nightly and tried…still didn’t work. Left early the next day to figure something out, and the moment I tried all the demos seemed to work. Yipee! – so the patch had landed and resolved all my issues. Now, I was looking through all my slides rehearsing for my talk and trying to shorten it to complete it within 30 mins or rather 25 mins as the last five minutes were for questions. So, I didn’t or rather couldn’t listen to Sankha talking before me. He had some technical issues with the projector and I was hoping the same wouldn’t happen to me. Now, it was my turn. I wentΒ  on stage feeling slightly nervous but once I began to speak I forgot it all πŸ™‚ I too had issues with the projector unfortunately but it was more fun talking so it didn’t disturb me much. Though, I couldn’t show everyone the code for PeerConnection and DataChannels due to flickering projector (to emphasize on how simple the JS api for WebRTC was) I think the talk went pretty well (Disclaimer : Audience poll not taken). I think it would have been great if the projector didn’t act funny but I still had a great time and I really have to appreciate the tech support by the HasGeek team. The bottomline is I that I discovered that I love speaking on stage and I can’t wait to do it once more (please don’t curse me guys – I’ll show better stuff next time) πŸ™‚

Arduino Bot

Again, it was a great day. I really enjoyed it and during lunch I got to talk to Aravind and Gaurav who were the founders of askabt, a new startup which was about to release their product at the end of this month. Learnt a lot from them and shared a lot of ideas. I also got a JSFoo T-shirt as I was a speaker…lucky me! I also talked to Jon after my talk as he seemed interested in WebRTC and also I was interested in his work at EPFL and Minsh. I also met Sunil after his talk and he is like this open and friendly super informative encyclopedia. His talk was fun and taught me new stuff. With Sunil and Jon’s help I am sure I can complete my Graphics mini project at college in an hour. As usual there was an awesome talk by Anand S on d3.js and it was simply entertaining and very informative. I think I am going to experiment with d3.js, it does look really tempting – what amazing visualizations it offers! In the end there was a talk by Sudar from Yahoo who also use JS to develop an Arduino bot which could be controlled from any laptop which was given access to it. Everyone controlled it and played around with it for a while. This was quite a fun session! After this it was the end of the conference and time for the vote of thanks and everyone to go back home. I wish everyone could stay back, I didn’t want to leave such a great crowd! Anyways I guess all good things have to come to an end. So, I really want to thank Zainab, Kiran, Sunil, Rakesh and all the other guys involved in organizing this brilliant event. Thanks guys! We should do this much more often!

Photos by : $hrink