Peer Connection Tutorial – Part1
Wow! I’ve had a huge break from blogging for a while now especially about WebRTC. I’m sure you all have been tracking how much WebRTC has been evolving over the past few months. The most recent and exciting news which you have got to know is the interop between Chrome and Mozilla has landed and it is undeniably super cool. You could check out the demo by following up this link. So, without any further delay let me start with the peer connection tutorial which I am sure is a component everyone is dying to try.
I have realized that a peer connection tutorial has got to have a minimum of two parts so that I do at least some justice to it. So here in part1, lets start with the basics of what you have got to know to get started with establishing your own peer connection i.e learn how a basic call is established.
Assumptions : I assume that you can understand the basics of HTML and JavaScript so that you can understand the other parts of what is happening and that you have already ready my tutorial on gUM or you know the basic usage of gUM. A little bit of going through the WebRTC blogs (I’ve listed a few in the last) would be really helpful as well.
I will be using the code from Anant’s demo to walk you through the steps of setting up a call. It provides an easy way of testing the peer connection component (especially when you are not hosting your app). This page basically has two video elements, one called Remote and the other called Local. The video captured by your local machine (using gUM) is displayed by your Local video component. The basic idea is to establish a call using WebRTC’s peer connection component between the Remote and Local video components on the page. Thus finally when the call is established the video captured by your local machine is streamed to the Remote Video element on that page through its connection with the Local Video element. I highly recommend you try out this demo yourself on Peer Connection Test Page before getting started with this tutorial so that you have a clear idea of what you are trying to build. (I’m not sure if it works on windows yet :|)
Note: For the demo to work you have to have the latest version of Mozilla Nightly. Now that the peer connection preferences are turned on by default I don’t think you need to tinker with your browser preferences.
Lets start of from the very beginning. When I want to initiate a call what should I do? ( what to do in the code when I click on the start button in the pc test page ) – The first and foremost step is to create the peer connection objects based on the number of connections that are required. For our example, we require only two, so we create pc1 and pc2 as follows:
pc1 = new mozRTCPeerConnection();
pc2 = new mozRTCPeerConnection();
The peer connection objects has several in built methods which can be used quite effectively as you will notice in this tutorial. The next important step is to capture the local video by making use of gUM as follows:
navigator.mozGetUserMedia({video:true}, function(video1) {
In the above function we have set the captured media by the gUM function to video1 i.e. currently video1 is the local machine’s captured video. The next part deals with setting the local video source as video1. Once the video begins to play (thanks to invoking play()) we add this video to the Stream attribute of the peer connection object by making use of the method addStream(video)
// Add stream obtained from gUM to <video> to start media flow.
localvideo.mozSrcObject = video1;
localvideo.play();
pc1.addStream(video1);
So now, we have the local video running. We will also create fake audio streams for both pc1 and pc2 (The reason we have pc1’s audio as a fake stream is due to the echo problem) . These can be used later on in case some captured audio has to be added to the stream.
navigator.mozGetUserMedia({audio:true, fake:true}, function(audio1) {
pc1.addStream(audio1);
pc2.addStream(audio1);
Similarly we also add a fake video stream to the pc2 object, so that once the call is established the video from pc1 can be added to this stream.
navigator.mozGetUserMedia({video:true, fake:true}, function(video2) {
pc2.addStream(video2);
You might be wondering if I’ve forgotten the method onaddstream. Don’t worry I don’t forget that easily 😛 So, this function is invoked everytime the addStream function is called. Let me explain this with respect to pc1.onaddstream. In this method we set the source stream for the pc2 video and we play it using the play method. Since, intially there is only a fake video stream nothing will play, but later on once the call is established – voila! pc2 will be playing the local video from pc1 i.e it plays the remote stream when a video call is established.
pc1.onaddstream = function(obj) {
log("pc1 got remote stream from pc2 " + obj.type);
pc2video.mozSrcObject = obj.stream;
pc2video.play();
}
The pc2.onaddstream does something similar, but this is for the case where pc2 is the local stream and pc1 is the remote stream.
pc2.onaddstream = function(obj) {
log("pc2 got remote stream from pc1 " + obj.type);
pc1video.mozSrcObject = obj.stream;
pc1video.play();
}
Now that we have everything ready, the only thing left is to establish a call. In WebRTC, calls are established through a process called Signalling which can roughly be defined as – “For a caller to be able to make a call he first has to make an offer to the callee (general procedure) and for the call to be established it is required that the callee accepts the offer and return an answer which is in turn accepted by the callee (yeah, yeah its the preliminary handshake). “ Signalling in itself is a huge topic to cover and is achieved through a protocol called Interactive Connectivity Establishment (ICE) and I plan to explain it in the next blogpost. For now, all you need to know about signalling is a higher level abstraction of it which is pretty much clearly stated in the definition. So the first step to signalling takes place here (creating an offer):
pc1.createOffer(step1, failed);
Here, tha basic idea is to make pc1 create an offer to pc2 as mentioned before. The offer will contain the Local Description of pc1 object and this Description is generated in the method setLocalDescription(..) in the function step1 as follows:
// pc1.createOffer finished, call pc1.setLocal
function step1(offer) {
pc1_offer = offer;
pc1.setLocalDescription(offer, step2, failed);
}
Now that pc1 has set its description, we have to set the Remote Description of pc2. This is done through the method setRemoteDescription(..) using the previously generated offer in the function step2 which is invoked while setting the Local Description of pc1 above. The function step2 looks like this:
// pc1.setLocal finished, call pc2.setRemote
function step2() {
pc2.setRemoteDescription(pc1_offer, step3, failed);
};
Now that the Local and Remote Descriptions are set for pc1 and pc2 respectively, it is required for pc2 to create an Answer which it will then send to pc1. This answer is created using the method pc2.createAnswer(..) in the function step3 invoked while creating the remote Description of pc2.
// pc2.setRemote finished, call pc2.createAnswer
function step3() {
pc2.createAnswer(step4, failed);
}
After creating the answer now pc2 also creates a Local Description using this answer. This is done through the method pc2.setLocalDescription(answer, step5, failed) as shown in the function step4 which is invoked previously while creating the answer:
function step4(answer) {
pc2_answer = answer;
pc2.setLocalDescription(answer, step5, failed);
}
Finally, the last step that remains is to set the Remote Description of pc1 using the answer generated previously by pc2. This is done in the method pc1.setRemoteDescription(pc2_answer, step6, failed) in the function step5 invoked while setting the Local Description of pc2:
// pc2.setLocal finished, call pc1.setRemote
function step5() {
pc1.setRemoteDescription(pc2_answer, step6, failed);
}
At long last our call is established – and your eyes will be filled with tears of joy 😛
// pc1.setRemote finished, media should be running!
function step6() {
log("HIP HIP HOORAY");
}
Now look back at all the steps – not so bad eh? – Doable? – Of course. This is the power of WebRTC – peerConnection established in a few simple steps. Though it does get a little tricky when it comes to real world apps (which I realized very recently), its still manageable right?! For those of you who think its not, there are a few great APIs (one of which is AddLive’s API) available and more coming 😉
I almost forgot – still a few final steps left.
In case of an error you can add the required error handlers and handle the errors differently/better depending upon your application/usage.But this is not a part that we shall cover here.
function failed(code) {
log("Failure callback: " + code);
}
Finally when you want to end the call you just close the peer connection using the syntax pc.close()
function stop() {
pc1.close();
pc2.close();
button.innerHTML = "Start!";
button.onclick = start;
}
That’s about it for this PeerConnection Overview.
In the next tutorial, I will try to explain what is going on underneath this neat JavaScript code. So it is enough if you have an abstract idea of how this works for now 😉 If possible I will also try to give you guys a brief idea of the underlying ICE protocol in the next blogpost. Hope you find this helpful. If you have any queries, feel free to comment here or ask on Twitter 🙂
The story of a boy, Reddit, The Dark Knight Rises, Open Source and Freedom !
Yes, this is the title I have chosen to tell you my story of how Open Source has changed my life over the past year. But, this is not where I shall begin. I had finished writing all my Entrance Examinations which would determine if I could study in India’s very best Engineering colleges or not. I had written a range of exams from IIT to COMEDK, and I was expecting pretty good results because I felt that I was pretty good at these exams. Though I was not very confident (actually quite sure) about getting into an IIT, I was sure that I would definitely make it into either BITS or an NIT. And I did make it into these universities, but alas! the only stream I was offered at NIT Surathkal was Metallurgy and in BITS, Goa was Chemical. I did do a good job in the COMEDK and Amrita Vishwavidyapeetham Entrance Examinations (against my wishes) and I did get into these universities with a choice of the branches that I wanted to get in – Mechanical or Computer Science. But, I was so devastated (yeah, I was a complete idiot back then) that I didn’t get into any of the universities I wanted, that I made what I would call the worst decision I had made till date – I chose to study in Amrita Vishwavidyapeetham University instead of universities in Bangalore like PESIT and MSRIT. I knew I was not doing the right thing, but it didn’t matter, I was completely lost and beaten at all levels. It does look like I’m making a big deal out of a real small issue but this is what happens when a kid who has never failed at what he has wanted to achieve since he could remember does not get something which he considers is his own, by nature. (also the effect of being locked up in a dungeon for more than three years)
That’s how I ended up in the serene, tranquil and beautiful place called Ettimadai, which is where my university is. But inside the university, its a completely different story. Yeah, maybe its a dream university for many people and maybe it really is good, but good and dream is always a matter of perspective. To me it was like a dungeon, not only physically but also mentally. I had lost all interest to learn and let go of all my dreams and ambitions when I had come here. Probably why I felt like everything was closed in all directions. I mean, everything else was usual or what you could call usual at Amrita, bitching about classes, talking about how bad life had got, playing some or the other sport everyday, studying and doing fun stuff once in a while, making plans to overthrow the power hungry bureaucrats, schemes to end the tyranny at university and finally go to sleep. But what lacked was a sense of purpose and satisfaction to what I was doing in addition to being mentally chained (And this I have realized is one of the few things that really matters in life). So, enough about the SAD life and lets start looking at what started the STORM. I really loved going to techfests and conferences (though I wasn’t really passionate about anything back then), thought they were like my occasional escape routes 😉 Now, I have heard quite a bit about Open Source from my friends at Bangalore and people around (who were still truly living unlike me and yeah, we don’t have internet or LAN connection in our rooms at uni – what’d you expect!) and so, I decide to find out what this thing is all about and I register for the Pengufest – A workshop on Open Source Software and Communities which is happening at NIT Thrichy’s annual techfest.
Its community
I and a couple of friends from my uni make it NIT Thrichy and we’re all quite excited to be there. We think we will finally be able to get an idea of what CS is all about, because we don’t know anything about it except some crap from the textbook and some labs in which we were tested on how well we could remember the syntax. Ah! I forgot the good part, a couple of weeks before I had looked up all the speakers and googled them (Yeah, we all have laptops and mobile broadband cards – where there is a will there is a way :P). This was when I looked into Reddit, because Alexis Ohanian was one of the speakers at the techfest. I began exploring Reddit and I really loved it, though I had to deactivate my account at a later stage due to addiction. Now that I’m almost done with uni, I’m gonna be back – excited and looking forward to it. This formed the central reason to why I was going to the techfest – to attend Alexis Ohanian’s talk. We did have fun at NIT Thrichy, though we found the infrastructure to be below the standards of our uni, they had something far more better – FREEDOM and a great community. Finally we are at the workshop and I learn a lot, the most important of which is what GSoc is all about. Then, there is a session on Git but the Alexis Ohanian talk is rescheduled to the same time as the Git workshop. So, I give up the git workshop, none of my friends want to come with me but still no one’s going to stop me from attending Alexis Ohanian’s talk. I dunno why but I just wanted to attend the talk so badly – call it fate, destiny or whatever but I just had to be there. And there I was – it was a skype talk over a projector and really bad bandwidth. But, the very presence of Alexis Ohanian just gave me the vibes. When, he started talking I just could feel the vibes and somehow the interest that I had lost, the passion and dreams that I had let go all started coming back to me. There’ s a lot of what he said that I really liked and was influenced by but that’s for another post. Basically, I was just taken off my feet by his open, friendly and calm nature. I felt he was just someone like me but I knew he had done a lot more (way way more) and hence, ever since then he has always been a very captivating role model for me. And this was the beginning, though I was low on self confidence, though I was going to step back into the real world after almost two years, I had a direction I could follow and this was definitely a start. I really believe Reddit and Alexis Ohanian were partially responsible for this. So thanks guys! I owe you one, big time! 🙂
The very next semester, I started daring to dream. I felt like I really had to get some stuff going my way and thanks to an amazing Networking Prof. I had then, I was allowed to do my Computer Networks mini project in Python – Yay! I decided to implement something completely out of the box (based on a friend’s advice) SNMP – Simple Network Management Protocol. Turns out there are different version of this which can be implemented through the PySNMP framework and we ended up experimenting with each one finally choosing v2. It was a great experience, though I couldn’t get everything that I wanted to work working, the proxy that we wrote in Python and the different SNMP bits we implemented was weirdly satisfying (I’d also tried out some bits of the twisted framework then). This is where, my learning curve was really starting to take shape. This time I really wanted to take part in GSoc. Though, I started off a bit late I started lurking on IRCs pestering a lot of people and trying to get to know how things worked in the Open Source Universe. This was when I discovered my community – a study group, a technical community with whom I could identify myself with. This was just like how people connected in a Metal concert or at home in a family, it was inbuilt – everyone just got together and built something and they enjoyed the process and what the community as a whole had to offer. This was HEAVEN for me! For metal concerts and technical conferences I had to travel and these happened once in a blue moon, but with Open Source everything was in my laptop, through it I was the king of my own little world. Getting back to GSoc, after a lot of IRC discussions in different channels I finally decided that at this stage two organizations suited me and my skill set to a really good level – Mozilla and Apertium.
I realized that GSoc was more about getting started in your area of interest, a project you are interested in and the best place to kickstart your journey as an Open Source Developer. The Apertium community was just great, lot of amazing people who were ready to help and more importantly teach in a way that I could learn by myself. They fueled my Open source desire even more and this is how I started off – trying to build a Machine Translation system through Apertium’s toolkit for the Hindi-English language pair (gotta really work on it seriously sometime).
At Mozilla again, lot of people helped me out to get started. Here, is where I made my first patch and to see it online and knowing that it was going to be part of Firefox was something different for me. Through these communities I had learned so much, a hundred times more than what I had learned at my university. Though I didn’t get through GSoc that year ( to save my pride – it was a narrow margin) I got into Open Source Development and this was a turning point in my life. And this was the FIRE rising, everything was falling into place. I was rediscovering all my interests and hobbies, and working on them too. Ah! This was bliss, this was what I had wanted – now I had a purpose, to change the world through my skills, add some value to it or at least point out where it has gone wrong and do something about it. (yeah, I know – overstatement but everyone loves to hear that , me included)
RISE
Finally, how can I not talk about Nolan’s films when I’ve written an thought provoking, hope rendering and what I think to be an inspirational blog. Obviously everyone has watched The Dark knight Rises and it is by far one of the best movies I’ve ever seen. I had given up an interview for this movie – by Mu Sigma, which might be good in terms of money but it just isn’t my kind of work. So, how has the Dark Knight rises had an impact on me,well – I’m now trying to climb out of the hole with no rope. Its made me introspect and has restored my lost confidence. Obviously everything I’ve done has resulted in what I am now but this movie served as the final trigger (which was pushed easily because I saw it at iMAX) which has brought back my confidence and self belief. Every time I feel a bit low, I just watch that scene where Bale climbs out of the well – thanks Hans Zimmer, that’s some really powerful music. Movies like this and lot of others which I have reserved for future posts have had a great influence on me. Fight for what you believe in and never look back (maybe except for mistakes so you don’t repeat them :P). That’s what I am trying to do now – An MS in Computer Science abroad, lets see what happens because I have no net to hold me. All I know is this, I can fight back and I will fight tooth and nail for what I believe in – so no worries 🙂
Though my physical freedom is still in question, I have become mentally free and ideas just started exploding, I’m transcending all the previously existing boundaries. I have begun to “Connect the dots” to find out what I really want. Now, I feel great, there’s always some kind of deeply satisfactory feeling inside everyday and it looks like each day has so much to offer. Wow! Now I feel stupid that I didn’t realize that all this was just so simple (hard work, lots of effort etc. but simple in terms of knowing what has to be done). I have another enemy, laziness – I somehow have to beat it but seems impossible right now 😛 Anyways, thought this was a good post to publish on Doomsday – was holding this back for that. I’m sure the Mayans will agree 😛
I’m finally LIBERATED !!!!!!
PS I wanted to write a longer story, but then it’ll get too boring.
WebRTC – Getting started !
I have got some exciting news – The WebRTC project is gaining some real momentum now. How do I know this? For starters the weekly Mozilla WebRTC meeting on Google+ has become crowded. A couple of months back when I first joined the hangout to see what WebRTC was all about there used to be hardly six people. Now, I have to fight for space on the hangout as only ten people can be accommodated at a time, and people have to keep dropping off after their relevant part is done to accommodate others (Yes! we need a better meeting solution – hopefully WebRTC will cater to our needs). The other reason for this gain in momentum is more people are getting involved as the components are getting stabler and better by the day thanks to the super enthusiastic, skilled and hardworking people involved in the project. Presently, there are a few developers working on creating some cool demos (and hopefully tutorials) for WebRTC which is exactly what we want now. Now, gUM is pretty much stable. Peer Connection has made real progress with a few demos where you could make video calls and Datachannels is being used to create a demo – this is a social app with filesharing also included as of its many awesome features. This is what I call real progress. Itching to try out WebRTC? Don’t know where to start? Want to know what can be done? Well, I am hoping this post will be of some use to all those who are deluded by these questions and maybe it will even fuel your interest to start experimenting with WebRTC.
A good place to start exploring WebRTC is gUM. My first reason for suggesting gUM is that it is the most stable and least buggy of all the WebRTC components. The second reason is that it is the easiest to use for a newbie as there is very little to learn about how it works and it can be easily integrated with whatever you are working on 😉 I forgot to mention the obvious, the reason I use it – its just so much FUN! 🙂 A simple demo on gUM created by Anant is the best place to try out gUM. Here, you can see that you can get audio and video independently or together, how cool is that? And of course for those of you obsessed with yourselves there is an option to take a photo of yourself! Yay! To get this demo to work you will need the Nightly version of Firefox from nightly.mozilla.org Then you will have to change the preferences which can be accessed by typing about:config in the address bar. Then toggle the following two preferences to true – media.navigator.enabled and media.navigator.permission.disabled. This should have you all set for the demo and voila! Your first look into WebRTC! (*Any bugs or problems you have, we would be happy to hear about them and fix them asap – you can report on bugzilla or just add a comment to this post*)
To get a better idea on how to write code check out the source of the demo on gUM that I had previously mentioned. Now, let us walk through the code or rather the part that matters the most and help you get a better idea on how this works! I am sure you’ll manage with the rest of the code. (My explanations might sound naive to a first time user but you are welcome to mail me or catch me on IRC)
try {
The gUM function starts here. You can specify the parameters you want to pass through the param argument. A valid set of parameters would be the ones which determine the values of video_status, audio_status and picture_status bool values used later on.
window.navigator.mozGetUserMedia(param, function(stream) {
message.innerHTML = "<p>Success!</p>";
Depending on the bool value of video_status the video stream is obtained and played or the code in the if block is ignored.
if (video_status) {
content.appendChild(video);
video.mozSrcObject = stream;
video.play();
Depending on the bool value of audio_status the audio stream is obtained and played or the code in the if block is ignored.
} else if (audio_status) {
content.appendChild(audio);
audio.mozSrcObject = stream;
audio.play();
Depending on the bool value of picture_status the picture is taken and displayed or the code in the if block is ignored.
} else if (picture_status) {
content.appendChild(picture);
picture.src = window.URL.createObjectURL(stream);
picture.onload = function(e) {
window.URL.revokeObjectURL(this.src);
}
}
The function and catch statements are used to display error messages in case gUM fails due to some of the commonly known ways of incorrect usage.
}, function(err) {
message.innerHTML = "<p class='error'>" + err + "</p>";
stopMedia();
});
} catch(e) {
message.innerHTML = "<p class='error'>" + e + "</p>";
stopMedia();
}
For more info on gUM you should check out the following links –
W3C Draft
Mozilla Factory
This page is dedicated to my Mozilla posts. Yes, I am a proud Mozilla volunteer though not really experienced(5 months I guess). I have learnt a lot after joining this wonderful community – about myself and the organization and in the process also improved upon my development skills and experience. So I will dedicate another post as to why I have fallen in love with Mozilla though that should be the starting post. Hence here I will share everything I know about Mozilla and anything interesting I find happening in the Mozilla community. So, stay tuned.
Yesterday while looking through some of the interesting pages of Mozilla on Facebook I cam across this really interesting one called Mozilla Factory. It is a project by Mozilla Japan. It has some really cool stuff going on (from what I have seen on their website). Not only are these projects cool but a couple of them are totally out of the box, they do stuff with the browser which you could have never thought of before. I highly recommend you take a look at the Mecha Mozilla Project. This is a project put in their words is – explore the possibility of web as thinking a good relationship between “Web” and “Real World”. I’m sure some of you will find this really crazy while some of you might be awestruck by it.
This project also has some great contributors. I was particularly interested in the profile of one of the contributors named Daisuke Akatsuka who was an ex-F1 racer and had done industrial engineering before settling down as a developer. I believe it is people like this who add variety to the web through their vast domain knowledge by bringing in their own experiences into development. As I had expected he is one of the leads on the Mecha Mozilla project. I am eager to see if I could talk to him on the IRC and get some ideas and learn something cool. For all of you guys who want to check this out here is the link: http://www.mozillafactory.org/en-US/about/
It is projects like this which stretch the boundaries of what we know and think to be impossible and help us understand how much more there is to learn and watch. This truly tells us the story of the evolution of the web. It has and will always be exponential – do you not agree?
The infinite universe of Infinity 