Open source behind a web request
CC Attribution 2.0 Belgium:
You are free to use, adapt and copy, distribute and transmit the work or content in adapted or unchanged form for any legal purpose as long as the work is attributed to the author in the manner specified by the author or licensor.
Molinari, Willian (PotHix)
In this talk, we will go through an end-to-end web request. Based on an opensource web application deployed in a hosting company, we will analyze manyopen source software that composes a web request. Using Chromium as the user browser, we will understand what it does when a URLis typed and how it resolves the domain. DNS and UDP protocols will beexplained in order to get the server IP. The IP received from DNS will be usedto create a TCP connection to the server and we will understand how the three-way handshake works for that matter. Instead of using only the HTTP protocol,this connection will use HTTPS and we will analyze the TLS handshake as well. From the user local network to the hosting server, there are many routes tofollow and we will analyze some of them. Getting to the server we willunderstand how the web server receives the request and forward to the webapplication. Web application talks to the web framework and your applicationthat creates the HTML response. Everything will be assembled and return to theweb browser that will process and render everything. Throughout the entire process, we will point to open source code wheneverpossible. ## Audience Everyone that knows a little bit about internet and has done some developmentfor the it
 Okay. So, hi everybody, I'm William Molinari and I'll talk about the open source behind of every quest. First thing is I'm Brazilian, so I'm not an English speaker thing, you know, we tend to destroy everyone's games on the internet, so that's it. We have a lot of Brazilian here by the way, so we would conquer the world sometime. And I have a hundred plus lives and I have just 30 minutes to say everything, so that would be fast. Yeah, I tend to speak fast, so sit tight and let's go have some water here and I'll need it. So, I'm William Molinari, I'm also known by this weird nickname called Potex. Yeah, that's something from Diablo 2 and some 10 years ago. I work for local app, so thanks local app for having me here, they are paying the expenses from Brazil to Brussels. I'm also the mayor of the neither for São Paulo Rubio's group in Brazil. It's the biggest Rubio's group in Brazil with meetings. It's a kind of cool thing and I tend to do HTML5 game development in my spare time just for fun, so you can check it out if you want to. That's it about me. Let's go to the motivation about this presentation. And the main motivation is that I wrote a book about that in Brazilian Portuguese. So yeah, it's not be so useful for you, but I tend to translate. I want to translate that to English in the future, so just tell me if you want to do that. If you think that is a good idea. But the book is about what happens in a web request, a lot of things that's happening from the operating system and networking and I know server, frameworks and some kind of things. And so I focus on desktop part and not going to so deep on server side. But that's it. So let's start with our user. I'm a big fan of JR talking. Yeah, I'm using some t-shirt of Gondor, but and I'm using a lot of his character for that. So let's imagine Gandalf as a user and he's trying to access my personal blog. I don't know why but he's trying to do that. So the first thing that will happen. We'll use a browser, of course, and we use Chromium in this case. So it's the first open source software here. And the first thing is to type a new URL there. And the first question that Chromium will ask is, is it really a new URL? And it may look silly, right? But why we will not ask type a new URL in the address bar? And you'll know that you can type whatever you want in the address bar. You just search for a search engine for you. In this case, we use Google to search for this. We will be RBR, you know. So but we will assume that that's the new URL and we can continue to the next question that is, is it in the HSTS list? And by HSTS, I mean strict transport security, that Chromium has a hard-coded version of a list of websites that can be accessed via HTTPS. Firstly, so if we will ask that if your URL is present on this list. And every server can send a strict transport security header. So it will be included there, dynamic. So there's a lot of links here. So it's a link for Chromium if you want to look to it. In this case, we will continue with HTTP for a while. The next question is, does it have cash about that? And we will assume in this presentation that there's no cash anywhere because it will be so easy, right? Just return the cash and thank you. But so we will continue without cash in. So in this case, there's no cash in. So browser just continue. Cash can be done via X-parts or cash control or ETag, a lot of HTTP headers here. So that there's no cash we can continue and browser will split the URL into three different parts. Protocol, domain and path. We have to continue through domain because the internet doesn't know what is Potex.com is really mean, right? We have to use IP addresses and this kind of thing. So we have to get the IP address for Potex.com. So it will check if we have a DNS caching for that. So Chromium has its own version of, own implementation of DNS. You can check into architecture of open source applications book. There's a chapter about that. So Chromium has its own implementation of DNS so it can go faster. But we will not continue with that. We will use get add DDR info that there are two main implementations with internal implementation and get add DDR info using the operating system. So we will use get add DDR info so we can get into the operating system right now. So by operating system I mean Potex and Glibc, right? That's Gnulibc. So Glibc will look to your ETC hosts. So maybe you have defined your, just like a DNS in your ETC host. So you have this domain points to a different IP address so it will not go to the real DNS. So it will just use your ETC hosts. There's a link for the implementation if you want to look to it. So let's get add DDR info. That's it. I'm not going to deep dive on the code because, you know, it's boring. I'm sure you have that in the presentation. So what get add DDR info will do is to, firstly, it search on a, ask name servers caching demon. So in case you have, this is another layer of caching so you may have a cache for your domain resolution. So it will ask name server caching demon. Obviously we are not using caching this case as well. So the big picture is we have a user. We are getting to the operating system to live to the internet, right? But to live to the internet we will have to go a little further to the, a little bit of theory to all the model. And I don't know about you, but every time I talk about the other model, I get one of those caps in my head because it's so academic, right? So boring to talk about that. So let's try to simplify this a little bit to not be so boring. Let's remove those two layers there and we use just five layers here. So we can, I split that into the operating system and the user and the user part. So we have application for the user and those other tools for the operating system. So we can, now you can see that as protocols and we look through each of those protocols one by one. So let's get started with DNS. As I said, just a parenthesis here. Chrome has its own implementation of DNS. You can check it out if you want. This is the link for the criminal source code. And getting back to the get a ddr info. We have two syscalls here, a call for socket and a call for connect. That there's a sort of code. So we are using UDP IP socket d-gram from UDP in IP protocol IP. We're using UDP IP and just attributing IP address via connect. So the connect, we just get this IP address and attribute to the socket. So we can check that with a trace if you want. Don't worry about the whole code. Chrono has a lot of processes, process for rendering for tabs for extensions for a lot of things. So I'm just getting all the PIGs there. Let's focus on this trace there. So we have minus F to get all the threads and we just using some syscalls there that socket connect. So what we have here, we have a sock for IPv6 with UDP IP. So we are trying UDP IP version of trying to get UDP IP. Trying to get the IP via IPv6, sorry. So we are trying to connect and I don't have IPv6 configured in the laptop. So network is unreachable. So we tried IPv4, PF IP net with UDP IP to get the DNS. Just we can see that we have here IPv6 for Google DNS and IPv6 for Google domain resolution from IPv4 and IPv6. So we get UDP IP and we get the IP from my personal blog and so we get the data that we want. So why is he trying IPv4 and IPv6? I didn't ask for that. And the reason is that we have this algorithm called happy eyeballs working behind the scenes. And we have the RFC for who don't want to read an RFC on this Sunday, right? So cool. So that's it if you want to read it. But what happy eyeballs does is just request both entries in the DNS. So we have a and for a IPv4 and IPv6. And so you can request do a request for both application IPv4 and IPv6. And you just use IPv4 in case IPv6 is not working or just use IPv6 if it's working. And why we need that we are still using that. But we don't know. It's because we are trying to transition from IPv4 to IPv6, right? And it's not all IPv6 connections are fully working. So we are doing that to use it when it's available. So we are not penalizing our user because of that. So that's it. Happy eyeballs is helping us. Curl implements that. Firefox, Chromium. A lot of the new Macs implement that iOS. So that's it. It's working. So let's get to the UDP IP in this case. We are in the DNS yet. We are not to the HTTP yet. So UDP is implemented inside the Linux kernel. So this is the operating system, right? UDP is implemented inside the Linux kernel. I'm not a kernel hacker. So I'm not sure what is happening there. But this is the first file that you can look if you want to understand some things. And what is happening with UDP is that we just attribute an IPv address and send it to the internet and you don't know if we will receive a response and confirmation about that. So that's UDP. We will get to TCP soon. So that's it. We can finally go to the physical layer. But let's let it for some time from here soon. So the DNS, I did this with DNS tracer. So what's happening with DNS here? Domain resolution. We are this user. We will ask our recursive name server, in this case, in my case, it's my personal router at home. So there's no cache. Remember that there's no cache. If we ask one of those two, those 13 root servers, that will send us the IP for one of those two top level domain servers. In my case, Potex.com is the top level domain for.com. That I will get my alternative server and I finally get my IP address. This is the 192 up there. So that's it. We finally got what we were looking for. That is my IP address. And we finally can get to the HTTP that is what we were looking for. So let's get started from scratch. So we have TCP IP with HTTP now. And let's get started with TCP first because it will be better to explain. So yeah, TCP is implemented inside the Linux kernel. So if you can look to the TCP if you want to. And TCP is really different than UDP. You have a lot of control and statuses. And yeah, we have a connection here, not just sending a pex, but we will not go into detail of that because it's boring. So let's look to the three-way handshake and how we establish a connection. So we will use Gandalf again for this case. What's happening here is that we have a server listening on the other side on port 80, for example. That's my blog is running port 80. So Gandalf in this case is the operating system. We will send a sync package to the server. This is the first step. So it tries to connect to the server. And the server says, nice. I want to expect a connection. I want to connect to you as well. So it sends an active confirmation and send to the other side. So this is the second step. And as a third step, it just confirms. So we have a connection from Gandalf to the server and from server to Gandalf. So it's a full-stab connection. And we have Gandalf happy, right? Yeah, that's it. So now that we have a connection, we can go to HTTP and ATTPS. We will start with ATTPS because we have to have an encrypted connection here. So ATTPS is just ATTPS over TLS. So we have an encryption tunnel between quotes here. So it's just like that, just like between HTTP and TCP IP. So it's based on this article, it's a dense article if you want to read it. Okay, let's imagine that we already have a connection here. You can remember that's Gandalf happy. So the first thing to do is to send a list of ciphers and a URL from client to server. And by a list of ciphers, I mean this thing. You can check your ciphers if you go to this website. But there's a lot of encryption algorithms and key exchange algorithms and yeah, that's it. A lot of things we are not going to do in detail here. So the server will pick one of those ciphers and send back with the certificate. So in this case, we are using TLS RCA with RCA4 and yeah, that's it. And what we can remove from that is that we will use RSA for asymmetric cryptography, RC4 for symmetric cryptography and MD5 for hashing. So we use the public key and private key here to exchange the key. Symmetric cryptography, we use this key to change information, to exchange information and we will use hashing just to compare package and CD if everything is okay. So that's it. Just a second. Let's go to awkward moment. Okay. So we will check the certificate because certificate is solving a trust problem, right? We don't know if this guy is Potex.com for sure. So we are asking a 30-party guy if he is really Potex.com and the certificate solves this problem. We will just check the sign of the certification with one of those public keys that we have on our laptop, for example. And if we sign in, the sign is the same as the 30-party that signed with the private key, we can be sure that this guy is really Potex.com in case we trust on a 30-party there. That may be, I don't know, a first sign or less encrypt. So we will check if we have a trustworth CA certificate authority. We will look for a valid date in case the certificate is not expired. We will have to check if the certificate is too valid. So we will have those checks. And so we can do a master key exchange. And there's a complicated process going on here. I'm not going into detail, but there's a rental number going from one site to another and a lot of, I don't know, cryptography. Those algorithms will go in action here, so we are not going so deep here. But there's a big key on both sides. They are generated on a secure way. So finally, they have a trustworth connection here. And it will use RC4, a symmetric cryptography, just like your Wi-Fi. We have a big key, and they use a symmetric cryptography for that. And MD5 for a content verifying, so every package will be verified with MD5 in this case, right, in this cipher that we chose. So that's it. Now we finally have TCP as a connection and TLS as an encryption connection as well. So we can use HTTP or HTTP2. HTTP2 is the best one, I think. I would recommend you to use HTTP2. It's really cool, and it's the future. But we use HTTP just as an example, because it's plain text, and it's easy to show there. There's a link for the Chrome implementation for HTTP and HTTP2 that's called Speedy on Chrome encoding, if you want to look at it. Okay. So this is HTTP that there's no rocket science here, right? It's just a get here and with the file that we want and the protocol number and name and number. And we are using host here just to, because I'm using GitHub page for this case, they may have a load balancer, and it will have to use that. But that's it. There's no rocket science. That's HTTP. And so we finally have that, and we go out to the Internet, so we use Internet or Wi-Fi and they have the physical layer. I'm just, that's not exactly like that, but we'll have those two layers down there. So it's based on a paper, so let's go inside the Linux kernel, which includes, so we have a socket library there, just like GLEBC, for example, that will talk to the TCP-IP stack inside Linux kernel that will talk to the frameworks and drivers developed there, so we can finally go to the hardware to leave the operating system, the machine, by the way. So this is the paper, I talked about the last slide, so if you want, you can read it if you want. So we are going to kernel on IPv4, TCP-PC. There's some frameworks to develop device drivers for wireless and that. Then we have IDWI-FI that is the driver for this laptop. And the only part that is not open source here is the firmware on my, the Intel firmware, so that's it, it's a bug. So everything else is open source. So we are finally leaving the operating system and we will go to our router to the Internet. I'm not talking about the router because I will not have enough time for that, but let's go to the Internet. And by Internet, I mean just a trace route because there's a lot of devices out there and we can't study that. So let's imagine a trace route with TCP because we lost a lot of packages if you use UDP here. So that's it, I'm going to my router that is volunteer, yeah, you know, it's another thing from Lord of the Rings. So we are going to the virtual that is Brazilian ISP, we are going to the backbone that goes to New York City probably and YK, it's going to USA backbone and GitHub base in this case. That's my personal blog. So that's it, let's imagine that's the Internet, right, just like I keep crowd just that little box, yeah, that's the Internet. So I'm not focusing on this part, the server part because we are in the desktop of that room, so I removed a lot of slides here, I was studying a framework, studying a lot of things here, so I just removed that. So let's imagine Nginx, Nginx may have a configuration for Unix sockets and for TCP, so in this case it's for TCP and this case is for Unix sockets, so I can check the documentation. And we imagine that Nginx is talking to a web application, web application server, so we are talking via TCP to this application server, we can use TCP or Unix socket in this case, we just chose TCP because of reason, right? So it just shows that. So Unicorn will talk to RAC that will talk to Rails, in this case it's just a Rails setup, right? We don't have to be familiar with Rails but what RAC does is just to make the protocol between Unicorn and the Rails app so you can change your framework to Sinatra, you know, your Unicorn to, I don't know, Poma and keep the same protocol there, so this is why RAC exists in this case. Another good question here is that why I have to put a web server instead of just using Unicorn, a web application server to the internet and the reason is that Unicorn is well prepared for that, Nginx is well prepared for that, Unicorn is just implementing HTTP but it's not so, I don't know, it's not a good guy to be on the internet because, you know, internet is difficult, we have a lot of Brazilians there. So what Rails we receive here is just Ruby hash with a lot of things here, we have CGI 1.1 there, we have, I don't know, this is what that's of localhost, so I was just asking here, HTTP 1.1, web rig, that's the web server, application server, that's it, so we have a lot of things here and Rails we receive that and we'll try to understand what we want with this URL, so we try to understand what is a controller and what is a model, what is a controller and what is a view. So let's imagine the MVC here, I would change a little bit just to get the Rails controller, so we get some data from model, build a view and we'll have an HTML here, so this is my personal blog finally, so we can finally get all the way back, all the way back until combo break, until we get to the user, so we will not get to the user right now because it's just plain text, right? We don't want to show plain text to our user, I don't want to, I don't know, understand the HTML for that, so that's it, that's the time when you have a spinner there, in your browser, it's just receiving a lot of information that is HTML and it will try to have to parse that and build the web page. So here is where the CS algorithms that we learned in the university comes to life, right? Because we have to build the DOM tree, there's document, object, model tree, so based on a lot of text there, and so there's a link for implementation for inside Chromium and inside Blink, that is the rendering engine for Chrome, so that's it, Chromium, we receive your HTML and fix a lot of errors for you, so I don't know about you, but I never saw an HTML syntax error, you can't see your Facebook comments because there is an HTML syntax error, that does not exist, right? And that does not exist because browser is doing a lot of magic for you, so that's it, thanks browser. We had more black magic in the past when we have IE6, you know, it's more magic than we wanted, just like again, the hobbit and a lot of it, yeah, okay. So we have a lot of tags here, so we are receiving images and scripts and CSS and this kind of thing, so rather than start downloading that, every image and every link will be downloaded in different thread, by thread I mean asynchronously, so that's been downloaded as synchronously, and every time it finds a JS file it will be downloaded synchronously, so it will block your whole handling and download the file, execute it, and so it will continue, so that's why you have to put your JS file at the end of the page, so that's the reason. And just remember that every one of those guys is a kind of happy thing, yeah, so you'll have a lot of, I don't know, three-way handshake and TLS thing, I know that's keep alive to save the day, but in theory you have a lot of things going on here. So that's it, and by using 80TP2 you will just use one connection and multiplexing through this connection, so use 80TP2 is the future. So we have to do the same thing with CSS, so we have the don't tree and the CSS on tree, so we have to do the same thing, there's not just text, we have to create this tree, and so we have those two trees and we have to merge them, so when we merge those two trees, we will have the render tree, and based on the render tree, we can finally build the whole application, website in this case, so there's a good article about that, that's master thesis I think, so just read it if you want, and that's really interesting, so you will find a render tree, and this render tree will have X and Y, so you have to drown this box with X100 and Y100, so browser will keep printing that on your display. So finally we finally have Gandalf in this case, we finally have my personal blog so I can read about Go and Ruby, I don't know. So that's it, and as a summary, is that OpenSearch powers the web, we have a lot of OpenSearch software out there, and that's why we have a, not that's why, but it's powering the web, it's really cool, and yeah, that's it, it's really easy, right? Okay, so I hope you enjoyed this presentation and it helped you to understand what's happening on our web request through OpenSearch software, and I'll be waiting for questions if we have time for that, and thank you. And I'll see you next time. Bye.
 Okay.  So, hi everybody, I'm William Molinari and I'll talk about the open source behind of every quest.  First thing is I'm Brazilian,  so I'm not an English speaker thing, you know, we tend to destroy everyone's games  on the internet, so that's it. We have a lot of Brazilian here by the way, so we would conquer the world sometime.  And I have a hundred plus lives and I have just 30 minutes to say everything, so that would be fast.  Yeah, I tend to speak fast, so  sit tight and let's go have some water here and I'll need it.  So, I'm William Molinari, I'm also known by this weird nickname called Potex.  Yeah, that's something from Diablo 2 and some 10 years ago.  I work for local app, so thanks local app for having me here, they are paying the expenses from Brazil to  Brussels.  I'm also the mayor of the neither for São Paulo Rubio's group in Brazil.  It's the biggest Rubio's group in Brazil with meetings.  It's a kind of cool thing and I tend to do HTML5 game development in my spare time just for fun,  so you can check it out if you want to.  That's it about me.  Let's go to the motivation about this presentation. And the main motivation is that I wrote a book about that in Brazilian Portuguese.  So yeah, it's not be so useful for you, but I tend to translate.  I want to translate that to English in the future, so just tell me if you want to do that.  If you think that is a good idea.  But the book is about  what happens in a web request, a lot of things that's happening from the operating system and networking and I know server,  frameworks and  some kind of things. And so I focus on desktop part and not going to so deep on  server side.  But that's it. So let's start with our user. I'm a big fan of JR talking.  Yeah, I'm using some t-shirt of Gondor, but and I'm using a lot of his character for that.  So let's imagine Gandalf as a user and he's trying to access my personal blog.  I don't know why but he's trying to do that. So the first thing that will happen. We'll use a  browser, of course, and we use Chromium in this case. So it's the first open source software here.  And the first thing is to type a new URL there.  And the first question that Chromium will ask is,  is it really a new URL? And it may look silly, right?  But why we will not ask type a new URL in the address bar?  And you'll know that you can type whatever you want in the address bar.  You just search for a search engine for you. In this case,  we use Google to search for this. We will be RBR, you know.  So but we will assume that that's the new URL and we can continue to the next question that is,  is it in the HSTS list? And by HSTS, I mean strict transport security,  that Chromium has a hard-coded version of a list of websites that can be accessed via HTTPS.  Firstly, so if we will ask that if your URL is present on this list.  And every server can send a strict transport security header.  So it will be included there, dynamic. So there's a lot of links here.  So it's a link for Chromium if you want to look to it.  In this case, we will continue with HTTP for a while.  The next question is, does it have cash about that?  And we will assume in this presentation that there's no cash anywhere because it will be so easy, right?  Just return the cash and thank you. But so we will continue without cash in.  So in this case, there's no cash in. So browser just continue.  Cash can be done via X-parts or cash control or ETag, a lot of HTTP headers here.  So that there's no cash we can continue and browser will split the URL into three different parts.  Protocol, domain and path. We have to continue through domain because the internet doesn't know  what is Potex.com is really mean, right? We have to use IP addresses and this kind of thing.  So we have to get the IP address for Potex.com.  So it will check if we have a DNS caching for that.  So Chromium has its own version of, own implementation of DNS.  You can check into architecture of open source applications book.  There's a chapter about that. So Chromium has its own implementation of DNS so it can go faster.  But we will not continue with that. We will use get add DDR info that there are two main implementations  with internal implementation and get add DDR info using the operating system.  So we will use get add DDR info so we can get into the operating system right now.  So by operating system I mean Potex and Glibc, right? That's Gnulibc.  So Glibc will look to your ETC hosts. So maybe you have defined your, just like a DNS in your ETC host.  So you have this domain points to a different IP address so it will not go to the real DNS.  So it will just use your ETC hosts. There's a link for the implementation if you want to look to it.  So let's get add DDR info. That's it. I'm not going to deep dive on the code because, you know, it's boring.  I'm sure you have that in the presentation. So what get add DDR info will do is to,  firstly, it search on a, ask name servers caching demon.  So in case you have, this is another layer of caching so you may have a cache for your domain resolution.  So it will ask name server caching demon. Obviously we are not using caching this case as well.  So the big picture is we have a user. We are getting to the operating system to live to the internet, right?  But to live to the internet we will have to go a little further to the, a little bit of theory to all the model.  And I don't know about you, but every time I talk about the other model, I get one of those caps in my head because it's so academic, right?  So boring to talk about that. So let's try to simplify this a little bit to not be so boring.  Let's remove those two layers there and we use just five layers here.  So we can, I split that into the operating system and the user and the user part.  So we have application for the user and those other tools for the operating system.  So we can, now you can see that as protocols and we look through each of those protocols one by one.  So let's get started with DNS.  As I said, just a parenthesis here.  Chrome has its own implementation of DNS.  You can check it out if you want.  This is the link for the criminal source code.  And getting back to the get a ddr info.  We have two syscalls here, a call for socket and a call for connect.  That there's a sort of code.  So we are using UDP IP socket d-gram from UDP in IP protocol IP.  We're using UDP IP and just attributing IP address via connect.  So the connect, we just get this IP address and attribute to the socket.  So we can check that with a trace if you want.  Don't worry about the whole code.  Chrono has a lot of processes, process for rendering for tabs for extensions for a lot of things.  So I'm just getting all the PIGs there.  Let's focus on this trace there.  So we have minus F to get all the threads and we just using some syscalls there that socket connect.  So what we have here, we have a sock for IPv6 with UDP IP.  So we are trying UDP IP version of trying to get UDP IP.  Trying to get the IP via IPv6, sorry.  So we are trying to connect and I don't have IPv6 configured in the laptop.  So network is unreachable.  So we tried IPv4, PF IP net with UDP IP to get the DNS.  Just we can see that we have here IPv6 for Google DNS and IPv6 for Google domain resolution from IPv4 and IPv6.  So we get UDP IP and we get the IP from my personal blog and so we get the data that we want.  So why is he trying IPv4 and IPv6?  I didn't ask for that.  And the reason is that we have this algorithm called happy eyeballs working behind the scenes.  And we have the RFC for who don't want to read an RFC on this Sunday, right?  So cool.  So that's it if you want to read it.  But what happy eyeballs does is just request both entries in the DNS.  So we have a and for a IPv4 and IPv6.  And so you can request do a request for both application IPv4 and IPv6.  And you just use IPv4 in case IPv6 is not working or just use IPv6 if it's working.  And why we need that we are still using that.  But we don't know.  It's because we are trying to transition from IPv4 to IPv6, right?  And it's not all IPv6 connections are fully working.  So we are doing that to use it when it's available.  So we are not penalizing our user because of that.  So that's it.  Happy eyeballs is helping us.  Curl implements that.  Firefox, Chromium.  A lot of the new Macs implement that iOS.  So that's it.  It's working.  So let's get to the UDP IP in this case.  We are in the DNS yet.  We are not to the HTTP yet.  So UDP is implemented inside the Linux kernel.  So this is the operating system, right?  UDP is implemented inside the Linux kernel.  I'm not a kernel hacker.  So I'm not sure what is happening there.  But this is the first file that you can look if you want to understand some things.  And what is happening with UDP is that we just attribute an IPv address and send it to  the internet and you don't know if we will receive a response and confirmation about  that.  So that's UDP.  We will get to TCP soon.  So that's it.  We can finally go to the physical layer.  But let's let it for some time from here soon.  So the DNS, I did this with DNS tracer.  So what's happening with DNS here?  Domain resolution.  We are this user.  We will ask our recursive name server, in this case, in my case, it's my personal router  at home.  So there's no cache.  Remember that there's no cache.  If we ask one of those two, those 13 root servers, that will send us the IP for one of  those two top level domain servers.  In my case, Potex.com is the top level domain for.com.  That I will get my alternative server and I finally get my IP address.  This is the 192 up there.  So that's it.  We finally got what we were looking for.  That is my IP address.  And we finally can get to the HTTP that is what we were looking for.  So let's get started from scratch.  So we have TCP IP with HTTP now.  And let's get started with TCP first because it will be better to explain.  So yeah, TCP is implemented inside the Linux kernel.  So if you can look to the TCP if you want to.  And TCP is really different than UDP.  You have a lot of control and statuses.  And yeah, we have a connection here, not just sending a pex, but we will not go into detail  of that because it's boring.  So let's look to the three-way handshake and how we establish a connection.  So we will use Gandalf again for this case.  What's happening here is that we have a server listening on the other side on port 80, for  example.  That's my blog is running port 80.  So Gandalf in this case is the operating system.  We will send a sync package to the server.  This is the first step.  So it tries to connect to the server.  And the server says, nice.  I want to expect a connection.  I want to connect to you as well.  So it sends an active confirmation and send to the other side.  So this is the second step.  And as a third step, it just confirms.  So we have a connection from Gandalf to the server and from server to Gandalf.  So it's a full-stab connection.  And we have Gandalf happy, right?  Yeah, that's it.  So now that we have a connection, we can go to HTTP and ATTPS.  We will start with ATTPS because we have to have an encrypted connection here.  So ATTPS is just ATTPS over TLS.  So we have an encryption tunnel between quotes here.  So it's just like that, just like between HTTP and TCP IP.  So it's based on this article, it's a dense article if you want to read it.  Okay, let's imagine that we already have a connection here.  You can remember that's Gandalf happy.  So the first thing to do is to send a list of ciphers and a URL from client to server.  And by a list of ciphers, I mean this thing.  You can check your ciphers if you go to this website.  But there's a lot of encryption algorithms and key exchange algorithms and yeah, that's  it.  A lot of things we are not going to do in detail here.  So the server will pick one of those ciphers and send back with the certificate.  So in this case, we are using TLS RCA with RCA4 and yeah, that's it.  And what we can remove from that is that we will use RSA for asymmetric cryptography,  RC4 for symmetric cryptography and MD5 for hashing.  So we use the public key and private key here to exchange the key.  Symmetric cryptography, we use this key to change information, to exchange information  and we will use hashing just to compare package and CD if everything is okay.  So that's it.  Just a second.  Let's go to awkward moment.  Okay.  So we will check the certificate because certificate is solving a trust problem, right?  We don't know if this guy is Potex.com for sure.  So we are asking a 30-party guy if he is really Potex.com and the certificate solves this  problem.  We will just check the sign of the certification with one of those public keys that we have  on our laptop, for example.  And if we sign in, the sign is the same as the 30-party that signed with the private key,  we can be sure that this guy is really Potex.com in case we trust on a 30-party there.  That may be, I don't know, a first sign or less encrypt.  So we will check if we have a trustworth CA certificate authority.  We will look for a valid date in case the certificate is not expired.  We will have to check if the certificate is too valid.  So we will have those checks.  And so we can do a master key exchange.  And there's a complicated process going on here.  I'm not going into detail, but there's a rental number going from one site to another and  a lot of, I don't know, cryptography.  Those algorithms will go in action here, so we are not going so deep here.  But there's a big key on both sides.  They are generated on a secure way.  So finally, they have a trustworth connection here.  And it will use RC4, a symmetric cryptography, just like your Wi-Fi.  We have a big key, and they use a symmetric cryptography for that.  And MD5 for a content verifying, so every package will be verified with MD5 in this  case, right, in this cipher that we chose.  So that's it.  Now we finally have TCP as a connection and TLS as an encryption connection as well.  So we can use HTTP or HTTP2.  HTTP2 is the best one, I think.  I would recommend you to use HTTP2.  It's really cool, and it's the future.  But we use HTTP just as an example, because it's plain text, and it's easy to show there.  There's a link for the Chrome implementation for HTTP and HTTP2 that's called Speedy on  Chrome encoding, if you want to look at it.  Okay.  So this is HTTP that there's no rocket science here, right?  It's just a get here and with the file that we want and the protocol number and name and  number.  And we are using host here just to, because I'm using GitHub page for this case, they  may have a load balancer, and it will have to use that.  But that's it.  There's no rocket science.  That's HTTP.  And so we finally have that, and we go out to the Internet, so we use Internet or Wi-Fi  and they have the physical layer.  I'm just, that's not exactly like that, but we'll have those two layers down there.  So it's based on a paper, so let's go inside the Linux kernel, which includes, so we have  a socket library there, just like GLEBC, for example, that will talk to the TCP-IP stack  inside Linux kernel that will talk to the frameworks and drivers developed there, so  we can finally go to the hardware to leave the operating system, the machine, by the  way.  So this is the paper, I talked about the last slide, so if you want, you can read it if  you want.  So we are going to kernel on IPv4, TCP-PC.  There's some frameworks to develop device drivers for wireless and that.  Then we have IDWI-FI that is the driver for this laptop.  And the only part that is not open source here is the firmware on my, the Intel firmware,  so that's it, it's a bug.  So everything else is open source.  So we are finally leaving the operating system and we will go to our router to the Internet.  I'm not talking about the router because I will not have enough time for that, but let's  go to the Internet.  And by Internet, I mean just a trace route because there's a lot of devices out there  and we can't study that.  So let's imagine a trace route with TCP because we lost a lot of packages if you use UDP here.  So that's it, I'm going to my router that is volunteer, yeah, you know, it's another  thing from Lord of the Rings.  So we are going to the virtual that is Brazilian ISP, we are going to the backbone that goes  to New York City probably and YK, it's going to USA backbone and GitHub base in this case.  That's my personal blog.  So that's it, let's imagine that's the Internet, right, just like I keep crowd just that little  box, yeah, that's the Internet.  So I'm not focusing on this part, the server part because we are in the desktop of that  room, so I removed a lot of slides here, I was studying a framework, studying a lot of  things here, so I just removed that.  So let's imagine Nginx, Nginx may have a configuration for Unix sockets and for TCP, so in this case  it's for TCP and this case is for Unix sockets, so I can check the documentation.  And we imagine that Nginx is talking to a web application, web application server, so  we are talking via TCP to this application server, we can use TCP or Unix socket in this  case, we just chose TCP because of reason, right?  So it just shows that.  So Unicorn will talk to RAC that will talk to Rails, in this case it's just a Rails setup,  right?  We don't have to be familiar with Rails but what RAC does is just to make the protocol  between Unicorn and the Rails app so you can change your framework to Sinatra, you know,  your Unicorn to, I don't know, Poma and keep the same protocol there, so this is why RAC  exists in this case.  Another good question here is that why I have to put a web server instead of just using  Unicorn, a web application server to the internet and the reason is that Unicorn is  well prepared for that, Nginx is well prepared for that, Unicorn is just implementing HTTP  but it's not so, I don't know, it's not a good guy to be on the internet because, you  know, internet is difficult, we have a lot of Brazilians there.  So what Rails we receive here is just Ruby hash with a lot of things here, we have CGI  1.1 there, we have, I don't know, this is what that's of localhost, so I was just  asking here, HTTP 1.1, web rig, that's the web server, application server, that's it,  so we have a lot of things here and Rails we receive that and we'll try to understand  what we want with this URL, so we try to understand what is a controller and what is a model,  what is a controller and what is a view.  So let's imagine the MVC here, I would change a little bit just to get the Rails controller,  so we get some data from model, build a view and we'll have an HTML here, so this is my  personal blog finally, so we can finally get all the way back, all the way back until  combo break, until we get to the user, so we will not get to the user right now because  it's just plain text, right?  We don't want to show plain text to our user, I don't want to, I don't know, understand  the HTML for that, so that's it, that's the time when you have a spinner there, in your  browser, it's just receiving a lot of information that is HTML and it will try to have to parse  that and build the web page.  So here is where the CS algorithms that we learned in the university comes to life, right?  Because we have to build the DOM tree, there's document, object, model tree, so based on  a lot of text there, and so there's a link for implementation for inside Chromium and  inside Blink, that is the rendering engine for Chrome, so that's it, Chromium, we receive  your HTML and fix a lot of errors for you, so I don't know about you, but I never saw  an HTML syntax error, you can't see your Facebook comments because there is an HTML syntax error,  that does not exist, right?  And that does not exist because browser is doing a lot of magic for you, so that's it,  thanks browser.  We had more black magic in the past when we have IE6, you know, it's more magic than  we wanted, just like again, the hobbit and a lot of it, yeah, okay.  So we have a lot of tags here, so we are receiving images and scripts and CSS and this kind of  thing, so rather than start downloading that, every image and every link will be downloaded  in different thread, by thread I mean asynchronously, so that's been downloaded as synchronously,  and every time it finds a JS file it will be downloaded synchronously, so it will block  your whole handling and download the file, execute it, and so it will continue, so that's  why you have to put your JS file at the end of the page, so that's the reason.  And just remember that every one of those guys is a kind of happy thing, yeah, so you'll  have a lot of, I don't know, three-way handshake and TLS thing, I know that's keep alive to  save the day, but in theory you have a lot of things going on here.  So that's it, and by using 80TP2 you will just use one connection and multiplexing through  this connection, so use 80TP2 is the future.  So we have to do the same thing with CSS, so we have the don't tree and the CSS on tree,  so we have to do the same thing, there's not just text, we have to create this tree, and  so we have those two trees and we have to merge them, so when we merge those two trees,  we will have the render tree, and based on the render tree, we can finally build the  whole application, website in this case, so there's a good article about that, that's  master thesis I think, so just read it if you want, and that's really interesting, so  you will find a render tree, and this render tree will have X and Y, so you have to drown  this box with X100 and Y100, so browser will keep printing that on your display.  So finally we finally have Gandalf in this case, we finally have my personal blog so  I can read about Go and Ruby, I don't know.  So that's it, and as a summary, is that OpenSearch powers the web, we have a lot of OpenSearch  software out there, and that's why we have a, not that's why, but it's powering the web,  it's really cool, and yeah, that's it, it's really easy, right?  Okay, so I hope you enjoyed this presentation and it helped you to understand what's happening  on our web request through OpenSearch software, and I'll be waiting for questions if we have  time for that, and thank you.  And I'll see you next time.  Bye.
Open_source_behind_a_web_request_explaining_a_web_request_through_open_source_software_0_131.png Open_source_behind_a_web_request_explaining_a_web_request_through_open_source_software_132_443.png Open_source_behind_a_web_request_explaining_a_web_request_through_open_source_software_444_647.png Open_source_behind_a_web_request_explaining_a_web_request_through_open_source_software_648_923.png Open_source_behind_a_web_request_explaining_a_web_request_through_open_source_software_924_1799.png Open_source_behind_a_web_request_explaining_a_web_request_through_open_source_software_1800_1943.png Open_source_behind_a_web_request_explaining_a_web_request_through_open_source_software_1944_2783.png Open_source_behind_a_web_request_explaining_a_web_request_through_open_source_software_2784_3203.png Open_source_behind_a_web_request_explaining_a_web_request_through_open_source_software_3204_8531.png Open_source_behind_a_web_request_explaining_a_web_request_through_open_source_software_8532_8735.png Open_source_behind_a_web_request_explaining_a_web_request_through_open_source_software_8736_8927.png Open_source_behind_a_web_request_explaining_a_web_request_through_open_source_software_8928_9095.png Open_source_behind_a_web_request_explaining_a_web_request_through_open_source_software_9096_9179.png Open_source_behind_a_web_request_explaining_a_web_request_through_open_source_software_9180_10703.png Open_source_behind_a_web_request_explaining_a_web_request_through_open_source_software_10704_12587.png Open_source_behind_a_web_request_explaining_a_web_request_through_open_source_software_12588_16595.png Open_source_behind_a_web_request_explaining_a_web_request_through_open_source_software_16596_16715.png Open_source_behind_a_web_request_explaining_a_web_request_through_open_source_software_16716_17591.png Open_source_behind_a_web_request_explaining_a_web_request_through_open_source_software_17592_18647.png Open_source_behind_a_web_request_explaining_a_web_request_through_open_source_software_18648_18719.png Open_source_behind_a_web_request_explaining_a_web_request_through_open_source_software_18720_24875.png Open_source_behind_a_web_request_explaining_a_web_request_through_open_source_software_24876_25487.png Open_source_behind_a_web_request_explaining_a_web_request_through_open_source_software_25488_27311.png Open_source_behind_a_web_request_explaining_a_web_request_through_open_source_software_27312_27623.png Open_source_behind_a_web_request_explaining_a_web_request_through_open_source_software_27624_28667.png Open_source_behind_a_web_request_explaining_a_web_request_through_open_source_software_28668_31103.png Open_source_behind_a_web_request_explaining_a_web_request_through_open_source_software_31104_31571.png Open_source_behind_a_web_request_explaining_a_web_request_through_open_source_software_31572_32303.png Open_source_behind_a_web_request_explaining_a_web_request_through_open_source_software_32304_32651.png Open_source_behind_a_web_request_explaining_a_web_request_through_open_source_software_32652_37727.png Open_source_behind_a_web_request_explaining_a_web_request_through_open_source_software_37728_38063.png Open_source_behind_a_web_request_explaining_a_web_request_through_open_source_software_38064_38435.png Open_source_behind_a_web_request_explaining_a_web_request_through_open_source_software_38436_38531.png Open_source_behind_a_web_request_explaining_a_web_request_through_open_source_software_38532_38553.png