Move fast and don't break things: High-performance networking in Rust
CC Attribution 3.0 Unported:
You are free to use, adapt and copy, distribute and transmit the work or content in adapted or unchanged form for any legal purpose as long as the work is attributed to the author in the manner specified by the author or licensor.
Liebow-Feeser, Joshua
What makes Rust different is not that you can write high-performance, bare-metal code. What makes Rust different is that when you write that code, it is clean and easy to use, and you are confident in its correctness. In this talk, we discuss a new, high-performance networking stack being written in pure Rust. We discuss how Rust has allowed us to squeeze every last drop of performance out of the stack without sacrificing usability, productivity, or the confidence that our code is bug-free. We focus specifically on packet parsing and serialization, which we accomplish with zero copying, zero heap allocation, and very little unsafe code.
 Thank you for that. All right, so I'm going to try to go through this quickly for the sake of not running over time. A little bit about me first. I work at security at Google. I get to work on open source Rust, which is really cool. If that sounds like something you would like to do, Rust or security or both, come talk to me. And I just started the secure code working group for Rust. If you're interested in that, also come check us out. The thing we're going to be talking about. So the thing is an experimental networking stack handles the link layer through the transport layer. It's like Ethernet through TCP and UDP. It is written in pure Rust, obviously. Design goals are for low resource environment, so low CPU utilization, low binary footprint. And finally, it is structured as a platform agnostic core and then some platform specific bindings. So if you are interested in using this thing for your kernel or your project or whatever, just let me know. It's obviously very early APIs are changing, but if it's just sort of experimental, that could be cool. A quick note on the subtitle that was advertised. High performance. This networking stack can currently respond to pings. That is all it can do. Yes, thank you. Yeah. So yeah, so when I say high performance, I'm not referring to actual benchmarks. I'm referring more to design to the fact that we can prove statically that a bunch of things are happening at compile time rather than runtime. I suspect that those will lead to actual good performance later, but I don't know. So we'll find out. So brief outline for the talk. First we're going to talk about design goals for the stack and the stuff we're going to be talking about. Give a little bit of background on packets and how they're laid out just to get everyone on the same page. First we're going to talk about parsing packets. Then we're going to talk about serializing packets. Then we're going to talk about forwarding, which is basically parsing and then serializing packets. Right, yeah. It's very fancy. And then finally we're going to talk about some nitty gritty details of how we do a zero copy in a safe way. So the goals for this design. So first of all, zero copying. When I say zero copying, I don't literally mean no memory reads or writes, obviously, but if we have a big buffer and there is a packet living in the buffer and we want to operate on it, so we want to parse it or modify it or serialize a new packet into the buffer, we're not going to pull it out into some scratch space and operate on it and then move it back. We're just going to do everything directly in place. As I said, we expect this to make it high performance. We'll find out. The second thing is zero heap allocation, so everything's on the stack. And then finally zero unsafe. So no use of the unsafe keyword. This may sound really cool and too good to be true. That's because it is. There's actually a little bit of unsafe. But in standard Rust fashion, that unsafe is self-contained in nice little small packages and it can be nicely composed. Another thing I want to clarify here, this is not a look at the cool things you can do in Rust Talk. This is a look at the cool things you can do safely in Rust Talk. Everything that we're going to describe here today can be done and is done in C and C++. The only difference there is it's very unsafe and you, the programmer, have to verify a lot of properties yourself. So the distinction here is not like we can do this thing, but rather we can do this thing differently. Quick note on terminology. When I say the word packet, I am aware for the patents in the audience that there are things like Ethernet frames and TCP segments. If you read the code base, you will find these things. But for the sake of simplicity, everything is a packet. I'm terribly sorry. So a little bit of background on how network packets are laid out. So packets are just a big sequence of bytes. All these diagrams that you're going to see today are beginning byte on the left, end byte on the right. Packets are recursive. So the analogy that we're going to be using throughout the entire talk, I hope you like onions. We're going to be talking a lot about onions. So basically you've got a packet which contains a packet which contains another packet which contains another packet and so on and so forth. The packet that we're going to be talking about are just a header and a body. I'm sorry for those of you who really like packet formats, no packets with footers. So there's a format header, there's a header at the beginning whose format is fixed. As in if you go and read the Ethernet spec, for example, you will see there is this field and then this field and blah, blah, blah. So the header has a well-known format that we can parse and then the body is completely unstructured and variable-length. We have no idea what's in there. It's completely opaque. Now once we know what is in there, that might be a different format which we can go parse. So for example, we might discover that, oh look, there's an IP packet in there. And then we can go pull out the IP spec and we can read it and say, oh, this is how you parse an IP packet. But from the perspective of Ethernet, it's completely opaque. So let's say that we have this IP packet here. We might say, well, we're going to parse this again, again, an IP packet just in the same way, there's a format header at the beginning, we know what that format is, and then there's an unstructured body, maybe this contains a UDP packet, and so on and so forth, we can keep going like this. An important thing to keep in mind here is that when we're parsing, we don't actually know ahead of time what all of these types are going to be. So we might get an Ethernet packet and we don't know what's inside of it. It might be an ARP packet, it might be an IP packet. We actually have to look at the header. The header tells us what's inside, so we actually don't know until runtime what the next thing that we're going to parse is. So this is unfortunately, it has to be less static than you might like. So let's talk very briefly about what our end goal is. If you recall, there are four parts, parsing, serialization, and then forwarding is our third part. We're going to build up to forwarding at the end, and I want to describe very briefly what that's going to look like, just to keep in mind what we're building towards. So in forwarding, we are going to allocate a buffer on the stack, we're going to receive an Ethernet packet into it. We're going to parse the Ethernet packet, we're going to parse the IP packet inside of the Ethernet packet, then we're going to decide to forward the packet. So we're actually just going to resend it to somebody else. And first we're going to update the header, we have to do some updates when we forward packets, and then we're going to serialize it inside of a new Ethernet packet, and all of that is going to happen in place on the stack, no copying. So that's the goal that we're building up to, but first we have to get through some other things, namely parsing. So the goal of this section is a lot simpler. We are going to allocate a buffer on the stack, and we're going to receive an Ethernet frame into that buffer. We're going to parse the packet, we're going to parse the packet, we're going to parse the packet, anyway, this is the, you get the idea. Very simple example, we just receive it and parse, parse, parse, parse, parse, that's the whole example for this section. So all right, the buffer trait. The buffer trait is a really important trait in the net stack. It is sort of like the core trait for parsing and serialization. The reason that it's a trait and not a concrete type is that we have a bunch of implementations of it, but the details there aren't important. A really important thing to keep in mind with buffers is that they are referencing. And by that I mean that when you say you have ownership of a buffer object, that's just a tiny little struct thing with some pointers, right? Maybe it's an own point, maybe it's owned, maybe it's a reference to something, but the point is that thing is tiny, the buffer bites themselves live somewhere else. So we're going to talk about like moving these things around a lot, keep in mind that that's a very cheap operation because this is basically just a pointer. So these things are basically just a sequence of bytes, just like a pack would be. You might ask the question reasonably, why is this not just a slice? Well, the answer is that we need to keep track of how much we've parsed so far or how much we have serialized. So for example, sorry, so the way that we do this is we split the buffer into a prefix and a body. The prefix is basically everything that I have not parsed or serialized so far, or sorry, everything that I have parsed or not serialized, and then the body is everything else. So it's essentially just a pointer into the buffer, right? Here's the whole buffer and here's where I am so far, right? So you can consume bytes from the body and they get added to the prefix, so I can shift this way. You can consume bytes from the prefix and they get added to the body, shift that way. And the invariant that we're trying to maintain here, and this is going to be really important through the whole thing, is that the body always contains, when you are parsing, the stuff that you haven't parsed yet. So if I have an ethernet packet and I chump the ethernet header off the beginning, then I have the body left, and then when I pick up where I left off, I call the parse method again, I'm naturally parsing the next most encapsulated thing. And then when I'm serializing, the exact inverse happens. The current thing in the body is everything that I have serialized so far, and when I ask, please serialize your header, it starts where we left off and goes this way and expands the body to include the now larger packet. So what is in the body is really important, where that body is versus the prefix, that's going to be sort of the core of everything here. So let's walk through an example. So this is a diagram here. On the bottom, we have the actual physical structure of the packet, or sorry, of the buffer, excuse me. So it starts off as being all body. On the top, we have the logical view. So logically, the contents of these bytes are an ethernet header and an ethernet body, but we don't know that yet. And so when we parse it, it's going to look a little something like this. So what we have here is the result of parsing is an ethernet packet object. An ethernet packet object is a tiny little struct that just has some references into the buffer. Again, this is a zero copy theme here. The ethernet packet object is just references. It has one reference into the header and then one reference into the body. An important thing to keep in mind here is that the header, so the body is just bytes, we don't know what it is, so that's just byte slice. The header, on the other hand, is actually structured. So we'll get into at the end of the talk how we actually achieve that without unsafe, but you should basically think of this as a struct reference. So we have a struct. It represents what the header actually looks like. We've got this field and then that field and so on and so forth. And this is just a reference into that struct, so we can just access it like plain vanilla rust. So we've got that struct.field, blah, blah, blah, blah. Last thing to note is because these are references, the ethernet packet object borrows the buffer. So if we want to actually move the buffer around, which we will want to do, we have to drop the packet first. So a pattern that you're going to see over and over again is first we parse, then we operate on that actual object, and then before we do anything else, we drop it. Then we pick up where we left off, but now we've forgotten about that thing. Okay, so as we said before, we've got this and then we're going to drop it. It goes away. And so what we're left with is that the body of the buffer now corresponds to the body of the ethernet packet. So as I mentioned before, we can just pick up where we left off and parse the next packet. So the exact same thing happens this time with the IP packet. We have a header. It references into the buffer and it is a structured reference to the actual, you know, it's a struct that we can access the fields. And then an unstructured reference to the body, it borrows the buffer and so when we're done with it, we have to drop it. Let's look at a little example of what this might look like in code. This is obviously a little pseudo-codesh because they're, you know, more like I would want to do more things if I receive an ethernet packet, but this really is the core of it. The buffer has a parse method and you literally say, here is a type of a packet, please parse it for me, done. And so you get a packet object back here and then you might operate on it and then you drop it and then you pass that buffer. Again, as I said, you pass the buffer by value. You're actually passing ownership. This will become important later. So you pass the buffer by ownership to the next layer of the stack. The body is left as it was before, so the next layer of the stack just picks up right where we left off. So let's actually look at what this might look like in memory on the stack so we get a sense of how all the memory is laid out. So let's say that we receive a, we have a buffer that is allocated on the stack, so it's in the stack from there. First we pass the buffer into receive ethernet packet. So this little black arrow here just means that the actual object is here but the actual bytes of the buffer are still in the original stack frame. First we parse the ethernet packet, just like we mentioned before. So the ethernet packet struct itself, which is this tiny little thing, it lives in this top stack frame here, but the references are all into the contents which live in the original stack frame, right? No copying. So we drop it and then we send the buffer again into the receive IP packet function this time. And again, we parse this time an IP packet. And the references, the tiny little struct lives in that stack frame and the references point into the bottom stack frame. So that's parsing. It's very straightforward. Let's get into serialization. So the goal here is that we are going to receive a request to send an IP, a UDP packet. So there is an application that says, you know, it's like a DNS client or something. It says here are the contents of a UDP packet that I would like you to send out onto the network. So we're going to compute some header information for the UDP packet. Then we're going to compute some routing information for IP, where are we sending this thing, and then some information about what the IP header will need to have. Then some ethernet routing and the header information. And then finally, after we've done all of this, only then do we actually compute the length and allocate the buffer. So all of the computation before, we haven't actually been doing anything. We've not been serializing. We've not been doing any kind of allocation. At the bottom, do we actually say, OK, now that we know how big the packet needs to be, now we can allocate the buffer? I know that I said stack allocation at the beginning. We're going to heap allocate in this example just to keep things simple. We'll actually stack allocate in the end. And then finally, once we have a buffer, then we can just serialize in one big pass UDP, UDPBotter, UDPHeader, IPHeader, ethernetHeader, and then we're done. So a little note on why this is hard. Some of you in the audience may be thinking, OK, well, if you know how big the packet is, why don't you just allocate it ahead of time? Why do you have to do all of this sort of weird stuff? And the answer is that you need to send the request to the next layer before you serialize. So for example, if I have a UDP body and I want to say, well, I need to know what buffer I'm going to serialize it into, I have to know, obviously, how big all of the headers are going to be so I know how big the buffer needs to be. But the IP layer needs to know where it's sending the thing in order to know which link layer protocol is being used. Am I sending it over an ethernet network that has a header of this size? Am I sending it over a Wi-Fi network that has a larger header? So you need to actually compute all the routing information before you know how big the buffer needs to be. And so there's this chicken and egg problem where in order to call the function that says, please send this thing for me, it cannot be serialized already. But of course, you can't get control flow back because the thing needs to be sent by the time you send you call the function. And so that's why this is difficult, and we'll see how we solve this problem. So the first trait that we're going to introduce to deal with parsing is the packet builder trait. Remember before that I said that all packets are onions, right? And they are just sort of a series of layers. The packet builder is like a single layer of the onion. Nothing inside of it, nothing outside of it, just one sort of shell, right? And it assumes that we already have a buffer allocated that already has enough prefix space to serialize everything. We'll solve that problem later. And what it does is it serializes itself into the buffer. So you give it a buffer like this. So here's a buffer that has a body. And you say, dear packet builder, so in this case an ethernet packet builder, please serialize yourself right here, right? We already gave you the body. We gave you enough space, just write yourself in, right? And so this contains all the metadata needed to figure out what that header needs to look like. Source address, destination address, so on and so forth. So that handles just a layer of the onion. Let's talk about the whole onion itself. So the serializer trait, on the other hand, represents the entire onion. So it represents a single layer and everything below it. And it's recursive. So if you take a serializer, which represents the whole onion, and you add a packet builder, which represents the next layer of the onion, you get a bigger onion. And this is how we're going to construct all of our serializers. Another important thing to note here, a serializer does not actually represent a packet directly. You can sort of think of it like a packet future. It describes all of the information necessary to figure out how to allocate and serialize a packet in the future later. Let's walk through a very simple example. So this is a sort of silly straw man example of a function that might construct a serializer. It takes a serializer in and then wraps it in IP and wraps that in Ethernet and then returns it. So we start off with a serializer called SIR. We encapsulate it in a new IP packet builder. Remember that builder is then the IP layer of the onion and we get a new larger onion back, right, a new larger serializer. We encapsulate it again by adding yet another field, which is this Ethernet packet builder. The struct just keeps growing. We get the Ethernet packet onion, an even bigger onion, and we can now return it. OK, so we know how to construct these things, but how do we actually use them? How do we actually use them to create a buffer and serialize into it? So the way that we do this is with the serializer trait again, and there is a method on the serializer trait called serialize. I'm good at naming. The serializer method is responsible for producing a buffer. So when you call it, it doesn't take an existing buffer. It actually creates and gives you back the buffer. And it might use a pool of buffers or it might use something that's already allocated. Implementation is up to the implementation of the trait, but it gives you back the buffer. And it takes the number of prefix bytes that you need so far. Remember that all of this is sort of recursive. So if I say I want to serialize an Ethernet packet, it says, OK, well, how many prefix bytes do you need for all of the packets outside of me? Because I don't know that. And so what you do is you start at the outside of the onion, and you work your way inwards building up the prefix length. So you start at the very outer layer, and you say, well, I need zero bytes so far. And then you go to the next layer, and you say, OK, well, this is an 18-byte header, so now I need 18 bytes. And then this next header and so on and so forth. And you just build up the number of prefix bytes that you need until you get to the inner core. And once you get to the core, now you know in total how many prefix bytes you need, and it's the core's responsibility to satisfy that request. So it has to give you a buffer whose body contains all of the stuff that the innermost packet needs to contain, and that has enough prefix length to handle all of the headers that came before it that you're going to need to serialize in the future. And so once you're in that innermost onion, and you've satisfied the request, you have a big buffer, it has the body, and so on, then you can return it. And now you start walking your way out. So remember, we went in calculating the prefix length, and then you walk out and actually serialize stuff. So you go to the first layer, serialize the first header. And then you expand the body, so now the body encapsulates the entire next packet. Then you return it again, and the next layer serializes its header, and so on and so forth. And by doing this, you make sure that you can just serialize in one shot. You allocate once, you serialize once, and you're done. Let's look at an example of what this looks like. So this diagram is wildly not to scale. Imagine that this struct here is really tiny, that body is really big, but diagrams are hard. So again, this packet, this serializer here represents the entire UDP packet, right? So it has all of the body bytes that the application asked us to send. And it has that packet builder for the UDP layer, right? That shell of the onion that just represents all the stuff that needs to go in the UDP header. And so when we want to send this, we're going to call this into the sendIP packet function, and we're going to pass it by value, right? This tiny little struct moves from this stack frame to this stack frame, but the body itself again doesn't move. We're going to shrink this just for brevity, because we're going to need to make these slides pretty big, so that thing is the same thing as that thing. The sendIP packet function then is going to calculate its own routing information, right? It needs to figure out where this stuff is going, so on and so forth. And on that basis, it's going to construct an IP packet builder for the IP layer of the onion, and boom, construct a bigger serializer, right? The IP packet builder represents the IP layer of the onion. This entire serializer represents the entire IP packet and everything inside of it. Again, send it to the next layer, so send it to the sendEthernet packet function, compress for brevity. Take this serializer and encapsulate it with Ethernet, right? So add another layer of the onion, now we have the Ethernet onion. All right, so now we've actually constructed this entire thing and we're ready to serialize it. Let's see how that works. This is a diagram that if we had a giant wide screen, would have been totally horizontal, but we don't. So each of these dot, dot, dots represents the entire next line, right? Just imagine that this is one big, long thing. It's one, you know, small, but conceptually big struct. So the first thing that we're going to do is call serialize0, right? The outermost layer is the outermost layer, so we don't need any prefix bytes ahead of it. And in order to figure out how many extra prefix bytes we need for this layer, when we call serialize into the next layer, we have to figure out how many bytes the Ethernet header is going to take up, so we ask it. We say, dear Ethernet packet builder, how many bytes will you need? And it says 18, so we go, okay, great. We needed zero, now we need 18, so we need 18 in total. And again, the next, the next innermost packet, or sorry, the next innermost serializer, this one for the IP layer, goes, okay, well. I'm supposed to give this person 18. My layer itself, the IP layer, right? The IP header is 20 bytes long, so I need an extra 20 bytes. So now I need 38 bytes in total. The UDP layer adds another 8 bytes, so now in total it's 46. But now we're in the middle, right? We've actually gotten into the inner serializer, and there are no more serializers to do. We can't recurse anymore, so we actually have to allocate the thing. So first we have to figure out how long the body is, so let's say it's 1,000 bytes. And now we know that we need 1,046 bytes, so we allocate them. Again, we'll stack allocate later for the time being we're doing it on the heap. We say, please give us a buffer, which is 1,046 bytes long. And that buffer starts off entirely as prefix, because there's nothing, we haven't serialized anything yet, so the body is completely empty. There's nothing there. So first we serialize the body that we were asked to into the buffer. So now the body of the buffer is the same thing as the body of the UDP packet. And now we can start rewinding back out, unwinding out the stack. So first we go to the UDP packet builder and we say, hey, you know how to serialize a header. Can you serialize yourself into this buffer, please? And it does. And it expands the body so that now the body contains both the UDP packet body and the UDP packet header. It's an entire UDP packet. And now we're done. This call to serialize has satisfied its constraint. It was asked to give a buffer, which contains the entire UDP packet, and 38 bytes of prefix. Well, we have that. So we can return it. Now this next call to serialize does the exact same thing. It goes, OK, great. I have a packet whose body is what it needs to be already. There is enough prefix space. So I will now serialize the IP layer in place. And again, now the body contains the entire IP packet and has 18 bytes of prefix space. So we can return that. Finally we can serialize the ethernet layer. And now the body contains the entire ethernet packet. Thank you. Yeah. And so we're entirely done. And we can return this thing. And now we have a buffer. So we talked about parsing. We talked about serialization. Let's put it together and forward. So the goal this time is, first of all, no heap allocation. We're actually going to allocate on the stack. We're going to receive an ethernet packet. We're going to parse it. We're going to parse the IP packet inside of it. We're going to decide that we want to forward the packet. So we're going to reserialize it and send it to somebody else. So first we have to update the header. There are some modifications that we have to make to the header. Then we have to serialize it in a new ethernet packet. All right, so let's get into it. So parsing is super easy. We saw in the parsing section how this works. This is going to be super straightforward. Again, we have the buffer. The actual bytes of it live on the stack. We send it into the receiveEthernetPacket function. We parse creating one of these ethernet packet objects, which consumes those bytes from the prefix. We drop the thing so we can continue operating on it. And we send the buffer into the receiveIPPacket function. Again, we parse this time getting an IP packet struct. And so now we have to modify this stuff in place. So this is where it gets interesting, because the modifications that we need to do operate on the sort of logical structure of the IP header. So it's really important that this header is structured and gives us access to struct fields and stuff like that for us to be able to update it. And so the updates that we're going to do are actually methods on the IP packet object. So here's some code. It's very simple. We just say packet, please decrement your TTL, just like a counter of how many hops the packet has gone through. And that just vanilla rust code, right? It's struct.field minus minus, or minus equals 1. Sorry. And yeah, so it's just normal rust code. So now we have this thing. Now we drop this thing. It's a little bit of a problem here. Does anyone spot the problem if we want to serialize the IP packet? What is the contents of the body? Sorry? It's not modified. Well, so the body hasn't been modified, but importantly, the body is the body of the IP packet. We don't want to forward the body of the IP packet. We want to forward the whole IP packet, including the header. But we just parsed the header and threw it away. So there's this cute little hack that we do. It's very straightforward. It's called undo parse. We say, how many header bytes did you just parse? Oops, please undo it. And so we do. It just takes the pointer and goes, oh, you just parsed 20 bytes. OK, oops, bump it 20 bytes. So we go from that to that. And now we have an IP packet that we can forward. And so let's do that. So we decrement the TTL. We figure out how many header bytes we need to undo, and we undo them, and then we serialize them. Very straightforward. Now, you might be wondering, I thought that sendEthernet packet was supposed to take a serializer. Well, it is. Here's the core idea that I want to get across. Buffers are serializers. The serializer trait simply says, can you give me a buffer that has these properties? If the buffer already has those properties, then it says, yes, here I am. It just returns itself by value. The serialize method you might have noticed if you were looking closely takes the serializer by value, takes it by ownership. So you basically take this buffer and you say, buy buffer. Give me a buffer. And it says, I'm already here. Here I am back. And that's what we're going to do. So first we take the buffer. We send it into receiveIP packet. That receiveIP packet function, sorry, excuse me, we're in receiveIP packet currently. We send it into sendEthernet packet. SendEthernet packet then says, OK, great. Here's a serializer. I need to now encapsulate it inside of Ethernet. So we do. Construct the serializer. I'm going to squash these stack frames just for brevity. So then the sendEthernet packet function says, OK, great. I've got my serializer and I need a serializer. So we call serialize of 0. We want to serialize this thing. It says, OK. Ethernet header asks for 18 bytes in the prefix. So we'll ask for serialize 18 of our serializer. Now the buffer is the serializer. So the buffer looks at itself and goes, well, I already have 18 bytes. So we're done. And it returns itself by value. And the call to serialize 0 goes, OK, great. I have a buffer that has 18 bytes of prefix space. I can just serialize this header directly in place. Then we have finally a buffer, which is still on the stack. We haven't ever copied it. It contains all of the bytes of the Ethernet packet. And we can just send this thing. So we've gone from receiving the packet on the stack, just in the stack. It parsed it all the way. We've decided that we want to re-serialize. We've gone, hey, we already have this buffer space. Why don't we just reuse it? And we reuse it. And finally, we can turn around and go, OK, great. Let's just forward it out. So this is what forwarding looks like. And this is sort of like, this is it. This is as simple as it gets. There's obviously, in practice, a little bit more complicated logic, because we have footers and other stuff like that. But this is fundamentally the core of the algorithm. And in none of this stuff is there any unsafe code. We'll talk about unsafe in a second, but none of this has any unsafe code. Thank you. Thank you. Thank you. All right. So that was all about how we do parsing serialization. I mentioned at the very beginning that our header references are actually struct references, which you might have said, how the hell do you do that? And the answer is unsafe. So this diagram actually looks like this diagram. It's just like, that's a header reference. So the way that we do this is with an unsafe marker trait called frombytes. Frombytes, if a type implements frombytes, it promises to satisfy the following properties. First. Oops. Any size of t bytes, any sequence of bytes, size of t long, is a valid instance of this type. What that means is that I can take random bytes that I got off the network and just go, this is a thing now. I can just treat it as a thing. This is not true of all types. If I take a random sequence of eight bytes, and I say this is a reference now, that is not guaranteed to be a valid reference. It might just point randomly into memory somewhere. So it's important that this only actually applies to a subset of types. So we have to be very careful about which types we apply this to. For composite types, like structs and arrays and unions, all the fields or the elements have to be just recursively frombytes. It just composes nicely. And then if you really want, for enums, they must be C-like and have a power of two number of variants. Exercise to the reader why that's true. All right. Finally, we have a custom derive for this thing. So if we just slap derive frombytes on something, we've got a custom derive that will analyze your type and go, that's not frombytes. What are you talking about? Or yeah, that's fine. We'll go ahead and emit the impulse. So that's one thing. The next thing is the unaligned trait. So the unaligned trait says that something is. This is fantastic. Yeah. So the unaligned trait says that something is unaligned. Fantastic at naming. And so you can slap a derive on something and say, great, this thing is unaligned. Cool. It's important to note that this has some little wonky implications, such as this ether type field really should be a U16. But it can't because U16s are not unaligned. They have an alignment of two. So it's a little bit of like you have to use some byte order type stuff. It's super simple, but it's kind of annoying. But anyway, so it has to be unaligned because we don't know where in the buffer we might be parsing something from. So once we have those two things, we can build this awesome function. Now, this function doesn't literally exist in the code base, but there's something very analogous to it. And basically what it does is called ref from bytes. You say, here is a sequence of bytes of the appropriate length. If it is not of the appropriate length, then we return none. But if it is of the appropriate length, then it just gives us back a reference to T because the type system already took care of making sure that all of this stuff was safe. So you don't have to do any work at runtime. Like ideally, this function compiles to nothing. This function shouldn't actually ever run. But it's a place for us to write the unsafe keeper. And what it does is it just says, here's bytes. Here's a T, cool. Everything is safe. The compiler verified it for us. What this allows us to do is build this really cool function which uses one of my favorite, favorite, favorite methods from the entire library or from the entire code base. So this is actually the parsing function that we really have in the ethernet stack. It's stripped down. There's a removed some error handling. But we have this function, this method called takeOBJ, which I just love. It is a method on buffer, and it says the following. Consume size of T bytes from the buffer, add them from the body to the prefix. Reinterpret them as though it was a reference to T and return that reference. And so parsing then just becomes, OK, you have a header. It is a struct. You have a header struct. Cool. Take it. Done. That's all parsing. So this is literally what the ethernet parsing function looks like. Similarly for all the other parsers, it's basically just take a struct, take a struct, take a struct, done. And from here on out, by the way, this is a structured reference. So now we can just access it. We can say header.srsmack or whatever. So that's all I've got. So the conclusions from all of this. Again, as I said at the beginning, in C and C++, you can absolutely do this stuff. It is done. If you look at most high performance networking stacks, most of which are written in C and C++, they do all of this stuff. But they just do it unsafely. All of the guarantees that I described that are given by lifetimes and references, immutability versus mutability, the front-bytes, unlined stuff, all of that is just reasoned in the head of a program. And sometimes it is so unsafe that it can't be done. Some of you probably know what I'm referring to here. Famously, Parallel CSS layout computation in Chrome was attempted a number of times and was never done because they just couldn't iron out all of the bugs in Parallel C++ code. It was just too hard. Of course, it exists in Firefox. Why? Because it's written in Rust. And yeah. So the takeaway that I want to leave you with is that safety brings speed and developer friendliness. When I'm working on this codebase, I can move quickly. All of the other developers that I work with, they can move quickly because we're not afraid that we're going to accidentally invalidate some subtle invariant on a list of invariants that isn't actually written down anywhere. And it's developer friendly. Because when we get new people to the codebase, in fact, I think a majority of the developers that we have on this codebase, this is their first Rust project. And what that means is that they can come to this project and they can say, OK, this little from byte stuff is terrifying, but the rest of the codebase is fine. It's all safe. There's no unsafe. I don't have to worry about keeping all of these invariants in mind. I can just hack on the thing. And they do. So that's all I've got. Here are some code links. This last one here doesn't actually exist yet, but it will hopefully by the time this stuff is up. There's my email if you want to email me about anything. Thank you very much. Appreciate it.
 Thank you for that.  All right, so I'm going to try to go through this quickly for the sake of not running over  time.  A little bit about me first.  I work at security at Google.  I get to work on open source Rust, which is really cool.  If that sounds like something you would like to do, Rust or security or both, come talk  to me.  And I just started the secure code working group for Rust.  If you're interested in that, also come check us out.  The thing we're going to be talking about.  So the thing is an experimental networking stack handles the link layer through the transport  layer.  It's like Ethernet through TCP and UDP.  It is written in pure Rust, obviously.  Design goals are for low resource environment, so low CPU utilization, low binary footprint.  And finally, it is structured as a platform agnostic core and then some platform specific  bindings.  So if you are interested in using this thing for your kernel or your project or whatever,  just let me know.  It's obviously very early APIs are changing, but if it's just sort of experimental, that  could be cool.  A quick note on the subtitle that was advertised.  High performance.  This networking stack can currently respond to pings.  That is all it can do.  Yes, thank you.  Yeah.  So yeah, so when I say high performance, I'm not referring to actual benchmarks.  I'm referring more to design to the fact that we can prove statically that a bunch of things  are happening at compile time rather than runtime.  I suspect that those will lead to actual good performance later, but I don't know.  So we'll find out.  So brief outline for the talk.  First we're going to talk about design goals for the stack and the stuff we're going to  be talking about.  Give a little bit of background on packets and how they're laid out just to get everyone  on the same page.  First we're going to talk about parsing packets.  Then we're going to talk about serializing packets.  Then we're going to talk about forwarding, which is basically parsing and then serializing  packets.  Right, yeah.  It's very fancy.  And then finally we're going to talk about some nitty gritty details of how we do a zero  copy in a safe way.  So the goals for this design.  So first of all, zero copying.  When I say zero copying, I don't literally mean no memory reads or writes, obviously,  but if we have a big buffer and there is a packet living in the buffer and we want to  operate on it, so we want to parse it or modify it or serialize a new packet into the buffer,  we're not going to pull it out into some scratch space and operate on it and then move it back.  We're just going to do everything directly in place.  As I said, we expect this to make it high performance.  We'll find out.  The second thing is zero heap allocation, so everything's on the stack.  And then finally zero unsafe.  So no use of the unsafe keyword.  This may sound really cool and too good to be true.  That's because it is.  There's actually a little bit of unsafe.  But in standard Rust fashion, that unsafe is self-contained in nice little small packages  and it can be nicely composed.  Another thing I want to clarify here, this is not a look at the cool things you can do  in Rust Talk.  This is a look at the cool things you can do safely in Rust Talk.  Everything that we're going to describe here today can be done and is done in C and C++.  The only difference there is it's very unsafe and you, the programmer, have to verify a  lot of properties yourself.  So the distinction here is not like we can do this thing, but rather we can do this thing  differently.  Quick note on terminology.  When I say the word packet, I am aware for the patents in the audience that there are  things like Ethernet frames and TCP segments.  If you read the code base, you will find these things.  But for the sake of simplicity, everything is a packet.  I'm terribly sorry.  So a little bit of background on how network packets are laid out.  So packets are just a big sequence of bytes.  All these diagrams that you're going to see today are beginning byte on the left, end  byte on the right.  Packets are recursive.  So the analogy that we're going to be using throughout the entire talk, I hope you like  onions.  We're going to be talking a lot about onions.  So basically you've got a packet which contains a packet which contains another packet which  contains another packet and so on and so forth.  The packet that we're going to be talking about are just a header and a body.  I'm sorry for those of you who really like packet formats, no packets with footers.  So there's a format header, there's a header at the beginning whose format is fixed.  As in if you go and read the Ethernet spec, for example, you will see there is this field  and then this field and blah, blah, blah.  So the header has a well-known format that we can parse and then the body is completely  unstructured and variable-length.  We have no idea what's in there.  It's completely opaque.  Now once we know what is in there, that might be a different format which we can go parse.  So for example, we might discover that, oh look, there's an IP packet in there.  And then we can go pull out the IP spec and we can read it and say, oh, this is how you  parse an IP packet.  But from the perspective of Ethernet, it's completely opaque.  So let's say that we have this IP packet here.  We might say, well, we're going to parse this again, again, an IP packet just in the same  way, there's a format header at the beginning, we know what that format is, and then there's  an unstructured body, maybe this contains a UDP packet, and so on and so forth, we can  keep going like this.  An important thing to keep in mind here is that when we're parsing, we don't actually  know ahead of time what all of these types are going to be.  So we might get an Ethernet packet and we don't know what's inside of it.  It might be an ARP packet, it might be an IP packet.  We actually have to look at the header.  The header tells us what's inside, so we actually don't know until runtime what the next thing  that we're going to parse is.  So this is unfortunately, it has to be less static than you might like.  So let's talk very briefly about what our end goal is.  If you recall, there are four parts, parsing, serialization, and then forwarding is our  third part.  We're going to build up to forwarding at the end, and I want to describe very briefly  what that's going to look like, just to keep in mind what we're building towards.  So in forwarding, we are going to allocate a buffer on the stack, we're going to receive  an Ethernet packet into it.  We're going to parse the Ethernet packet, we're going to parse the IP packet inside  of the Ethernet packet, then we're going to decide to forward the packet.  So we're actually just going to resend it to somebody else.  And first we're going to update the header, we have to do some updates when we forward  packets, and then we're going to serialize it inside of a new Ethernet packet, and all  of that is going to happen in place on the stack, no copying.  So that's the goal that we're building up to, but first we have to get through some  other things, namely parsing.  So the goal of this section is a lot simpler.  We are going to allocate a buffer on the stack, and we're going to receive an Ethernet frame  into that buffer.  We're going to parse the packet, we're going to parse the packet, we're going to parse  the packet, anyway, this is the, you get the idea.  Very simple example, we just receive it and parse, parse, parse, parse, parse, that's  the whole example for this section.  So all right, the buffer trait.  The buffer trait is a really important trait in the net stack.  It is sort of like the core trait for parsing and serialization.  The reason that it's a trait and not a concrete type is that we have a bunch of implementations  of it, but the details there aren't important.  A really important thing to keep in mind with buffers is that they are referencing.  And by that I mean that when you say you have ownership of a buffer object, that's just  a tiny little struct thing with some pointers, right?  Maybe it's an own point, maybe it's owned, maybe it's a reference to something, but the  point is that thing is tiny, the buffer bites themselves live somewhere else.  So we're going to talk about like moving these things around a lot, keep in mind that that's  a very cheap operation because this is basically just a pointer.  So these things are basically just a sequence of bytes, just like a pack would be.  You might ask the question reasonably, why is this not just a slice?  Well, the answer is that we need to keep track of how much we've parsed so far or how much  we have serialized.  So for example, sorry, so the way that we do this is we split the buffer into a prefix  and a body.  The prefix is basically everything that I have not parsed or serialized so far, or sorry,  everything that I have parsed or not serialized, and then the body is everything else.  So it's essentially just a pointer into the buffer, right?  Here's the whole buffer and here's where I am so far, right?  So you can consume bytes from the body and they get added to the prefix, so I can shift  this way.  You can consume bytes from the prefix and they get added to the body, shift that way.  And the invariant that we're trying to maintain here, and this is going to be really important  through the whole thing, is that the body always contains, when you are parsing, the  stuff that you haven't parsed yet.  So if I have an ethernet packet and I chump the ethernet header off the beginning, then  I have the body left, and then when I pick up where I left off, I call the parse method  again, I'm naturally parsing the next most encapsulated thing.  And then when I'm serializing, the exact inverse happens.  The current thing in the body is everything that I have serialized so far, and when I  ask, please serialize your header, it starts where we left off and goes this way and expands  the body to include the now larger packet.  So what is in the body is really important, where that body is versus the prefix, that's  going to be sort of the core of everything here.  So let's walk through an example.  So this is a diagram here.  On the bottom, we have the actual physical structure of the packet, or sorry, of the  buffer, excuse me.  So it starts off as being all body.  On the top, we have the logical view.  So logically, the contents of these bytes are an ethernet header and an ethernet body,  but we don't know that yet.  And so when we parse it, it's going to look a little something like this.  So what we have here is the result of parsing is an ethernet packet object.  An ethernet packet object is a tiny little struct that just has some references into  the buffer.  Again, this is a zero copy theme here.  The ethernet packet object is just references.  It has one reference into the header and then one reference into the body.  An important thing to keep in mind here is that the header, so the body is just bytes,  we don't know what it is, so that's just byte slice.  The header, on the other hand, is actually structured.  So we'll get into at the end of the talk how we actually achieve that without unsafe, but  you should basically think of this as a struct reference.  So we have a struct.  It represents what the header actually looks like.  We've got this field and then that field and so on and so forth.  And this is just a reference into that struct, so we can just access it like plain vanilla  rust.  So we've got that struct.field, blah, blah, blah, blah.  Last thing to note is because these are references, the ethernet packet object borrows the buffer.  So if we want to actually move the buffer around, which we will want to do, we have  to drop the packet first.  So a pattern that you're going to see over and over again is first we parse, then we  operate on that actual object, and then before we do anything else, we drop it.  Then we pick up where we left off, but now we've forgotten about that thing.  Okay, so as we said before, we've got this and then we're going to drop it.  It goes away.  And so what we're left with is that the body of the buffer now corresponds to the body  of the ethernet packet.  So as I mentioned before, we can just pick up where we left off and parse the next packet.  So the exact same thing happens this time with the IP packet.  We have a header.  It references into the buffer and it is a structured reference to the actual, you know,  it's a struct that we can access the fields.  And then an unstructured reference to the body, it borrows the buffer and so when we're  done with it, we have to drop it.  Let's look at a little example of what this might look like in code.  This is obviously a little pseudo-codesh because they're, you know, more like I would want  to do more things if I receive an ethernet packet, but this really is the core of it.  The buffer has a parse method and you literally say, here is a type of a packet, please parse  it for me, done.  And so you get a packet object back here and then you might operate on it and then you  drop it and then you pass that buffer.  Again, as I said, you pass the buffer by value.  You're actually passing ownership.  This will become important later.  So you pass the buffer by ownership to the next layer of the stack.  The body is left as it was before, so the next layer of the stack just picks up right  where we left off.  So let's actually look at what this might look like in memory on the stack so we get  a sense of how all the memory is laid out.  So let's say that we receive a, we have a buffer that is allocated on the stack, so  it's in the stack from there.  First we pass the buffer into receive ethernet packet.  So this little black arrow here just means that the actual object is here but the actual  bytes of the buffer are still in the original stack frame.  First we parse the ethernet packet, just like we mentioned before.  So the ethernet packet struct itself, which is this tiny little thing, it lives in this  top stack frame here, but the references are all into the contents which live in the original  stack frame, right?  No copying.  So we drop it and then we send the buffer again into the receive IP packet function this  time.  And again, we parse this time an IP packet.  And the references, the tiny little struct lives in that stack frame and the references  point into the bottom stack frame.  So that's parsing.  It's very straightforward.  Let's get into serialization.  So the goal here is that we are going to receive a request to send an IP, a UDP packet.  So there is an application that says, you know, it's like a DNS client or something.  It says here are the contents of a UDP packet that I would like you to send out onto the  network.  So we're going to compute some header information for the UDP packet.  Then we're going to compute some routing information for IP, where are we sending this thing, and  then some information about what the IP header will need to have.  Then some ethernet routing and the header information.  And then finally, after we've done all of this, only then do we actually compute the  length and allocate the buffer.  So all of the computation before, we haven't actually been doing anything.  We've not been serializing.  We've not been doing any kind of allocation.  At the bottom, do we actually say, OK, now that we know how big the packet needs to be,  now we can allocate the buffer?  I know that I said stack allocation at the beginning.  We're going to heap allocate in this example just to keep things simple.  We'll actually stack allocate in the end.  And then finally, once we have a buffer, then we can just serialize in one big pass UDP,  UDPBotter, UDPHeader, IPHeader, ethernetHeader, and then we're done.  So a little note on why this is hard.  Some of you in the audience may be thinking, OK, well, if you know how big the packet is,  why don't you just allocate it ahead of time?  Why do you have to do all of this sort of weird stuff?  And the answer is that you need to send the request to the next layer before you serialize.  So for example, if I have a UDP body and I want to say, well, I need to know what buffer  I'm going to serialize it into, I have to know, obviously, how big all of the headers  are going to be so I know how big the buffer needs to be.  But the IP layer needs to know where it's sending the thing in order to know which link layer  protocol is being used.  Am I sending it over an ethernet network that has a header of this size?  Am I sending it over a Wi-Fi network that has a larger header?  So you need to actually compute all the routing information before you know how big the buffer  needs to be.  And so there's this chicken and egg problem where in order to call the function that says,  please send this thing for me, it cannot be serialized already.  But of course, you can't get control flow back because the thing needs to be sent by  the time you send you call the function.  And so that's why this is difficult, and we'll see how we solve this problem.  So the first trait that we're going to introduce to deal with parsing is the packet builder  trait.  Remember before that I said that all packets are onions, right?  And they are just sort of a series of layers.  The packet builder is like a single layer of the onion.  Nothing inside of it, nothing outside of it, just one sort of shell, right?  And it assumes that we already have a buffer allocated that already has enough prefix space  to serialize everything.  We'll solve that problem later.  And what it does is it serializes itself into the buffer.  So you give it a buffer like this.  So here's a buffer that has a body.  And you say, dear packet builder, so in this case an ethernet packet builder, please serialize  yourself right here, right?  We already gave you the body.  We gave you enough space, just write yourself in, right?  And so this contains all the metadata needed to figure out what that header needs to look  like.  Source address, destination address, so on and so forth.  So that handles just a layer of the onion.  Let's talk about the whole onion itself.  So the serializer trait, on the other hand, represents the entire onion.  So it represents a single layer and everything below it.  And it's recursive.  So if you take a serializer, which represents the whole onion, and you add a packet builder,  which represents the next layer of the onion, you get a bigger onion.  And this is how we're going to construct all of our serializers.  Another important thing to note here, a serializer does not actually represent a packet directly.  You can sort of think of it like a packet future.  It describes all of the information necessary to figure out how to allocate and serialize  a packet in the future later.  Let's walk through a very simple example.  So this is a sort of silly straw man example of a function that might construct a serializer.  It takes a serializer in and then wraps it in IP and wraps that in Ethernet and then returns  it.  So we start off with a serializer called SIR.  We encapsulate it in a new IP packet builder.  Remember that builder is then the IP layer of the onion and we get a new larger onion  back, right, a new larger serializer.  We encapsulate it again by adding yet another field, which is this Ethernet packet builder.  The struct just keeps growing.  We get the Ethernet packet onion, an even bigger onion, and we can now return it.  OK, so we know how to construct these things, but how do we actually use them?  How do we actually use them to create a buffer and serialize into it?  So the way that we do this is with the serializer trait again, and there is a method on the  serializer trait called serialize.  I'm good at naming.  The serializer method is responsible for producing a buffer.  So when you call it, it doesn't take an existing buffer.  It actually creates and gives you back the buffer.  And it might use a pool of buffers or it might use something that's already allocated.  Implementation is up to the implementation of the trait, but it gives you back the buffer.  And it takes the number of prefix bytes that you need so far.  Remember that all of this is sort of recursive.  So if I say I want to serialize an Ethernet packet, it says, OK, well, how many prefix  bytes do you need for all of the packets outside of me?  Because I don't know that.  And so what you do is you start at the outside of the onion, and you work your way inwards  building up the prefix length.  So you start at the very outer layer, and you say, well, I need zero bytes so far.  And then you go to the next layer, and you say, OK, well, this is an 18-byte header,  so now I need 18 bytes.  And then this next header and so on and so forth.  And you just build up the number of prefix bytes that you need until you get to the inner  core.  And once you get to the core, now you know in total how many prefix bytes you need, and  it's the core's responsibility to satisfy that request.  So it has to give you a buffer whose body contains all of the stuff that the innermost  packet needs to contain, and that has enough prefix length to handle all of the headers  that came before it that you're going to need to serialize in the future.  And so once you're in that innermost onion, and you've satisfied the request, you have  a big buffer, it has the body, and so on, then you can return it.  And now you start walking your way out.  So remember, we went in calculating the prefix length, and then you walk out and actually  serialize stuff.  So you go to the first layer, serialize the first header.  And then you expand the body, so now the body encapsulates the entire next packet.  Then you return it again, and the next layer serializes its header, and so on and so forth.  And by doing this, you make sure that you can just serialize in one shot.  You allocate once, you serialize once, and you're done.  Let's look at an example of what this looks like.  So this diagram is wildly not to scale.  Imagine that this struct here is really tiny, that body is really big, but diagrams are  hard.  So again, this packet, this serializer here represents the entire UDP packet, right?  So it has all of the body bytes that the application asked us to send.  And it has that packet builder for the UDP layer, right?  That shell of the onion that just represents all the stuff that needs to go in the UDP header.  And so when we want to send this, we're going to call this into the sendIP packet function,  and we're going to pass it by value, right?  This tiny little struct moves from this stack frame to this stack frame, but the body itself  again doesn't move.  We're going to shrink this just for brevity, because we're going to need to make these  slides pretty big, so that thing is the same thing as that thing.  The sendIP packet function then is going to calculate its own routing information, right?  It needs to figure out where this stuff is going, so on and so forth.  And on that basis, it's going to construct an IP packet builder for the IP layer of the  onion, and boom, construct a bigger serializer, right?  The IP packet builder represents the IP layer of the onion.  This entire serializer represents the entire IP packet and everything inside of it.  Again, send it to the next layer, so send it to the sendEthernet packet function, compress  for brevity.  Take this serializer and encapsulate it with Ethernet, right?  So add another layer of the onion, now we have the Ethernet onion.  All right, so now we've actually constructed this entire thing and we're ready to serialize  it.  Let's see how that works.  This is a diagram that if we had a giant wide screen, would have been totally horizontal,  but we don't.  So each of these dot, dot, dots represents the entire next line, right?  Just imagine that this is one big, long thing.  It's one, you know, small, but conceptually big struct.  So the first thing that we're going to do is call serialize0, right?  The outermost layer is the outermost layer, so we don't need any prefix bytes ahead of  it.  And in order to figure out how many extra prefix bytes we need for this layer, when  we call serialize into the next layer, we have to figure out how many bytes the Ethernet  header is going to take up, so we ask it.  We say, dear Ethernet packet builder, how many bytes will you need?  And it says 18, so we go, okay, great.  We needed zero, now we need 18, so we need 18 in total.  And again, the next, the next innermost packet, or sorry, the next innermost serializer, this  one for the IP layer, goes, okay, well.  I'm supposed to give this person 18.  My layer itself, the IP layer, right?  The IP header is 20 bytes long, so I need an extra 20 bytes.  So now I need 38 bytes in total.  The UDP layer adds another 8 bytes, so now in total it's 46.  But now we're in the middle, right?  We've actually gotten into the inner serializer, and there are no more serializers to do.  We can't recurse anymore, so we actually have to allocate the thing.  So first we have to figure out how long the body is, so let's say it's 1,000 bytes.  And now we know that we need 1,046 bytes, so we allocate them.  Again, we'll stack allocate later for the time being we're doing it on the heap.  We say, please give us a buffer, which is 1,046 bytes long.  And that buffer starts off entirely as prefix, because there's nothing, we haven't serialized  anything yet, so the body is completely empty.  There's nothing there.  So first we serialize the body that we were asked to into the buffer.  So now the body of the buffer is the same thing as the body of the UDP packet.  And now we can start rewinding back out, unwinding out the stack.  So first we go to the UDP packet builder and we say, hey, you know how to serialize a header.  Can you serialize yourself into this buffer, please?  And it does.  And it expands the body so that now the body contains both the UDP packet body and the  UDP packet header.  It's an entire UDP packet.  And now we're done.  This call to serialize has satisfied its constraint.  It was asked to give a buffer, which contains the entire UDP packet, and 38 bytes of prefix.  Well, we have that.  So we can return it.  Now this next call to serialize does the exact same thing.  It goes, OK, great.  I have a packet whose body is what it needs to be already.  There is enough prefix space.  So I will now serialize the IP layer in place.  And again, now the body contains the entire IP packet and has 18 bytes of prefix space.  So we can return that.  Finally we can serialize the ethernet layer.  And now the body contains the entire ethernet packet.  Thank you.  Yeah.  And so we're entirely done.  And we can return this thing.  And now we have a buffer.  So we talked about parsing.  We talked about serialization.  Let's put it together and forward.  So the goal this time is, first of all, no heap allocation.  We're actually going to allocate on the stack.  We're going to receive an ethernet packet.  We're going to parse it.  We're going to parse the IP packet inside of it.  We're going to decide that we want to forward the packet.  So we're going to reserialize it and send it to somebody else.  So first we have to update the header.  There are some modifications that we have to make to the header.  Then we have to serialize it in a new ethernet packet.  All right, so let's get into it.  So parsing is super easy.  We saw in the parsing section how this works.  This is going to be super straightforward.  Again, we have the buffer.  The actual bytes of it live on the stack.  We send it into the receiveEthernetPacket function.  We parse creating one of these ethernet packet objects, which  consumes those bytes from the prefix.  We drop the thing so we can continue operating on it.  And we send the buffer into the receiveIPPacket function.  Again, we parse this time getting an IP packet struct.  And so now we have to modify this stuff in place.  So this is where it gets interesting, because the modifications  that we need to do operate on the sort of logical structure  of the IP header.  So it's really important that this header is structured  and gives us access to struct fields and stuff like that  for us to be able to update it.  And so the updates that we're going to do  are actually methods on the IP packet object.  So here's some code.  It's very simple.  We just say packet, please decrement your TTL,  just like a counter of how many hops the packet has gone through.  And that just vanilla rust code, right?  It's struct.field minus minus, or minus equals 1.  Sorry.  And yeah, so it's just normal rust code.  So now we have this thing.  Now we drop this thing.  It's a little bit of a problem here.  Does anyone spot the problem if we want to serialize the IP packet?  What is the contents of the body?  Sorry?  It's not modified.  Well, so the body hasn't been modified,  but importantly, the body is the body of the IP packet.  We don't want to forward the body of the IP packet.  We want to forward the whole IP packet, including the header.  But we just parsed the header and threw it away.  So there's this cute little hack that we do.  It's very straightforward.  It's called undo parse.  We say, how many header bytes did you just parse?  Oops, please undo it.  And so we do.  It just takes the pointer and goes, oh, you just parsed 20 bytes.  OK, oops, bump it 20 bytes.  So we go from that to that.  And now we have an IP packet that we can forward.  And so let's do that.  So we decrement the TTL.  We figure out how many header bytes we need to undo,  and we undo them, and then we serialize them.  Very straightforward.  Now, you might be wondering, I thought  that sendEthernet packet was supposed to take a serializer.  Well, it is.  Here's the core idea that I want to get across.  Buffers are serializers.  The serializer trait simply says,  can you give me a buffer that has these properties?  If the buffer already has those properties,  then it says, yes, here I am.  It just returns itself by value.  The serialize method you might have noticed  if you were looking closely takes the serializer by value,  takes it by ownership.  So you basically take this buffer and you say, buy buffer.  Give me a buffer.  And it says, I'm already here.  Here I am back.  And that's what we're going to do.  So first we take the buffer.  We send it into receiveIP packet.  That receiveIP packet function, sorry,  excuse me, we're in receiveIP packet currently.  We send it into sendEthernet packet.  SendEthernet packet then says, OK, great.  Here's a serializer.  I need to now encapsulate it inside of Ethernet.  So we do.  Construct the serializer.  I'm going to squash these stack frames just for brevity.  So then the sendEthernet packet function says, OK, great.  I've got my serializer and I need a serializer.  So we call serialize of 0.  We want to serialize this thing.  It says, OK.  Ethernet header asks for 18 bytes in the prefix.  So we'll ask for serialize 18 of our serializer.  Now the buffer is the serializer.  So the buffer looks at itself and goes, well,  I already have 18 bytes.  So we're done.  And it returns itself by value.  And the call to serialize 0 goes, OK, great.  I have a buffer that has 18 bytes of prefix space.  I can just serialize this header directly in place.  Then we have finally a buffer, which is still on the stack.  We haven't ever copied it.  It contains all of the bytes of the Ethernet packet.  And we can just send this thing.  So we've gone from receiving the packet on the stack,  just in the stack.  It parsed it all the way.  We've decided that we want to re-serialize.  We've gone, hey, we already have this buffer space.  Why don't we just reuse it?  And we reuse it.  And finally, we can turn around and go, OK, great.  Let's just forward it out.  So this is what forwarding looks like.  And this is sort of like, this is it.  This is as simple as it gets.  There's obviously, in practice, a little bit more complicated  logic, because we have footers and other stuff like that.  But this is fundamentally the core of the algorithm.  And in none of this stuff is there any unsafe code.  We'll talk about unsafe in a second,  but none of this has any unsafe code.  Thank you.  Thank you.  Thank you.  All right.  So that was all about how we do parsing serialization.  I mentioned at the very beginning  that our header references are actually  struct references, which you might have said,  how the hell do you do that?  And the answer is unsafe.  So this diagram actually looks like this diagram.  It's just like, that's a header reference.  So the way that we do this is with an unsafe marker  trait called frombytes.  Frombytes, if a type implements frombytes,  it promises to satisfy the following properties.  First.  Oops.  Any size of t bytes, any sequence of bytes, size of t long,  is a valid instance of this type.  What that means is that I can take random bytes that I got off  the network and just go, this is a thing now.  I can just treat it as a thing.  This is not true of all types.  If I take a random sequence of eight bytes,  and I say this is a reference now,  that is not guaranteed to be a valid reference.  It might just point randomly into memory somewhere.  So it's important that this only actually applies  to a subset of types.  So we have to be very careful about which types we apply this  to.  For composite types, like structs and arrays and unions,  all the fields or the elements have  to be just recursively frombytes.  It just composes nicely.  And then if you really want, for enums,  they must be C-like and have a power of two number of variants.  Exercise to the reader why that's true.  All right.  Finally, we have a custom derive for this thing.  So if we just slap derive frombytes on something,  we've got a custom derive that will analyze your type  and go, that's not frombytes.  What are you talking about?  Or yeah, that's fine.  We'll go ahead and emit the impulse.  So that's one thing.  The next thing is the unaligned trait.  So the unaligned trait says that something is.  This is fantastic.  Yeah.  So the unaligned trait says that something is unaligned.  Fantastic at naming.  And so you can slap a derive on something and say, great,  this thing is unaligned.  Cool.  It's important to note that this has some little wonky  implications, such as this ether type field really  should be a U16.  But it can't because U16s are not unaligned.  They have an alignment of two.  So it's a little bit of like you have to use some byte order  type stuff.  It's super simple, but it's kind of annoying.  But anyway, so it has to be unaligned  because we don't know where in the buffer we might be parsing  something from.  So once we have those two things, we can build this  awesome function.  Now, this function doesn't literally exist in the code  base, but there's something very analogous to it.  And basically what it does is called ref from bytes.  You say, here is a sequence of bytes of the appropriate  length.  If it is not of the appropriate length, then we  return none.  But if it is of the appropriate length, then it just  gives us back a reference to T because the type system  already took care of making sure that all of this stuff was  safe.  So you don't have to do any work at runtime.  Like ideally, this function compiles to nothing.  This function shouldn't actually ever run.  But it's a place for us to write the unsafe keeper.  And what it does is it just says, here's bytes.  Here's a T, cool.  Everything is safe.  The compiler verified it for us.  What this allows us to do is build this really cool function  which uses one of my favorite, favorite, favorite methods  from the entire library or from the entire code base.  So this is actually the parsing function that we really  have in the ethernet stack.  It's stripped down.  There's a removed some error handling.  But we have this function, this method called takeOBJ,  which I just love.  It is a method on buffer, and it says the following.  Consume size of T bytes from the buffer, add them from the  body to the prefix.  Reinterpret them as though it was a reference to T and  return that reference.  And so parsing then just becomes, OK, you have a header.  It is a struct.  You have a header struct.  Cool.  Take it.  Done.  That's all parsing.  So this is literally what the ethernet parsing function  looks like.  Similarly for all the other parsers, it's basically just  take a struct, take a struct, take a struct, done.  And from here on out, by the way, this is a structured  reference.  So now we can just access it.  We can say header.srsmack or whatever.  So that's all I've got.  So the conclusions from all of this.  Again, as I said at the beginning, in C and C++, you  can absolutely do this stuff.  It is done.  If you look at most high performance networking stacks,  most of which are written in C and C++, they do  all of this stuff.  But they just do it unsafely.  All of the guarantees that I described that are given by  lifetimes and references, immutability versus  mutability, the front-bytes, unlined stuff, all of that is  just reasoned in the head of a program.  And sometimes it is so unsafe that it can't be done.  Some of you probably know what I'm referring to here.  Famously, Parallel CSS layout computation in Chrome was  attempted a number of times and was never done because they  just couldn't iron out all of the bugs in Parallel C++ code.  It was just too hard.  Of course, it exists in Firefox.  Why?  Because it's written in Rust.  And yeah.  So the takeaway that I want to leave you with is that  safety brings speed and developer friendliness.  When I'm working on this codebase, I can move quickly.  All of the other developers that I work with, they can move  quickly because we're not afraid that we're going to  accidentally invalidate some subtle invariant on a list of  invariants that isn't actually written down anywhere.  And it's developer friendly.  Because when we get new people to the codebase, in fact, I  think a majority of the developers that we have on  this codebase, this is their first Rust project.  And what that means is that they can come to this project  and they can say, OK, this little from byte stuff is  terrifying, but the rest of the codebase is fine.  It's all safe.  There's no unsafe.  I don't have to worry about keeping all of these invariants  in mind.  I can just hack on the thing.  And they do.  So that's all I've got.  Here are some code links.  This last one here doesn't actually exist yet, but it  will hopefully by the time this stuff is up.  There's my email if you want to email me about anything.  Thank you very much.  Appreciate it.
Move_fast_and_don_t_break_things__High-performance_UfMOOxOGCmA_0_59.png Move_fast_and_don_t_break_things__High-performance_UfMOOxOGCmA_60_131.png Move_fast_and_don_t_break_things__High-performance_UfMOOxOGCmA_132_191.png Move_fast_and_don_t_break_things__High-performance_UfMOOxOGCmA_192_263.png Move_fast_and_don_t_break_things__High-performance_UfMOOxOGCmA_264_743.png Move_fast_and_don_t_break_things__High-performance_UfMOOxOGCmA_744_1163.png Move_fast_and_don_t_break_things__High-performance_UfMOOxOGCmA_1164_1607.png Move_fast_and_don_t_break_things__High-performance_UfMOOxOGCmA_1608_5267.png Move_fast_and_don_t_break_things__High-performance_UfMOOxOGCmA_5268_8207.png Move_fast_and_don_t_break_things__High-performance_UfMOOxOGCmA_8208_8891.png Move_fast_and_don_t_break_things__High-performance_UfMOOxOGCmA_8892_8963.png Move_fast_and_don_t_break_things__High-performance_UfMOOxOGCmA_8964_9395.png Move_fast_and_don_t_break_things__High-performance_UfMOOxOGCmA_9396_11375.png Move_fast_and_don_t_break_things__High-performance_UfMOOxOGCmA_11376_11459.png Move_fast_and_don_t_break_things__High-performance_UfMOOxOGCmA_11460_17939.png Move_fast_and_don_t_break_things__High-performance_UfMOOxOGCmA_17940_21203.png Move_fast_and_don_t_break_things__High-performance_UfMOOxOGCmA_21204_24539.png Move_fast_and_don_t_break_things__High-performance_UfMOOxOGCmA_24540_24983.png Move_fast_and_don_t_break_things__High-performance_UfMOOxOGCmA_24984_27179.png Move_fast_and_don_t_break_things__High-performance_UfMOOxOGCmA_27180_34319.png Move_fast_and_don_t_break_things__High-performance_UfMOOxOGCmA_34320_34475.png Move_fast_and_don_t_break_things__High-performance_UfMOOxOGCmA_34476_35087.png Move_fast_and_don_t_break_things__High-performance_UfMOOxOGCmA_35088_39347.png Move_fast_and_don_t_break_things__High-performance_UfMOOxOGCmA_39348_42191.png Move_fast_and_don_t_break_things__High-performance_UfMOOxOGCmA_42192_42551.png Move_fast_and_don_t_break_things__High-performance_UfMOOxOGCmA_42552_42755.png Move_fast_and_don_t_break_things__High-performance_UfMOOxOGCmA_42756_45911.png Move_fast_and_don_t_break_things__High-performance_UfMOOxOGCmA_45912_48179.png Move_fast_and_don_t_break_things__High-performance_UfMOOxOGCmA_48180_48407.png Move_fast_and_don_t_break_things__High-performance_UfMOOxOGCmA_48408_51107.png Move_fast_and_don_t_break_things__High-performance_UfMOOxOGCmA_51108_51179.png Move_fast_and_don_t_break_things__High-performance_UfMOOxOGCmA_51180_51227.png Move_fast_and_don_t_break_things__High-performance_UfMOOxOGCmA_51228_51347.png Move_fast_and_don_t_break_things__High-performance_UfMOOxOGCmA_51348_51419.png Move_fast_and_don_t_break_things__High-performance_UfMOOxOGCmA_51420_51467.png Move_fast_and_don_t_break_things__High-performance_UfMOOxOGCmA_51468_51533.png