Drone Control - "Controlling a drone using Rust over WiFi"
CC Attribution 3.0 Unported:
You are free to use, adapt and copy, distribute and transmit the work or content in adapted or unchanged form for any legal purpose as long as the work is attributed to the author in the manner specified by the author or licensor.
Lechev, Lachezar
Have you flown a drone? What about writing Rust? Let me tell you about our efforts to hack on a drone using Rust and build a demo project that will showcase some practical capabilities. The drone we are using has a couple of ways you can communicate with. We are focusing our efforts on the WiFi (UDP) communication and decoding the Frames used by the drone using the open-source C SDK provided by the manufacturer. We will share with you what were the hurdles we've encountered as well as what we are planning for the future of the project.
 Hello everyone and welcome to my talk about drone control or controlling a drone using Rust. My name is Lachazar Lechif. I know it's hard to pronounce or remember so don't worry if you get it wrong or forget it. I'm from Bulgaria and I'm currently working at ADEX during the day. I'm using Rust for the last one and a half years. Yay! And it's full-time job. So this was my dream only two years ago and I finally achieved it. Our work is centered around the Ethereum network blockchain at the moment and we are building a display at network for reduced outfruits and increased user privacy. At night however I play around the IRIS working group that we have started at the beginning of this year and the goal is to help and push the open source community into the growing aerospace industry by providing information, materials, tools, crates and etc. to the industry itself and to hobbyists of course. One of the main reasons that I wanted to start this working group was in fact that I couldn't find enough information at a reasonable level that I can understand. At my job and at the working group I make and build open source projects. I love to travel a lot and combined with the fact that I'm location independent it's awesome. However, due to the current outbreak, traveling is not a certain thing anymore. Combined with traveling I love to go to festivals and parties, especially like trance music. After all we have to blow some steam somehow from time to time apart from being software engineers and developers. As you can also see I'm a heavy user of emojis as well. They're part of me and I just like it. Today we're going to take a look at the Parrot Anafi 4K drone. The company Parrot makes a bunch of drones, some of them fly, some of them don't. Initially when the whole project started, Ignition, one of the other co-founders of IroRust started hacking on a different drone. It is called the Jumping Zoom and it's a remote controlled two-wheeler that can also jump. However, we as in I thought it would be awesome if we make it work for the flying drones and try to control them. The picture you see here is one of the packages that the Parrot company has of the Anafi 4K drone. It includes the Sky Controller as well that you can use to control the drone. It is a good time to mention that none of this is sponsored by them in any means, at least not for now. It is an accessible small drone, especially because of the price and it's easy to work with. And why did we choose it in the first place? Well, we chose it because it has a lot of features including Wi-Fi over which you control the drone. The drone has its own network that you can connect to. It has a 4K camera and the Parrot company provides a series of K to be used for the drones. It's open sourced and it's the main way we find many of the things that we need in order to hack on the drone using Rust. The company provides other SDKs as well for building Android and iOS apps. They also include a Python version but it's using the CSDK and that's where the other SDK, PyParrot, comes into play. PyParrot is a Python SDK and it's very, very helpful when things don't work and you wonder why. There is a simulation tool called Swinks. It has many features that I haven't reviewed or used yet. However, since I don't actually own enough 4K drone, it's very useful to play around with it and build the Rust SDK with it. There is also the free flight app which is for iOS and Android that gives you the ability to connect to any Parrot drone and you can use it even for the simulation. I tried it one day but that's not what we are here for. We want to use Rust to control it. There is the SCAS controller that I already mentioned and the list of features goes on. Now that we know what we are working with, it's time to take a look at how does it work. The main point is that it all has to do with communication. The way we communicate with the drone is over Wi-Fi, the same way as the apps and the SCAS controller. The SCAS controller is a to your phone. It has its own separate Wi-Fi antennas and stuff and I think the phone is used for the video stream itself instead of controlling the drone. Regarding the connection and the communication, we need to talk about three main points. The first one is the handshake. After connecting to the Wi-Fi of the drone, we need to establish communication between our controller that is our program and the drone. The Parrot drones use their own protocol that we are going to take a look at. It includes different mechanisms and things that we are still finding out how to work with. The protocol uses what we call frames. We will see what they look like and represent in a bit. This is all incorporated in the AR SDK errors create, which is in heavy development stage right now, aka work in progress. Now let's start with the handshake. After we have connected to the drone over Wi-Fi, we need to establish the connection between our program, aka the controller and the drone. Each drone has its own initial port. It's usually 44444 and we use TCP and make sure that we handle the handshake correctly, especially the response from the drone. Through the established TCP connection, we send the request, which is serialized with Serde and Serde JSON into a JSON stream. It includes the D2C port, which means drone controller port, alongside the controller name and type, which I'm not really sure how the drone uses. There are other ports for the video streams, which I haven't bothered to check yet. After sending the request, we should get a response in JSON, which we serialize again with Serde and Serde JSON. We get a controller to drone port from the drone and its response, which is basically how the controller will speak to the drone. It also includes status field, and if the status field is zero, then we have a connection. Otherwise, we need to try the handshake again. The response provides other fields as well, regarding other options, ports, etc. We currently don't use them, and honestly, at this point, I don't know what they are for yet. By this, we have a connection. Hooray! To get a visual representation of what I just explained, I've included the request logs here that you can see, and we use the same 444444 port to connect the drone and send the request to the drone, as well as the field we sent to it. And finally, we have the successful response from the drone. Again, you can see the controller to drone port and the status fields, as well as the other fields that the drone sends. And now that we have this connection with the drone established, it's time to talk about the protocol. The protocol uses what we call frames. And the first part of the protocol is the pingpong frames slash mechanism. Then we have the listener that listens to the provided drone controller port for incoming from the drone messages and frames, and serializes them. Then finally, we have the command sender, which we will use to send frames to control the drone. The pingpong frames and mechanism in a nutshell means that the drone sends us a pink frame with some gibberish data, which I actually found out a couple of days ago. Before that, I thought it also combines an actual frame. However, this is not the case. It's just gibberish. And the drone expects a response of that pink frame called Pong, which we will see how it works when we talk about the frames themselves. The Pong frame should contain the same data, the same gibberish data that was received from the ping. And if we don't send any Pong response in seven seconds, the drone usually disconnects the controller, as in our program, and we need to do the whole handshake again. The listener at this point does some basic stuff which are crucial to the communication, the connection itself, and debugging and so on. It uses a UDP socket with the controller to drone ports that was provided by the drone during the handshake. And each received message can contain more than one frame. And we will see how we cope with this and extract each frame from the message. The listener also deserializes all known and unknown frames, since it's still work in progress. Not all frame deserialization is implemented. And this gives us an easy way to gradually decode each frame, either by using the C or the PyParat SDKs and looking into them and trying to understand what each byte means. At this point, the listener is handling ping responses by sending a new Pong frame to the command sender so that it can also be sent to the drone, as well as the so-called acknowledge frames, which will get to in a bit, which will get to it in a bit. I've also included some logs from the listener to better understand what's happening. As we can see, we receive some bytes from the drone, and then we see the deserialized version of the bytes. Yeah, we are going to take a look at each individual element in a bit. On the other hand, the command sender handles the sending of frames from our program to the drone. It uses UDP socket as well, and we expect it to send frames from the controller to the drone on the drone to controller port. This is the same port that we have provided on the request when making the handshake. Right now, it sends the raw bytes to the drone. The actual serialization is happening on a different place. The logging, again, is straightforward. This example is for the takeoff frame, which will tell the Anafi 4K drone how to magically take off and how we're above the ground waiting for further frames and commands. The earlier logs that we saw from the listener were actually the acknowledged response from the drone for the takeoff frame that we saw in the previous slide. Basically, it is saying that it received the frame and executed it. In order to fully understand and know what each thing means, we need to take out the frames. As we mentioned before, we can receive more than one frame from the drone and the way we distinguish between every frame, like shown here, is the way the frame and its bytes are arranged. I've personally found that it's easier to understand the frame by representing the bytes as U8 or unsigned 8-bit numbers. This message can roughly be translated to this. The first frame we have, we can decode, has a type data, which means that it's a frame containing some data. The buffer ID is pink, and following the buffer ID, we have the sequence ID. This sequence ID is based on each individual buffer ID, and it starts from zero. When the count gets to the limit of U8, it overflows and starts from zero again. This way, we can send an acknowledged frame to confirm to the drone that we received a specific frame. But we can also make sure that the drone returns an acknowledged frame that guarantees that a command we sent is executed. For example, we might want to know if the drone has actually landed and executed the landing command, otherwise it can still hover over the surface. The next four bytes, or U32 in a nutshell, is the length of the full frame. It is serialized using a little Indian or the less significant byte first. The length includes the headers type buffer sequence ID, each represented by one byte. The U32 that the length represents, this is four bytes, and the feature length of the frame if there is one. And by this length, we can split each and every individual frame if we have received more than one. In similar matter, we can decode the second frame in the first one. And the type here is data with acknowledge, which means that the drone requires us to send an acknowledge for this frame, meaning that we need to confirm that we have received it. Then we have the buffer ID DC event, which means drone to controller event. And this is an event that the drone sends us to notify us about something happening. In this case, it's sending us a calibration state that the drone is into. And as we saw in the previous slides, the message had 35 bytes. And we can see that the length of the first frame is 23, and the length of the second one is 12, which is a total of 35 bytes. And with the current implemented types, this looks like so. To elaborate a bit on the whole buffer ID and sequence ID, you can see that the pink frame has a sequence ID one, and the DC event also has a sequence ID of one. Each buffer ID has its own counter. The next time we receive a pink frame, it should be with a sequence of two. The same goes for the frames that we are sending. As you can also see, not all types are currently implemented and defined. And although the pink in the pink frame, we see that the feature is unknown. And with the value of three, this is not used since it's part of the gibberish date of pink, and not an actual feature value. Now let's take a look at the type and the buffer ID. The type basically means that what type of frame we have, what kind. As we can see, we have acknowledged data. Most of them are easy to understand, but some of them we still don't know how to use, or we haven't looked at them. On the other side, we see the buffer ID. They are usually separated into distinct categories. From the drone to the controller, they are usually prefixed with DC, not all. The pink and the acknowledge from send with acknowledge, the last one. They are not prefixed yet. And we can see that the punk is also not prefixed, and it's part of the controller to drone buffer IDs. These are the buffer IDs that we're going to use to send frames to the drone. And we already saw the DC event in action when we took a look at the calibration state example from the previous slide. Now that we know at least what the headers do and how what the frame looks like and represents, it's now time to take a look at the frame itself and build a whole frame, which we can use to send to the drone. The frame struct is pretty simple and straightforward. The bulk of the work, however, is in the features. Those features represent each and every command, event, setting, state, etc. That the drone understands. Some features include only simple values that we saw in the previous example. Some of the features include C strings and special parsing, like for example the setting, the date and time of the drone. Even the frame for sending piloting commands to the drone includes a timestamp that is specially encoded, that is in a specific format, let's say, and we need to parse it for both serialization and serialization. It is important to mention though that when we are serializing the frame, we serialize the feature first of the frame and then we can calculate the total length of the frame in bytes. When we have the total length, we can encode it at the right bytes in order for the frame to be fully ready to be sent to the drone. As we saw in the messages example, it's usually like the third byte to the seven or something like that. For the actual serialization and the serialization of the frames, we are using the create scroll. This means that for each struct or enum, we have to implement try into context and try from context, which gives us the ability to serialize and serialize the bytes as we saw earlier. This does require a lot of research and boilerplate at this point in order to map every single feature that we can have. This is why we still have a lot to implement and track down in order to finish the whole serialization and the serialization frames. Now I was going to show you a GIF of a working example. However, let's see how the God Supply demo, if they are going to allow me to show it live to you instead of actual GIF. First we need to start the Sphinx tool. It's a pretty simple command. It does steal your Wi-Fi cards because it needs to create the same Wi-Fi as in the drone. It uses the same firmware as the drone and it also includes this dot drone file that has some settings you can tweak and the 3D model and stuff like that. When we start it, it's going to give us this gazebo UI and we can see the drone right here in the middle. Maybe I can. And if I rotate it, we can see the camera here. So it's pointing at the back right now. For reference, we can insert a 3D model of some building, an apartment building. And usually there is a video stream of the drone. However, because I don't have a dedicated GPU, I'm not sure if it's going to work very well. It usually doesn't. It gets to the initial view of the webcam of the camera, actually. And then it doesn't update until the drone has landed. However, let's try it and see if it works. So we have the drone here. I have this example for taking off with the drone and I should take off hover for a couple of seconds. Then it should fly up and down and land. As we can see here, the logs that I showed earlier with all the frames and the actual drone now moving and accepting the commands we're sending. Let's try the video. Nope, the video is broken. So I'll just close it. And we can see that we're currently sending the take down command. And now we've just sent the landing command. And the drone has landed successfully. The drone, we're still keeping the connection with the drone and it's still sending some information although it has landed. And we can still start it up again very easily if we send another take off command. And thankfully this worked. Yay! Demo time. Yes. Oops. This is still work in progress. Pretty much. And we have a long, long way to go. However, it was really fun to build this and we are exploring new things that we can make and do with the drone. I've been in contact, for example, with the Rust Computer Vision working group and I hope we can make some post-processing of the video from the drone. We still have a lot of flying to do, especially to figure out what is a good approach for the API to make it easier, more idiomatic to use and to control the drone in each aspect of the flying itself. There are, of course, much improvements. It is just hacking at this point and trying to make things work. There are definitely improvements to be made, especially separating the frames by a receiving frames and sending frames to make it more comfortable to work with them and build the correct frames instead of sending something that the drone won't understand. And some of the other mechanisms that I haven't mentioned here, especially around the flying and sending the commands and receiving outputs of them or confirmations. We still have a lot of implementations to do, all of the features and we need to be able to distinguish between the receiving and the sending because when we receive an unknown frame and we don't know how to handle it, this can crash the drone and we should know all the correct values for that. I hope this whole experience is inspiring because we had a lot of fun building it, especially thanks to Ignition who started the whole journey, starting with the small jumping zoom and then I just, let's try the flying drone. We had a lot of frustration. Why doesn't work? What's broken? Why does it disconnect? However, with enough time and how to say it, with enough, yeah, when you want to do something and you want it really hard, you can achieve it. It's just a matter of time. And this project is part of the Aerobrust Working Group. You can find the SDK repository below as well as some other cool projects we are working on in our GitHub repository and organization. I've provided some links to some of the Parrots 2s as well as the Rust Computer Vision organization if you're interested in that. And thank you for your time. Oh, okay, perfect. Oh, couple slides. Sorry. Oh, yeah. I'm open for questions. So please share them on whatever streaming portal you're looking this at. Question on Twitch. R.P. Ruiz. Sorry if I missed this at the beginning, but is this an OS project? Anyway, one can collaborate. Yes, the software is free and open source right now. And I have provided you in the last slide with the actual repository that you can get involved in. Just jumping into the Discord server, ask questions. There are a lot of people there, especially people with backgrounds in whatever, they're aerospace engineers and material engineers. We have separate groups of people working on different projects in different areas. Okay, another question. You too. Sorry, would it be possible to run Rust Flight Controller software directly on the drone without the network round trip? Yeah, I haven't checked that. Probably it might be possible to flash some custom firmware on it, but at this point we're still playing with Wi-Fi. Question on Twitch. Do you welcome contributors without much experience? Yes, we do. The whole point of the IroRust community, not only a working group, we're a whole community, is to share knowledge and information on various topics about Iro space and engineering and whatever space even. So it's definitely encouraging to see new people coming in and trying to bring them up to speed or try to help them find their place. Well, another question. Maybe I can return the slides that we missed. YouTube, Boris, Tilan, is there interest in space industry to use Rust? Yes, yes, there is. There is quite a big interest right now, especially in the Rust community, as far as I know. On another conference, on Oxydise, they were talking about how they want to bring the Rust and to verify the compiler or at least part of it to be used in space, because there are really tight things that have verification of the tools. Each tool must be verified that it does what it's meant to be and it's defined somehow. Question on YouTube. Michael Lazowski, have you tried to write it in other languages? For example, C or C++? No, mainly working with Rust nowadays and I don't even know C and C++ a lot. However, the main SDK is actually in C, so it might be interesting to take a look at it. Okay, so I need to pick somebody with the most interesting question. I definitely like questions about Rust because the main idea is to share information and although I don't know anything about drones or space, I can actually go somewhere and ask somebody and start preparing myself even for a career there. In that case, I would like to switch another question. Final question. I want to have a compiler purimate from Rust before we can verify it. Targeting LLVM would be hard to verify. If I remember correctly, we don't need pure Rust compiler to do so. There are subsets of the C or C++ compilers that are verified as far as I know. We can do the same thing for Rust, especially given that we now have two backends or frontends to keep forgetting. With CraneLift, we can take a subset of Rust and try to verify it bit by bit until we can start using it in space applications. And as I said, I do like questions regarding Rust. So I would say XCcage. I choose you. Best question. Yeah, from Twitch. So thank you, everybody. Sorry, I'm a bit nervous talking in front of people. However, it was really fun and I hope you had some inspirational things that you found in my talk. Bye-bye.
 Hello everyone and welcome to my talk about drone control or controlling a drone using  Rust.  My name is Lachazar Lechif.  I know it's hard to pronounce or remember so don't worry if you get it wrong or forget  it.  I'm from Bulgaria and I'm currently working at ADEX during the day.  I'm using Rust for the last one and a half years.  Yay!  And it's full-time job.  So this was my dream only two years ago and I finally achieved it.  Our work is centered around the Ethereum network blockchain at the moment and we are building  a display at network for reduced outfruits and increased user privacy.  At night however I play around the IRIS working group that we have started at the beginning  of this year and the goal is to help and push the open source community into the growing  aerospace industry by providing information, materials, tools, crates and etc. to the industry  itself and to hobbyists of course.  One of the main reasons that I wanted to start this working group was in fact that I couldn't  find enough information at a reasonable level that I can understand.  At my job and at the working group I make and build open source projects.  I love to travel a lot and combined with the fact that I'm location independent it's  awesome.  However, due to the current outbreak, traveling is not a certain thing anymore.  Combined with traveling I love to go to festivals and parties, especially like trance music.  After all we have to blow some steam somehow from time to time apart from being software  engineers and developers.  As you can also see I'm a heavy user of emojis as well.  They're part of me and I just like it.  Today we're going to take a look at the Parrot Anafi 4K drone.  The company Parrot makes a bunch of drones, some of them fly, some of them don't.  Initially when the whole project started, Ignition, one of the other co-founders of  IroRust started hacking on a different drone.  It is called the Jumping Zoom and it's a remote controlled two-wheeler that can also jump.  However, we as in I thought it would be awesome if we make it work for the flying drones and  try to control them.  The picture you see here is one of the packages that the Parrot company has of the Anafi 4K  drone.  It includes the Sky Controller as well that you can use to control the drone.  It is a good time to mention that none of this is sponsored by them in any means, at  least not for now.  It is an accessible small drone, especially because of the price and it's easy to work  with.  And why did we choose it in the first place?  Well, we chose it because it has a lot of features including Wi-Fi over which you control  the drone.  The drone has its own network that you can connect to.  It has a 4K camera and the Parrot company provides a series of K to be used for the drones.  It's open sourced and it's the main way we find many of the things that we need in order  to hack on the drone using Rust.  The company provides other SDKs as well for building Android and iOS apps.  They also include a Python version but it's using the CSDK and that's where the other  SDK, PyParrot, comes into play.  PyParrot is a Python SDK and it's very, very helpful when things don't work and you wonder  why.  There is a simulation tool called Swinks.  It has many features that I haven't reviewed or used yet.  However, since I don't actually own enough 4K drone, it's very useful to play around  with it and build the Rust SDK with it.  There is also the free flight app which is for iOS and Android that gives you the ability  to connect to any Parrot drone and you can use it even for the simulation.  I tried it one day but that's not what we are here for.  We want to use Rust to control it.  There is the SCAS controller that I already mentioned and the list of features goes on.  Now that we know what we are working with, it's time to take a look at how does it work.  The main point is that it all has to do with communication.  The way we communicate with the drone is over Wi-Fi, the same way as the apps and the SCAS  controller.  The SCAS controller is a to your phone.  It has its own separate Wi-Fi antennas and stuff and I think the phone is used for the  video stream itself instead of controlling the drone.  Regarding the connection and the communication, we need to talk about three main points.  The first one is the handshake.  After connecting to the Wi-Fi of the drone, we need to establish communication between  our controller that is our program and the drone.  The Parrot drones use their own protocol that we are going to take a look at.  It includes different mechanisms and things that we are still finding out how to work  with.  The protocol uses what we call frames.  We will see what they look like and represent in a bit.  This is all incorporated in the AR SDK errors create, which is in heavy development stage  right now, aka work in progress.  Now let's start with the handshake.  After we have connected to the drone over Wi-Fi, we need to establish the connection  between our program, aka the controller and the drone.  Each drone has its own initial port.  It's usually 44444 and we use TCP and make sure that we handle the handshake correctly,  especially the response from the drone.  Through the established TCP connection, we send the request, which is serialized with  Serde and Serde JSON into a JSON stream.  It includes the D2C port, which means drone controller port, alongside the controller  name and type, which I'm not really sure how the drone uses.  There are other ports for the video streams, which I haven't bothered to check yet.  After sending the request, we should get a response in JSON, which we serialize again  with Serde and Serde JSON.  We get a controller to drone port from the drone and its response, which is basically  how the controller will speak to the drone.  It also includes status field, and if the status field is zero, then we have a connection.  Otherwise, we need to try the handshake again.  The response provides other fields as well, regarding other options, ports, etc.  We currently don't use them, and honestly, at this point, I don't know what they are  for yet.  By this, we have a connection.  Hooray!  To get a visual representation of what I just explained, I've included the request logs  here that you can see, and we use the same 444444 port to connect the drone and send the request  to the drone, as well as the field we sent to it.  And finally, we have the successful response from the drone.  Again, you can see the controller to drone port and the status fields, as well as the  other fields that the drone sends.  And now that we have this connection with the drone established, it's time to talk about  the protocol.  The protocol uses what we call frames.  And the first part of the protocol is the pingpong frames slash mechanism.  Then we have the listener that listens to the provided drone controller port for incoming  from the drone messages and frames, and serializes them.  Then finally, we have the command sender, which we will use to send frames to control  the drone.  The pingpong frames and mechanism in a nutshell means that the drone sends us a pink frame  with some gibberish data, which I actually found out a couple of days ago.  Before that, I thought it also combines an actual frame.  However, this is not the case.  It's just gibberish.  And the drone expects a response of that pink frame called Pong, which we will see how it  works when we talk about the frames themselves.  The Pong frame should contain the same data, the same gibberish data that was received from  the ping.  And if we don't send any Pong response in seven seconds, the drone usually disconnects  the controller, as in our program, and we need to do the whole handshake again.  The listener at this point does some basic stuff which are crucial to the communication,  the connection itself, and debugging and so on.  It uses a UDP socket with the controller to drone ports that was provided by the drone  during the handshake.  And each received message can contain more than one frame.  And we will see how we cope with this and extract each frame from the message.  The listener also deserializes all known and unknown frames, since it's still work in progress.  Not all frame deserialization is implemented.  And this gives us an easy way to gradually decode each frame, either by using the C or  the PyParat SDKs and looking into them and trying to understand what each byte means.  At this point, the listener is handling ping responses by sending a new Pong frame to the  command sender so that it can also be sent to the drone, as well as the so-called acknowledge  frames, which will get to in a bit, which will get to it in a bit.  I've also included some logs from the listener to better understand what's happening.  As we can see, we receive some bytes from the drone, and then we see the deserialized  version of the bytes.  Yeah, we are going to take a look at each individual element in a bit.  On the other hand, the command sender handles the sending of frames from our program to  the drone.  It uses UDP socket as well, and we expect it to send frames from the controller to the  drone on the drone to controller port.  This is the same port that we have provided on the request when making the handshake.  Right now, it sends the raw bytes to the drone.  The actual serialization is happening on a different place.  The logging, again, is straightforward.  This example is for the takeoff frame, which will tell the Anafi 4K drone how to magically  take off and how we're above the ground waiting for further frames and commands.  The earlier logs that we saw from the listener were actually the acknowledged response from  the drone for the takeoff frame that we saw in the previous slide.  Basically, it is saying that it received the frame and executed it.  In order to fully understand and know what each thing means, we need to take out the  frames.  As we mentioned before, we can receive more than one frame from the drone and the way  we distinguish between every frame, like shown here, is the way the frame and its bytes are  arranged.  I've personally found that it's easier to understand the frame by representing the  bytes as U8 or unsigned 8-bit numbers.  This message can roughly be translated to this.  The first frame we have, we can decode, has a type data, which means that it's a frame  containing some data.  The buffer ID is pink, and following the buffer ID, we have the sequence ID.  This sequence ID is based on each individual buffer ID, and it starts from zero.  When the count gets to the limit of U8, it overflows and starts from zero again.  This way, we can send an acknowledged frame to confirm to the drone that we received a  specific frame.  But we can also make sure that the drone returns an acknowledged frame that guarantees that  a command we sent is executed.  For example, we might want to know if the drone has actually landed and executed the  landing command, otherwise it can still hover over the surface.  The next four bytes, or U32 in a nutshell, is the length of the full frame.  It is serialized using a little Indian or the less significant byte first.  The length includes the headers type buffer sequence ID, each represented by one byte.  The U32 that the length represents, this is four bytes, and the feature length of the  frame if there is one.  And by this length, we can split each and every individual frame if we have received  more than one.  In similar matter, we can decode the second frame in the first one.  And the type here is data with acknowledge, which means that the drone requires us to send  an acknowledge for this frame, meaning that we need to confirm that we have received it.  Then we have the buffer ID DC event, which means drone to controller event.  And this is an event that the drone sends us to notify us about something happening.  In this case, it's sending us a calibration state that the drone is into.  And as we saw in the previous slides, the message had 35 bytes.  And we can see that the length of the first frame is 23, and the length of the second  one is 12, which is a total of 35 bytes.  And with the current implemented types, this looks like so.  To elaborate a bit on the whole buffer ID and sequence ID, you can see that the pink  frame has a sequence ID one, and the DC event also has a sequence ID of one.  Each buffer ID has its own counter.  The next time we receive a pink frame, it should be with a sequence of two.  The same goes for the frames that we are sending.  As you can also see, not all types are currently implemented and defined.  And although the pink in the pink frame, we see that the feature is unknown.  And with the value of three, this is not used since it's part of the gibberish date of pink,  and not an actual feature value.  Now let's take a look at the type and the buffer ID.  The type basically means that what type of frame we have, what kind.  As we can see, we have acknowledged data.  Most of them are easy to understand, but some of them we still don't know how to use, or  we haven't looked at them.  On the other side, we see the buffer ID.  They are usually separated into distinct categories.  From the drone to the controller, they are usually prefixed with DC, not all.  The pink and the acknowledge from send with acknowledge, the last one.  They are not prefixed yet.  And we can see that the punk is also not prefixed, and it's part of the controller to drone buffer  IDs.  These are the buffer IDs that we're going to use to send frames to the drone.  And we already saw the DC event in action when we took a look at the calibration state  example from the previous slide.  Now that we know at least what the headers do and how what the frame looks like and represents,  it's now time to take a look at the frame itself and build a whole frame, which we can  use to send to the drone.  The frame struct is pretty simple and straightforward.  The bulk of the work, however, is in the features.  Those features represent each and every command, event, setting, state, etc.  That the drone understands.  Some features include only simple values that we saw in the previous example.  Some of the features include C strings and special parsing, like for example the setting,  the date and time of the drone.  Even the frame for sending piloting commands to the drone includes a timestamp that is  specially encoded, that is in a specific format, let's say, and we need to parse it for both  serialization and serialization.  It is important to mention though that when we are serializing the frame, we serialize  the feature first of the frame and then we can calculate the total length of the frame  in bytes.  When we have the total length, we can encode it at the right bytes in order for the frame  to be fully ready to be sent to the drone.  As we saw in the messages example, it's usually like the third byte to the seven or something  like that.  For the actual serialization and the serialization of the frames, we are using the create scroll.  This means that for each struct or enum, we have to implement try into context and try  from context, which gives us the ability to serialize and serialize the bytes as we saw  earlier.  This does require a lot of research and boilerplate at this point in order to map every single  feature that we can have.  This is why we still have a lot to implement and track down in order to finish the whole  serialization and the serialization frames.  Now I was going to show you a GIF of a working example.  However, let's see how the God Supply demo, if they are going to allow me to show it live  to you instead of actual GIF.  First we need to start the Sphinx tool.  It's a pretty simple command.  It does steal your Wi-Fi cards because it needs to create the same Wi-Fi as in the drone.  It uses the same firmware as the drone and it also includes this dot drone file that has  some settings you can tweak and the 3D model and stuff like that.  When we start it, it's going to give us this gazebo UI and we can see the drone right here  in the middle.  Maybe I can.  And if I rotate it, we can see the camera here.  So it's pointing at the back right now.  For reference, we can insert a 3D model of some building, an apartment building.  And usually there is a video stream of the drone.  However, because I don't have a dedicated GPU, I'm not sure if it's going to work very  well.  It usually doesn't.  It gets to the initial view of the webcam of the camera, actually.  And then it doesn't update until the drone has landed.  However, let's try it and see if it works.  So we have the drone here.  I have this example for taking off with the drone and I should take off hover for a couple  of seconds.  Then it should fly up and down and land.  As we can see here, the logs that I showed earlier with all the frames and the actual  drone now moving and accepting the commands we're sending.  Let's try the video.  Nope, the video is broken.  So I'll just close it.  And we can see that we're currently sending the take down command.  And now we've just sent the landing command.  And the drone has landed successfully.  The drone, we're still keeping the connection with the drone and it's still sending some  information although it has landed.  And we can still start it up again very easily if we send another take off command.  And thankfully this worked.  Yay!  Demo time.  Yes.  Oops.  This is still work in progress.  Pretty much.  And we have a long, long way to go.  However, it was really fun to build this and we are exploring new things that we can make  and do with the drone.  I've been in contact, for example, with the Rust Computer Vision working group and I hope  we can make some post-processing of the video from the drone.  We still have a lot of flying to do, especially to figure out what is a good approach for  the API to make it easier, more idiomatic to use and to control the drone in each aspect  of the flying itself.  There are, of course, much improvements.  It is just hacking at this point and trying to make things work.  There are definitely improvements to be made, especially separating the frames by a receiving  frames and sending frames to make it more comfortable to work with them and build the  correct frames instead of sending something that the drone won't understand.  And some of the other mechanisms that I haven't mentioned here, especially around the flying  and sending the commands and receiving outputs of them or confirmations.  We still have a lot of implementations to do, all of the features and we need to be  able to distinguish between the receiving and the sending because when we receive an  unknown frame and we don't know how to handle it, this can crash the drone and we should  know all the correct values for that.  I hope this whole experience is inspiring because we had a lot of fun building it, especially  thanks to Ignition who started the whole journey, starting with the small jumping  zoom and then I just, let's try the flying drone.  We had a lot of frustration.  Why doesn't work?  What's broken?  Why does it disconnect?  However, with enough time and how to say it, with enough, yeah, when you want to do something  and you want it really hard, you can achieve it.  It's just a matter of time.  And this project is part of the Aerobrust Working Group.  You can find the SDK repository below as well as some other cool projects we are working  on in our GitHub repository and organization.  I've provided some links to some of the Parrots 2s as well as the Rust Computer Vision organization  if you're interested in that.  And thank you for your time.  Oh, okay, perfect.  Oh, couple slides.  Sorry.  Oh, yeah.  I'm open for questions.  So please share them on whatever streaming portal you're looking this at.  Question on Twitch.  R.P.  Ruiz.  Sorry if I missed this at the beginning, but is this an OS project?  Anyway, one can collaborate.  Yes, the software is free and open source right now.  And I have provided you in the last slide with the actual repository that you can get  involved in.  Just jumping into the Discord server, ask questions.  There are a lot of people there, especially people with backgrounds in whatever, they're  aerospace engineers and material engineers.  We have separate groups of people working on different projects in different areas.  Okay, another question.  You too.  Sorry, would it be possible to run Rust Flight Controller software directly on the drone  without the network round trip?  Yeah, I haven't checked that.  Probably it might be possible to flash some custom firmware on it, but at this point  we're still playing with Wi-Fi.  Question on Twitch.  Do you welcome contributors without much experience?  Yes, we do.  The whole point of the IroRust community, not only a working group, we're a whole community,  is to share knowledge and information on various topics about Iro space and engineering and  whatever space even.  So it's definitely encouraging to see new people coming in and trying to bring them  up to speed or try to help them find their place.  Well, another question.  Maybe I can return the slides that we missed.  YouTube, Boris, Tilan, is there interest in space industry to use Rust?  Yes, yes, there is.  There is quite a big interest right now, especially in the Rust community, as far as I know.  On another conference, on Oxydise, they were talking about how they want to bring the Rust  and to verify the compiler or at least part of it to be used in space, because there are  really tight things that have verification of the tools.  Each tool must be verified that it does what it's meant to be and it's defined somehow.  Question on YouTube.  Michael Lazowski, have you tried to write it in other languages?  For example, C or C++?  No, mainly working with Rust nowadays and I don't even know C and C++ a lot.  However, the main SDK is actually in C, so it might be interesting to take a look at  it.  Okay, so I need to pick somebody with the most interesting question.  I definitely like questions about Rust because the main idea is to share information and  although I don't know anything about drones or space, I can actually go somewhere and  ask somebody and start preparing myself even for a career there.  In that case, I would like to switch another question.  Final question.  I want to have a compiler purimate from Rust before we can verify it.  Targeting LLVM would be hard to verify.  If I remember correctly, we don't need pure Rust compiler to do so.  There are subsets of the C or C++ compilers that are verified as far as I know.  We can do the same thing for Rust, especially given that we now have two backends or frontends  to keep forgetting.  With CraneLift, we can take a subset of Rust and try to verify it bit by bit until we can  start using it in space applications.  And as I said, I do like questions regarding Rust.  So I would say XCcage.  I choose you.  Best question.  Yeah, from Twitch.  So thank you, everybody.  Sorry, I'm a bit nervous talking in front of people.  However, it was really fun and I hope you had some inspirational things that you found  in my talk.  Bye-bye.
Rusty_Days_2020_-_Lachezar_Lechev__Drone_Control_-_e2mJq9qm_Io_0_3467.png Rusty_Days_2020_-_Lachezar_Lechev__Drone_Control_-_e2mJq9qm_Io_3468_3599.png Rusty_Days_2020_-_Lachezar_Lechev__Drone_Control_-_e2mJq9qm_Io_3600_7931.png Rusty_Days_2020_-_Lachezar_Lechev__Drone_Control_-_e2mJq9qm_Io_7932_12539.png Rusty_Days_2020_-_Lachezar_Lechev__Drone_Control_-_e2mJq9qm_Io_12540_12959.png Rusty_Days_2020_-_Lachezar_Lechev__Drone_Control_-_e2mJq9qm_Io_12960_14027.png Rusty_Days_2020_-_Lachezar_Lechev__Drone_Control_-_e2mJq9qm_Io_14028_15659.png Rusty_Days_2020_-_Lachezar_Lechev__Drone_Control_-_e2mJq9qm_Io_15660_16895.png Rusty_Days_2020_-_Lachezar_Lechev__Drone_Control_-_e2mJq9qm_Io_16896_17015.png Rusty_Days_2020_-_Lachezar_Lechev__Drone_Control_-_e2mJq9qm_Io_17016_17999.png Rusty_Days_2020_-_Lachezar_Lechev__Drone_Control_-_e2mJq9qm_Io_18000_19751.png Rusty_Days_2020_-_Lachezar_Lechev__Drone_Control_-_e2mJq9qm_Io_19752_23735.png Rusty_Days_2020_-_Lachezar_Lechev__Drone_Control_-_e2mJq9qm_Io_23736_24407.png Rusty_Days_2020_-_Lachezar_Lechev__Drone_Control_-_e2mJq9qm_Io_24408_24539.png Rusty_Days_2020_-_Lachezar_Lechev__Drone_Control_-_e2mJq9qm_Io_24540_24803.png Rusty_Days_2020_-_Lachezar_Lechev__Drone_Control_-_e2mJq9qm_Io_24804_25079.png Rusty_Days_2020_-_Lachezar_Lechev__Drone_Control_-_e2mJq9qm_Io_25080_25595.png Rusty_Days_2020_-_Lachezar_Lechev__Drone_Control_-_e2mJq9qm_Io_25596_25691.png Rusty_Days_2020_-_Lachezar_Lechev__Drone_Control_-_e2mJq9qm_Io_25692_25931.png Rusty_Days_2020_-_Lachezar_Lechev__Drone_Control_-_e2mJq9qm_Io_25932_36131.png Rusty_Days_2020_-_Lachezar_Lechev__Drone_Control_-_e2mJq9qm_Io_36132_40415.png Rusty_Days_2020_-_Lachezar_Lechev__Drone_Control_-_e2mJq9qm_Io_40416_42215.png Rusty_Days_2020_-_Lachezar_Lechev__Drone_Control_-_e2mJq9qm_Io_42216_45227.png Rusty_Days_2020_-_Lachezar_Lechev__Drone_Control_-_e2mJq9qm_Io_45228_45275.png Rusty_Days_2020_-_Lachezar_Lechev__Drone_Control_-_e2mJq9qm_Io_45276_47375.png Rusty_Days_2020_-_Lachezar_Lechev__Drone_Control_-_e2mJq9qm_Io_47376_47615.png Rusty_Days_2020_-_Lachezar_Lechev__Drone_Control_-_e2mJq9qm_Io_47616_48671.png Rusty_Days_2020_-_Lachezar_Lechev__Drone_Control_-_e2mJq9qm_Io_48672_49943.png Rusty_Days_2020_-_Lachezar_Lechev__Drone_Control_-_e2mJq9qm_Io_49944_51035.png Rusty_Days_2020_-_Lachezar_Lechev__Drone_Control_-_e2mJq9qm_Io_51036_52847.png Rusty_Days_2020_-_Lachezar_Lechev__Drone_Control_-_e2mJq9qm_Io_52848_53171.png Rusty_Days_2020_-_Lachezar_Lechev__Drone_Control_-_e2mJq9qm_Io_53172_53351.png Rusty_Days_2020_-_Lachezar_Lechev__Drone_Control_-_e2mJq9qm_Io_53352_53423.png Rusty_Days_2020_-_Lachezar_Lechev__Drone_Control_-_e2mJq9qm_Io_53424_53819.png Rusty_Days_2020_-_Lachezar_Lechev__Drone_Control_-_e2mJq9qm_Io_53820_53915.png Rusty_Days_2020_-_Lachezar_Lechev__Drone_Control_-_e2mJq9qm_Io_53916_54323.png Rusty_Days_2020_-_Lachezar_Lechev__Drone_Control_-_e2mJq9qm_Io_54324_55103.png Rusty_Days_2020_-_Lachezar_Lechev__Drone_Control_-_e2mJq9qm_Io_55104_55175.png Rusty_Days_2020_-_Lachezar_Lechev__Drone_Control_-_e2mJq9qm_Io_55176_55223.png Rusty_Days_2020_-_Lachezar_Lechev__Drone_Control_-_e2mJq9qm_Io_55224_55331.png Rusty_Days_2020_-_Lachezar_Lechev__Drone_Control_-_e2mJq9qm_Io_55332_56555.png Rusty_Days_2020_-_Lachezar_Lechev__Drone_Control_-_e2mJq9qm_Io_56556_56915.png Rusty_Days_2020_-_Lachezar_Lechev__Drone_Control_-_e2mJq9qm_Io_56916_57155.png Rusty_Days_2020_-_Lachezar_Lechev__Drone_Control_-_e2mJq9qm_Io_57156_57359.png Rusty_Days_2020_-_Lachezar_Lechev__Drone_Control_-_e2mJq9qm_Io_57360_57827.png Rusty_Days_2020_-_Lachezar_Lechev__Drone_Control_-_e2mJq9qm_Io_57828_57971.png Rusty_Days_2020_-_Lachezar_Lechev__Drone_Control_-_e2mJq9qm_Io_57972_58151.png Rusty_Days_2020_-_Lachezar_Lechev__Drone_Control_-_e2mJq9qm_Io_58152_58847.png Rusty_Days_2020_-_Lachezar_Lechev__Drone_Control_-_e2mJq9qm_Io_58848_59291.png Rusty_Days_2020_-_Lachezar_Lechev__Drone_Control_-_e2mJq9qm_Io_59292_59411.png Rusty_Days_2020_-_Lachezar_Lechev__Drone_Control_-_e2mJq9qm_Io_59412_59567.png Rusty_Days_2020_-_Lachezar_Lechev__Drone_Control_-_e2mJq9qm_Io_59568_59807.png Rusty_Days_2020_-_Lachezar_Lechev__Drone_Control_-_e2mJq9qm_Io_59808_59879.png Rusty_Days_2020_-_Lachezar_Lechev__Drone_Control_-_e2mJq9qm_Io_59880_60119.png Rusty_Days_2020_-_Lachezar_Lechev__Drone_Control_-_e2mJq9qm_Io_60120_60203.png Rusty_Days_2020_-_Lachezar_Lechev__Drone_Control_-_e2mJq9qm_Io_60204_60479.png Rusty_Days_2020_-_Lachezar_Lechev__Drone_Control_-_e2mJq9qm_Io_60480_60635.png Rusty_Days_2020_-_Lachezar_Lechev__Drone_Control_-_e2mJq9qm_Io_60636_60863.png Rusty_Days_2020_-_Lachezar_Lechev__Drone_Control_-_e2mJq9qm_Io_60864_61067.png Rusty_Days_2020_-_Lachezar_Lechev__Drone_Control_-_e2mJq9qm_Io_61068_61199.png Rusty_Days_2020_-_Lachezar_Lechev__Drone_Control_-_e2mJq9qm_Io_61200_61283.png Rusty_Days_2020_-_Lachezar_Lechev__Drone_Control_-_e2mJq9qm_Io_61284_61439.png Rusty_Days_2020_-_Lachezar_Lechev__Drone_Control_-_e2mJq9qm_Io_61440_63486.png