Reverse Engineering the Tesla Battery Management System
CC Attribution 3.0 Unported:
You are free to use, adapt and copy, distribute and transmit the work or content in adapted or unchanged form for any legal purpose as long as the work is attributed to the author in the manner specified by the author or licensor.
Kiley, Patrick
Question and answer session for Patrick Kiley - Reverse Engineering the Tesla Battery Management System Q&A
 There we go. All right, welcome back everybody. We are here with another of our great DEF CON speakers. We have Patrick Kiley here and you can check out his presentation on YouTube where he talked about reverse engineering the Tesla battery management system for more power. Welcome Patrick, how are you doing today? I'm great, how are you guys? Doing all right. Pretty good. So, can you just kind of give us a little bit of an overview of who you are, let people hear a little bit about your presentation just in case they haven't seen it or kind of give a little bit of an overview recap? Sure. So, my name is Patrick Kiley. I'm a security consultant for Rapid7, part of the penetration testing team. And I got interested in this project back when I just started looking at how the Tesla kind of worked inside the model specifically. You know, I'd seen how some people had hacked them. And once I got a chance to peek under the hood, I just wanted to learn more and more. I was already into car hacking and learning more about this. And once I realized that there was an upgrade that this car had that nobody had really researched and actually published information about, it kind of gave me a path forward. So I want to see like, okay, how did Tesla actually make this car capable of ludicrous? And it sent me down a path that was probably the most complicated project I've ever worked on. That's awesome. And I believe this is your first time being a main track speaker for Defconn, is that right? That's correct, yes. All right. Well, for anyone that is familiar with Defconn, we have a tradition that you should be very familiar with by now. It's called Shot the New. New speakers are welcomed into the Defconn speaker collective with a shot of a drink of their choice to bring them into the community, celebrate their joining, the people that are giving back to the community knowledge. So thank you. Here's to you. Bottoms up. Cheers. Congrats. Thank you. All right. Good. Loosen you up before the questions start getting serious. Okay. So first off, like I was blown away by the amount of different like techniques and skill sets that you cruise through in your talk. You had hardware reversing, hardware debugging, assembly reversing, a decompiling, compiled Python, diagnosing, obscure third party issues, binary extraction, just like a crazy amount of things. Like, how did you go about like piecing all these things together? Like how did you find your next steps to go through everything? Yeah. So piece by piece. I already had like the mechanical skills, so you know, in the past I've had like a Mustang that I've either replaced the heads on or put a supercharger on or, you know, things along those lines replaced exhaust systems on. So I had the mechanical knowledge already. I've been doing that for longer than I care to admit. I think even when I was a teenager, I was tinkering around with automobiles and taking them apart, taking carburetors apart. So complex small items like that. And then just trying to take things apart and figure out how they work has been something I've been doing since I was a child. My parents told me about how the time I locked them out of the bathroom and then just to disassemble the toilet. And I don't remember that, but they remind me of it on pretty much every occasion when I talked about something I've been working on. So there's that. So really once I started getting into it, you know, I already had the reverse engineering skills from just doing some other stuff and had the CAN bus skills. You know, if you look at some of the previous presentations I've done, I've been poking around CAN bus for some time. So having the DBC files, it was really trivial to just, you know, diagnose the stuff going on within the CAN bus. From there, it was just, okay, how do I do these other things? I know that CAN has this protocol that sits on top of, you know, the thing itself for rain diagnostics called ODX. So I found those called UDS. I found those ODX files and worked with a car fucker who helped show me how to use them. He's one of the main guys of the car hacking village. He showed me how to actually, you know, import those into vehicle spy and play around with them, helped me figure out some of the security access stuff. I had already seen some of that from Craig Smith's work. Nori, he figured out how to pop airbags on cars. So I knew about security access and I understand the algorithm. But Tesla's algorithm, fortunately for this vehicle was incredibly simple. It's a static seeding key. There's no transform required. It's like you request the seed and you reply with a fixed reply and boom, you're in. So that made that part easy and then the rest of it was just piece by piece digging through the Python code and then building a test bench, which for any reverse engineering project when you're doing the very expensive piece of hardware, try and replicate it on a bench because that was invaluable. I never would have attempted half the stuff I did if I hadn't proofed it out using a much cheaper variant of it that I just wired together and figured out how to work. There was one jump in particular that it was right after you had your Tesla towed home, not figuring out what those particular, like what was going on with there. Do you remember what was the final thing that got you past that hump? Oh, I remember it really well. I imagine that there was some stress involved there. So, yeah. So I mean, I was into day two of trying to... Oh. You're having a little video loss. It might be some Wi-Fi interference or something. I said, no, I'm not here for the weekend. I'm over in Irvine. You're here in Rancho Cucamonga. I'm not coming here. You can come on Monday. I'm like, no. I'm going to bail out and figure the rest of this out at home. So I only needed your garage to drop the battery and do that stuff. If I have to come back for that, I'm really in a SOL-type situation. So stressing out that for a day and a half, I just started capturing a bunch of logs. So just from past experience, I was like, maybe the logs are telling me something. And the logs weren't great, but they did mention this one file and it was a firmware.rc. Mentioned that a couple of times. It kind of just had that around the point where it said error and I could see that it failed. So basically, I can stamped. Okay. So when the car said, fuck you, I'm not actually going to upgrade or let you redeploy the software, I kind of noted where that time was and looked right there in the logs and saw an error about that. And then I just did some searching. It turns out the 10 cent guys, when they did their Tesla gateway reversing, they mentioned that file. They mentioned how that file was loaded into memory. I'm like, I've never seen that file before. I've seen it mentioned. I searched through all my stuff, found a couple references to it, and then I just like, well, let's just go to the gateway and see if it'll give it to me and just give you a little background. This is not why I have the car. I had already spent the two days. I was already flustered. I had to buy a last minute ticket home, flew home, and I was messing around on my bench. So I was like, I went to my bench version that had the gateway and said, give me firmware at rc. It gave it to me. I'm like, cool. Okay. You've got this BMS error. I've already replicated that. Look to the values in it, and I actually have a copy of the map here, but there's no real way to share it, but there's basically a tab separated value file. Within that file, it tells you the firmware that that particular battery pack ID needs, and it has a little CRC code. That CRC code needs to be in that firmware.rc file. Once that was updated with the correct CRC code and the BMS I knew was first, it was the version of firmware, and I could also see that CRC code being broadcast by the BMS again. I did that, and then I was like, okay, well, there's this final CRC file. How do I figure that out? I asked some people that I knew, it's like, hey, I don't know a ton about CRC32s, but here's where I have, and here's the file itself with the CRC value. And then someone said, oh, it's a jam CRC. Here's a file. Here's a website you can go to to recreate it. So I recreated the last CRC line, uploaded it back to the gateway, rebooted the gateway, and the error cleared. So I'm like, good, I've got to pass forward. I'm going to go through the car to actually try all this. And the car was basically being an asshole as well, because the battery wasn't engaging, it was just dealing with 12 volts. So a lot of the components, it would just basically shut down after a little while. So I basically had to make sure I had a good battery maintain on it. So it sat there for a few minutes to make sure that the 12 volts had enough juice. And then I tried to get the gateway to go through its process. And as soon as that car woke up and I heard the clicks of the contactors in the rear, it was like, hallelujah. There was one other thing that you said you missed that you would only say over drinks. We did just have a drink. Can you tell us? No? Okay. I was basically messing around with the car and putting on a feature that it didn't really need to have. It was causing the error. And it was just, different times I was screwing around with something I shouldn't have been. So it was my own fault, is really the crux of what I want to say there. Fair. So to catch you in person over drinks, I tried to slip that in. Didn't expect it to really work. That was pretty slight. Nice try, Pastis. Yep. So, go ahead. I've got one question coming in for you. That you would have had to pay Tesla about $5,000 to make this transition. What was the approximate cost for you to do it yourself? Counting the towing or not? Let's say not. Because maybe somebody will learn from your lesson. So the fuse itself, where I found on the user market for about $350, I found the contactors for about $200. The rest of that was just basically my time and effort and labor. So just basically me being my labor. The rest of the stuff didn't really cost me anything other than just time. And then of course there were all the components to my bench. You can buy a Tesla MCU for about $700 to $1,000 on eBay or some of the other markets. And the BMS is running anywhere between $150 and $300. I wanted to make sure I got a BMS with the shunt. That was a little bit harder. I just happened to get lucky and saw one. But the bench parts, I guess you could consider were major portions of that all. And then there is a second kit that I bought just to be safe that was actually the exact part number that Tesla used to do the upgrade. So basically people would pay for this kit and maybe part of their whole upgrade package. I bought one of those as well to make sure there wasn't something I was missing. And that was about another $1,000. Can people just go to their local junkyard and pull pieces themselves to create a test bench? And if so, do you think the junkyards really understand the value of this equipment that you're using? Well, so many people are doing weird things with Teslas. So I'm sure they can. But I'm not sure how many junkyards you may find that actually have Teslas out there. I found the secondary market to be really all. But if you can get one, if you can get to a junkyard and you have one that has an MCU great, getting the BMS would be a pain in the... I don't even want to think about it. You basically have to drop the battery pack out of the vehicle to get at that BMS. There's no way to get at it from any other means. So doing that in a junkyard would be problematic. But if you get that center display and it's not already been snagged, that's the part that you would start with with any test bench because that's where you learn how to route the car. That's where you learn to do all the changes. That's where the gateway is stored. It really is kind of a central hub of the vehicle. Yes. Go ahead, Vist. I was going to say, I keep hearing you mention the gateway. What exactly is the gateway in the car? Can you talk about that? Is it actually like a network router or is this something else? It's called the security gateway. It's a function you'll see on newer CAN bus automobiles. It basically is a device that exists like a firewall, but not sort of like a firewall that exists between multiple CAN buses. And in this place, the infotainment unit, the part that actually connects to the internet. So it acts as an ethernet to CAN bus gateway. So the ethernet side is what connects to the central display, connects to the instrument cluster, and then from the central display, there's like the cellular connection, the Wi-Fi connection, the Bluetooth connection, the USB ports, and then has all the logic for how it communicates out on the internet and the rest of it communicates with the rest of the Tesla mothership. But then the gateway also bridges the various CAN buses. So it can take a message from, say, the powertrain CAN bus and copy it onto either the ethernet CAN bus or the chassis CAN bus. Consequently, it can take messages from those other CAN buses and push them onto the BMS as well. Because sometimes devices that are not on the same bus need to communicate with one another. So when you're requesting the firmware, is it coming directly from the gateway or is it gateway like asking these sub-devices to, hey, send your firmware? So just to clarify your question, are you talking about the firmware and RSC file, or are you talking about the firmware itself? Well, kind of both. I think you did both in your, throughout your talk. There was one particular place where I noticed that you were issuing a command line to retrieve a file and then you made some changes. I think it was even used VIM, which represent, and then you pushed it back. Yeah. So that was all from the CID. So the CID on the vehicle that we're talking about is an NVIDIA Tegra running Ubuntu. And it's basically, it's an ARM version of Ubuntu. All the firmware for all the modules of the vehicle sit within that firmware image. And then when people talk about how their Tesla gets updates, it has all new firmware and it pushes out firmware updates, all the various modules that need it. Some module way over here may not need an update, so it may not get updated. But that's part of what the gateway and the main system do when they do an update. The gateway itself does store a few files, but it doesn't store a copy of the entire vehicle's firmware. The entire vehicle's firmware is stored on the central display, on that Tegra-based Ubuntu system on that little EMMC chip. That's the same EMC that people complain about wearing out because of all the logging that's going on. But the, the, the image itself has that entire copy of the firmware. The gateway only has things like the firmware at RC file, the internal.dat file, and a few others, like there's like hwids.ecq, which is all the hardware IDs of the vehicle, and it gets that from the CAN bus. So the gateway does that, does queries during an upgrade. But there are also some like, like crash files that are stored on the gateway because it's easier for it to store on the gateway. I don't know why it does that, but some of the crash files are stored there as well because I saw on my bench system some images right before the vehicle was crashed. Oh. Okay. So, so when you say crash, it's not like software crashing. It's like, like black box from the actual crash. Yeah. And actually, I think there's another like actual black box device, but the gateway stores quite a bit of that. I'm not going to pretend to understand how the whole, you know, emergency data recorder function of the vehicle works, but it's, it's there. Some of the files are on the gateway. Yeah, that might be an interesting future talk for someone. Yes. Just perversing the black box of Tesla. Yeah. So you also do a lot of stuff with the car hacking village. Is there anything interesting going on that you want to sort of announce while you're here to get people to come and show up in the car hacking village? Anything like that. Your question broke up right at the point where you asked the critical part. So do you mind repeating it? Oh, yeah. Sorry. Sorry. So you're very active in the car hacking village. Is there anything like interesting going on, like maybe Tesla related or related to your talk that you're going to continue working on something that you want to like pimp out? So tomorrow at 10 o'clock, I've got a deep dive into many of the techniques they covered in the main doc. And that's basically because, you know, due to the whole change of the online format, some of the stuff got cut off to the talk. So I have some, some of the stuff that was cut from my, you know, talk itself, but then I also expanded on some additional topics as well. So I go into some of the binary analysis. I load up the former insight IDA. I don't get very far, but I show that I was able to get an IDA. I do some of the like live UDS techniques, you know, the security access as well as the shunt calibration. And then it's just another brief overview. So it's like three different things that I'm kind of talking about. And then, you know, there's some can't be DC stuff as well. Other than like the setting up a bench itself, is there any particular like software that or tools that people are required to do this kind of like investigation and playgrounds work with car hacking? So I was using a not free tool called vehicle spy only because it's very powerful and may go a lot easier. Only how are they available? But you can use tools and some of the other stuff out there pretty much make it possible to do whatever you want to out there. There are a ton of can bus interfaces out there. One of my favorites is the panda. So for this, I'm going to give a shout out to the comment at AI folks for actually making such an awesome can bus interface. Come on.ai is a open source self driving tool that I'll just let people go out and check it out on their own. But they also they make a tool called the panda works really well with the Tesla can connect to three can buses at once. A lot of people that have done hacking on the Tesla use that. And I'm very impressed with it. It's a very inexpensive tool that'll give you physical interface. And from that you can connect over Wi-Fi or USB and from there directly, you know, start messing with Linux tools that are all free. And those are all I can can use. What's cool. What's been Tesla's reaction to the kind of work that you've been doing to get you kind of ludicrous speed? So when this talk was accepted, I actually reached out several times and Tesla was very supportive. They actually just asked to review the slides to make sure they're wearing any surprises. I kind of told them, hey, I'm going to talk about this, this and this. They said, yeah, just do us a favor, send us a copy of your slides, white paper, et cetera. So make sure there are any gotchas in there. And they said, yeah, everything looks good. You're good to go. So very supportive. They actually have a program called the security vehicle research program in addition to their bug mounting. So a vehicle that you're interested in doing security research on can be registered with them and you won't violate any warranties. So basically, and supposedly I haven't tried this, they'll even help you with some breaking situations. I did need their help on this one, fortunately, but they do some of the other stuff. Sorry, excuse me. I've got a really annoying thing going on. There we go. Sorry for anyone that was on the stream. I was just constantly getting calendar alerts for the next things that I'm responsible for. Are there any other things that you're aware of in a Tesla that you could possibly try to unlock or bypass or anything else that either you want to look into or you think that other people might want to look into to build upon your research? I'd have to think about that one for a bit. Yeah, there are some things. I would like to see if where the actual limits are in the BMS, I would like to understand those, but I don't really have a desire to push past them because you're starting to get into dangerous territory at that point. I just want to find where they are. So it's like, all right, here's the variable that actually controls max power. Here's the power curve that's basically defined state of charge, battery temperature and how much power is available for the various drive inverters and the battery itself. I'd like to find those. So that's hence the additional item stuff. And then just more or less actually get more underneath the hood of how the battery management itself and the drive inverters work. The drive inverters are not something that I actually have hardware on a bench for, but because they're very expensive, the whole drive inverter is inside the drive unit and drive unit is a multi-thousand dollar piece of equipment. But yeah, that's where I would like to get from there. I want to see where the steps are for the P90D and the P100D because those cars are faster. So we did get a really good question that was, for those not familiar with the whole gateway can bus stuff, would the car be able to phone home after you've made the ludicrous speed modification? Yeah. So the vehicle within its firmware itself has a function where actually it uploads the config to Tesla on a regular basis. I think it's like vitals.json and puts it in JSON format. So basically they take the vehicles config and whenever I think it's when the car goes to sleep, it actually uploads that information to Tesla. So anytime someone makes a change that wasn't done by someone else or I'm sure even when the service center makes a change, that is going to be seen on Tesla's end unless of course you figure out a way around that. I imagine that you're probably no longer under warranty as soon as you route the front panel or something like that. Magnuson Moss Act affects a lot of stuff. But I'm not a lawyer so I'm not even going to go there and try and figure it out. But it's like if you route the center display and you have a problem with your brakes, how are the two related? Yeah. But if you route your center display and you break your center display, it's a totally different situation. That's all a great legal area that I'm not going to dare dealt with. And to follow up on that question, if Tesla does get made aware of this bypass that you put into place, could they then reverse it and remove it from you? I'm sure they have. There are stories out there about how people had a ludicrous speed vehicle and then after they purchased it, it was removed because Tesla said they audited it and found out that it shouldn't have had it and removed it. And then of course the person who bought the vehicle was pretty pissed off. So don't know what the resolution on that was. But that's also, there's a huge amount of controversy around the whole super charging thing. So if you buy a salvage Tesla, it has super charging disabled. Tesla does that. It's their prerogative, it's their supercharger network, but it makes it very difficult to go on trips. And people figured out using the same techniques that I'm talking about here on how to re-enable that again. And then there was a thing that came out recently that basically says, you're opening yourselves up to getting sued by us because you're basically getting something that you shouldn't be. My only suggestion for Tesla is why not verify the safety of it and then only enable it to where they have to pay for it? Because now you've got a source of revenue. That was going to be my next question because I know that after a certain point they stopped allowing superchargers to be free for Teslas. But it sounds like it's completely disabled on these. On the salvage vehicles, it's disabled here. The way I understand it is if the vehicle has free supercharging and has free supercharging. As soon as the vehicle is sold back to Tesla, it's lost. This particular vehicle has free supercharging because it was purchased with free supercharging. So it should always have that. But as I understand it, they haven't taken it away from anymore. They said, this vehicle will have free supercharging for life unless it has been in an accident and been totaled out by an insurance company. So it wasn't just like Tesla purchasing it back and then reselling it with it disabled or something like that. It's like people trade in their cars. People trade in their Teslas. I think when they sell them out again, they don't have free supercharging. I don't know. That makes sense. I don't keep track of them. I didn't know. It seems like a really sketchy area. So any kind of details that I can pull out of you. Their frog is. Yeah. So it seems as though with the where electric cars are going now that this is going to be a really great area of research. And what kind of advice would you give to people that want to get started with car hacking research? Like if somebody wants to start from the ground up, what sorts of things could they do? Where should they start looking? Start learning Canvas. That seems to be a real lot of it. Gets from there learn UDS and from there learn about like binary reversing and just reversing engineering in general. Go to a junkyard. Find a car that you find interesting. Verify. And start ripping modules out. We did something with our company called a junkyard hackathon where a bunch of us went out and just ripped apart various various vehicles. Craig Smith helped us. He's the writer of the car hackers handbook. And we got like several vehicles worth of modules. Even if you just get one module, take it apart. Figure out what CPU it has. Learn how to hardware hack it. And from there have fun. Yes, I thought I've seen in some of the videos with people hacking on electric cars. Maybe I'm remembering this wrong, but is there any risk to cars sitting in a junkyard still holding any kind of electrical charge that people should be aware of? Yeah, because high voltage is high voltage. It'll hurt you either way. I personally have never been to a junkyard where I've seen electric cars. I've only seen the salvage yards where they're like the warranty auction places and you see those online. But the actual junkyards themselves, I don't think I've ever seen an electric vehicle in a junkyard. Even a hybrid would be dangerous though because the hybrid still has high voltage because it has an electric motor. We got one question. It's kind of more like, Hawkeye is wondering if Tesla's have a problem with shutting down if their error message is reach a critical amount of storage space. I'm guessing that has to do with the EMC login thing. I've never seen one run out of storage space. It's just the volume of logging. I believe at a certain point it overwrites itself, but it's the volume of constantly writing to the EMMC that wears it down, not for the device storage actually filling up. It's unfortunate because it's like a $20 part. The EMMC module itself on that dagger is a very inexpensive part, but it's a $2,000 repair if you take it to a service center. As we start wrapping this up, what sorts of takeaways do you want people to have from your presentation and what would be your call to action or what would you like to see come about based on your research and your presentation that you put out there? I'd love for Tesla owners to be able to work on their own vehicles. There to be a consumer version of the tools that are used to work on and diagnose the vehicles because right now you just have to take it to the garage and pay the service center to do that. I can understand why you want to do that, but as these cars age and continue to be out there in the fleet, there needs to be another method. In addition to that, this is the next version of hot rodding. People are going to figure this out. Now that electric cars are becoming more and more mainstream, there are going to be people that want to buy them and make them faster. You can make them faster by either lightening them or you can make them faster by tweaking what's under the hood just like you would on a standard internal combustion engine. This is the next phase where that's going to go. Awesome. Thank you so much for doing this. Thanks so much for your presentation that people can go see on the YouTube with the Defcon channel. Thanks for doing this, Patrick. Really enjoyed you discussing this with us. Thank you.
 There we go.  All right, welcome back everybody.  We are here with another of our great DEF CON speakers.  We have Patrick Kiley here and you can check out his presentation on YouTube where he talked  about reverse engineering the Tesla battery management system for more power.  Welcome Patrick, how are you doing today?  I'm great, how are you guys?  Doing all right.  Pretty good.  So, can you just kind of give us a little bit of an overview of who you are, let people  hear a little bit about your presentation just in case they haven't seen it or kind of give  a little bit of an overview recap?  Sure.  So, my name is Patrick Kiley.  I'm a security consultant for Rapid7, part of the penetration testing team.  And I got interested in this project back when I just started looking at how the Tesla  kind of worked inside the model specifically.  You know, I'd seen how some people had hacked them.  And once I got a chance to peek under the hood, I just wanted to learn more and more.  I was already into car hacking and learning more about this.  And once I realized that there was an upgrade that this car had that nobody had really researched  and actually published information about, it kind of gave me a path forward.  So I want to see like, okay, how did Tesla actually make this car capable of ludicrous?  And it sent me down a path that was probably the most complicated project I've ever worked  on.  That's awesome.  And I believe this is your first time being a main track speaker for Defconn, is that right?  That's correct, yes.  All right.  Well, for anyone that is familiar with Defconn, we have a tradition that you should be very  familiar with by now.  It's called Shot the New.  New speakers are welcomed into the Defconn speaker collective with a shot of a drink  of their choice to bring them into the community, celebrate their joining, the people that are  giving back to the community knowledge.  So thank you.  Here's to you.  Bottoms up.  Cheers.  Congrats.  Thank you.  All right.  Good.  Loosen you up before the questions start getting serious.  Okay.  So first off, like I was blown away by the amount of different like techniques and skill  sets that you cruise through in your talk.  You had hardware reversing, hardware debugging, assembly reversing, a decompiling, compiled  Python, diagnosing, obscure third party issues, binary extraction, just like a crazy amount  of things.  Like, how did you go about like piecing all these things together?  Like how did you find your next steps to go through everything?  Yeah.  So piece by piece.  I already had like the mechanical skills, so you know, in the past I've had like a  Mustang that I've either replaced the heads on or put a supercharger on or, you know,  things along those lines replaced exhaust systems on.  So I had the mechanical knowledge already.  I've been doing that for longer than I care to admit.  I think even when I was a teenager, I was tinkering around with automobiles and taking  them apart, taking carburetors apart.  So complex small items like that.  And then just trying to take things apart and figure out how they work has been something  I've been doing since I was a child.  My parents told me about how the time I locked them out of the bathroom and then just to  disassemble the toilet.  And I don't remember that, but they remind me of it on pretty much every occasion when  I talked about something I've been working on.  So there's that.  So really once I started getting into it, you know, I already had the reverse engineering  skills from just doing some other stuff and had the CAN bus skills.  You know, if you look at some of the previous presentations I've done, I've been poking around  CAN bus for some time.  So having the DBC files, it was really trivial to just, you know, diagnose the stuff going  on within the CAN bus.  From there, it was just, okay, how do I do these other things?  I know that CAN has this protocol that sits on top of, you know, the thing itself for  rain diagnostics called ODX.  So I found those called UDS.  I found those ODX files and worked with a car fucker who helped show me how to use them.  He's one of the main guys of the car hacking village.  He showed me how to actually, you know, import those into vehicle spy and play around with  them, helped me figure out some of the security access stuff.  I had already seen some of that from Craig Smith's work.  Nori, he figured out how to pop airbags on cars.  So I knew about security access and I understand the algorithm.  But Tesla's algorithm, fortunately for this vehicle was incredibly simple.  It's a static seeding key.  There's no transform required.  It's like you request the seed and you reply with a fixed reply and boom, you're in.  So that made that part easy and then the rest of it was just piece by piece digging through  the Python code and then building a test bench, which for any reverse engineering project  when you're doing the very expensive piece of hardware, try and replicate it on a bench  because that was invaluable.  I never would have attempted half the stuff I did if I hadn't proofed it out using a much  cheaper variant of it that I just wired together and figured out how to work.  There was one jump in particular that it was right after you had your Tesla towed home,  not figuring out what those particular, like what was going on with there.  Do you remember what was the final thing that got you past that hump?  Oh, I remember it really well.  I imagine that there was some stress involved there.  So, yeah.  So I mean, I was into day two of trying to...  Oh.  You're having a little video loss.  It might be some Wi-Fi interference or something.  I said, no, I'm not here for the weekend.  I'm over in Irvine.  You're here in Rancho Cucamonga.  I'm not coming here.  You can come on Monday.  I'm like, no.  I'm going to bail out and figure the rest of this out at home.  So I only needed your garage to drop the battery and do that stuff.  If I have to come back for that, I'm really in a SOL-type situation.  So stressing out that for a day and a half, I just started capturing a bunch of logs.  So just from past experience, I was like, maybe the logs are telling me something.  And the logs weren't great, but they did mention this one file and it was a firmware.rc.  Mentioned that a couple of times.  It kind of just had that around the point where it said error and I could see that it  failed.  So basically, I can stamped.  Okay.  So when the car said, fuck you, I'm not actually going to upgrade or let you redeploy the software,  I kind of noted where that time was and looked right there in the logs and saw an error about  that.  And then I just did some searching.  It turns out the 10 cent guys, when they did their Tesla gateway reversing, they mentioned  that file.  They mentioned how that file was loaded into memory.  I'm like, I've never seen that file before.  I've seen it mentioned.  I searched through all my stuff, found a couple references to it, and then I just like, well,  let's just go to the gateway and see if it'll give it to me and just give you a little background.  This is not why I have the car.  I had already spent the two days.  I was already flustered.  I had to buy a last minute ticket home, flew home, and I was messing around on my bench.  So I was like, I went to my bench version that had the gateway and said, give me firmware  at rc.  It gave it to me.  I'm like, cool.  Okay.  You've got this BMS error.  I've already replicated that.  Look to the values in it, and I actually have a copy of the map here, but there's no real  way to share it, but there's basically a tab separated value file.  Within that file, it tells you the firmware that that particular battery pack ID needs,  and it has a little CRC code.  That CRC code needs to be in that firmware.rc file.  Once that was updated with the correct CRC code and the BMS I knew was first, it was  the version of firmware, and I could also see that CRC code being broadcast by the BMS  again.  I did that, and then I was like, okay, well, there's this final CRC file.  How do I figure that out?  I asked some people that I knew, it's like, hey, I don't know a ton about CRC32s, but  here's where I have, and here's the file itself with the CRC value.  And then someone said, oh, it's a jam CRC.  Here's a file.  Here's a website you can go to to recreate it.  So I recreated the last CRC line, uploaded it back to the gateway, rebooted the gateway,  and the error cleared.  So I'm like, good, I've got to pass forward.  I'm going to go through the car to actually try all this.  And the car was basically being an asshole as well, because the battery wasn't engaging,  it was just dealing with 12 volts.  So a lot of the components, it would just basically shut down after a little while.  So I basically had to make sure I had a good battery maintain on it.  So it sat there for a few minutes to make sure that the 12 volts had enough juice.  And then I tried to get the gateway to go through its process.  And as soon as that car woke up and I heard the clicks of the contactors in the rear,  it was like, hallelujah.  There was one other thing that you said you missed that you would only say over drinks.  We did just have a drink.  Can you tell us?  No?  Okay.  I was basically messing around with the car and putting on a feature that it didn't really  need to have.  It was causing the error.  And it was just, different times I was screwing around with something I shouldn't have been.  So it was my own fault, is really the crux of what I want to say there.  Fair.  So to catch you in person over drinks, I tried to slip that in.  Didn't expect it to really work.  That was pretty slight.  Nice try, Pastis.  Yep.  So, go ahead.  I've got one question coming in for you.  That you would have had to pay Tesla about $5,000 to make this transition.  What was the approximate cost for you to do it yourself?  Counting the towing or not?  Let's say not.  Because maybe somebody will learn from your lesson.  So the fuse itself, where I found on the user market for about $350, I found the contactors  for about $200.  The rest of that was just basically my time and effort and labor.  So just basically me being my labor.  The rest of the stuff didn't really cost me anything other than just time.  And then of course there were all the components to my bench.  You can buy a Tesla MCU for about $700 to $1,000 on eBay or some of the other markets.  And the BMS is running anywhere between $150 and $300.  I wanted to make sure I got a BMS with the shunt.  That was a little bit harder.  I just happened to get lucky and saw one.  But the bench parts, I guess you could consider were major portions of that all.  And then there is a second kit that I bought just to be safe that was actually the exact  part number that Tesla used to do the upgrade.  So basically people would pay for this kit and maybe part of their whole upgrade package.  I bought one of those as well to make sure there wasn't something I was missing.  And that was about another $1,000.  Can people just go to their local junkyard and pull pieces themselves to create a test  bench?  And if so, do you think the junkyards really understand the value of this equipment that  you're using?  Well, so many people are doing weird things with Teslas.  So I'm sure they can.  But I'm not sure how many junkyards you may find that actually have Teslas out there.  I found the secondary market to be really all.  But if you can get one, if you can get to a junkyard and you have one that has an MCU  great, getting the BMS would be a pain in the...  I don't even want to think about it.  You basically have to drop the battery pack out of the vehicle to get at that BMS.  There's no way to get at it from any other means.  So doing that in a junkyard would be problematic.  But if you get that center display and it's not already been snagged, that's the part  that you would start with with any test bench because that's where you learn how to route  the car.  That's where you learn to do all the changes.  That's where the gateway is stored.  It really is kind of a central hub of the vehicle.  Yes.  Go ahead, Vist.  I was going to say, I keep hearing you mention the gateway.  What exactly is the gateway in the car?  Can you talk about that?  Is it actually like a network router or is this something else?  It's called the security gateway.  It's a function you'll see on newer CAN bus automobiles.  It basically is a device that exists like a firewall, but not sort of like a firewall  that exists between multiple CAN buses.  And in this place, the infotainment unit, the part that actually connects to the internet.  So it acts as an ethernet to CAN bus gateway.  So the ethernet side is what connects to the central display, connects to the instrument  cluster, and then from the central display, there's like the cellular connection, the  Wi-Fi connection, the Bluetooth connection, the USB ports, and then has all the logic  for how it communicates out on the internet and the rest of it communicates with the rest  of the Tesla mothership.  But then the gateway also bridges the various CAN buses.  So it can take a message from, say, the powertrain CAN bus and copy it onto either the ethernet  CAN bus or the chassis CAN bus.  Consequently, it can take messages from those other CAN buses and push them onto the BMS  as well.  Because sometimes devices that are not on the same bus need to communicate with one another.  So when you're requesting the firmware, is it coming directly from the gateway or is  it gateway like asking these sub-devices to, hey, send your firmware?  So just to clarify your question, are you talking about the firmware and RSC file, or  are you talking about the firmware itself?  Well, kind of both.  I think you did both in your, throughout your talk.  There was one particular place where I noticed that you were issuing a command line to retrieve  a file and then you made some changes.  I think it was even used VIM, which represent, and then you pushed it back.  Yeah.  So that was all from the CID.  So the CID on the vehicle that we're talking about is an NVIDIA Tegra running Ubuntu.  And it's basically, it's an ARM version of Ubuntu.  All the firmware for all the modules of the vehicle sit within that firmware image.  And then when people talk about how their Tesla gets updates, it has all new firmware  and it pushes out firmware updates, all the various modules that need it.  Some module way over here may not need an update, so it may not get updated.  But that's part of what the gateway and the main system do when they do an update.  The gateway itself does store a few files, but it doesn't store a copy of the entire  vehicle's firmware.  The entire vehicle's firmware is stored on the central display, on that Tegra-based Ubuntu  system on that little EMMC chip.  That's the same EMC that people complain about wearing out because of all the logging that's  going on.  But the, the, the image itself has that entire copy of the firmware.  The gateway only has things like the firmware at RC file, the internal.dat file, and a few  others, like there's like hwids.ecq, which is all the hardware IDs of the vehicle, and  it gets that from the CAN bus.  So the gateway does that, does queries during an upgrade.  But there are also some like, like crash files that are stored on the gateway because it's  easier for it to store on the gateway.  I don't know why it does that, but some of the crash files are stored there as well because  I saw on my bench system some images right before the vehicle was crashed.  Oh.  Okay.  So, so when you say crash, it's not like software crashing.  It's like, like black box from the actual crash.  Yeah.  And actually, I think there's another like actual black box device, but the gateway stores  quite a bit of that.  I'm not going to pretend to understand how the whole, you know, emergency data recorder  function of the vehicle works, but it's, it's there.  Some of the files are on the gateway.  Yeah, that might be an interesting future talk for someone.  Yes.  Just perversing the black box of Tesla.  Yeah.  So you also do a lot of stuff with the car hacking village.  Is there anything interesting going on that you want to sort of announce while you're  here to get people to come and show up in the car hacking village?  Anything like that.  Your question broke up right at the point where you asked the critical part.  So do you mind repeating it?  Oh, yeah.  Sorry.  Sorry.  So you're very active in the car hacking village.  Is there anything like interesting going on, like maybe Tesla related or related to your  talk that you're going to continue working on something that you want to like pimp out?  So tomorrow at 10 o'clock, I've got a deep dive into many of the techniques they covered  in the main doc.  And that's basically because, you know, due to the whole change of the online format,  some of the stuff got cut off to the talk.  So I have some, some of the stuff that was cut from my, you know, talk itself, but then  I also expanded on some additional topics as well.  So I go into some of the binary analysis.  I load up the former insight IDA.  I don't get very far, but I show that I was able to get an IDA.  I do some of the like live UDS techniques, you know, the security access as well as the  shunt calibration.  And then it's just another brief overview.  So it's like three different things that I'm kind of talking about.  And then, you know, there's some can't be DC stuff as well.  Other than like the setting up a bench itself, is there any particular like software that  or tools that people are required to do this kind of like investigation and playgrounds  work with car hacking?  So I was using a not free tool called vehicle spy only because it's very powerful and may  go a lot easier.  Only how are they available?  But you can use tools and some of the other stuff out there pretty much make it possible  to do whatever you want to out there.  There are a ton of can bus interfaces out there.  One of my favorites is the panda.  So for this, I'm going to give a shout out to the comment at AI folks for actually making  such an awesome can bus interface.  Come on.ai is a open source self driving tool that I'll just let people go out and check  it out on their own.  But they also they make a tool called the panda works really well with the Tesla can  connect to three can buses at once.  A lot of people that have done hacking on the Tesla use that.  And I'm very impressed with it.  It's a very inexpensive tool that'll give you physical interface.  And from that you can connect over Wi-Fi or USB and from there directly, you know, start  messing with Linux tools that are all free.  And those are all I can can use.  What's cool.  What's been Tesla's reaction to the kind of work that you've been doing to get you kind  of ludicrous speed?  So when this talk was accepted, I actually reached out several times and Tesla was very  supportive.  They actually just asked to review the slides to make sure they're wearing any surprises.  I kind of told them, hey, I'm going to talk about this, this and this.  They said, yeah, just do us a favor, send us a copy of your slides, white paper, et cetera.  So make sure there are any gotchas in there.  And they said, yeah, everything looks good.  You're good to go.  So very supportive.  They actually have a program called the security vehicle research program in addition to their  bug mounting.  So a vehicle that you're interested in doing security research on can be registered with  them and you won't violate any warranties.  So basically, and supposedly I haven't tried this, they'll even help you with some breaking  situations.  I did need their help on this one, fortunately, but they do some of the other stuff.  Sorry, excuse me.  I've got a really annoying thing going on.  There we go.  Sorry for anyone that was on the stream.  I was just constantly getting calendar alerts for the next things that I'm responsible for.  Are there any other things that you're aware of in a Tesla that you could possibly try  to unlock or bypass or anything else that either you want to look into or you think  that other people might want to look into to build upon your research?  I'd have to think about that one for a bit.  Yeah, there are some things.  I would like to see if where the actual limits are in the BMS, I would like to understand  those, but I don't really have a desire to push past them because you're starting to  get into dangerous territory at that point.  I just want to find where they are.  So it's like, all right, here's the variable that actually controls max power.  Here's the power curve that's basically defined state of charge, battery temperature and how  much power is available for the various drive inverters and the battery itself.  I'd like to find those.  So that's hence the additional item stuff.  And then just more or less actually get more underneath the hood of how the battery management  itself and the drive inverters work.  The drive inverters are not something that I actually have hardware on a bench for, but  because they're very expensive, the whole drive inverter is inside the drive unit and  drive unit is a multi-thousand dollar piece of equipment.  But yeah, that's where I would like to get from there.  I want to see where the steps are for the P90D and the P100D because those cars are faster.  So we did get a really good question that was, for those not familiar with the whole  gateway can bus stuff, would the car be able to phone home after you've made the ludicrous  speed modification?  Yeah.  So the vehicle within its firmware itself has a function where actually it uploads the  config to Tesla on a regular basis.  I think it's like vitals.json and puts it in JSON format.  So basically they take the vehicles config and whenever I think it's when the car goes  to sleep, it actually uploads that information to Tesla.  So anytime someone makes a change that wasn't done by someone else or I'm sure even when  the service center makes a change, that is going to be seen on Tesla's end unless of  course you figure out a way around that.  I imagine that you're probably no longer under warranty as soon as you route the front  panel or something like that.  Magnuson Moss Act affects a lot of stuff.  But I'm not a lawyer so I'm not even going to go there and try and figure it out.  But it's like if you route the center display and you have a problem with your brakes, how  are the two related?  Yeah.  But if you route your center display and you break your center display, it's a totally  different situation.  That's all a great legal area that I'm not going to dare dealt with.  And to follow up on that question, if Tesla does get made aware of this bypass that you  put into place, could they then reverse it and remove it from you?  I'm sure they have.  There are stories out there about how people had a ludicrous speed vehicle and then after  they purchased it, it was removed because Tesla said they audited it and found out that  it shouldn't have had it and removed it.  And then of course the person who bought the vehicle was pretty pissed off.  So don't know what the resolution on that was.  But that's also, there's a huge amount of controversy around the whole super charging  thing.  So if you buy a salvage Tesla, it has super charging disabled.  Tesla does that.  It's their prerogative, it's their supercharger network, but it makes it very difficult to  go on trips.  And people figured out using the same techniques that I'm talking about here on how to re-enable  that again.  And then there was a thing that came out recently that basically says, you're opening yourselves  up to getting sued by us because you're basically getting something that you shouldn't be.  My only suggestion for Tesla is why not verify the safety of it and then only enable it to  where they have to pay for it?  Because now you've got a source of revenue.  That was going to be my next question because I know that after a certain point they stopped  allowing superchargers to be free for Teslas.  But it sounds like it's completely disabled on these.  On the salvage vehicles, it's disabled here.  The way I understand it is if the vehicle has free supercharging and has free supercharging.  As soon as the vehicle is sold back to Tesla, it's lost.  This particular vehicle has free supercharging because it was purchased with free supercharging.  So it should always have that.  But as I understand it, they haven't taken it away from anymore.  They said, this vehicle will have free supercharging for life unless it has been in an accident  and been totaled out by an insurance company.  So it wasn't just like Tesla purchasing it back and then reselling it with it disabled  or something like that.  It's like people trade in their cars.  People trade in their Teslas.  I think when they sell them out again, they don't have free supercharging.  I don't know.  That makes sense.  I don't keep track of them.  I didn't know.  It seems like a really sketchy area.  So any kind of details that I can pull out of you.  Their frog is.  Yeah.  So it seems as though with the where electric cars are going now that this is going to be  a really great area of research.  And what kind of advice would you give to people that want to get started with car hacking  research?  Like if somebody wants to start from the ground up, what sorts of things could they do?  Where should they start looking?  Start learning Canvas.  That seems to be a real lot of it.  Gets from there learn UDS and from there learn about like binary reversing and just  reversing engineering in general.  Go to a junkyard.  Find a car that you find interesting.  Verify.  And start ripping modules out.  We did something with our company called a junkyard hackathon where a bunch of us went  out and just ripped apart various various vehicles.  Craig Smith helped us.  He's the writer of the car hackers handbook.  And we got like several vehicles worth of modules.  Even if you just get one module, take it apart.  Figure out what CPU it has.  Learn how to hardware hack it.  And from there have fun.  Yes, I thought I've seen in some of the videos with people hacking on electric cars.  Maybe I'm remembering this wrong, but is there any risk to cars sitting in a junkyard still  holding any kind of electrical charge that people should be aware of?  Yeah, because high voltage is high voltage.  It'll hurt you either way.  I personally have never been to a junkyard where I've seen electric cars.  I've only seen the salvage yards where they're like the warranty auction places and you see  those online.  But the actual junkyards themselves, I don't think I've ever seen an electric vehicle in  a junkyard.  Even a hybrid would be dangerous though because the hybrid still has high voltage because  it has an electric motor.  We got one question.  It's kind of more like, Hawkeye is wondering if Tesla's have a problem with shutting down  if their error message is reach a critical amount of storage space.  I'm guessing that has to do with the EMC login thing.  I've never seen one run out of storage space.  It's just the volume of logging.  I believe at a certain point it overwrites itself, but it's the volume of constantly  writing to the EMMC that wears it down, not for the device storage actually filling up.  It's unfortunate because it's like a $20 part.  The EMMC module itself on that dagger is a very inexpensive part, but it's a $2,000 repair  if you take it to a service center.  As we start wrapping this up, what sorts of takeaways do you want people to have from  your presentation and what would be your call to action or what would you like to see come  about based on your research and your presentation that you put out there?  I'd love for Tesla owners to be able to work on their own vehicles.  There to be a consumer version of the tools that are used to work on and diagnose the  vehicles because right now you just have to take it to the garage and pay the service  center to do that.  I can understand why you want to do that, but as these cars age and continue to be out  there in the fleet, there needs to be another method.  In addition to that, this is the next version of hot rodding.  People are going to figure this out.  Now that electric cars are becoming more and more mainstream, there are going to be people  that want to buy them and make them faster.  You can make them faster by either lightening them or you can make them faster by tweaking  what's under the hood just like you would on a standard internal combustion engine.  This is the next phase where that's going to go.  Awesome.  Thank you so much for doing this.  Thanks so much for your presentation that people can go see on the YouTube with the  Defcon channel.  Thanks for doing this, Patrick.  Really enjoyed you discussing this with us.  Thank you.
DEF_CON_Safe_Mode_-_Patrick_Kiley_-_Reverse_Engine_dAft0m4oL_Q_0_1283.png DEF_CON_Safe_Mode_-_Patrick_Kiley_-_Reverse_Engine_dAft0m4oL_Q_1284_1343.png DEF_CON_Safe_Mode_-_Patrick_Kiley_-_Reverse_Engine_dAft0m4oL_Q_1344_3167.png DEF_CON_Safe_Mode_-_Patrick_Kiley_-_Reverse_Engine_dAft0m4oL_Q_3168_3395.png DEF_CON_Safe_Mode_-_Patrick_Kiley_-_Reverse_Engine_dAft0m4oL_Q_3396_3455.png DEF_CON_Safe_Mode_-_Patrick_Kiley_-_Reverse_Engine_dAft0m4oL_Q_3456_3503.png DEF_CON_Safe_Mode_-_Patrick_Kiley_-_Reverse_Engine_dAft0m4oL_Q_3504_3563.png DEF_CON_Safe_Mode_-_Patrick_Kiley_-_Reverse_Engine_dAft0m4oL_Q_3564_6827.png DEF_CON_Safe_Mode_-_Patrick_Kiley_-_Reverse_Engine_dAft0m4oL_Q_6828_11723.png DEF_CON_Safe_Mode_-_Patrick_Kiley_-_Reverse_Engine_dAft0m4oL_Q_11724_12755.png DEF_CON_Safe_Mode_-_Patrick_Kiley_-_Reverse_Engine_dAft0m4oL_Q_12756_12863.png DEF_CON_Safe_Mode_-_Patrick_Kiley_-_Reverse_Engine_dAft0m4oL_Q_12864_13067.png DEF_CON_Safe_Mode_-_Patrick_Kiley_-_Reverse_Engine_dAft0m4oL_Q_13068_13799.png DEF_CON_Safe_Mode_-_Patrick_Kiley_-_Reverse_Engine_dAft0m4oL_Q_13800_14003.png DEF_CON_Safe_Mode_-_Patrick_Kiley_-_Reverse_Engine_dAft0m4oL_Q_14004_14171.png DEF_CON_Safe_Mode_-_Patrick_Kiley_-_Reverse_Engine_dAft0m4oL_Q_14172_14243.png DEF_CON_Safe_Mode_-_Patrick_Kiley_-_Reverse_Engine_dAft0m4oL_Q_14244_14339.png DEF_CON_Safe_Mode_-_Patrick_Kiley_-_Reverse_Engine_dAft0m4oL_Q_14340_14387.png DEF_CON_Safe_Mode_-_Patrick_Kiley_-_Reverse_Engine_dAft0m4oL_Q_14388_14519.png DEF_CON_Safe_Mode_-_Patrick_Kiley_-_Reverse_Engine_dAft0m4oL_Q_14520_14807.png DEF_CON_Safe_Mode_-_Patrick_Kiley_-_Reverse_Engine_dAft0m4oL_Q_14808_14891.png DEF_CON_Safe_Mode_-_Patrick_Kiley_-_Reverse_Engine_dAft0m4oL_Q_14892_15503.png DEF_CON_Safe_Mode_-_Patrick_Kiley_-_Reverse_Engine_dAft0m4oL_Q_15504_19007.png DEF_CON_Safe_Mode_-_Patrick_Kiley_-_Reverse_Engine_dAft0m4oL_Q_19008_19799.png DEF_CON_Safe_Mode_-_Patrick_Kiley_-_Reverse_Engine_dAft0m4oL_Q_19800_30539.png DEF_CON_Safe_Mode_-_Patrick_Kiley_-_Reverse_Engine_dAft0m4oL_Q_30540_32807.png DEF_CON_Safe_Mode_-_Patrick_Kiley_-_Reverse_Engine_dAft0m4oL_Q_32808_32855.png DEF_CON_Safe_Mode_-_Patrick_Kiley_-_Reverse_Engine_dAft0m4oL_Q_32856_33563.png DEF_CON_Safe_Mode_-_Patrick_Kiley_-_Reverse_Engine_dAft0m4oL_Q_33564_34079.png DEF_CON_Safe_Mode_-_Patrick_Kiley_-_Reverse_Engine_dAft0m4oL_Q_34080_34583.png DEF_CON_Safe_Mode_-_Patrick_Kiley_-_Reverse_Engine_dAft0m4oL_Q_34584_35843.png DEF_CON_Safe_Mode_-_Patrick_Kiley_-_Reverse_Engine_dAft0m4oL_Q_35844_38051.png DEF_CON_Safe_Mode_-_Patrick_Kiley_-_Reverse_Engine_dAft0m4oL_Q_38052_39575.png DEF_CON_Safe_Mode_-_Patrick_Kiley_-_Reverse_Engine_dAft0m4oL_Q_39576_40751.png DEF_CON_Safe_Mode_-_Patrick_Kiley_-_Reverse_Engine_dAft0m4oL_Q_40752_45839.png DEF_CON_Safe_Mode_-_Patrick_Kiley_-_Reverse_Engine_dAft0m4oL_Q_45840_45886.png