IoT Village - Privacy leaks in smart devices: Extracting data from used smart home devices
CC Attribution 3.0 Unported:
You are free to use, adapt and copy, distribute and transmit the work or content in adapted or unchanged form for any legal purpose as long as the work is attributed to the author in the manner specified by the author or licensor.
Giese, Dennis
Remember the good old fun sport, where people bought random hard drives from eBay and did forensics on them? Did you know you can do the same thing with used IoT devices too? Most end-users have no idea what kind of information their devices are storing and how to securely clean their devices (if that even is possible). Lets explore together what the risks are and how we can extract that data. Many IoT devices collect a lot of data and log files. Of course, most of this data is sent to the Cloud. However, often this data is also stored locally on the device and never deleted in the lifetime of those devices, not even on a factory reset (in contrast to Smart Phones nowadays). This might surprise many people, and especially end users might not be aware of that. Due to the design of IoT devices, there is usually no real way, like for notebooks or PCs, for end users to clean the devices before they sell them on eBay or discard them. The devices may hold sensitive information like Wi-Fi credentials, nearby access points, cloud communication log files, maps, or audio samples. In this talk I will show some examples of interesting IoT devices from various vendors and how to extract the corresponding information. We will use software methods (rooting) and hardware methods (flash dumping). Using this information, I will show how I am able to find the original owner of the device. Also I discuss various challenges and tricks of the methods, and how to prevent this kind of data leakage for yourself.
 Thank you everyone for coming. This is our penultimate talk of the day. This is Dennis. He's going to be doing a talk on, you know, privacy leaks by dumpster diving, more or less, correct? Yeah, so if you ever get, you know, cool technology at eBay or the Goodwill Store or in the dumpster, maybe there's some stuff that got left over even after like a factory reset. So he's gonna get into all that today. I think it's gonna be pretty cool. Can everyone in the back hear me okay? If I talked like this, would you really hear me okay? Okay, so you have to talk as loud as that. Because he's gonna start talking and he's gonna be like, can you hear me? It's pretty good talk, right guys? See, negative thumbs. So you gotta get real close. All right, I try. Can you hear me in the back? Yes, maybe. Should I speak louder? Louder? Louder enough? What about now? Yeah, okay, great. All right, so I'll welcome to my talk about privacy leaks in smart devices. The introduction was mostly correct in terms of like dumpster diving. What I do more is like a shop on Amazon warehouse deals or like eBay. And the main idea of the talk is about to speak about the extraction of data of used smart home devices. So the outline for the talk is the following. I start with the motivation when I will speak about how data is stored or what kind of data is stored on IoT devices. Then how is it stored and what kind of reset states you can imagine if you get like a used device. What kind of data is maybe there or won't be there. In the next topic I will talk about the data extraction methods. So how you get the data off the device and later on I will have some example devices where I will tell you how I got the data off and how I found for example like the previous owner of that. So some information about me. I'm a PhD student at Norfison University and I'm working there with Professor Givrauner Bier. So we're doing mostly wireless security, but I'm kind of into this IoT stuff. I'm also right now a grad student at the TU Darmstadt in Germany and working there with Mathias Hadek. And my main interest is like reverse engineering of any kind of interesting devices can be nearly everything, but at the moment it's more like IoT and smart logs, but I do also physical logs, which is kind of like the same thing and so you find me also in the tool of lock picking. Some side notes about my talk. It's not a Xiaomi bashing talk. So maybe some of you know me from last year where I was giving a lot of talks about Xiaomi. The problem is I have only a lot of this devices, so basically I need to talk about them. Most of the methods I present are already known and used for other things. There are some ethical and legal questions in this presentation, so I had to censor most of the data and like mark it up a little bit. If you use any of the methods, for example, if you want to do like BGA on soldering and you break it, then it's kind of on you. I'm not responsible for that. And a lot of technical aspects I have totally simplified. So basically, NAND flash is way more complex, but I did it for the sake of the talk to simplify. And there's particular things which are out of the scope of the stock, for example, how to interpret NAND data or reassemble it, or for example, how to do device specific routing. I mean, there's a lot of manuals out there, so I won't cover that. So let's start with the motivation of the stock. Traditionally, we had like nearly forever the problem that there's a lot of second hand devices which still contain data. This issue literally existed forever since the hard drive exists, but it was increased as soon as people started to buy or sell hard drives on ebay. And most of these hard drives on ebay, they still contain data like some personal information, emails, pictures, or any other media, sometimes even sensitive documents if the government sold hard drives. And the awareness for this kind of topic was raised like in the beginning of the 2000s. There was, for example, like a paper where the researchers analyzed like what kind of data is still there and what you have to do to basically get rid of all the data. Interestingly, this not only affected like computers, but it also affected like multifunction printers or lab instruments. And the NIST kind of recognized that in 2006 and they published a standard how to kind of wipe your devices to get rid of the data. And some of the solution was, for example, like you need to wipe your device and actually verify it, or you sell devices without hard drive and destroy the hard drive like in the one picture, which is like the example there you just shredded. But there was still like a big problem and the problem was that the lack of knowledge and awareness of the users. So for example, most of the people didn't know how to securely reshark drives. They format their USB stick and they thought like, okay, everything's fine. Some people didn't care. They said like, yeah, I have nothing to hide. And there's one particular problem if you have a broken device. For example, a broken laptop. Most people don't know how to remove the hard drive and to format it. For a long time, I thought this problem was actually kind of solved. But this year, we have been like a study where someone bought like 159 hard drives from ebay and 66 of them still contain data. So it's like still apparently like an issue. The whole problem become a little bit more interesting with smartphones because smartphones also contain a lot of data, sometimes even more than your personal computer pictures, all kind of pictures, messages, I can't create an answer as a call list. And one thing which people didn't know for a long time is that the phones are not encrypted by default, at least not up to 2014, 2015. So very long time, the phones were not encrypted. And this was only introduced with iOS 8, for example, by Apple in 2014 or with Android 6 in 2015. There have been some researchers who didn't analyze on like how the devices do the factory reset and they figured out that most devices actually don't do it correctly, but still there's a lot of data. And this was kind of addressed again by NIST in 2014. They figured out, okay, smartphones are a big risk, so we should address that in this new standard. So now the next topic comes with IoT. IoT in contrast to smartphones and PCs has like a big difference. And there's literally no, a small or like no user interface available. So as a user, you cannot really access directly the data which is on the device, like you go in the file browser or something, you look at that. So you don't know what kind of data is also collected by this device. So you literally don't know anything about your device. If you do a factory reset, then you cannot really verify that this device is really empty. The other thing is you kind of don't know what kind of implementation of the factory reset was done by the vendor. So and this depends from vendor to vendor, from device to device, and sometimes even from one version to another. So you could have a version which does it correctly and then the next version may do something completely different. So the main motivation for the stock is when I was writing my master thesis at Zemo and was analyzing a lot of security, the security of a lot of IoT devices, I had to do like a lot of root access matters and had to factory reset them regularly. And I figured out every time I do a factory reset with this device, all the data is still there or there's still data left. So I was kind of figuring out like, hey, maybe I should look into that. Another important thing is not only we as kind of hackers are interested in the data, but other people are also interested in the data. And this is like an example from Germany where the police figured out like, hmm, smart devices that might be interesting to look at, like for forensic, to solve crimes. So more and more people come now on the idea to actually do forensics analysis of IoT devices. So this kind of talk is about like if you get a use device or if the cops bust your door and actually take your device, what kind of data you can expect there. All right, let's talk about the data on IoT devices. In general, the data that you find on a device is basically dependent on the device type. There's different device types which contain more or less data. However, there's one important thing which is like the same for all IoT devices. And this is basically we need to have bifurc credentials. Before bifurc credentials, it's not IoT because it's kind of connected to the Internet. The other thing what you most of the time have is like some kind of drought credentials because the device needs to authenticate itself against the cloud to like upload data to download data, whatever. As a rule of thumb, what I found is the more performance functions or storage a device has, the more data is on it or will be collected. So this is like a thing if you have a very powerful device, you can expect to definitely collect something. So that's what the first kind of group of devices and those are the vacuum cleaning robots, which most of the time additional to critical credentials contain connection lock files, maps, cleaning locks, user IDs. And you see like an example for like a map, like they're on the screen. I did analysis over a lot of vacuum cleaners. I have a lot of vacuum cleaners. You see like a small collection there, but I have even more. So I had a lot of fun with them. Another second group is smart home gateways. This can be anything which controls any functions in your home. They also contain connection lock files, sensor actuators binding. So for example, if you have like any sensors in your home, they know obviously what kind of sensors you have. Sometimes they collect also data in terms of like if sensors are triggered like temperature sensors, you have like lock files about temperatures in your home. Depending on the device, you might have also key material on it or like user ID again. The next group cameras here again, depending on what kind of camera you have, it might contain cash snapshots or video clips, recorded video event logs. For example, if you have this nice double cameras and they detect if someone is in front of it, then most of the time this device is sometimes or sometimes this device is locked that it might contain again user IDs. And for cameras in particular, they might have a cloud storage credentials because we need to upload like video files to some kind of space. And sometimes you can use the storage credentials to download data again. So it's like depends on the device. Another thing which is not directly smart home, but which is also important are routers. You can buy a lot of routers out of eBay, by the way. So people update them on a regular base, but what they forgot is forget actually is to erase them. And interesting information, for example, is there a DCP leases. So you see the MAC address to IP binding. You see timestamps. You see like lock files. You can potentially track down like when a person is home or not home because if you go home, your smartphone is logging into your Wi-Fi. So this kind of stuff is stored in your device. Firewall configurations, if you have any interesting service behind like your firewall, media files, there's like very powerful like media routers, which contain also like media files and all kind of other lock files, connections, DNS. Some people activate actually DNS logging on their router and sell it later, which is maybe not the best idea because you see all the DNS requests. And especially parents are a little paranoid and said like, you know, parenting filters and other stuff. And also other credentials. So some people put whatever in their router. Another interesting example, media players. This is a kind of critical thing for me. So they contain, again, connection locks. Sometimes, or most of the time, media libraries, playlists, caches. Some people are using this kind of devices also for in their home to browse on the TV. So basically, we have Firefox and some other browser installed. We have their emails installed and so on. And they contain also credentials like for Google Play Store or for network shares. So people are using them to network shares. There's one important thing which I have to change in my presentation. And it's basically theoretical reasons. I need to skip this device because this device contains some interesting information or interesting pictures, let's say it's a. Very interesting taste. I heard from someone else that this is not that uncommon that someone bought a PlayStation 3 and it contained also like a lot of adult movies and whatever. Another thing which I added also are toys. Most of the people don't know that toys can also record a lot of data. For example, I have this nice drone. And this devices can also record audio and video data. And if you crash it, you have no way to reset it. And most of the time, like this device, you cannot reset it anyway in the first place. So yeah. Right. This is like a little bit more technical now, the storage of IoT devices. But this is like a little bit, this is actually important to understand why you kind of screwed up in particular situations. If you talk about storage on IoT devices, you have more or less two kind of flash types which you use. You use the raw flash and the block devices. For raw flash, the typical things which most of you probably know or see in IoT devices is SPI flash. Which can come in two flavors, NAND and NOR. For this presentation, we just stick to NAND. And the parallel NAND flash. Paranand flash is in the green box model. It's also like different flavors of that. The other group are the block devices which are EMMC, EMCP which is like in your smartphone or SD cards. And depending what kind of storage type you have chosen, it affects more or less the selection of file systems which you can use for your IoT device. And this becomes more or less later, this becomes important later for the forensic analysis. Let's start first with the raw NAND flash. If you have SPI flash for example, most of the time it's smaller than 64 megabyte. And you see it in small packages like, let's see, like in this kind of like 8 pin package. If you have bigger sized NAND flash, it's most of the time parallel NAND flash. It's a raw NAND. This can vary from 128 megabyte to 4 gigabyte and it can come in all kind of different packages like BGA or TESAP or whatever. The reason why vendors are liking this kind of flash is because it's cheap and fast. But the problem with that is that it has produced a lot of bit errors which you need to correct. And you see on the right how the access is going. So basically we have the raw NAND flash and this is connected to the host processor. And the host processor in this case need to take care of the layer leveling, the ECC correction and the bed block management. If you work with Linux, this is usually done by the memory technology devices subsystem which takes care of that. And it basically converts the character device into a block device and it kind of takes care of all the things for you. If we take a closer look like to the properties of NAND flash, then what you see at some point is that it's organized in blocks, which are then organized in blocks and pages and the block contains multiple pages. The problem with NAND flash is if you want to erase data, you need to erase the whole block which contains a lot of pages. And the erasing of a block means basically you set all the things to one. The typical size of these blocks are between 16 and 512 kilobytes. So if you want to erase one particular small area of memory, then you need to erase like 512 kilobytes and you need to copy them somewhere. Interestingly, the programming works on page level, so the programming you can do it on 2 kilobytes, but if you want to erase it, you need to erase a big block. The other thing, in addition to the data space, there's also like an out-of-band data space which is for management and for ECC reasons. Because NAND flash is so instable, a lot of vendors are adding some like 2% of additional blocks for spare. So if you have like a lot of bad blocks, then basically they are used for that. And again, like I said, the ECC is computed by a host CPU. And the problem here at some point is that depending on what kind of host CPU you use or NAND controller, it can do particular computations differently. So you cannot really swap one flash to another maybe and read it out. So let's talk about wear leveling of raw flash. And the problem with raw flash is basically that individuals flash cells have very limited lifetimes and writes. So you can write on one cell only 1000 times. And file systems like X2, X3, X4 are not wear leveling aware. So basically if you run an X2 or X3 system, a file system on an NAND flash, without any other layer in between, it will just destroy it with like multiple writes as soon as you write too much. And the solution for that is basically that you have flash aware file system, so additional layer. So on the right side you see like one example, you might have like a JFFS2 for example, it's like a file system which is like flash aware, which works on a partition layer, a level. And the other thing what you can do is like you can add an additional level between the flash and the file system. And this is an example for that is like UB or UBFS, and this works on the whole level. There's a particular pro and cons for the decision, but this is like more or less depends on the vendor what we want to do. This is not that important for now. But the thing here is that the support for the bad block management and wear leveling is again done in the operation system. And the main idea of that is, of this wear leveling in particular, deleted blocks are not erased because that would cause again like many writes. Instead we are just marked as dirty or deleted. And the actual change information is copied into a new block. And at some point, if necessary, the garbage collector will take care of like the old data, but if you don't need to take care of it, then it stays there. I did a graph for that. So this is again simplified. So bear with me. It's like not totally correct in terms of like terms. But imagine we have like a data block, this one, which we want to change. And it's assigned to the use block, the physical use block there. And now what happens is if you want to change it, the original, the old use block is read, then the information with the modified information is written to a new free block. The link is more or less set to the new block and the old block is just marked as dirty. And the important thing here is the data is still present. The data is not erased. It's still there. The interesting thing, particularly for IoT, is if you change the particular file more often, then you have more and more copies of that. So the thing is you have multiple copies on flash. And this data is also not erased as long as this whole block is not erased. And if you have some data in there which is never touched, then it's probably never erased. The interesting thing for us is to reverse engineer or like to extract data is basically that the copies have like a size bigger than two kilobytes. So you can put like a text file, credentials file, whatever into that. And the more data gets changed, for example, log files or Wafir credentials or pictures, the more copies you have in then flash. So basically it's like kind of like a backup just without being a backup. So this was super simplified. If you are more interested in that, there's a great talk from the back at USA 2014 about reverse engineering flash memory for fun and benefit from Mat.0. And you have given like an introduction into the communication protocol of NAND, the answering of NAND flash and how to reverse engineer the NAND formats. Another interesting talk is also like from, or it's a block entry actually, from NAND chips to files from John Michael Picotte. And there's also like a recommended thing. As a side note, by the way, even vendors of devices are not aware of particular features of their file systems. For example, this is like from, this next example from my talk last year at DevCon, where developers leaked actually like a key, where developer keys in the JFFS2 file system. So we basically deleted the file and just imaged the whole file system and put it on millions of devices. And all the millions of devices still had like some really fragments of the key. And some vendors are not safe from that. Let's talk about block devices. So block devices are in general known as managed NAND. They have multiple standards, EMMC4, EMMC5. The higher the standard, the more difficult it gets for us as reverse engineers or like forensics. The classical thing that most people know is like EMMC. And EMMC in technically speaking is a flash with integrated controllers. So basically the controller is like on the chip, so it's not necessary anymore that you have, that you do the bare leveling or the other things in like, in like on the operation system. Another flavor of this kind of memory is EMCP. And those are typical things which you find on your smartphone, because it's basically EMMC with an additional DRAM on one chip. And the advantages for the vendor, for the supply chain, basically you have only one chip. It contains DRAM and flash at the same time. You see it gets a little bit more complicated with the pins down there. Under Linux, these devices are kind of seen as normal block storage devices like an SD card. And so they support all file systems X2, X3, X4. And the advantage of VAMKINOR is like for complexity reasons that the bare leveling, the ECC and bad block management is already integrated in them, so you don't have to take care of that. Now, if you want to access deleted data, then it gets a little bit more complicated, because the EMMC controller doesn't really allow us to get raw access on the device like with a raw NAND. So, but the thing is that the just kind of chips they're using raw NAND internally, and there are like possibilities where you can just bypass the EMMC controller and directly attach to the NAND internally and just bypass all the security things. The challenge is just that some EMMC controllers are doing some kind of weird format, so you need to figure out what we do internally. Again, recommended talk here is the EMMC chips, data recovery beyond the controller by a company who is doing like a lot of recovery things, raw salute, I think. And the summary is basically even if you delete all the data from the EMMC, all the data is still in there in the chip, so you can extract it. But this is a little bit out of scope for this talk. All right, let's talk about reset states which you can find if you buy like a used device or if you do dumpster diving or if your neighbor gives you some broken device. So, the reset states most of the time depend on the previous owner. So, some previous owner think like, yeah, I have nothing to hide, so for example, for this media box, for the critical one, I don't need to reset it at all. So, basically this device contains still all the data, all the configurations. And one of the reasons why it might be the case is that the knowledge is just missing, so the people then know how to reset this thing. Another level which can be the Wi-Fi configuration reset. So, this device might still contain all the data or most of the data, but it's like in an unprovisioned state. That means if you power it on, it won't connect to any Wi-Fi to expect more or less you to provision it again to a new Wi-Fi. And interestingly, many devices only offer this Wi-Fi reset. So, basically if you see a reset button on your device, then most of the time it's a Wi-Fi reset, it's not the device vibe, which is the next thing. Device vibes in contrast to that, they just delete all the data, put the device back in a factory state, and but even in this case, there might be still traces of data there. Interestingly, not all the devices support that because it's basically some technical reasons if they built the device incorrectly when they can't just vibe the device. So, in comparison, both, so some devices actually support both features. So, Wi-Fi reset is usually marked by a special button. If you have a reset button, you click it, it's basically doing a Wi-Fi reset. Device vibe is usually available over the app. If you have an app or you need to press multiple keys. And now the idea of this Wi-Fi reset is like a very logical one. So, basically you don't want to erase the whole device every time you do want to change the Wi-Fi. So, basically this device can be reconnected quickly to a new Wi-Fi. So, you don't have to wait like for five minutes or ten minutes until the flash gets erased. So, you want to just move the thing. In contrast to that, I mean, as with the device vibe, you want to erase all the data and you want to get rid of everything and bring it back in a factory mode. But this is like, these things are like not necessarily the same in terms of like what people understand under reset, you know. So, it's basically if the people think they reset the device, most of the time it's a Wi-Fi reset. All right, let's talk about that data extraction methods. For me, the idea is extract all available data, everything which I can get from this device. And there are multiple methods to do that. For example, if I somehow can get root access on a device, then I can extract it by software. Sometimes I can dump the flash contents without actually disordering anything. In the worst case, I need to disorder like the flash for example. So, this is like typically the chip of method. So, if you talk about software methods, for many IoT devices, especially for the cheap ones, where like some public methods are available where people tell you how to access like the file system, how to root a device, how to do like whatever. And most devices support also like installation of custom firmware or access over USB, for example, ADB or UART. For some devices, especially for cameras, there's like some special modes where you can root a system from the SD card for example. And if you have now a root chart at some point, the thing what you want to do is like basically dump all the flash contents via DD. The advantage for DD is basically DD is no flash away, so it doesn't know anything about out of data blocks, out of band blocks or ECC, so it just dumps everything, which is very helpful for us. Usually you shouldn't use DD by the way if you ever do backups of NAND flashes, because you can't restore them again without breaking a system. You extract the data usually by external media, SSH or Netcat. Netcat is probably the most easiest thing to do. Most methods of software work especially good for file systems if it's like JFS2 or UBFS. The disadvantage here is you can only do things with the kernels allowing you. So basically sometimes the kernel is not allowing you low level access on the flash, so you might be a little bit limited there. The other thing which you can do is like you can dump flash without disordering, and it works mainly very good for SPI flash or EMMC. Some of the devices, if you get them, they allow in-system programming, and how it works basically you take your flash reader and you attach it to particular test pins, and then you can directly access the memory. The problem is the processor shouldn't interfere with this process, so you need to make sure that the processor or the SOC is not working, and you can do that sometimes by pulling the reset pin, or you just ground the clock, the crystal, so that it doesn't do anything. The advantage of this thing is you have a reduced risk of destroying the hardware, if you're not very good in BGA resoldering or unsoldering, then you probably break your device. The disadvantage is you need to fight the test pins first, or you get the traces, and some vendors do with full-air PCBs, they do so that they hide things like in the middle layers, and this kind of nasty. The last method of resort is more or less dumping the flash by the soldering, so this works literally for auto flash methods, for the auto flash chips. I'm confused. Well, okay, so how does it work? How does it work? Usually the recommended thing is you heat up the PCB before you do any soldering, because if you don't heat up the PCB, then at some point you might accidentally pull pins, because you have a ground pin. If you have accessible pins for T-SOP, one trick you can do is basically create a low-temperature alloy, and a low-temperature alloy, you just take a soldering alloy which is like 137 degrees Celsius, and then you just mix it with the existing alloy, and you can do that with a normal soldering iron, so you don't need too much special tools. For BGA chips, it gets a little bit more complicated. Most of the time you need some hot air station, infrared or a flow oven, a soldering station. Some people use also a pizza oven, but I might not recommend that. In general, this advantage of BGA chips is if you want to reuse the device later, you need to re-ball the chip and to resort it, and this might be complicated, and might require a lot of patience. One thing about NAND devices is that you might need a special adapter to use them, because of the high-pink count. Let's start with the tools which you can use for particular things. If you want to access SPI flash, you can literally take any device which can do big bit banging on GPIOs. You can use a Raspberry Pi, you can use Adrinos, you can use bus pirates. For Raspberry Pi, you have the software flash run, which can natively speak to most of the flash chips, so that's kind of easy. My favorite tool, because I use it also for other things, is FlashCAD USB, which is, I don't know, a couple, like, 30, 40 bucks or so. This is also that purpose. If you want to access EMMC flash, the advantage of EMMC flash is it speaks the same protocol like SD cards, so you can use potentially like SD card readers. Here, the important thing is you want to double check the datasheet of the EMMC flash, because some of them are low voltage, so it's like 1.8 volt if you connected the 3.3, when you have a very hard time and you need to buy new devices. One very cheap method is here, there's like an exploit tiers EMMC adapter for 10 bucks, which you can get, you can just connect it into a SD card reader, you solder the cables to the trip and you can read it out. The disadvantage of that is the chips are very small, and so you might have good eyes to do that, or you need a microscope or something. The disadvantage of this particular method is also that you might not be able to access auto partitions, which is actually not a real problem for us in this RU's case of getting data from these devices, but if you do more reverse engineering, you also want to access the special partitions like the boot partitions, but this doesn't matter here. Another method, by the way, for EMMC flash is if you get some cheap stuff from China, like this BGA adapter, where you can just put your BGA chip in there and jiggle a little bit around, and then get a very good connection. The advantage of this is that they support dual voltage chips, so you should be good to go with that. This advantage is with that, by the way, you need to find the correct position of the chips, so you need to try a little bit. I wouldn't recommend to use the original software because it's detected as malware, and the typical recommendation of the developers of the software is, just to say, you're anti-virus and doesn't sound very trustworthy. If you want to access raw NAND, this gets a little bit more complicated because the pin count gets higher, so you need at least, I think, 16 pins, which you need to connect to the chip, and this requires some kind of NAND controller. You can, again, look into the talk about reverse engineering flash memory for fun and benefit. They have very cheap methods to do that, but, again, here, there's more or less professional devices available where you can just use a socket to put the chip in and read it out and dump all the data. The problem with that kind of devices is most of the time that they cannot interpret the ECC or the band data, so it might be a hit or miss. Another method, if you have access to that, evaluation boards. Just take a look at the processor which is used in your device. I did a lot of vacuum cleaners. I have some development boards which I have exactly the same chip, so I can just sort the chip onto this evaluation board and can just use the processor like it is to read and dump all the information. I can also use it to write information to that without thinking about how to calculate the ECC and how to do other things. The disadvantage of this method is, obviously, you need to get this board somehow, and sometimes they're not available, or if they're available, they're extremely expensive, so it's kind of, yeah, difficult. At the moment, we extracted the data. There's a whole zoo of tools which you can use. The typical thing that most people just do is bin work. If you just want to extract some data like credentials, then you can just use a hex editor. At some point, if you want to interpret NAND flash dumps, there's some tools like the dump flash tool or NAND dump tool. The problem with this particular tool is, even though they're open source, they don't handle exotic out-of-band data sizes correctly sometimes, or also the ECC data is kind of like difficult, so it depends more or less on the stock which was used. If you at some point have like a UB image, you can just dump the whole content of that. JFFS images, it's also easy, there's like also tools for that. Basically, for most of the things you have open source tools available, you should be good to go with that. All right, let's start with the device analysis. Some of the methods which I use, where basically disassemble the device, unsolder the flash, dump the flash, put the flash on again, then power the device on again and maybe try to route it. Sometimes it works, sometimes it doesn't work. One thing which I try to do is basically try to connect the device also to the app. As a new user, I connect the device to the app and hope that I still get some old information. The other thing is like as a kind of test, I use the devices, for example, I ran vacuum cleaners in my lab, and then I reset them and then I just compare the data which I had before and after that, so to get an idea of how good the factory set is. The first device where I would start is the EcoVox D-Bot 900. I think it's mostly, it's very popular in Europe, I'm not sure how popular it is in America. I got this device in 2019 and the previous owner told me that he erased it, so I was kind of curious about that. This thing runs Linux, it has this weird rock chip, a sock, and they use a 128 megabyte of NAND flash in TESOP 48 format, so you see this green thing on the left. The approach which I use is basically I dump the NAND flash and then I connect it to where you are to get some additional information to figure out what's going on. After I unsoldered the flash, I wanted to verify that this device was actually factory-setted because otherwise it's kind of like a little bit unfair. And the thing is what I found is actually that I found a lock file which is where it says that this device was erased before I got it. So basically it was newing for sure that this device was erased. The next thing what I found there is like lots of fragments of lock files, keys, maps, Wi-Fi credentials. The only problem that I had with this particular device is that I didn't know how to read the out-of-band data to reassign all the blocks correctly. So this is like still something which I'm working on. The next step what I did is I connected it to a UART and was hoping that I can get some shell to extract the data very easy, but unfortunately they did protect that by making the UART read only, but I found some information about it. So the one information I could figure out is that the Rock Trip is using this very specific like driver to communicate with the flash and they do a very level link by themselves. And the other thing what I knew is that the root partition is squash-afface and that the data partition is X4. So basically I could expect that I find more traces. So this is like the example with the credentials. So basically this device, at some point you need obviously to have files which store the credentials. And what you can see here from the top to the bottom, this was the original file which was created in the factory. So you still see the factory credentials of the SDSS ID and the Wi-Fi password for the factory somewhere in Changseng or wherever this vacuum cleaner was designed. And over the time when the previous owner was adding Wi-Fi's or was connecting this device to the Wi-Fi, you see that more and more Wi-Fi's were like added to this. The other, I had to censor things, so the blue things are censored. The other thing what you see out of the lock files is every time when the vacuum cleaner was connected to the Wi-Fi or even the original point where the owner was provisioning this device first time to the Wi-Fi. So in this case I knew that the person had like a Galaxy phone because you can see it in the lock files that the Galaxy phone was connected to this. So now we have some information about locating the former owner. And there's like a very useful API for that, the Google geolocation API. And what it does is you give it two MAC addresses and the signal strength and it gives you back the coordinates and the accuracy rating. The problem with this particular device was unfortunately it had only one MAC address so I couldn't use this API to find the original position of that. But there's another website where you can just enter the SSID and then you can just look around the world where the SSID is coming from. And the thing about this particular SSID is that it was like a provider modem which had like a random number SSID so I found exactly only one position where the SSID was used. At the second SSID from the same owner I double checked it and figured out it's somewhere in the middle of Germany. So I have also the exact address but I didn't have it here. Right. So as a summary for this particular device, most of the user data is still existing on this device even though it was wiped. I could see the XMMP lock files. This device is using XMMP for the communication with the cloud. I could see the maps, the credentials. Because the maps are using some kind of weird format I couldn't really reassemble them but I might at some point. I also tried to reset the device three times in a row and still could get all the fragments so factory resetting is not very effective there. One interesting aspect is I also found the factory locks so where the sensors and all the other stuff was tested I think these files are actually not existing anymore in the logical space but they're still on this thing. Again I could track the previous owner of this device. However there's one small good news. If you connect this vacuum cleaner now to the Wi-Fi and to your account then basically the app doesn't leak previous information so it's kind of, this is kind of okay. So the data is apparently not there anymore in the form that everyone can see it. Interesting thing is I found the similar results also for other vacuum cleaners. Not sure if, I mean everyone who's using this kind of raw N-ships just kind of like want to save money so a lot of devices have exactly the same issue. Right. The second device which I analyzed because I do many research on Xiaomi so I bought random used devices and broken devices out of eBay. This is a device which I got in 2018. I didn't really know what kind of condition it had because it was broken. In general the Xiaomi devices we have, we run an Ubuntu 14.04 with the quad core ARM and they have 4GB of EMMC. And here the approach was that I connect over UART and dump the partitions over UART and then later on I try to connect it again to the cloud. The good thing about the Xiaomi vacuum cleaners is that routing methods exist. I found them I think two years ago and presented them. You can get root shell over UART or you just can push your custom firmware onto that. The good thing with pushing custom firmware is it doesn't destroy the data so the data is still there so it's kind of good. And later on you can exfuturate all the data via SSH. The alternative method is if this device is totally broken you can still remove and dump the flash content but still works. To give you an idea of what to look for this system this is like the EMMC layout how it looks like so you see that it has three copies of the operation system then on the pre-last line there's the reserve partition which contains all the usage data so how often this device was used what is the status of all the sensors, the HEPA filter and so on and then there's a user partition which is the ut disk partition which contains all the locks, maps, Wi-Fi configuration and user ID. And this is like the partition which we're interested for. So one thing about this vacuum cleaners in particular is that the devices actually support two modes of reset. So the first thing is a Wi-Fi reset which is a button which is called reset and the second is like the factory reset. The Wi-Fi reset what it does is just it leads to Wi-Fi credentials but everything else remains there and the factory reset needs a special procedure where you need to press multiple buttons to wait and then it resets the device in the factory. It's mentioned in the manual but I figured out that nearly no one knows about it so probably the first thing you do is if you get the manual you just throw it away. What this factory reset actually is doing is it restores the operation system from the recovery and formats the data partition. But interestingly here it doesn't wipe it really so there might be still some traces. The partition with the usage data is not erased so basically all the usage data is still there but not kind of user data. So what I did is I just connected it to the cloud and I figured out oh hey all the data is still visible in the app so I could figure out when was the apartment cleaned of the person, what the apartment is looking like and so the assumption was that this device was only Wi-Fi resetted so they pressed only the reset button, nothing more. And even though I connected it to a totally different account this device was re-uploading all the data from the vacuum cleaner up in the cloud so I had it as a new owner, I had all the previous data before. Also all the lock files were still locally available and I was kind of curious what happens if I do a factory reset so at the end of the day I did the factory reset, the good news is if you do like a real factory reset and press the re-buttons then the data is at least not visible in the app anymore. I was looking at the lock files and figured out hey I have two MAC addresses and actually the Google Geo API have given me some coordinates back and I figured out I mean I again censored it so it's like somewhere in the area but I knew the exact address. As soon as I knew the address I was looking at the street and I figured out that the Wi-Fi credentials have SSID had part of the street name in it and the street number because I don't know someone figured out it's maybe a good idea to use the street name and street number as a Wi-Fi SSID and the password contains some personal information like probably Doc's name, I don't know, not sure. However because this data is like stored in plaintext in the device you can just read it out. Another magic trick is by the way the device has a user ID of the user and if you use the app you can share devices with other users and one thing which I did is basically I created a fake account connected to a fake device and then I was just sharing this device with this user with the previous owner and had also the profile picture and the name so yeah. So as a summary for this particular device all the data was still there the device was not wiped, only the Wi-Fi reset was done the reset button is very misleading like for many devices so reset is not doing actually reset however in this case the correct procedure is actually documented in the menu but most of the people don't read that. Again here I could track the owner due to lock files and this kind of devices that create a lot of lock files and very, very both so you have like a nice idea of what's going on there. Alright, some other example which I have here are like this door belts. I figured out like I have like now I think eight door belts and seven of them are more or less from the same I think SDK. In terms of like if you buy like eight door belts from Amazon you have probably internally seven times the same hardware but different exterior. Most of them are based on the high 3518 chip. They have like an SPI NorthLash in there. They use again JFS2 and UbfS so you can really recover all the credentials. They have some lock files so you might get one MAC address off to the WIFI access point so it's kind of like a little bit more difficult but I actually could track it down. The sad thing about this kind of devices is we have unfortunately like a SD card slot so they don't contain video data so it was like a little bit sad about that. Not necessarily all IoT devices are bad so I found actually one device which I didn't disclose yet which was also WakingCleaner from a different company who used TrustZone for key storage. They encrypted all the user partitions with looks. The keys were managed by this trusted execution engine and they're very specific and every time this thing boots up it checks like some states and then unlocks the partitions. The interesting thing here is if you do a factory set it just throws away the keys and recreates the partitions. So this kind of case it would be kind of safe but this is like this is one example I saw so far so it's very rare. As a conclusion in general I had to like a sample size of like 26 devices which I found in like some swap fests. I figured out that apparently secure or correct factory sets are very difficult to implement especially if you use raw nand flash it nearly defeats all the time like full vibes and there's no real way to ensure that the device has actually been re-baked. Also many vendors actually know to re-order user generated data so you have usage data which still remains where log files are not erased. The only thing that sometimes is erased is like the bifur confirmation files but all the other information are more or less still on this device. Another problem what you have again is the missing knowledge of the user so if the people don't know how to reset devices then in the first place then we cannot even complain about the vendors because the user don't really set them. As a recommendation if you ever want to get rid of devices don't sell them and throw them away especially if you expect that they contain any sensitive information so especially don't throw away like this media stuff, the media players and also don't throw away devices if you cannot verify that they have been done full-vibed. One example is destroy the physical memory if it's a cheap device or just use a device to practice soldering, that unsolder resorder things. In general it might be a good idea if you had to throw away devices maybe change your Wi-Fi credentials so at least that someone who has the credentials cannot break into your local network or the other thing is just put all your devices into a separate IoT Wi-Fi which kind of also limits the impact if someone gets the credentials. Alright, that's it. For here I want to thank Professor Nubir and Professor John Manfredelli from the Nuffisen University and also the Zemo team from Tiu Darmstadt and if you have any questions I'm happy here to answer them and I have also some devices if you want to take a look at this kind of weird adapters I'm here for a while. Thank you. Thank you.
 Thank you everyone for coming. This is our penultimate talk of the day.  This is Dennis. He's going to be doing a talk on, you know, privacy leaks by dumpster diving, more or less, correct?  Yeah, so if you ever get, you know, cool technology at eBay or  the Goodwill Store or in the dumpster, maybe there's some stuff that got left over even after like a factory reset.  So he's gonna get into all that today. I think it's gonna be pretty cool.  Can everyone in the back hear me okay?  If I talked like this, would you really hear me okay?  Okay, so you have to talk as loud as that.  Because he's gonna start talking and he's gonna be like, can you hear me? It's pretty good talk, right guys?  See, negative thumbs. So you gotta get real close.  All right, I try. Can you hear me in the back?  Yes, maybe. Should I speak louder? Louder? Louder enough?  What about now?  Yeah, okay, great.  All right, so I'll welcome to my talk about privacy leaks in smart devices.  The introduction was mostly correct in terms of like dumpster diving. What I do more is like a shop on Amazon  warehouse deals or like eBay.  And the main idea of the talk is about to speak about the extraction of data of used smart home devices.  So the outline for the talk is the following.  I start with the motivation when I will speak about how data is stored or what kind of data is stored on IoT devices.  Then how is it stored and what kind of reset states you can imagine if you get like a used device.  What kind of data is maybe there or won't be there.  In the next  topic I will talk about the data extraction methods. So how you get the data off the device and later on I will have some  example devices where I will tell you  how I got the data off and how I found for example like the previous owner of that.  So some information about me. I'm a PhD student at Norfison University and I'm working there with Professor Givrauner Bier.  So we're doing mostly wireless security, but I'm kind of into this IoT stuff.  I'm also right now a grad student at the TU Darmstadt in Germany and working there with Mathias Hadek.  And my main interest is like reverse engineering of any kind of interesting devices can be nearly everything, but at the moment it's more like  IoT and smart logs, but I do also physical logs, which is kind of like the same thing and  so you find me also in the tool of lock picking.  Some side notes about my talk. It's not a Xiaomi bashing talk.  So maybe some of you know me from last year where I was giving a lot of talks about Xiaomi.  The problem is I have only a lot of this devices, so basically I need to talk about them.  Most of the methods I present are already known and used for other things.  There are some ethical and legal questions in this presentation, so I had to censor most of the data and like mark it up a little bit.  If you use any of the methods, for example, if you want to do like BGA on soldering and you break it, then it's kind of on you.  I'm not responsible for that.  And a lot of technical aspects I  have totally simplified. So basically, NAND flash is way more complex, but I did it for the sake of the talk to simplify.  And there's particular things which are out of the scope of the stock, for example, how to interpret NAND data or reassemble it,  or for example, how to do device specific routing. I mean, there's a lot of manuals out there, so I won't cover that.  So let's start with the motivation of the stock.  Traditionally,  we had like nearly forever the problem that there's a lot of second hand devices which still contain data.  This issue literally existed forever since the hard drive exists,  but it was increased as soon as people started to buy or sell  hard drives on ebay. And most of these hard drives on ebay, they still contain data like some personal information,  emails, pictures, or any other media, sometimes even sensitive documents if the government sold hard drives.  And the awareness for this kind of topic was raised like in the beginning of  the 2000s. There was, for example, like a paper where the researchers analyzed like what kind of data is still there and  what you have to do to basically get rid of all the data.  Interestingly, this not only affected like computers, but it also affected like multifunction printers or lab instruments.  And the NIST kind of recognized that in 2006 and they published a standard how to kind of  wipe your devices to get rid of the data. And some of the solution was, for example, like you need to wipe your device and actually verify it,  or you sell devices without hard drive and  destroy the hard drive like in the one picture, which is like the example there you just shredded.  But there was still like a big problem and the problem was that  the lack of knowledge and awareness of the users. So for example, most of the people didn't know how to securely reshark drives.  They format their USB stick and they thought like, okay, everything's fine.  Some people didn't care. They said like, yeah, I have nothing to hide.  And there's one particular problem if you have a broken device. For example, a broken laptop.  Most people don't know how to remove the hard drive and to format it.  For a long time, I thought this problem was actually kind of solved. But this year,  we have been like a study where someone bought like 159 hard drives from ebay and 66 of them still contain data.  So it's like still apparently like an issue.  The whole problem become a little bit more interesting with smartphones because smartphones also contain a lot of data,  sometimes even more than your personal computer pictures, all kind of pictures,  messages, I can't create an answer as a call list. And one thing which people didn't know for a long time is that the phones are not encrypted by default,  at least not up to 2014, 2015. So very long time, the phones were not encrypted.  And this was only introduced with iOS 8, for example, by Apple in 2014 or with Android 6 in 2015.  There have been some researchers who didn't analyze on like how the devices do the factory reset and they figured out that most devices actually don't do it correctly,  but still there's a lot of data. And this was kind of addressed again by NIST in 2014.  They figured out, okay, smartphones are a big risk, so we should address that in this new standard.  So now the next topic comes with IoT. IoT in contrast to smartphones and PCs has like a big difference.  And there's literally no, a small or like no user interface available.  So as a user, you cannot really access directly the data which is on the device, like you go in the file browser or something, you look at that.  So you don't know what kind of data is also collected by this device.  So you literally don't know anything about your device.  If you do a factory reset, then you cannot really verify that this device is really empty.  The other thing is you kind of don't know what kind of implementation of the factory reset was done by the vendor.  So and this depends from vendor to vendor, from device to device, and sometimes even from one version to another.  So you could have a version which does it correctly and then the next version may do something completely different.  So the main motivation for the stock is when I was writing my master thesis at Zemo and was analyzing a lot of security, the security of a lot of IoT devices,  I had to do like a lot of root access matters and had to factory reset them regularly.  And I figured out every time I do a factory reset with this device, all the data is still there or there's still data left.  So I was kind of figuring out like, hey, maybe I should look into that.  Another important thing is not only we as kind of hackers are interested in the data, but other people are also interested in the data.  And this is like an example from Germany where the police figured out like, hmm, smart devices that might be interesting to look at, like for forensic, to solve crimes.  So more and more people come now on the idea to actually do forensics analysis of IoT devices.  So this kind of talk is about like if you get a use device or if the cops bust your door and actually take your device, what kind of data you can expect there.  All right, let's talk about the data on IoT devices.  In general, the data that you find on a device is basically dependent on the device type.  There's different device types which contain more or less data.  However, there's one important thing which is like the same for all IoT devices.  And this is basically we need to have bifurc credentials.  Before bifurc credentials, it's not IoT because it's kind of connected to the Internet.  The other thing what you most of the time have is like some kind of drought credentials because the device needs to authenticate itself against the cloud to like upload data to download data, whatever.  As a rule of thumb, what I found is the more performance functions or storage a device has, the more data is on it or will be collected.  So this is like a thing if you have a very powerful device, you can expect to definitely collect something.  So that's what the first kind of group of devices and those are the vacuum cleaning robots, which most of the time additional to critical credentials contain connection lock files, maps, cleaning locks, user IDs.  And you see like an example for like a map, like they're on the screen.  I did analysis over a lot of vacuum cleaners.  I have a lot of vacuum cleaners.  You see like a small collection there, but I have even more.  So I had a lot of fun with them.  Another second group is smart home gateways.  This can be anything which controls any functions in your home.  They also contain connection lock files, sensor actuators binding.  So for example, if you have like any sensors in your home, they know obviously what kind of sensors you have.  Sometimes they collect also data in terms of like if sensors are triggered like temperature sensors, you have like lock files about temperatures in your home.  Depending on the device, you might have also key material on it or like user ID again.  The next group cameras here again, depending on what kind of camera you have, it might contain cash snapshots or video clips, recorded video event logs.  For example, if you have this nice double cameras and they detect if someone is in front of it, then most of the time this device is sometimes or sometimes this device is locked that it might contain again user IDs.  And for cameras in particular, they might have a cloud storage credentials because we need to upload like video files to some kind of space.  And sometimes you can use the storage credentials to download data again.  So it's like depends on the device.  Another thing which is not directly smart home, but which is also important are routers.  You can buy a lot of routers out of eBay, by the way.  So people update them on a regular base, but what they forgot is forget actually is to erase them.  And interesting information, for example, is there a DCP leases.  So you see the MAC address to IP binding.  You see timestamps.  You see like lock files.  You can potentially track down like when a person is home or not home because if you go home, your smartphone is logging into your Wi-Fi.  So this kind of stuff is stored in your device.  Firewall configurations, if you have any interesting service behind like your firewall, media files, there's like very powerful like media routers,  which contain also like media files and all kind of other lock files, connections, DNS.  Some people activate actually DNS logging on their router and sell it later, which is maybe not the best idea because you see all the DNS requests.  And especially parents are a little paranoid and said like, you know, parenting filters and other stuff.  And also other credentials.  So some people put whatever in their router.  Another interesting example, media players.  This is a kind of critical thing for me.  So they contain, again, connection locks.  Sometimes, or most of the time, media libraries, playlists, caches.  Some people are using this kind of devices also for in their home to browse on the TV.  So basically, we have Firefox and some other browser installed.  We have their emails installed and so on.  And they contain also credentials like for Google Play Store or for network shares.  So people are using them to network shares.  There's one important thing which I have to change in my presentation.  And it's basically theoretical reasons.  I need to skip this device because this device contains some interesting information or interesting pictures, let's say it's a.  Very interesting taste.  I heard from someone else that this is not that uncommon that someone bought a PlayStation 3 and it contained also like a lot of adult movies and whatever.  Another thing which I added also are toys.  Most of the people don't know that toys can also record a lot of data.  For example, I have this nice drone.  And this devices can also record audio and video data.  And if you crash it, you have no way to reset it.  And most of the time, like this device, you cannot reset it anyway in the first place.  So yeah.  Right.  This is like a little bit more technical now, the storage of IoT devices.  But this is like a little bit, this is actually important to understand why you kind of screwed up in particular situations.  If you talk about storage on IoT devices, you have more or less two kind of flash types which you use.  You use the raw flash and the block devices.  For raw flash, the typical things which most of you probably know or see in IoT devices is SPI flash.  Which can come in two flavors, NAND and NOR.  For this presentation, we just stick to NAND.  And the parallel NAND flash.  Paranand flash is in the green box model.  It's also like different flavors of that.  The other group are the block devices which are EMMC, EMCP which is like in your smartphone or SD cards.  And depending what kind of storage type you have chosen, it affects more or less the selection of file systems which you can use for your IoT device.  And this becomes more or less later, this becomes important later for the forensic analysis.  Let's start first with the raw NAND flash.  If you have SPI flash for example, most of the time it's smaller than 64 megabyte.  And you see it in small packages like, let's see, like in this kind of like 8 pin package.  If you have bigger sized NAND flash, it's most of the time parallel NAND flash.  It's a raw NAND.  This can vary from 128 megabyte to 4 gigabyte and it can come in all kind of different packages like BGA or TESAP or whatever.  The reason why vendors are liking this kind of flash is because it's cheap and fast.  But the problem with that is that it has produced a lot of bit errors which you need to correct.  And you see on the right how the access is going.  So basically we have the raw NAND flash and this is connected to the host processor.  And the host processor in this case need to take care of the layer leveling, the ECC correction and the bed block management.  If you work with Linux, this is usually done by the memory technology devices subsystem which takes care of that.  And it basically converts the character device into a block device and it kind of takes care of all the things for you.  If we take a closer look like to the properties of NAND flash, then what you see at some point is that it's organized in blocks,  which are then organized in blocks and pages and the block contains multiple pages.  The problem with NAND flash is if you want to erase data, you need to erase the whole block which contains a lot of pages.  And the erasing of a block means basically you set all the things to one.  The typical size of these blocks are between 16 and 512 kilobytes.  So if you want to erase one particular small area of memory, then you need to erase like 512 kilobytes and you need to copy them somewhere.  Interestingly, the programming works on page level, so the programming you can do it on 2 kilobytes,  but if you want to erase it, you need to erase a big block.  The other thing, in addition to the data space, there's also like an out-of-band data space which is for management and for ECC reasons.  Because NAND flash is so instable, a lot of vendors are adding some like 2% of additional blocks for spare.  So if you have like a lot of bad blocks, then basically they are used for that.  And again, like I said, the ECC is computed by a host CPU.  And the problem here at some point is that depending on what kind of host CPU you use or NAND controller,  it can do particular computations differently.  So you cannot really swap one flash to another maybe and read it out.  So let's talk about wear leveling of raw flash.  And the problem with raw flash is basically that individuals flash cells have very limited lifetimes and writes.  So you can write on one cell only 1000 times.  And file systems like X2, X3, X4 are not wear leveling aware.  So basically if you run an X2 or X3 system, a file system on an NAND flash,  without any other layer in between, it will just destroy it with like multiple writes as soon as you write too much.  And the solution for that is basically that you have flash aware file system, so additional layer.  So on the right side you see like one example, you might have like a JFFS2 for example,  it's like a file system which is like flash aware, which works on a partition layer, a level.  And the other thing what you can do is like you can add an additional level between the flash and the file system.  And this is an example for that is like UB or UBFS, and this works on the whole level.  There's a particular pro and cons for the decision, but this is like more or less depends on the vendor what we want to do.  This is not that important for now.  But the thing here is that the support for the bad block management and wear leveling is again done in the operation system.  And the main idea of that is, of this wear leveling in particular, deleted blocks are not erased because that would cause again like many writes.  Instead we are just marked as dirty or deleted.  And the actual change information is copied into a new block.  And at some point, if necessary, the garbage collector will take care of like the old data,  but if you don't need to take care of it, then it stays there.  I did a graph for that.  So this is again simplified. So bear with me. It's like not totally correct in terms of like terms.  But imagine we have like a data block, this one, which we want to change.  And it's assigned to the use block, the physical use block there.  And now what happens is if you want to change it, the original, the old use block is read,  then the information with the modified information is written to a new free block.  The link is more or less set to the new block and the old block is just marked as dirty.  And the important thing here is the data is still present.  The data is not erased. It's still there.  The interesting thing, particularly for IoT, is if you change the particular file more often,  then you have more and more copies of that.  So the thing is you have multiple copies on flash.  And this data is also not erased as long as this whole block is not erased.  And if you have some data in there which is never touched, then it's probably never erased.  The interesting thing for us is to reverse engineer or like to extract data is basically that the copies  have like a size bigger than two kilobytes.  So you can put like a text file, credentials file, whatever into that.  And the more data gets changed, for example, log files or Wafir credentials or pictures,  the more copies you have in then flash.  So basically it's like kind of like a backup just without being a backup.  So this was super simplified.  If you are more interested in that, there's a great talk from the back at USA 2014  about reverse engineering flash memory for fun and benefit from Mat.0.  And you have given like an introduction into the communication protocol of NAND,  the answering of NAND flash and how to reverse engineer the NAND formats.  Another interesting talk is also like from, or it's a block entry actually,  from NAND chips to files from John Michael Picotte.  And there's also like a recommended thing.  As a side note, by the way, even vendors of devices are not aware of particular features of their file systems.  For example, this is like from, this next example from my talk last year at DevCon,  where developers leaked actually like a key, where developer keys in the JFFS2 file system.  So we basically deleted the file and just imaged the whole file system  and put it on millions of devices.  And all the millions of devices still had like some really fragments of the key.  And some vendors are not safe from that.  Let's talk about block devices.  So block devices are in general known as managed NAND.  They have multiple standards, EMMC4, EMMC5.  The higher the standard, the more difficult it gets for us as reverse engineers or like forensics.  The classical thing that most people know is like EMMC.  And EMMC in technically speaking is a flash with integrated controllers.  So basically the controller is like on the chip, so it's not necessary anymore that you have,  that you do the bare leveling or the other things in like, in like on the operation system.  Another flavor of this kind of memory is EMCP.  And those are typical things which you find on your smartphone,  because it's basically EMMC with an additional DRAM on one chip.  And the advantages for the vendor, for the supply chain, basically you have only one chip.  It contains DRAM and flash at the same time.  You see it gets a little bit more complicated with the pins down there.  Under Linux, these devices are kind of seen as normal block storage devices like an SD card.  And so they support all file systems X2, X3, X4.  And the advantage of VAMKINOR is like for complexity reasons that the bare leveling,  the ECC and bad block management is already integrated in them, so you don't have to take care of that.  Now, if you want to access deleted data, then it gets a little bit more complicated,  because the EMMC controller doesn't really allow us to get raw access on the device like with a raw NAND.  So, but the thing is that the just kind of chips they're using raw NAND internally,  and there are like possibilities where you can just bypass the EMMC controller  and directly attach to the NAND internally and just bypass all the security things.  The challenge is just that some EMMC controllers are doing some kind of weird format,  so you need to figure out what we do internally.  Again, recommended talk here is the EMMC chips, data recovery beyond the controller  by a company who is doing like a lot of recovery things, raw salute, I think.  And the summary is basically even if you delete all the data from the EMMC,  all the data is still in there in the chip, so you can extract it.  But this is a little bit out of scope for this talk.  All right, let's talk about reset states which you can find if you buy like a used device  or if you do dumpster diving or if your neighbor gives you some broken device.  So, the reset states most of the time depend on the previous owner.  So, some previous owner think like, yeah, I have nothing to hide,  so for example, for this media box, for the critical one, I don't need to reset it at all.  So, basically this device contains still all the data, all the configurations.  And one of the reasons why it might be the case is that the knowledge is just missing,  so the people then know how to reset this thing.  Another level which can be the Wi-Fi configuration reset.  So, this device might still contain all the data or most of the data,  but it's like in an unprovisioned state.  That means if you power it on, it won't connect to any Wi-Fi  to expect more or less you to provision it again to a new Wi-Fi.  And interestingly, many devices only offer this Wi-Fi reset.  So, basically if you see a reset button on your device,  then most of the time it's a Wi-Fi reset, it's not the device vibe, which is the next thing.  Device vibes in contrast to that, they just delete all the data,  put the device back in a factory state, and but even in this case,  there might be still traces of data there.  Interestingly, not all the devices support that because it's basically  some technical reasons if they built the device incorrectly  when they can't just vibe the device.  So, in comparison, both, so some devices actually support both features.  So, Wi-Fi reset is usually marked by a special button.  If you have a reset button, you click it, it's basically doing a Wi-Fi reset.  Device vibe is usually available over the app.  If you have an app or you need to press multiple keys.  And now the idea of this Wi-Fi reset is like a very logical one.  So, basically you don't want to erase the whole device every time you do want to change the Wi-Fi.  So, basically this device can be reconnected quickly to a new Wi-Fi.  So, you don't have to wait like for five minutes or ten minutes until the flash gets erased.  So, you want to just move the thing.  In contrast to that, I mean, as with the device vibe, you want to erase all the data  and you want to get rid of everything and bring it back in a factory mode.  But this is like, these things are like not necessarily the same in terms of like  what people understand under reset, you know.  So, it's basically if the people think they reset the device, most of the time it's a Wi-Fi reset.  All right, let's talk about that data extraction methods.  For me, the idea is extract all available data, everything which I can get from this device.  And there are multiple methods to do that.  For example, if I somehow can get root access on a device, then I can extract it by software.  Sometimes I can dump the flash contents without actually disordering anything.  In the worst case, I need to disorder like the flash for example.  So, this is like typically the chip of method.  So, if you talk about software methods, for many IoT devices, especially for the cheap ones,  where like some public methods are available where people tell you how to access like the file system,  how to root a device, how to do like whatever.  And most devices support also like installation of custom firmware or access over USB, for example,  ADB or UART.  For some devices, especially for cameras, there's like some special modes where you can root a system  from the SD card for example.  And if you have now a root chart at some point, the thing what you want to do is like basically dump  all the flash contents via DD.  The advantage for DD is basically DD is no flash away, so it doesn't know anything about  out of data blocks, out of band blocks or ECC, so it just dumps everything, which is very helpful for us.  Usually you shouldn't use DD by the way if you ever do backups of NAND flashes,  because you can't restore them again without breaking a system.  You extract the data usually by external media, SSH or Netcat.  Netcat is probably the most easiest thing to do.  Most methods of software work especially good for file systems if it's like JFS2 or UBFS.  The disadvantage here is you can only do things with the kernels allowing you.  So basically sometimes the kernel is not allowing you low level access on the flash, so you might be a little bit limited there.  The other thing which you can do is like you can dump flash without disordering,  and it works mainly very good for SPI flash or EMMC.  Some of the devices, if you get them, they allow in-system programming,  and how it works basically you take your flash reader and you attach it to particular test pins,  and then you can directly access the memory.  The problem is the processor shouldn't interfere with this process,  so you need to make sure that the processor or the SOC is not working,  and you can do that sometimes by pulling the reset pin, or you just ground the clock, the crystal,  so that it doesn't do anything.  The advantage of this thing is you have a reduced risk of destroying the hardware,  if you're not very good in BGA resoldering or unsoldering, then you probably break your device.  The disadvantage is you need to fight the test pins first, or you get the traces,  and some vendors do with full-air PCBs, they do so that they hide things like in the middle layers, and this kind of nasty.  The last method of resort is more or less dumping the flash by the soldering,  so this works literally for auto flash methods, for the auto flash chips.  I'm confused.  Well, okay, so how does it work?  How does it work?  Usually the recommended thing is you heat up the PCB before you do any soldering,  because if you don't heat up the PCB, then at some point you might accidentally pull pins,  because you have a ground pin.  If you have accessible pins for T-SOP, one trick you can do is basically create a low-temperature alloy,  and a low-temperature alloy, you just take a soldering alloy which is like 137 degrees Celsius,  and then you just mix it with the existing alloy, and you can do that with a normal soldering iron,  so you don't need too much special tools.  For BGA chips, it gets a little bit more complicated.  Most of the time you need some hot air station, infrared or a flow oven, a soldering station.  Some people use also a pizza oven, but I might not recommend that.  In general, this advantage of BGA chips is if you want to reuse the device later,  you need to re-ball the chip and to resort it, and this might be complicated,  and might require a lot of patience.  One thing about NAND devices is that you might need a special adapter to use them,  because of the high-pink count.  Let's start with the tools which you can use for particular things.  If you want to access SPI flash, you can literally take any device which can do big bit banging on GPIOs.  You can use a Raspberry Pi, you can use Adrinos, you can use bus pirates.  For Raspberry Pi, you have the software flash run, which can natively speak to most of the flash chips,  so that's kind of easy.  My favorite tool, because I use it also for other things, is FlashCAD USB,  which is, I don't know, a couple, like, 30, 40 bucks or so.  This is also that purpose.  If you want to access EMMC flash, the advantage of EMMC flash is it speaks the same protocol like SD cards,  so you can use potentially like SD card readers.  Here, the important thing is you want to double check the datasheet of the EMMC flash,  because some of them are low voltage, so it's like 1.8 volt if you connected the 3.3,  when you have a very hard time and you need to buy new devices.  One very cheap method is here, there's like an exploit tiers EMMC adapter for 10 bucks,  which you can get, you can just connect it into a SD card reader,  you solder the cables to the trip and you can read it out.  The disadvantage of that is the chips are very small,  and so you might have good eyes to do that, or you need a microscope or something.  The disadvantage of this particular method is also that you might not be able to access auto partitions,  which is actually not a real problem for us in this RU's case of getting data from these devices,  but if you do more reverse engineering, you also want to access the special partitions like the boot partitions,  but this doesn't matter here.  Another method, by the way, for EMMC flash is if you get some cheap stuff from China,  like this BGA adapter, where you can just put your BGA chip in there and jiggle a little bit around,  and then get a very good connection.  The advantage of this is that they support dual voltage chips, so you should be good to go with that.  This advantage is with that, by the way, you need to find the correct position of the chips,  so you need to try a little bit.  I wouldn't recommend to use the original software because it's detected as malware,  and the typical recommendation of the developers of the software is,  just to say, you're anti-virus and doesn't sound very trustworthy.  If you want to access raw NAND, this gets a little bit more complicated because the pin count gets higher,  so you need at least, I think, 16 pins, which you need to connect to the chip,  and this requires some kind of NAND controller.  You can, again, look into the talk about reverse engineering flash memory for fun and benefit.  They have very cheap methods to do that, but, again, here, there's more or less professional devices  available where you can just use a socket to put the chip in and read it out and dump all the data.  The problem with that kind of devices is most of the time that they cannot interpret the ECC or the band data,  so it might be a hit or miss.  Another method, if you have access to that, evaluation boards.  Just take a look at the processor which is used in your device.  I did a lot of vacuum cleaners. I have some development boards which I have exactly the same chip,  so I can just sort the chip onto this evaluation board and can just use the processor like it is to read and dump all the information.  I can also use it to write information to that without thinking about how to calculate the ECC and how to do other things.  The disadvantage of this method is, obviously, you need to get this board somehow,  and sometimes they're not available, or if they're available, they're extremely expensive, so it's kind of, yeah, difficult.  At the moment, we extracted the data. There's a whole zoo of tools which you can use.  The typical thing that most people just do is bin work.  If you just want to extract some data like credentials, then you can just use a hex editor.  At some point, if you want to interpret NAND flash dumps, there's some tools like the dump flash tool or NAND dump tool.  The problem with this particular tool is, even though they're open source, they don't handle exotic out-of-band data sizes correctly sometimes,  or also the ECC data is kind of like difficult, so it depends more or less on the stock which was used.  If you at some point have like a UB image, you can just dump the whole content of that.  JFFS images, it's also easy, there's like also tools for that.  Basically, for most of the things you have open source tools available, you should be good to go with that.  All right, let's start with the device analysis.  Some of the methods which I use, where basically disassemble the device, unsolder the flash, dump the flash, put the flash on again,  then power the device on again and maybe try to route it.  Sometimes it works, sometimes it doesn't work.  One thing which I try to do is basically try to connect the device also to the app.  As a new user, I connect the device to the app and hope that I still get some old information.  The other thing is like as a kind of test, I use the devices, for example, I ran vacuum cleaners in my lab,  and then I reset them and then I just compare the data which I had before and after that,  so to get an idea of how good the factory set is.  The first device where I would start is the EcoVox D-Bot 900.  I think it's mostly, it's very popular in Europe, I'm not sure how popular it is in America.  I got this device in 2019 and the previous owner told me that he erased it,  so I was kind of curious about that.  This thing runs Linux, it has this weird rock chip, a sock,  and they use a 128 megabyte of NAND flash in TESOP 48 format, so you see this green thing on the left.  The approach which I use is basically I dump the NAND flash and then I connect it to where you are  to get some additional information to figure out what's going on.  After I unsoldered the flash, I wanted to verify that this device was actually factory-setted  because otherwise it's kind of like a little bit unfair.  And the thing is what I found is actually that I found a lock file which is where it says that this device was erased before I got it.  So basically it was newing for sure that this device was erased.  The next thing what I found there is like lots of fragments of lock files, keys, maps, Wi-Fi credentials.  The only problem that I had with this particular device is that I didn't know how to read the out-of-band data to reassign all the blocks correctly.  So this is like still something which I'm working on.  The next step what I did is I connected it to a UART and was hoping that I can get some shell to extract the data very easy,  but unfortunately they did protect that by making the UART read only, but I found some information about it.  So the one information I could figure out is that the Rock Trip is using this very specific like driver to communicate with the flash  and they do a very level link by themselves.  And the other thing what I knew is that the root partition is squash-afface and that the data partition is X4.  So basically I could expect that I find more traces.  So this is like the example with the credentials.  So basically this device, at some point you need obviously to have files which store the credentials.  And what you can see here from the top to the bottom, this was the original file which was created in the factory.  So you still see the factory credentials of the SDSS ID and the Wi-Fi password for the factory somewhere in Changseng or wherever this vacuum cleaner was designed.  And over the time when the previous owner was adding Wi-Fi's or was connecting this device to the Wi-Fi,  you see that more and more Wi-Fi's were like added to this.  The other, I had to censor things, so the blue things are censored.  The other thing what you see out of the lock files is every time when the vacuum cleaner was connected to the Wi-Fi  or even the original point where the owner was provisioning this device first time to the Wi-Fi.  So in this case I knew that the person had like a Galaxy phone  because you can see it in the lock files that the Galaxy phone was connected to this.  So now we have some information about locating the former owner.  And there's like a very useful API for that, the Google geolocation API.  And what it does is you give it two MAC addresses and the signal strength  and it gives you back the coordinates and the accuracy rating.  The problem with this particular device was unfortunately it had only one MAC address  so I couldn't use this API to find the original position of that.  But there's another website where you can just enter the SSID  and then you can just look around the world where the SSID is coming from.  And the thing about this particular SSID is that it was like a provider modem  which had like a random number SSID so I found exactly only one position where the SSID was used.  At the second SSID from the same owner I double checked it  and figured out it's somewhere in the middle of Germany.  So I have also the exact address but I didn't have it here.  Right. So as a summary for this particular device,  most of the user data is still existing on this device even though it was wiped.  I could see the XMMP lock files. This device is using XMMP for the communication with the cloud.  I could see the maps, the credentials.  Because the maps are using some kind of weird format I couldn't really reassemble them  but I might at some point.  I also tried to reset the device three times in a row and still could get all the fragments  so factory resetting is not very effective there.  One interesting aspect is I also found the factory locks  so where the sensors and all the other stuff was tested  I think these files are actually not existing anymore in the logical space  but they're still on this thing.  Again I could track the previous owner of this device.  However there's one small good news.  If you connect this vacuum cleaner now to the Wi-Fi and to your account  then basically the app doesn't leak previous information  so it's kind of, this is kind of okay.  So the data is apparently not there anymore in the form that everyone can see it.  Interesting thing is I found the similar results also for other vacuum cleaners.  Not sure if, I mean everyone who's using this kind of raw N-ships  just kind of like want to save money so a lot of devices have exactly the same issue.  Right.  The second device which I analyzed because I do many research on Xiaomi  so I bought random used devices and broken devices out of eBay.  This is a device which I got in 2018.  I didn't really know what kind of condition it had because it was broken.  In general the Xiaomi devices we have, we run an Ubuntu 14.04  with the quad core ARM and they have 4GB of EMMC.  And here the approach was that I connect over UART  and dump the partitions over UART  and then later on I try to connect it again to the cloud.  The good thing about the Xiaomi vacuum cleaners is that routing methods exist.  I found them I think two years ago and presented them.  You can get root shell over UART  or you just can push your custom firmware onto that.  The good thing with pushing custom firmware is it doesn't destroy the data  so the data is still there so it's kind of good.  And later on you can exfuturate all the data via SSH.  The alternative method is if this device is totally broken  you can still remove and dump the flash content but still works.  To give you an idea of what to look for  this system this is like the EMMC layout how it looks like  so you see that it has three copies of the operation system  then on the pre-last line there's the reserve partition  which contains all the usage data so how often this device was used  what is the status of all the sensors, the HEPA filter and so on  and then there's a user partition which is the ut disk partition  which contains all the locks, maps, Wi-Fi configuration and user ID.  And this is like the partition which we're interested for.  So one thing about this vacuum cleaners in particular  is that the devices actually support two modes of reset.  So the first thing is a Wi-Fi reset which is a button which is called reset  and the second is like the factory reset.  The Wi-Fi reset what it does is just it leads to Wi-Fi credentials  but everything else remains there and the factory reset needs a special procedure  where you need to press multiple buttons to wait and then it resets the device in the factory.  It's mentioned in the manual but I figured out that nearly no one knows about it  so probably the first thing you do is if you get the manual you just throw it away.  What this factory reset actually is doing is it restores the operation system  from the recovery and formats the data partition.  But interestingly here it doesn't wipe it really so there might be still some traces.  The partition with the usage data is not erased so basically all the usage data is still there  but not kind of user data.  So what I did is I just connected it to the cloud and I figured out  oh hey all the data is still visible in the app so I could figure out  when was the apartment cleaned of the person, what the apartment is looking like  and so the assumption was that this device was only Wi-Fi resetted  so they pressed only the reset button, nothing more.  And even though I connected it to a totally different account this device was re-uploading  all the data from the vacuum cleaner up in the cloud so I had it as a new owner,  I had all the previous data before. Also all the lock files were still locally available  and I was kind of curious what happens if I do a factory reset so at the end of the day  I did the factory reset, the good news is if you do like a real factory reset and press the  re-buttons then the data is at least not visible in the app anymore.  I was looking at the lock files and figured out hey I have two MAC addresses  and actually the Google Geo API  have given me some coordinates back and I figured out I mean  I again censored it so it's like somewhere in the area but I knew the exact address.  As soon as I knew the address I was looking at the street and I figured out that the Wi-Fi credentials  have SSID had part of the street name in it and the street number  because I don't know someone figured out it's maybe a good idea to use the street name  and street number as a Wi-Fi SSID and the password contains some  personal information like probably Doc's name, I don't know, not sure.  However because this data is like stored in plaintext in the device you can just read it out.  Another magic trick is by the way the device has a user ID of the user  and if you use the app you can share devices with other users  and one thing which I did is basically I created a fake account connected to a fake device  and then I was just sharing this device with this user  with the previous owner and had also the profile picture and the name  so yeah. So as a summary  for this particular device all the data was still there  the device was not wiped, only the Wi-Fi reset was done  the reset button is very misleading like for many devices  so reset is not doing actually reset however in this case the  correct procedure is actually documented in the menu but most of the people don't read that.  Again here I could track the owner due to  lock files and this kind of devices that create a lot of lock files  and very, very both so you have like a nice idea of what's going on there.  Alright, some other example which I have here  are like this door belts. I figured out like I have like  now I think eight door belts and seven of them are more or less from the same  I think SDK. In terms of like if you buy like  eight door belts from Amazon you have probably internally  seven times the same hardware but different exterior. Most of them are based  on the high 3518 chip. They have like an SPI NorthLash  in there. They use again JFS2 and UbfS so you can  really recover all the credentials. They have some lock files  so you might get one MAC address off to the WIFI access point so it's  kind of like a little bit more difficult but I actually could track it down. The sad thing about this  kind of devices is we have unfortunately like a SD card slot  so they don't contain video data so it was like a little bit sad about that.  Not necessarily all IoT devices are bad  so I found actually one device which I didn't disclose yet which was also WakingCleaner  from a different company who used TrustZone for key storage. They encrypted  all the user partitions with looks. The  keys were managed by this trusted execution engine and they're very  specific and every time this thing boots up it checks like some states and then unlocks  the partitions. The interesting thing here is if you do a factory set it just throws away the keys  and recreates the partitions. So this kind of case it would be kind of safe but this is like this is one  example I saw so far so it's very rare.  As a conclusion in general I had to like a sample size of like 26 devices which I  found in like some swap fests.  I figured out that apparently secure or correct  factory sets are very difficult to implement especially if you use raw nand  flash it nearly defeats all the time like full vibes  and there's no real way to ensure that the device has actually  been re-baked. Also many vendors actually know to re-order user generated data so you have  usage data which still remains where log files are not erased. The only thing that sometimes  is erased is like the bifur confirmation files but all the other information  are more or less still on this device. Another problem what you have  again is the missing knowledge of the user so if the people don't know how to reset devices  then in the first place then we cannot even complain about the vendors because the user don't  really set them. As a recommendation if you ever want to get rid of  devices don't sell them and throw them away  especially if you expect that they contain any sensitive information so especially  don't throw away like this media stuff, the media players  and also don't throw away devices if you cannot  verify that they have been done full-vibed. One example is destroy the physical memory  if it's a cheap device or just use a device to  practice soldering, that unsolder resorder things.  In general it might be a good idea if you had to throw away devices maybe change your  Wi-Fi credentials so at least that someone who has the credentials cannot break into your  local network or the other thing is just put all your devices into a separate  IoT Wi-Fi which kind of also limits the impact if someone gets the credentials.  Alright, that's it. For here  I want to thank Professor Nubir and Professor  John Manfredelli from the Nuffisen University and also the Zemo team  from Tiu Darmstadt and if you have any questions I'm happy here  to answer them and I have also some devices if you want to take a look at this kind of weird adapters  I'm here for a while. Thank you.  Thank you.
Dennis_Giese_-_Privacy_leaks_Extracting_data_from__Iit4SmzQ2Uo_0_47.png Dennis_Giese_-_Privacy_leaks_Extracting_data_from__Iit4SmzQ2Uo_48_1019.png Dennis_Giese_-_Privacy_leaks_Extracting_data_from__Iit4SmzQ2Uo_1020_1103.png Dennis_Giese_-_Privacy_leaks_Extracting_data_from__Iit4SmzQ2Uo_1104_1223.png Dennis_Giese_-_Privacy_leaks_Extracting_data_from__Iit4SmzQ2Uo_1224_2123.png Dennis_Giese_-_Privacy_leaks_Extracting_data_from__Iit4SmzQ2Uo_2124_2939.png Dennis_Giese_-_Privacy_leaks_Extracting_data_from__Iit4SmzQ2Uo_2940_3743.png Dennis_Giese_-_Privacy_leaks_Extracting_data_from__Iit4SmzQ2Uo_3744_5087.png Dennis_Giese_-_Privacy_leaks_Extracting_data_from__Iit4SmzQ2Uo_5088_5183.png Dennis_Giese_-_Privacy_leaks_Extracting_data_from__Iit4SmzQ2Uo_5184_6179.png Dennis_Giese_-_Privacy_leaks_Extracting_data_from__Iit4SmzQ2Uo_6180_7523.png Dennis_Giese_-_Privacy_leaks_Extracting_data_from__Iit4SmzQ2Uo_7524_7763.png Dennis_Giese_-_Privacy_leaks_Extracting_data_from__Iit4SmzQ2Uo_7764_8951.png Dennis_Giese_-_Privacy_leaks_Extracting_data_from__Iit4SmzQ2Uo_8952_10127.png Dennis_Giese_-_Privacy_leaks_Extracting_data_from__Iit4SmzQ2Uo_10128_10643.png Dennis_Giese_-_Privacy_leaks_Extracting_data_from__Iit4SmzQ2Uo_10644_11447.png Dennis_Giese_-_Privacy_leaks_Extracting_data_from__Iit4SmzQ2Uo_11448_11543.png Dennis_Giese_-_Privacy_leaks_Extracting_data_from__Iit4SmzQ2Uo_11544_12551.png Dennis_Giese_-_Privacy_leaks_Extracting_data_from__Iit4SmzQ2Uo_12552_13115.png Dennis_Giese_-_Privacy_leaks_Extracting_data_from__Iit4SmzQ2Uo_13116_13871.png Dennis_Giese_-_Privacy_leaks_Extracting_data_from__Iit4SmzQ2Uo_13872_14795.png Dennis_Giese_-_Privacy_leaks_Extracting_data_from__Iit4SmzQ2Uo_14796_16391.png Dennis_Giese_-_Privacy_leaks_Extracting_data_from__Iit4SmzQ2Uo_16392_17195.png Dennis_Giese_-_Privacy_leaks_Extracting_data_from__Iit4SmzQ2Uo_17196_17735.png Dennis_Giese_-_Privacy_leaks_Extracting_data_from__Iit4SmzQ2Uo_17736_18227.png Dennis_Giese_-_Privacy_leaks_Extracting_data_from__Iit4SmzQ2Uo_18228_18479.png Dennis_Giese_-_Privacy_leaks_Extracting_data_from__Iit4SmzQ2Uo_18480_19715.png Dennis_Giese_-_Privacy_leaks_Extracting_data_from__Iit4SmzQ2Uo_19716_21335.png Dennis_Giese_-_Privacy_leaks_Extracting_data_from__Iit4SmzQ2Uo_21336_23387.png Dennis_Giese_-_Privacy_leaks_Extracting_data_from__Iit4SmzQ2Uo_23388_25583.png Dennis_Giese_-_Privacy_leaks_Extracting_data_from__Iit4SmzQ2Uo_25584_25859.png Dennis_Giese_-_Privacy_leaks_Extracting_data_from__Iit4SmzQ2Uo_25860_26243.png Dennis_Giese_-_Privacy_leaks_Extracting_data_from__Iit4SmzQ2Uo_26244_26699.png Dennis_Giese_-_Privacy_leaks_Extracting_data_from__Iit4SmzQ2Uo_26700_27587.png Dennis_Giese_-_Privacy_leaks_Extracting_data_from__Iit4SmzQ2Uo_27588_28367.png Dennis_Giese_-_Privacy_leaks_Extracting_data_from__Iit4SmzQ2Uo_28368_29027.png Dennis_Giese_-_Privacy_leaks_Extracting_data_from__Iit4SmzQ2Uo_29028_30863.png Dennis_Giese_-_Privacy_leaks_Extracting_data_from__Iit4SmzQ2Uo_30864_31043.png Dennis_Giese_-_Privacy_leaks_Extracting_data_from__Iit4SmzQ2Uo_31044_32123.png Dennis_Giese_-_Privacy_leaks_Extracting_data_from__Iit4SmzQ2Uo_32124_32363.png Dennis_Giese_-_Privacy_leaks_Extracting_data_from__Iit4SmzQ2Uo_32364_34115.png Dennis_Giese_-_Privacy_leaks_Extracting_data_from__Iit4SmzQ2Uo_34116_35495.png Dennis_Giese_-_Privacy_leaks_Extracting_data_from__Iit4SmzQ2Uo_35496_35579.png Dennis_Giese_-_Privacy_leaks_Extracting_data_from__Iit4SmzQ2Uo_35580_36179.png Dennis_Giese_-_Privacy_leaks_Extracting_data_from__Iit4SmzQ2Uo_36180_38171.png Dennis_Giese_-_Privacy_leaks_Extracting_data_from__Iit4SmzQ2Uo_38172_39635.png Dennis_Giese_-_Privacy_leaks_Extracting_data_from__Iit4SmzQ2Uo_39636_42047.png Dennis_Giese_-_Privacy_leaks_Extracting_data_from__Iit4SmzQ2Uo_42048_42767.png Dennis_Giese_-_Privacy_leaks_Extracting_data_from__Iit4SmzQ2Uo_42768_44375.png Dennis_Giese_-_Privacy_leaks_Extracting_data_from__Iit4SmzQ2Uo_44376_45431.png Dennis_Giese_-_Privacy_leaks_Extracting_data_from__Iit4SmzQ2Uo_45432_45995.png Dennis_Giese_-_Privacy_leaks_Extracting_data_from__Iit4SmzQ2Uo_45996_46475.png Dennis_Giese_-_Privacy_leaks_Extracting_data_from__Iit4SmzQ2Uo_46476_47351.png Dennis_Giese_-_Privacy_leaks_Extracting_data_from__Iit4SmzQ2Uo_47352_48671.png Dennis_Giese_-_Privacy_leaks_Extracting_data_from__Iit4SmzQ2Uo_48672_48719.png Dennis_Giese_-_Privacy_leaks_Extracting_data_from__Iit4SmzQ2Uo_48720_49763.png Dennis_Giese_-_Privacy_leaks_Extracting_data_from__Iit4SmzQ2Uo_49764_50591.png Dennis_Giese_-_Privacy_leaks_Extracting_data_from__Iit4SmzQ2Uo_50592_52319.png Dennis_Giese_-_Privacy_leaks_Extracting_data_from__Iit4SmzQ2Uo_52320_53171.png Dennis_Giese_-_Privacy_leaks_Extracting_data_from__Iit4SmzQ2Uo_53172_53711.png Dennis_Giese_-_Privacy_leaks_Extracting_data_from__Iit4SmzQ2Uo_53712_54299.png Dennis_Giese_-_Privacy_leaks_Extracting_data_from__Iit4SmzQ2Uo_54300_54971.png Dennis_Giese_-_Privacy_leaks_Extracting_data_from__Iit4SmzQ2Uo_54972_56939.png Dennis_Giese_-_Privacy_leaks_Extracting_data_from__Iit4SmzQ2Uo_56940_57695.png Dennis_Giese_-_Privacy_leaks_Extracting_data_from__Iit4SmzQ2Uo_57696_58403.png Dennis_Giese_-_Privacy_leaks_Extracting_data_from__Iit4SmzQ2Uo_58404_59279.png Dennis_Giese_-_Privacy_leaks_Extracting_data_from__Iit4SmzQ2Uo_59280_60707.png Dennis_Giese_-_Privacy_leaks_Extracting_data_from__Iit4SmzQ2Uo_60708_61751.png Dennis_Giese_-_Privacy_leaks_Extracting_data_from__Iit4SmzQ2Uo_61752_62051.png Dennis_Giese_-_Privacy_leaks_Extracting_data_from__Iit4SmzQ2Uo_62052_62099.png Dennis_Giese_-_Privacy_leaks_Extracting_data_from__Iit4SmzQ2Uo_62100_63323.png Dennis_Giese_-_Privacy_leaks_Extracting_data_from__Iit4SmzQ2Uo_63324_64199.png Dennis_Giese_-_Privacy_leaks_Extracting_data_from__Iit4SmzQ2Uo_64200_65471.png Dennis_Giese_-_Privacy_leaks_Extracting_data_from__Iit4SmzQ2Uo_65472_66347.png Dennis_Giese_-_Privacy_leaks_Extracting_data_from__Iit4SmzQ2Uo_66348_67631.png Dennis_Giese_-_Privacy_leaks_Extracting_data_from__Iit4SmzQ2Uo_67632_68843.png Dennis_Giese_-_Privacy_leaks_Extracting_data_from__Iit4SmzQ2Uo_68844_69167.png Dennis_Giese_-_Privacy_leaks_Extracting_data_from__Iit4SmzQ2Uo_69168_69490.png