Fuzzing the phone in the iPhone
CC Attribution 4.0 International:
You are free to use, adapt and copy, distribute and transmit the work or content in adapted or unchanged form for any legal purpose as long as the work is attributed to the author in the manner specified by the author or licensor.
Classen, Jiska
How secure is the interface between baseband chips and iOS? While this interface should protect against escalations from the baseband into operating system components, its implementation is full of bugs. Fuzzing this interface is not only relevant to security, but also results in various funny effects, since the iPhone looses information about its identity and location, and eventually ends up in a state with a few thousand unread SMS that can no longer be deleted. The baseband chip is the phone within a smartphone. While users might disable Bluetooth or not join untrusted Wi-Fis to increase security, mobile data and phone calls are always on. Some baseband features remain enabled even without an active data pass or SIM card installed. Thus, baseband chips are a popular target for Remote Code Execution (RCE) attacks. Launched over-the-air, they do not leave any trace on intermediate servers. With recently published tools, emulating and fuzzing baseband firmware has become accessible and gained a lot of attention by security researchers. Yet, a critical part of an exploit chain remains escalating from the baseband chip into the operating system. An attacker with code execution in the baseband chip could modify network traffic and escalate with a Web browser exploit. However, wireless communication is already susceptible to manipulation without a baseband exploit due to missing user data integrity protection, as shown by the IMP4GT attacks on LTE. Moreover, network traffic manipulation still requires the attacker to wait until their target creates traffic in the context of a Web browser. More stealthy attacks can be achieved via baseband activity that does not require any user interaction, such as retrieving information about nearby base stations. On an iPhone, the interface forwarding this information comes in two variants for Intel and Qualcomm baseband chips. Both interfaces interact with CommCenter, which is the phone component of the iPhone that parses and forwards phone calls, SMS, and more. This talk demonstrates how to fuzz these interfaces as well as various challenges and solutions in fuzzing these. The baseband interface is such an integral part of iOS that fuzzing results are not limited to finding issues within CommCenter but also led to discovery of bugs in various daemons connecting to it. Some of these issues might also be triggered by a rouge base station, though, fuzzing the baseband interface locally results in higher performance and more control. Fuzzing the baseband interface demonstrates fuzzing in a way that is easy to understand even for those who are not familiar with security research. The iPhone under test receives SMS so fast that it will not finish playingthe Dimm notification sound before the next SMS arrives, and, thus, this actually sounds like D-d-d-di-di-d-d-di-d-di-d-di-d-dimm! The sound effect when the audio controller hangs up is also quite fun. Exposure notifications are no longer available due to the location change, since the iPhone uses baseband information to determine its region. And due to all the confusion about its identity, it regularly asks for activation. Besides security analysis, understanding the baseband interface means that it can also be controlled on jailbroken iPhones. Moreover, passive observations are possible on non-jailbroken devices with a baseband debug profile. This opens up these devices for new wireless research opportunities.
 Hello everyone and welcome to my talk, Puzzling the Phone and the iPhone. The phone and the iPhone is the component that receives SMS, sends SMS, receives phone calls, makes phone calls and also manages your internet connection when you are not on Wi-Fi. However, you might now wonder what is it exactly? So I'm talking about Comcenter and fuzzing it via the QMI and ARI interfaces, but this is a bit too technical for most of you. So I will first introduce you to the concept of fuzzing in general and protocol fuzzing before I dive into further details. For those of you who have not yet heard about the concept of fuzzing, you can send a lot of random messages and then try to test the security of an interface with this. And in this video, you can see how I sent SMS over a three-day-based fuzzer with something like 400 fast cases per second. And then the IMH and Perceives them, catches them and sends a couple of them also to the smartphone. Let's dive into the motivation and an explanation to the attacker model. So if you look into a modern smartphone, you have two components. If you want to show it in a simple way, so a boost of all the RST hardware part with a lot of chips. And then on top of this, there's an operating system and applications. However, it's not as simple as this because even those chips are so complex that they run their own little real-time operating systems to preprocess data. So this means that you can even get code execution on such a chip. And this is usually much easier than in the operating system itself because those chips cannot have that many mitigations. However, so what do you even do if you have code execution in such a chip? So if you are in a baseband chip, then one escalation strategy from the chip towards the operating system might be to manipulate traffic in the browser. However, I don't think that this is the case because if you look at the zero-diam price list, then actually the browser exploits are much more expensive. So it's probably not done like this. And there must be other ways to escalate from this chip into the operating system. In general, the traffic manipulation is something that you can always do in wireless transmission or also on the internet. So if you look how those systems work these days, so you have something like the internet in general that serves websites and so on and also the core network of your mobile provider. And there are many, many ways to manipulate traffic, either if you are a state-level actor who is able to have something in the core network or just by sending around websites or modifying websites. And then there is the base station subsystem. There might also be dragons, we don't know exactly. And of course, there are over-the-air transmissions. And wireless transmissions are very special because if there is something just slightly broken in the encryption, for example, then it's also possible to manipulate traffic there if you have a software-defined radio, for example. So all of this could be attacked to manipulate traffic. And I don't think that for this one would craft a baseband exploit. Already in 2014 at the CCC, there have been two talks about the SS7 protocol, which is run in the core network and is actually meant to connect different mobile carriers to each other. And this can also be used to intercept phone calls, for example. And this also has been exploited recently. So even though there have been some mitigations, et cetera, since then, it's still exploited for the same purpose to spy on people. So really, really, really, basement exploits only exist to escalate from the chip into the operating system. But now the question is, what are the strategies? So if it's not via the browser, what else could it be? So the browser, really, I'm sure it is not because also you need to have some traffic and so on. It doesn't really work instant. You need to visit a website to replace traffic on the website and so on. So yeah, there must be something else. So if you are on the chip with remote code execution and want to go into the operating system, there is some interface. And this means that something in those interfaces needs to be exploitable so that you can escalate the privileges from the chip into the system. And also, those interfaces are very interesting from a reverse engineer's perspective. So even if you don't want to attack anything, just understanding how they work is also a goal of this work. So for example, if you have a basement debug profile, you can just download this onto your iPhone and then you open your iDevice syslog. You can already see a lot of management messages that are exchanged between the chip and the iPhone. And if you have a jailbreak and Frida, you can even inject packets or modify packets to change the behavior of your modem. But if you want to start to work on such a thing, the question is like, how do we even start? Where do you start? Well, browsing is actually a method that can be used to understand such an interface. So initially, if you identified an interface just to check if it is the correct interface, so if it really changes behavior, if you flip some bytes, but also how powerful this interface is, so what are the features, what breaks instantly. And if things break, also, you can check if the whole interface has been designed with security in mind. Now let's start with an introduction to wireless protocol fuzzing. This will also be a short rant because the current tooling for fuzzing is usually not made to fuzz a protocol. So let's start with a very simple fuzzer, a fuzzer that is just an image parser. So you browse your smartphone for unicorn pictures or PNGs or JPEGs, and then you send them to the image parser. And in the image parser, you might be able to observe which functions are executed in the form of basic blocks. And then during this initialization, the image parser can even report which bytes were executed and you can just start the image parser again and again with different images and get this basic block coverage back. In the next step, you can then combine existing images or flip bits in these images and send them to the image parser and again observe the coverage. Most of the time, it won't generate any new coverage. So you just say you are not looking into this image in particular, but sometimes you might get new coverage like here and then you add this image to your corpus. So over time, you will increase your corpus and increase your coverage. Another method can be if you know how exactly an image format looks like. So you might know the JPEG specification and because of this, you could just generate images that are more or less specification compliant and they look more artificial like this. So you just generate images and send them to the image parser. And at some point, you might observe a crash. So that also depends again on your harnessing. Maybe you can observe basic blocks. Maybe you can just observe crashes. And then you know at which image you had a crash. You might even be able to combine these two approaches just depending on what you know about your input and how you can harness your target. Now it looks a bit different for a protocol. So in a protocol, you can have a very complex state. Let's say you are in an active phone call or just something like you receive an SMS. You can actually force the iPhone to receive SMS if you have a second iPhone and send SMS. And then during the fuzzing, you can replace some bits and bytes like this. And then you would have a modification. So this is a very simple approach and it preserves the state. So no matter how complex the thing is that you're currently doing, it's very simple to flip a bit here and there in an active interaction. But it's also a bit annoying because you need to have these active phone calls, etc. So something that's more efficient is injection. So you would observe certain messages and then just send them again. And then you don't even need this second phone to make calls, etc. You can just send a lot, a lot, a lot of data. And this is the effect when your iPhone goes to the dim or something because of all the notifications and all the data that is sent. But the issue here is that this does not preserve state. So there might be actions where the iPhone requests something that is then answered. So the iPhone might request, for example, a date and only then the chip would reply with a date and only then the iPhone would accept the date. But it's still very interesting to do this. So even though you cannot reach certain states because you can do this without a SIM card and you can do this very, very fast. So just to summarize the issues here, if you pass a virus protocol, you can have very significant state differences and just injecting packets cannot reach all states. The fact that you cannot reach all states also shows in very simple stuff like a trace replay. So a trace is something that you record. So let's say I have an active phone call, I record all the packets. And I can also observe the coverage. So with Frida, you can observe coverage on an iPhone while the phone call is active. And then in a second step, you would do some injection. But the only thing that you can inject are the packets sent from the baseband to the smartphone, not the opposite direction. And this resides usually in much less coverage. So you are missing a lot of things due to a missing state. And even worse, if you do the same thing again, you might be in a different state and you might observe a different coverage. So you do the exact same thing, but you get different coverage. So even replaying recorded messages results in less or inconsistent coverage. Anyway, let's take a look into an injection example. So in this video, you can see how I'm in the Unicor network on an iPhone 8, which has obviously 5G, but also does a lot of fuzzing. And in the fuzzing, what is interesting is that you might do a lot of states in a combination that are not usually possible, like you have a lost network connection, while you have to confirm a pin or you have a network connection during this, et cetera. So to summarize my rant, some states cannot be reached solely by injecting packets. So even if we have a very good corpus and do very good mutations, we might just miss 80% of the code, but we can just fuzz anyway. But we need to keep in mind that some stuff is just not fuzzable. We looked into a lot of virus protocols at CMO in the past, so it's worth to also consider which tooling we already had available for fuzzing protocols. The most advanced tooling that we have is Frankenstein and it's built by Jan. So what Jan did is he emulated a firmware and attached it to a virtual modem and also a Linux host. For this, he first looked into the firmware that's here and we had some partial symbols for this and also some information about registers. Then Frankenstein is actually taking a snapshot that you can see here, including some of those registers of the modem. And with this, you can build a virtual modem and fuzz input as if it would come over the air. Then Frankenstein also emulates the whole firmware, including thread switches, so it gets into very complex states and it's even attached to a Linux host. So it also fuzzes a bit of Linux while actually fuzzing the firmware itself. Now the issue with this is that basement firmware is usually a 10 times the size of Bluetooth firmware or even more and we don't have any symbols for this. So it's a lot of work to customize this. And even if one would do all those steps and put all the work into this, it's only, so to say, code execution in the basement. It's not yet a privileged escalation into the operating system. The next interesting tooling was built by Stefan and what Stefan did, he built a fuzzle based on detrace and AFL. Detrace is a tool that can provide function level coverage in the macOS kernel and user space. With some modifications, you can even get basic block coverage in the user space, which is required for AFL to work. So in the end, you have AFL or AFL++ as a fuzzle on any program on macOS. It's even slightly faster than Frida, at least the version that he used. And he gets a couple of thousand fuzz cases per second, even on a very old iMac. So yeah, in our lab, we just had an old iMac 2012 for this and it works on this. But the issue is that Wi-Fi and Bluetooth, which he fuzz are very complex protocols, so he couldn't find any new bugs with AFL. And also in the kernel space, you only get this function level coverage. He still, despite not finding any bugs in Wi-Fi or Bluetooth, got a CVE because detrace also has bugs. So at least some finding. But on iOS, this is not supported out of the box. So it might be possible to get detrace working with some tweaks, but it's a lot of work. So probably it's easier to just use Frida in the iOS user space. Also during this, so while Stefan was building all this very advanced tooling, Wang Yu found issues in the macOS Bluetooth and Wi-Fi drivers. And so he was very, very successful in comparison to us. That's really a pity. And I think what he did is much better state modeling of how the messages interact and what is important to reach certain functions. So what is still left? So usually fuzzing the baseband means that you need to modify firmware or also emulate firmware. You need to implement very complex specifications on a subtly defined radio if you want to fuzz over the air or build proof of concepts. And for everything that's somewhat proprietary, you need to do protocol reverse engineering. So you can spend a lot of time and money just to do very, very basic research. Or well, you can also use Frida. So you can fuzz with Frida. And all you need to do for this is write a few lines of code in JavaScript. So I kid you not, the option is Frida. Dennis was the first in our team who has advised us as a thesis student who built a Frida-based fuzzer and it's called Toothpicker. It's based on Frida and Radamsa. So what it does is, well, it hooks into these connections or into the protocols of the Bluetooth daemon, you could also think of this upper part here as a one block. So the protocols are implemented in the Bluetooth daemon, but we want to fuzz certain protocol handlers. And to increase the coverage, he creates a virtual connection. So virtual connection holds a connection and pretends to the Bluetooth daemon that there would be an active connection to a device. And of course, the chip would then say, I don't know anything about this connection. So there are also some abstractions in here so that the connection is not terminated. So that's a very simple tool, but it really found a lot of bugs and issues. And even there were some issues in the protocols themselves that also apply to macOS. So it's not just iOS bugs, but also protocol bugs in macOS that Dennis found. And this really got me thinking because two picker runs with only 20 fuzz cases per second. So it's really, really slow. And we were still able to find Bluetooth vulnerabilities at this speed. So why is this? So first of all, if you try to fuzz Bluetooth over the air, then the over the air connections are terminated after something like five invalid packets. So over the air fuzzing is really, really inefficient and with Frida you can actually patch this function so it's gone. Then the virtual connections are very important factor. So they are really, really important for having coverage. It's still a lot of coverage that we miss during your play and fuzzing. But yeah, so it's really an advantage compared to the other fuzzing approaches where you just inject packets. And in addition, there is an issue here because if you have a virtual connection, it might be that this virtual connection triggers behavior that you cannot reproduce over the air. So that means that everything that you find, you need also to confirm that it works over the air. At least inconsistent coverage is also fixed in Toothpicker because Toothpicker replace all packets five times in a row. But the issue here is that it also means that if you have a sequence of packets that is like generating a certain bug, so you need multiple packets, this is nothing that the mutator is aware of and also nothing that's locked properly in Toothpicker. And because of this, I got a bit anxious like, so yeah, maybe we missed a lot of things. So once I got the intuition that we are actually missing certain state information, I had the idea to replace bytes in active connections. And this is one project that you can see on a keyboard. So I'm just replacing bytes on keyboard input and see what happens. And I let this run for a couple of weeks, also for different protocols and so on to see if there are further bugs or not that we didn't find previously. So here you can see the same for airports with SCO and then they produce some crack sounds for the replaced bytes. It's even worse for ACL, so actual music because then you can hear very noisy chirps. I let this further run for multiple weeks and it didn't find any bugs that Toothpicker hadn't discovered before. So I think the reason for this is that I mainly passed in active connections like the one with the audio or the keyboard, but I only passed a few active pairings because this requires me to actually perform those pairings by hand. So nothing really interesting. The only bad thing that I could produce with this but not worth a CVE is that the sound quality of my airports is now really, really bad. Well, okay, and also the Broadcom chips on iOS don't check the UI length, but that's not that bad. So I mean, if you consider that they removed the right RAM recently, then you might now still be able to write into the RAM via UI, buffer overflows, but yeah, nothing too interesting. So after all of this, I asked myself what is still left for fuzzing if we cannot find a new Bluetooth or Wi-Fi box? Well, the iPhone baseband or actually the iPhone basebands because there are two. The first variant of iPhone baseband that you can get are the Qualcomm chips and they are in the US devices. They use the Qualcomm MSM interface and this interface comes with some documentation and there are even open source implementations for it. So it's something that's probably easy to understand and easy to pass. On the other hand, in almost all devices that I had on my table were Intel chips. Intel has been recently bought by Apple at least the part that does the baseband chips and these are the chips in the European devices. That's the reason why almost all my devices had Intel chips and they use a special protocol. It's called Apple Remote Invocation and if you search for this on the internet, I even checked it like just today, there are no Google hits at all. So it really hasn't been researched before, at least not publicly. It's completely undocumented and it's a very custom interface. So it's not even used for Android. It's really an interface just for Apple. The component that we are going to fuzz in the following is Comcenter. So Comcenter is the equivalent of, for example, the Bluetooth or Wi-Fi Demon, but for Telephony. It's sandboxed as the user wireless, but it comes with a lot of XPC interfaces and this is something that we will also see later in the fuzzing results. The next part is that there are two flavors of libraries. So depending on if you have a Qualcomm or an Intel chip, different libraries will be used before certain actions or data actually is then processed by the Comcenter itself. So we have different code paths here. But all of this runs in user space and this means that both libraries can be hooked with Frida and can be fuzz with Frida. So that's very interesting. There's still a lot of stuff that goes on in the kernel, so what you can see here is that Qmai and Ari have some management information that is sent to Comcenter, but they don't contain the raw network or audio data. So they don't contain your phone call. They don't contain your website that you are opening. And the next issue is that Qmai and Ari are not directly sent over the air, but what is sent over the air are normal baseband interactions and these generate Qmai and Ari messages. So there's still some section in between, but of course there are now two ways. Either you have interaction that you can do over the air that is causing Ari and Qmai messages directly that are something that causes an issue in the upper layers or you might have this full exploit chain requirement that you first need to exploit the chip over the air and then from the chip break the interface into the Comcenter. Now Qmai, the code has a lot of assertions. So it's really asserting everything about the protocol, the length, the TLV format and so on. And if anything goes wrong, it really terminates Comcenter. So if you just send one invalid packet, Comcenter is terminated. This doesn't matter a lot because if your protocol is stable and you usually don't send any invalid packets, then you know an attack is ongoing. So it's valid to terminate the Comcenter. And furthermore, it doesn't matter a lot to the user. So the worst thing that happens when Comcenter crashes, for example, where you have an active phone call, it's just that the phone call gets lost or your LTE connection is reestablished. So you don't really notice it. It just feels like your internet connection breaks for a short moment. In contrast, there is the Ari protocol and this is a Pyrza that works just very, very, very different. So whatever it's getting, it just pyrzes it and it doesn't terminate Comcenter. So you can send many, many, many fancy things and it just continues, continues, continues because the developers were probably very, very happy once they got their special protocol for Apple working and then they never touched it again. But what does it look like? So it has a very basic format also with some TLEs. And the first thing that I noticed when I fasted is that in the iDevices log, it always complained about this sequence number being wrong. So it just said, I expected to follow up sequence number so and so. So I started to fix this. And if you open it in Ida, you can see that the range that is expected is between 0 and 7FF hexadecimal. So you know at least the range. And then it gets weird. So the sequence number is spread over three different bytes in single bits and shifted around and so on and it's not even continuous. So very weird code. Probably they just added those sequence numbers to confirm some race conditions or something I really don't know or out of order packets. Something weird going on there. But I wrote the code, I fixed the sequence number and then during the replay of packets I noticed well, it doesn't even matter. It doesn't matter if your sequence number is valid or invalid, parsing continues and even worse, even packets with the wrong sequence number are parsed. Probably because otherwise there would be too many issues because the protocol implementation is too buggy. And there are also a couple of other things. So for example, if you send the first four magic bytes wrong or a wrong length or something, then the packet is potentially ignored. But the parsing continues and commCenter is not terminated like in QMI. Since it's a proprietary protocol, there's currently no tooling available, but Tobias is working on a Weischach sector and once he finishes his thesis, it will also be publicly released. So you need to wait a while, but then you will have a tool for this. Anyway, let's also talk about parsing this. So I would not recommend to pass this because you might pre-cure device or at least get into Weischach state. So just don't do this on your productive iPhone. I mean, obviously I know what I'm doing. So yeah, just parsing packets, right? But I'm not so sure about what exactly I'm doing. So the only direction that I pass is from the baseband to the iPhone here, not the opposite direction. So I hopefully do prevent anything weird on the chip, right? But the iPhone might still answer with something invalid and this might confuse the baseband or cause other crashes. And so I actually had to call for help like me, me, me, me, me. I brought my iPhone. I mean, just one of my research devices, but still, so it booted into PongoOS, but no longer into iOS and it didn't tell me any debug message that was useful. Well, it turns out at least on the Qualcomm chips and that's where this happens. It just boots after a couple of hours again, but before it's just entering a boot loop. And on the Intel iPhones, I also almost break the 918, but luckily it didn't completely break. So the issue there is if you enable the baseband debug profile, then it writes a lot of stuff to the ISTP files. So that's some debug format of Intel. And every few minutes, it's just create something like 500 megabytes of data, at least on the iPhone 8. On the newer iPhones, this debug format is a bit shorter, so it doesn't create as much data, but still a lot. And if you don't delete this regularly, then of course your disk will be full and an iPhone behaves quite strange if it has a full disk. So you can still interact with the user interface, but you can no longer delete photos because deleting a photo, it seems it just needs some file interaction. Also you can no longer log in with SSH, which is also an issue because it somehow seems to create a file when logging in, so you can no longer delete any files. And I was just rebooting the iPhone after trying a couple of things, and luckily it came back and deleted some files and I was able to log in and remove the baseband logs, but be careful when doing this. And of course, all the iPhones are very confused from the fudging, so they really lose everything about their identity and location and they want to be activated again. So here you can see a smartphone that lost its location and really wants to be activated, activated, activated. During SMS fuzzing you might even get LESH messages and if you click on the head menu on the direct theme, they are displayed black on gray, so probably nobody ever tested it. Also great if you have a locked iPhone, you can still display SIM menus and SIM messages on top of the lock. Okay, so I guess I have to revise my first instruction, so pass this, really, really fast this. It's a lot of fun, maybe just not on your primary device, but you will enjoy fuzzing these interfaces. But first of all, you obviously need to build a fuzzer, so how do you build a fuzzer? The first fuzzer that I used was the one that I also used for Bluetooth that just uses the existing byte stream of the protocol and then flips single bits and bytes. So it has this high state awareness, but it also means that like some kind of monkey I was just calling myself, writing SMS to myself, enabling flight mode, everything that you could just imagine and it's a very boring task. But it also found very fancy bugs that I couldn't reprove with the other fuzzers yet, because it can reach states that just injection of packets cannot reach. So at least it was quite successful. And well, I fast-whiped this for something like three days and it already found bugs. That's very different to the Bluetooth fuzzer, so there seem to be more bugs in Comcenter. And so I just wrote to Apple, yeah, hi there, I wrote this really, really ugly 10 lines of code fuzzer and see what it found, awesome, awesome, awesome and crash logs are attached. And obviously this is simple to reproduce because I only fast for three days, got most of these crashes multiple times. So here you go and draw my fuzzer. And this was probably quite stupid because it's not that simple. So it's really not easy to reproduce the crashes. Most of all, well, of course, the script is so generic that it runs on all iPhones with an Intel chip. So no matter if I take an iPhone 7 or an iPhone 11, it will just work. But the crash logs that you get are very different depending on if you fast on pre-A12, so iPhone 7 and 8 or on the later versions like the iPhone 11 and SE2. So you cannot reproduce the same crash logs that easy. And also it depends a lot on the SIM, so even on a passive iPhone, if you don't do any active phone calls and so on, you would get different results. So I started my parsing actually with a Singaporean SIM card without any data contract or phone contract on top of it. And it already found a couple of things. But yeah, it might just behave very different on just a slightly different configuration. Anyway, let's listen to a null pointer that it found. And this null pointer has been fixed in iOS 14.2 and it's in the audio controller. So you can hear some loop going on there. What you can see here is me calling the Deutsche Telekom and so on, so they have this variable in text. Guten Tag und herzlich willkommen beim Kundenservice der Telekom. And then I call again and have a crash. And now let's listen to the crash. First what is that? In fact, I also recorded another one. So this one is with Aldi Talk. And now let's listen to a special offer by Aldi Talk in 3, 2, 1. Since these first parsing results were very promising, I decided to use the latest toothpicker version and extend it for parsing Arri. And I called it ice pickery because the little chips are also called ice. So I just cloned Dennis' latest toothpicker alpha, which is very, very unstable, but this one actually runs on the iPhone locally without any interaction with MacOS or Linux. So it doesn't need to exchange any payload via USB. And also it's using AFL++, which is a much faster mutated and radamsa. So from a speed consideration, this is a much better design. However, AFL++ didn't turn out to be the best fuzzer for a protocol. So most of the time it actually spent trying to brute-pause the first magic bytes, the first four bytes, because it tries to shorten inputs. It's also not aware of something like a packet order. So it was just brute-posing those first four bytes. And well, the next issue is that for some reason, if the first four bytes are invalid, the Arri parser slows down a lot. So I was suddenly down to something like less than 10 fast cases per second. And also there is no awareness of the ice picker in this case of the Arri host state. So Arri sometimes shuts down this interface if it thinks that something is very invalid. And the parser would just continue. So I looked into the ID by syslog after the parser couldn't find any new coverage for more than six hours. And I was wondering, what is the issue here? Is the implementation wrong or is it the fuzzer? And it really looks like the fuzzer is producing inputs that are not good for protocol fuzzing. Of course, this is stuff that you can optimize. So AFL++ can do a lot here. So you can tell it a bit how the protocol looks like and also get it to not brute-force the first four magic bytes. But for this, I would have to recompile the whole thing. And it was something that compiled on Dennis machine, but it didn't compile on my machine because I had my Xcode beta in a weird state. And well, of course, some of you might now say, yeah, just download and install a new Xcode. But this takes so long that actually writing the next fuzzer seemed to be easier. Still, this variant of ice picker was interesting to me because it was the first time when I saw that the fuzzer in the situation works, including coverage. And also my replay works across my multiple iPhone version. So my corpus collected on an iPhone SE 2 was replayable on an iPhone 7. So it was not useless in that sense, but I just decided to not use this configuration. So I just wrote a very simple fuzzer again, and I didn't do the porting of everything to run locally on iOS. I just kept the design a bit simpler or at least easier to code and had my fuzzer running on Linux and then using only freedom iOS. It cannot reproduce all the states and crashes that I observed with my very first fuzzer, but most crashes could be reproduced. I didn't do any coverage. I didn't do any smart mutations, just very stupid mutations. And basically I just did a very blind injection. But this was super fast. So instead of the 20 fast cases per second, I already had something like 400 fast cases per second on an iPhone 7, which was about the same speed or even faster than the AFL++ variant. And I can at least correct the length, field sequence number and so on before injecting the payload. Since it doesn't do that great mutations, at least I need to collect a good corpus with many sims, many cores. And I'm also logging the packet order with this. So it's at least a bear of a packet sequence in the sense of I can reproduce the sequence later on. I had this fuzzer running on a couple of iPhones in parallel for multiple weeks, and it found a lot of interesting crashes. So that's my go-to fuzzer. I still wanted to confirm that not collecting coverage wasn't an issue. So I also cloned a publicly released tooth picker, which definitely finds new coverage. And it's using the Radamsa Mutator, which is very, very slow, but it does a bit smarter mutations at least in terms of protocol fuzzing. It's still only a bear of single packets, and it's only using the same packet five times in the road, confirm coverage, et cetera. And also an issue is that it cannot catch a lot of the crashes of comms sentry. So it happens quite often that comms send the crashes. And then if you cannot catch the crash with Frida and everything crashes, then you need to start the fuzzer again. But you also need to delete the files in the corpus that led to the crash, because otherwise you would just run into the same crash very fast. So it needs a lot of babysitting. I also had a droning for a couple of weeks, but sadly it didn't find any new crashes. So at least I can be sure that fuzzing much slower, but with coverage is not any improvement. Still the mutations it creates are quite useful, as you can see in the following. So you can even see this phone number scrolling here and so on, so it generated a very long phone number correctly into some TLV structure here. And that's quite interesting to see. So this is something that you could not reach by just flipping bits and bytes. There's one big shortcoming that all of these fuzzers have, including the initial toothpicker, which is they don't have any kind of memory sanitization. So the framework that you would usually use in user space on iOS is the Manlox Stack Logging Framework. I even got this running for comms center, so it's a bit of command line juggling, but in the end you can enable Manlox Stack Logging also for comms center. The issue here is that it increases the memory usage a lot, and even if you configure comms center to have a higher memory allowance, it is so high that it's just immediately killed by the out-of-memory killer. So this doesn't work. Then there is also the PgManlog. It doesn't exist for iOS, it's just exist in Xcode. I got one of the Xcode libraries running on one of my iPhones. I have no idea if this is an expected configuration or not. At least I could execute smaller programs. And then when you use this on comms center, it just crashes with a lipgManlog error on parsing some of the configuration files very, very early when starting the comms center. So all of this didn't work, and this also means that the fuzzer cannot find certain bug types or crashes much later when encountering bugs. So all of the fuzzers that I created are not perfect, but at least they found a lot of different crashes. Let's look into this. I mean the first obvious number that you see here is the 42, so I stopped fuzzing after 42 crashes. At least crashes that I think are individual crashes and that are not caused by Frida, so I tried to filter out Frida crashes. And this corresponds to the total amount of crashes, but only some of them are replayable by either one or multiple packets. And for the replayable crashes, I can also check if they were fixed in recent iOS versions, so the most recent iOS 14.3 or not. Then I also marked two colors here because there is the Intel libraries, but there is also the Qualcomm libraries. And for the Qualcomm libraries, I didn't spend as much time fuzzing because I have less Qualcomm phones, but also all the asserts in the code prevent a lot of issues from being reached. So the libraries themselves have less issues and also within Comcenter less of the code that has improper state handling is reached. The Location Demon is marked also with a big gray box here because the Location Demon is similar to the Comcenter using some of the raw packet inputs and parses them, so it has special parses for Qualcomm and Intel. And it's also an interesting target because of this. Other than this, I got really a lot, a lot, a lot of different demons crashing. Some of them even with replayable behavior. So for example, there is the BioDest radio manager demon that you can just crash via one Intel packet, but this has been fixed. And then there is one interesting crash that I actually got via Qualcomm and Intel libraries. So in the mobile Internet sharing demon, this also has been fixed. And some of the crashes only happened via Qualcomm, but I'm not sure if that's like a Qualcomm specific thing or it's just randomness of the father. So the mobile Internet sharing demon has an issue where it accesses memory at configuration strings. So there's different strings in this memory address. And I found this quite early, but I was not aware of the fact that so many other demons are actually crashing when I fast-com center. So I didn't look into this in the very beginning. And when I reported it to Apple, they said, yeah, yeah, we already know about this and we fixed it in a better prior to report. So sadly, nothing that I got a CVE for. Another interesting crash is in the cell monitor, but only of the Intel library. The cell monitor is something that is running passively in the background all the time and it passes, for example, GSM and UMTS cell information. I already found this on the Singaporean Sim without any active data plan in my very first round of fuzzing and reported it back then to Apple. I don't know if it's triggerable over the air or not. So I guess it's something that you first need chip code execution for. And it has been fixed in iOS 14.2. And I wrote a lot of emails with Apple because I thought that they didn't fix it. And the reason for this is that both the GSM cell info and the UMTS cell info function, when they parse data, they have two different bugs. So I still got crashes in the same functions. And I thought, okay, same function, still a crash. The bug is not fixed. But actually it's very high quality code and it's just multiple bugs per function. And there is even one more issue in the cell monitor, even though I think the remaining bugs are very simple crashes. So nothing that could be exploitable at all, but still hints to the great code quality. And the same story is that there are even more bugs to be fixed. So most of them are probably just stability improvements, but some of them are still interesting. So let's see how this goes. So since I told that it's a very simple fuzzer, some of you might have already started coding those 10 lines of code for fuzzing while I continue talking and scrapped their old iPhones that they are willing to lose if something goes wrong. So how can we actually build a fuzzer that is performant and replicates some of the bugs that I found just within a day? Let's take a look. When you do freedom fuzzing, a lot of the stuff that you do is limited by the processing power of the iPhone. So your iPhone will get very, very, very hot and it might even drain more battery than it can get via the USB port. So it might even discharge by fuzzing and performance is really key. So you need to identify bottlenecks. So yeah, I said toothpicker or ice picker. The initial version is just 20 fast cases per second. And you can tune this to something like 20,000 fast cases per second. So I already told that I tuned it to something like 400 or 500 fast cases per second, but why the 20,000? So initially a student of mine did some fuzzing in a very different part of it and said on my iPhone 6S, it's running with 20,000 fast cases per second. I was like, no way, no way. But actually you can do this. So this depends a lot on the freedom design. The first variant, how most freedom scripts are written is that you have some Python script that runs on Linux or Mac OS and it has a couple of functions that you can see here. So first of all, it has this on message callback. So this on message callback is something that we need later and we just register it to our free descript. The free descript I'm going to show you in a second and you load the script and the script can then even call functions on your iPhone. For this, you load a second script on your iPhone. So this is JavaScript injected into the iOS target process and it can, for example, use the send function to send something back to the on message function and it can export functions via IPC. So you can then call them. All this happens via JSON and so it needs serialization and deserialization, which means you cannot send hex data or binary data directly. So you have a hex string that you encode into JSON, which is then parsed as binary data. And also it's all via USB. So you also have to speed limitation by USB. And of course, if you use the free to see bindings locally on the iOS smartphone, it is a bit faster, but it's still not perfect. So the more you can prevent from this JSON part and USB part, the better. The actual pattern looks a bit like this. So you are in the lib-iris server. So that's the slowest library from the diagram before. And then you define this inbound message callback function, which has two arguments, which are the payload and the length. So this looks a bit cryptic, but that's basically it. And then you can, but you don't have to add this interceptor here because you might want to fix your sequence number or add basic block coverage to your father, et cetera. So this is also done there. And then you can just call this inbound message callback of Ari and send Ari payloads. So this already can be very different. So if you now call this wire RPC, export via a Python script on your laptop, you can reach something like 500 fast cases per second if you inject SMS, which are quite processing intensive payload. Or if you just do the same thing, and if you just run this inbound message callback in a loop locally with JavaScript without any external Python script, then you would get 22,000 fast cases per second on the very same device. So this is the speed difference that the JSON serialization, the serialization and the USB in between make. So I did a few more measurements. And sadly on the iPhone 8, there is a bug that prevents me from collecting coverage. But what you can see is so the first part here is if you have just a bit flipper in a loop that calls the target function, you can get 17,000 fast cases per second on an iPhone 7. As soon as you start collecting basic block coverage, not processing it, just collecting, you drop to 250 fast cases per second. So you need to ask yourself if your puzzle gets really that much better from collecting coverage. And another thing is that's this line above. So if you just print the packet that you first or injected and print this via Python to your laptop, you also have a huge slowdown, which is not as large as the coverage slowdown. But still you can see every print and every sending of a message in between a Python script and JavaScript takes a lot of time. Now if you have this remote SMS injection that I had before, then you drop to 400 fast cases per second. There's a blind injection without any coverage. If you collect coverage but don't process coverage, then you are down to 100 fast cases per second. So for the initial toothpick design, this would be the optimum. But because the radams and mutators very slow and because you also need to process the coverage information, et cetera, that's down to 20 fast cases per second. So this is the comparison here and now you can imagine why collecting coverage probably isn't always useful and why also having your laptop calculating better mutation because it's easier to write a mutator there than directly in JavaScript is not always the best idea. So let's watch one last demo video. What you can see here is when you try to delete SMS after all of the fuzzing, it really doesn't break neither by the settings nor via the SMS app. So you really need to reset your iPhone after fuzzing it for too long. No other chance than this to delete the messages. With this, we are already at the end of this talk. But of course, there will be a Q&A session and if you missed the Q&A session, you can also ask me on Twitter or write me an email. Thanks for watching. We'll see you in the next video.
 Hello everyone and welcome to my talk, Puzzling the Phone and the iPhone.  The phone and the iPhone is the component that receives SMS, sends SMS, receives phone  calls, makes phone calls and also manages your internet connection when you are not  on Wi-Fi.  However, you might now wonder what is it exactly?  So I'm talking about Comcenter and fuzzing it via the QMI and ARI interfaces, but this  is a bit too technical for most of you.  So I will first introduce you to the concept of fuzzing in general and protocol fuzzing  before I dive into further details.  For those of you who have not yet heard about the concept of fuzzing, you can send a lot  of random messages and then try to test the security of an interface with this.  And in this video, you can see how I sent SMS over a three-day-based fuzzer with something  like 400 fast cases per second.  And then the IMH and Perceives them, catches them and sends a couple of them also to the  smartphone.  Let's dive into the motivation and an explanation to the attacker model.  So if you look into a modern smartphone, you have two components.  If you want to show it in a simple way, so a boost of all the RST hardware part with  a lot of chips.  And then on top of this, there's an operating system and applications.  However, it's not as simple as this because even those chips are so complex that they  run their own little real-time operating systems to preprocess data.  So this means that you can even get code execution on such a chip.  And this is usually much easier than in the operating system itself because those chips  cannot have that many mitigations.  However, so what do you even do if you have code execution in such a chip?  So if you are in a baseband chip, then one escalation strategy from the chip towards  the operating system might be to manipulate traffic in the browser.  However, I don't think that this is the case because if you look at the zero-diam price  list, then actually the browser exploits are much more expensive.  So it's probably not done like this.  And there must be other ways to escalate from this chip into the operating system.  In general, the traffic manipulation is something that you can always do in wireless transmission  or also on the internet.  So if you look how those systems work these days, so you have something like the internet  in general that serves websites and so on and also the core network of your mobile provider.  And there are many, many ways to manipulate traffic, either if you are a state-level actor  who is able to have something in the core network or just by sending around websites  or modifying websites.  And then there is the base station subsystem.  There might also be dragons, we don't know exactly.  And of course, there are over-the-air transmissions.  And wireless transmissions are very special because if there is something just slightly  broken in the encryption, for example, then it's also possible to manipulate traffic there  if you have a software-defined radio, for example.  So all of this could be attacked to manipulate traffic.  And I don't think that for this one would craft a baseband exploit.  Already in 2014 at the CCC, there have been two talks about the SS7 protocol, which is  run in the core network and is actually meant to connect different mobile carriers to each  other.  And this can also be used to intercept phone calls, for example.  And this also has been exploited recently.  So even though there have been some mitigations, et cetera, since then, it's still exploited  for the same purpose to spy on people.  So really, really, really, basement exploits only exist to escalate from the chip into  the operating system.  But now the question is, what are the strategies?  So if it's not via the browser, what else could it be?  So the browser, really, I'm sure it is not because also you need to have some traffic  and so on.  It doesn't really work instant.  You need to visit a website to replace traffic on the website and so on.  So yeah, there must be something else.  So if you are on the chip with remote code execution and want to go into the operating  system, there is some interface.  And this means that something in those interfaces needs to be exploitable so that you can escalate  the privileges from the chip into the system.  And also, those interfaces are very interesting from a reverse engineer's perspective.  So even if you don't want to attack anything, just understanding how they work is also a  goal of this work.  So for example, if you have a basement debug profile, you can just download this onto your  iPhone and then you open your iDevice syslog.  You can already see a lot of management messages that are exchanged between the chip and the  iPhone.  And if you have a jailbreak and Frida, you can even inject packets or modify packets  to change the behavior of your modem.  But if you want to start to work on such a thing, the question is like, how do we even  start?  Where do you start?  Well, browsing is actually a method that can be used to understand such an interface.  So initially, if you identified an interface just to check if it is the correct interface,  so if it really changes behavior, if you flip some bytes, but also how powerful this interface  is, so what are the features, what breaks instantly.  And if things break, also, you can check if the whole interface has been designed with  security in mind.  Now let's start with an introduction to wireless protocol fuzzing.  This will also be a short rant because the current tooling for fuzzing is usually not  made to fuzz a protocol.  So let's start with a very simple fuzzer, a fuzzer that is just an image parser.  So you browse your smartphone for unicorn pictures or PNGs or JPEGs, and then you send  them to the image parser.  And in the image parser, you might be able to observe which functions are executed in  the form of basic blocks.  And then during this initialization, the image parser can even report which bytes were executed  and you can just start the image parser again and again with different images and get this  basic block coverage back.  In the next step, you can then combine existing images or flip bits in these images and send  them to the image parser and again observe the coverage.  Most of the time, it won't generate any new coverage.  So you just say you are not looking into this image in particular, but sometimes you might  get new coverage like here and then you add this image to your corpus.  So over time, you will increase your corpus and increase your coverage.  Another method can be if you know how exactly an image format looks like.  So you might know the JPEG specification and because of this, you could just generate  images that are more or less specification compliant and they look more artificial like  this.  So you just generate images and send them to the image parser.  And at some point, you might observe a crash.  So that also depends again on your harnessing.  Maybe you can observe basic blocks.  Maybe you can just observe crashes.  And then you know at which image you had a crash.  You might even be able to combine these two approaches just depending on what you know  about your input and how you can harness your target.  Now it looks a bit different for a protocol.  So in a protocol, you can have a very complex state.  Let's say you are in an active phone call or just something like you receive an SMS.  You can actually force the iPhone to receive SMS if you have a second iPhone and send  SMS.  And then during the fuzzing, you can replace some bits and bytes like this.  And then you would have a modification.  So this is a very simple approach and it preserves the state.  So no matter how complex the thing is that you're currently doing, it's very simple to  flip a bit here and there in an active interaction.  But it's also a bit annoying because you need to have these active phone calls, etc.  So something that's more efficient is injection.  So you would observe certain messages and then just send them again.  And then you don't even need this second phone to make calls, etc.  You can just send a lot, a lot, a lot of data.  And this is the effect when your iPhone goes to the dim or something because of all the  notifications and all the data that is sent.  But the issue here is that this does not preserve state.  So there might be actions where the iPhone requests something that is then answered.  So the iPhone might request, for example, a date and only then the chip would reply with  a date and only then the iPhone would accept the date.  But it's still very interesting to do this.  So even though you cannot reach certain states because you can do this without a SIM card  and you can do this very, very fast.  So just to summarize the issues here, if you pass a virus protocol, you can have very significant  state differences and just injecting packets cannot reach all states.  The fact that you cannot reach all states also shows in very simple stuff like a trace  replay.  So a trace is something that you record.  So let's say I have an active phone call, I record all the packets.  And I can also observe the coverage.  So with Frida, you can observe coverage on an iPhone while the phone call is active.  And then in a second step, you would do some injection.  But the only thing that you can inject are the packets sent from the baseband to the  smartphone, not the opposite direction.  And this resides usually in much less coverage.  So you are missing a lot of things due to a missing state.  And even worse, if you do the same thing again, you might be in a different state and you  might observe a different coverage.  So you do the exact same thing, but you get different coverage.  So even replaying recorded messages results in less or inconsistent coverage.  Anyway, let's take a look into an injection example.  So in this video, you can see how I'm in the Unicor network on an iPhone 8, which has obviously  5G, but also does a lot of fuzzing.  And in the fuzzing, what is interesting is that you might do a lot of states in a combination  that are not usually possible, like you have a lost network connection, while you have  to confirm a pin or you have a network connection during this, et cetera.  So to summarize my rant, some states cannot be reached solely by injecting packets.  So even if we have a very good corpus and do very good mutations, we might just miss  80% of the code, but we can just fuzz anyway.  But we need to keep in mind that some stuff is just not fuzzable.  We looked into a lot of virus protocols at CMO in the past, so it's worth to also consider  which tooling we already had available for fuzzing protocols.  The most advanced tooling that we have is Frankenstein and it's built by Jan.  So what Jan did is he emulated a firmware and attached it to a virtual modem and also  a Linux host.  For this, he first looked into the firmware that's here and we had some partial symbols  for this and also some information about registers.  Then Frankenstein is actually taking a snapshot that you can see here, including some of those  registers of the modem.  And with this, you can build a virtual modem and fuzz input as if it would come over the  air.  Then Frankenstein also emulates the whole firmware, including thread switches, so it  gets into very complex states and it's even attached to a Linux host.  So it also fuzzes a bit of Linux while actually fuzzing the firmware itself.  Now the issue with this is that basement firmware is usually a 10 times the size of Bluetooth  firmware or even more and we don't have any symbols for this.  So it's a lot of work to customize this.  And even if one would do all those steps and put all the work into this, it's only, so  to say, code execution in the basement.  It's not yet a privileged escalation into the operating system.  The next interesting tooling was built by Stefan and what Stefan did, he built a fuzzle  based on detrace and AFL.  Detrace is a tool that can provide function level coverage in the macOS kernel and user  space.  With some modifications, you can even get basic block coverage in the user space, which  is required for AFL to work.  So in the end, you have AFL or AFL++ as a fuzzle on any program on macOS.  It's even slightly faster than Frida, at least the version that he used.  And he gets a couple of thousand fuzz cases per second, even on a very old iMac.  So yeah, in our lab, we just had an old iMac 2012 for this and it works on this.  But the issue is that Wi-Fi and Bluetooth, which he fuzz are very complex protocols,  so he couldn't find any new bugs with AFL.  And also in the kernel space, you only get this function level coverage.  He still, despite not finding any bugs in Wi-Fi or Bluetooth, got a CVE because detrace  also has bugs.  So at least some finding.  But on iOS, this is not supported out of the box.  So it might be possible to get detrace working with some tweaks, but it's a lot of work.  So probably it's easier to just use Frida in the iOS user space.  Also during this, so while Stefan was building all this very advanced tooling, Wang Yu found  issues in the macOS Bluetooth and Wi-Fi drivers.  And so he was very, very successful in comparison to us.  That's really a pity.  And I think what he did is much better state modeling of how the messages interact and  what is important to reach certain functions.  So what is still left?  So usually fuzzing the baseband means that you need to modify firmware or also emulate  firmware.  You need to implement very complex specifications on a subtly defined radio if you want to fuzz  over the air or build proof of concepts.  And for everything that's somewhat proprietary, you need to do protocol reverse engineering.  So you can spend a lot of time and money just to do very, very basic research.  Or well, you can also use Frida.  So you can fuzz with Frida.  And all you need to do for this is write a few lines of code in JavaScript.  So I kid you not, the option is Frida.  Dennis was the first in our team who has advised us as a thesis student who built a Frida-based  fuzzer and it's called Toothpicker.  It's based on Frida and Radamsa.  So what it does is, well, it hooks into these connections or into the protocols of the Bluetooth  daemon, you could also think of this upper part here as a one block.  So the protocols are implemented in the Bluetooth daemon, but we want to fuzz certain protocol  handlers.  And to increase the coverage, he creates a virtual connection.  So virtual connection holds a connection and pretends to the Bluetooth daemon that there  would be an active connection to a device.  And of course, the chip would then say, I don't know anything about this connection.  So there are also some abstractions in here so that the connection is not terminated.  So that's a very simple tool, but it really found a lot of bugs and issues.  And even there were some issues in the protocols themselves that also apply to macOS.  So it's not just iOS bugs, but also protocol bugs in macOS that Dennis found.  And this really got me thinking because two picker runs with only 20 fuzz cases per second.  So it's really, really slow.  And we were still able to find Bluetooth vulnerabilities at this speed.  So why is this?  So first of all, if you try to fuzz Bluetooth over the air, then the over the air connections  are terminated after something like five invalid packets.  So over the air fuzzing is really, really inefficient and with Frida you can actually  patch this function so it's gone.  Then the virtual connections are very important factor.  So they are really, really important for having coverage.  It's still a lot of coverage that we miss during your play and fuzzing.  But yeah, so it's really an advantage compared to the other fuzzing approaches where you  just inject packets.  And in addition, there is an issue here because if you have a virtual connection, it might  be that this virtual connection triggers behavior that you cannot reproduce over the air.  So that means that everything that you find, you need also to confirm that it works over  the air.  At least inconsistent coverage is also fixed in Toothpicker because Toothpicker replace  all packets five times in a row.  But the issue here is that it also means that if you have a sequence of packets that is  like generating a certain bug, so you need multiple packets, this is nothing that the  mutator is aware of and also nothing that's locked properly in Toothpicker.  And because of this, I got a bit anxious like, so yeah, maybe we missed a lot of things.  So once I got the intuition that we are actually missing certain state information, I had the  idea to replace bytes in active connections.  And this is one project that you can see on a keyboard.  So I'm just replacing bytes on keyboard input and see what happens.  And I let this run for a couple of weeks, also for different protocols and so on to  see if there are further bugs or not that we didn't find previously.  So here you can see the same for airports with SCO and then they produce some crack  sounds for the replaced bytes.  It's even worse for ACL, so actual music because then you can hear very noisy chirps.  I let this further run for multiple weeks and it didn't find any bugs that Toothpicker  hadn't discovered before.  So I think the reason for this is that I mainly passed in active connections like the one  with the audio or the keyboard, but I only passed a few active pairings because this  requires me to actually perform those pairings by hand.  So nothing really interesting.  The only bad thing that I could produce with this but not worth a CVE is that the sound  quality of my airports is now really, really bad.  Well, okay, and also the Broadcom chips on iOS don't check the UI length, but that's  not that bad.  So I mean, if you consider that they removed the right RAM recently, then you might now  still be able to write into the RAM via UI, buffer overflows, but yeah, nothing too interesting.  So after all of this, I asked myself what is still left for fuzzing if we cannot find  a new Bluetooth or Wi-Fi box?  Well, the iPhone baseband or actually the iPhone basebands because there are two.  The first variant of iPhone baseband that you can get are the Qualcomm chips and they  are in the US devices.  They use the Qualcomm MSM interface and this interface comes with some documentation and  there are even open source implementations for it.  So it's something that's probably easy to understand and easy to pass.  On the other hand, in almost all devices that I had on my table were Intel chips.  Intel has been recently bought by Apple at least the part that does the baseband chips  and these are the chips in the European devices.  That's the reason why almost all my devices had Intel chips and they use a special protocol.  It's called Apple Remote Invocation and if you search for this on the internet, I even  checked it like just today, there are no Google hits at all.  So it really hasn't been researched before, at least not publicly.  It's completely undocumented and it's a very custom interface.  So it's not even used for Android.  It's really an interface just for Apple.  The component that we are going to fuzz in the following is Comcenter.  So Comcenter is the equivalent of, for example, the Bluetooth or Wi-Fi Demon, but for Telephony.  It's sandboxed as the user wireless, but it comes with a lot of XPC interfaces and this  is something that we will also see later in the fuzzing results.  The next part is that there are two flavors of libraries.  So depending on if you have a Qualcomm or an Intel chip, different libraries will be  used before certain actions or data actually is then processed by the Comcenter itself.  So we have different code paths here.  But all of this runs in user space and this means that both libraries can be hooked with  Frida and can be fuzz with Frida.  So that's very interesting.  There's still a lot of stuff that goes on in the kernel, so what you can see here is  that Qmai and Ari have some management information that is sent to Comcenter, but they don't contain  the raw network or audio data.  So they don't contain your phone call.  They don't contain your website that you are opening.  And the next issue is that Qmai and Ari are not directly sent over the air, but what is  sent over the air are normal baseband interactions and these generate Qmai and Ari messages.  So there's still some section in between, but of course there are now two ways.  Either you have interaction that you can do over the air that is causing Ari and Qmai  messages directly that are something that causes an issue in the upper layers or you  might have this full exploit chain requirement that you first need to exploit the chip over  the air and then from the chip break the interface into the Comcenter.  Now Qmai, the code has a lot of assertions.  So it's really asserting everything about the protocol, the length, the TLV format and  so on.  And if anything goes wrong, it really terminates Comcenter.  So if you just send one invalid packet, Comcenter is terminated.  This doesn't matter a lot because if your protocol is stable and you usually don't send  any invalid packets, then you know an attack is ongoing.  So it's valid to terminate the Comcenter.  And furthermore, it doesn't matter a lot to the user.  So the worst thing that happens when Comcenter crashes, for example, where you have an active  phone call, it's just that the phone call gets lost or your LTE connection is reestablished.  So you don't really notice it.  It just feels like your internet connection breaks for a short moment.  In contrast, there is the Ari protocol and this is a Pyrza that works just very, very,  very different.  So whatever it's getting, it just pyrzes it and it doesn't terminate Comcenter.  So you can send many, many, many fancy things and it just continues, continues, continues  because the developers were probably very, very happy once they got their special protocol  for Apple working and then they never touched it again.  But what does it look like?  So it has a very basic format also with some TLEs.  And the first thing that I noticed when I fasted is that in the iDevices log, it always  complained about this sequence number being wrong.  So it just said, I expected to follow up sequence number so and so.  So I started to fix this.  And if you open it in Ida, you can see that the range that is expected is between 0 and  7FF hexadecimal.  So you know at least the range.  And then it gets weird.  So the sequence number is spread over three different bytes in single bits and shifted  around and so on and it's not even continuous.  So very weird code.  Probably they just added those sequence numbers to confirm some race conditions or something  I really don't know or out of order packets.  Something weird going on there.  But I wrote the code, I fixed the sequence number and then during the replay of packets  I noticed well, it doesn't even matter.  It doesn't matter if your sequence number is valid or invalid, parsing continues and  even worse, even packets with the wrong sequence number are parsed.  Probably because otherwise there would be too many issues because the protocol implementation  is too buggy.  And there are also a couple of other things.  So for example, if you send the first four magic bytes wrong or a wrong length or something,  then the packet is potentially ignored.  But the parsing continues and commCenter is not terminated like in QMI.  Since it's a proprietary protocol, there's currently no tooling available, but Tobias  is working on a Weischach sector and once he finishes his thesis, it will also be publicly  released.  So you need to wait a while, but then you will have a tool for this.  Anyway, let's also talk about parsing this.  So I would not recommend to pass this because you might pre-cure device or at least get  into Weischach state.  So just don't do this on your productive iPhone.  I mean, obviously I know what I'm doing.  So yeah, just parsing packets, right?  But I'm not so sure about what exactly I'm doing.  So the only direction that I pass is from the baseband to the iPhone here, not the opposite  direction.  So I hopefully do prevent anything weird on the chip, right?  But the iPhone might still answer with something invalid and this might confuse the baseband  or cause other crashes.  And so I actually had to call for help like me, me, me, me, me.  I brought my iPhone.  I mean, just one of my research devices, but still, so it booted into PongoOS, but no longer  into iOS and it didn't tell me any debug message that was useful.  Well, it turns out at least on the Qualcomm chips and that's where this happens.  It just boots after a couple of hours again, but before it's just entering a boot loop.  And on the Intel iPhones, I also almost break the 918, but luckily it didn't completely  break.  So the issue there is if you enable the baseband debug profile, then it writes a lot of stuff  to the ISTP files.  So that's some debug format of Intel.  And every few minutes, it's just create something like 500 megabytes of data, at least on the  iPhone 8.  On the newer iPhones, this debug format is a bit shorter, so it doesn't create as much  data, but still a lot.  And if you don't delete this regularly, then of course your disk will be full and an iPhone  behaves quite strange if it has a full disk.  So you can still interact with the user interface, but you can no longer delete photos because  deleting a photo, it seems it just needs some file interaction.  Also you can no longer log in with SSH, which is also an issue because it somehow seems  to create a file when logging in, so you can no longer delete any files.  And I was just rebooting the iPhone after trying a couple of things, and luckily it  came back and deleted some files and I was able to log in and remove the baseband logs,  but be careful when doing this.  And of course, all the iPhones are very confused from the fudging, so they really lose everything  about their identity and location and they want to be activated again.  So here you can see a smartphone that lost its location and really wants to be activated,  activated, activated.  During SMS fuzzing you might even get LESH messages and if you click on the head menu  on the direct theme, they are displayed black on gray, so probably nobody ever tested it.  Also great if you have a locked iPhone, you can still display SIM menus and SIM messages  on top of the lock.  Okay, so I guess I have to revise my first instruction, so pass this, really, really  fast this.  It's a lot of fun, maybe just not on your primary device, but you will enjoy fuzzing  these interfaces.  But first of all, you obviously need to build a fuzzer, so how do you build a fuzzer?  The first fuzzer that I used was the one that I also used for Bluetooth that just uses  the existing byte stream of the protocol and then flips single bits and bytes.  So it has this high state awareness, but it also means that like some kind of monkey I  was just calling myself, writing SMS to myself, enabling flight mode, everything that you  could just imagine and it's a very boring task.  But it also found very fancy bugs that I couldn't reprove with the other fuzzers yet, because  it can reach states that just injection of packets cannot reach.  So at least it was quite successful.  And well, I fast-whiped this for something like three days and it already found bugs.  That's very different to the Bluetooth fuzzer, so there seem to be more bugs in Comcenter.  And so I just wrote to Apple, yeah, hi there, I wrote this really, really ugly 10 lines  of code fuzzer and see what it found, awesome, awesome, awesome and crash logs are attached.  And obviously this is simple to reproduce because I only fast for three days, got most  of these crashes multiple times.  So here you go and draw my fuzzer.  And this was probably quite stupid because it's not that simple.  So it's really not easy to reproduce the crashes.  Most of all, well, of course, the script is so generic that it runs on all iPhones with  an Intel chip.  So no matter if I take an iPhone 7 or an iPhone 11, it will just work.  But the crash logs that you get are very different depending on if you fast on pre-A12, so iPhone  7 and 8 or on the later versions like the iPhone 11 and SE2.  So you cannot reproduce the same crash logs that easy.  And also it depends a lot on the SIM, so even on a passive iPhone, if you don't do any active  phone calls and so on, you would get different results.  So I started my parsing actually with a Singaporean SIM card without any data contract or phone  contract on top of it.  And it already found a couple of things.  But yeah, it might just behave very different on just a slightly different configuration.  Anyway, let's listen to a null pointer that it found.  And this null pointer has been fixed in iOS 14.2 and it's in the audio controller.  So you can hear some loop going on there.  What you can see here is me calling the Deutsche Telekom and so on, so they have this variable  in text.  Guten Tag und herzlich willkommen beim Kundenservice der Telekom.  And then I call again and have a crash.  And now let's listen to the crash.  First what is that?  In fact, I also recorded another one.  So this one is with Aldi Talk.  And now let's listen to a special offer by Aldi Talk in 3, 2, 1.  Since these first parsing results were very promising, I decided to use the latest toothpicker  version and extend it for parsing Arri.  And I called it ice pickery because the little chips are also called ice.  So I just cloned Dennis' latest toothpicker alpha, which is very, very unstable, but this  one actually runs on the iPhone locally without any interaction with MacOS or Linux.  So it doesn't need to exchange any payload via USB.  And also it's using AFL++, which is a much faster mutated and radamsa.  So from a speed consideration, this is a much better design.  However, AFL++ didn't turn out to be the best fuzzer for a protocol.  So most of the time it actually spent trying to brute-pause the first magic bytes, the first  four bytes, because it tries to shorten inputs.  It's also not aware of something like a packet order.  So it was just brute-posing those first four bytes.  And well, the next issue is that for some reason, if the first four bytes are invalid,  the Arri parser slows down a lot.  So I was suddenly down to something like less than 10 fast cases per second.  And also there is no awareness of the ice picker in this case of the Arri host state.  So Arri sometimes shuts down this interface if it thinks that something is very invalid.  And the parser would just continue.  So I looked into the ID by syslog after the parser couldn't find any new coverage for more  than six hours.  And I was wondering, what is the issue here?  Is the implementation wrong or is it the fuzzer?  And it really looks like the fuzzer is producing inputs that are not good for protocol fuzzing.  Of course, this is stuff that you can optimize.  So AFL++ can do a lot here.  So you can tell it a bit how the protocol looks like and also get it to not brute-force  the first four magic bytes.  But for this, I would have to recompile the whole thing.  And it was something that compiled on Dennis machine, but it didn't compile on my machine  because I had my Xcode beta in a weird state.  And well, of course, some of you might now say, yeah, just download and install a new  Xcode.  But this takes so long that actually writing the next fuzzer seemed to be easier.  Still, this variant of ice picker was interesting to me because it was the first time when I  saw that the fuzzer in the situation works, including coverage.  And also my replay works across my multiple iPhone version.  So my corpus collected on an iPhone SE 2 was replayable on an iPhone 7.  So it was not useless in that sense, but I just decided to not use this configuration.  So I just wrote a very simple fuzzer again, and I didn't do the porting of everything  to run locally on iOS.  I just kept the design a bit simpler or at least easier to code and had my fuzzer running  on Linux and then using only freedom iOS.  It cannot reproduce all the states and crashes that I observed with my very first fuzzer,  but most crashes could be reproduced.  I didn't do any coverage.  I didn't do any smart mutations, just very stupid mutations.  And basically I just did a very blind injection.  But this was super fast.  So instead of the 20 fast cases per second, I already had something like 400 fast cases  per second on an iPhone 7, which was about the same speed or even faster than the AFL++  variant.  And I can at least correct the length, field sequence number and so on before injecting  the payload.  Since it doesn't do that great mutations, at least I need to collect a good corpus with  many sims, many cores.  And I'm also logging the packet order with this.  So it's at least a bear of a packet sequence in the sense of I can reproduce the sequence  later on.  I had this fuzzer running on a couple of iPhones in parallel for multiple weeks, and it found  a lot of interesting crashes.  So that's my go-to fuzzer.  I still wanted to confirm that not collecting coverage wasn't an issue.  So I also cloned a publicly released tooth picker, which definitely finds new coverage.  And it's using the Radamsa Mutator, which is very, very slow, but it does a bit smarter  mutations at least in terms of protocol fuzzing.  It's still only a bear of single packets, and it's only using the same packet five times  in the road, confirm coverage, et cetera.  And also an issue is that it cannot catch a lot of the crashes of comms sentry.  So it happens quite often that comms send the crashes.  And then if you cannot catch the crash with Frida and everything crashes, then you need  to start the fuzzer again.  But you also need to delete the files in the corpus that led to the crash, because otherwise  you would just run into the same crash very fast.  So it needs a lot of babysitting.  I also had a droning for a couple of weeks, but sadly it didn't find any new crashes.  So at least I can be sure that fuzzing much slower, but with coverage is not any improvement.  Still the mutations it creates are quite useful, as you can see in the following.  So you can even see this phone number scrolling here and so on, so it generated a very long  phone number correctly into some TLV structure here.  And that's quite interesting to see.  So this is something that you could not reach by just flipping bits and bytes.  There's one big shortcoming that all of these fuzzers have, including the initial toothpicker,  which is they don't have any kind of memory sanitization.  So the framework that you would usually use in user space on iOS is the Manlox Stack Logging  Framework.  I even got this running for comms center, so it's a bit of command line juggling, but  in the end you can enable Manlox Stack Logging also for comms center.  The issue here is that it increases the memory usage a lot, and even if you configure comms  center to have a higher memory allowance, it is so high that it's just immediately killed  by the out-of-memory killer.  So this doesn't work.  Then there is also the PgManlog.  It doesn't exist for iOS, it's just exist in Xcode.  I got one of the Xcode libraries running on one of my iPhones.  I have no idea if this is an expected configuration or not.  At least I could execute smaller programs.  And then when you use this on comms center, it just crashes with a lipgManlog error on  parsing some of the configuration files very, very early when starting the comms center.  So all of this didn't work, and this also means that the fuzzer cannot find certain  bug types or crashes much later when encountering bugs.  So all of the fuzzers that I created are not perfect, but at least they found a lot of  different crashes.  Let's look into this.  I mean the first obvious number that you see here is the 42, so I stopped fuzzing after  42 crashes.  At least crashes that I think are individual crashes and that are not caused by Frida,  so I tried to filter out Frida crashes.  And this corresponds to the total amount of crashes, but only some of them are replayable  by either one or multiple packets.  And for the replayable crashes, I can also check if they were fixed in recent iOS versions,  so the most recent iOS 14.3 or not.  Then I also marked two colors here because there is the Intel libraries, but there is  also the Qualcomm libraries.  And for the Qualcomm libraries, I didn't spend as much time fuzzing because I have less  Qualcomm phones, but also all the asserts in the code prevent a lot of issues from being  reached.  So the libraries themselves have less issues and also within Comcenter less of the code  that has improper state handling is reached.  The Location Demon is marked also with a big gray box here because the Location Demon is  similar to the Comcenter using some of the raw packet inputs and parses them, so it has  special parses for Qualcomm and Intel.  And it's also an interesting target because of this.  Other than this, I got really a lot, a lot, a lot of different demons crashing.  Some of them even with replayable behavior.  So for example, there is the BioDest radio manager demon that you can just crash via  one Intel packet, but this has been fixed.  And then there is one interesting crash that I actually got via Qualcomm and Intel libraries.  So in the mobile Internet sharing demon, this also has been fixed.  And some of the crashes only happened via Qualcomm, but I'm not sure if that's like  a Qualcomm specific thing or it's just randomness of the father.  So the mobile Internet sharing demon has an issue where it accesses memory at configuration  strings.  So there's different strings in this memory address.  And I found this quite early, but I was not aware of the fact that so many other demons  are actually crashing when I fast-com center.  So I didn't look into this in the very beginning.  And when I reported it to Apple, they said, yeah, yeah, we already know about this and  we fixed it in a better prior to report.  So sadly, nothing that I got a CVE for.  Another interesting crash is in the cell monitor, but only of the Intel library.  The cell monitor is something that is running passively in the background all the time and  it passes, for example, GSM and UMTS cell information.  I already found this on the Singaporean Sim without any active data plan in my very first  round of fuzzing and reported it back then to Apple.  I don't know if it's triggerable over the air or not.  So I guess it's something that you first need chip code execution for.  And it has been fixed in iOS 14.2.  And I wrote a lot of emails with Apple because I thought that they didn't fix it.  And the reason for this is that both the GSM cell info and the UMTS cell info function,  when they parse data, they have two different bugs.  So I still got crashes in the same functions.  And I thought, okay, same function, still a crash.  The bug is not fixed.  But actually it's very high quality code and it's just multiple bugs per function.  And there is even one more issue in the cell monitor, even though I think the remaining  bugs are very simple crashes.  So nothing that could be exploitable at all, but still hints to the great code quality.  And the same story is that there are even more bugs to be fixed.  So most of them are probably just stability improvements, but some of them are still interesting.  So let's see how this goes.  So since I told that it's a very simple fuzzer, some of you might have already started coding  those 10 lines of code for fuzzing while I continue talking and scrapped their old iPhones  that they are willing to lose if something goes wrong.  So how can we actually build a fuzzer that is performant and replicates some of the bugs  that I found just within a day?  Let's take a look.  When you do freedom fuzzing, a lot of the stuff that you do is limited by the processing  power of the iPhone.  So your iPhone will get very, very, very hot and it might even drain more battery than  it can get via the USB port.  So it might even discharge by fuzzing and performance is really key.  So you need to identify bottlenecks.  So yeah, I said toothpicker or ice picker.  The initial version is just 20 fast cases per second.  And you can tune this to something like 20,000 fast cases per second.  So I already told that I tuned it to something like 400 or 500 fast cases per second, but  why the 20,000?  So initially a student of mine did some fuzzing in a very different part of it and said on  my iPhone 6S, it's running with 20,000 fast cases per second.  I was like, no way, no way.  But actually you can do this.  So this depends a lot on the freedom design.  The first variant, how most freedom scripts are written is that you have some Python script  that runs on Linux or Mac OS and it has a couple of functions that you can see here.  So first of all, it has this on message callback.  So this on message callback is something that we need later and we just register it to our  free descript.  The free descript I'm going to show you in a second and you load the script and the script  can then even call functions on your iPhone.  For this, you load a second script on your iPhone.  So this is JavaScript injected into the iOS target process and it can, for example, use  the send function to send something back to the on message function and it can export  functions via IPC.  So you can then call them.  All this happens via JSON and so it needs serialization and deserialization, which means  you cannot send hex data or binary data directly.  So you have a hex string that you encode into JSON, which is then parsed as binary data.  And also it's all via USB.  So you also have to speed limitation by USB.  And of course, if you use the free to see bindings locally on the iOS smartphone, it  is a bit faster, but it's still not perfect.  So the more you can prevent from this JSON part and USB part, the better.  The actual pattern looks a bit like this.  So you are in the lib-iris server.  So that's the slowest library from the diagram before.  And then you define this inbound message callback function, which has two arguments, which are  the payload and the length.  So this looks a bit cryptic, but that's basically it.  And then you can, but you don't have to add this interceptor here because you might want  to fix your sequence number or add basic block coverage to your father, et cetera.  So this is also done there.  And then you can just call this inbound message callback of Ari and send Ari payloads.  So this already can be very different.  So if you now call this wire RPC, export via a Python script on your laptop, you can reach  something like 500 fast cases per second if you inject SMS, which are quite processing  intensive payload.  Or if you just do the same thing, and if you just run this inbound message callback in  a loop locally with JavaScript without any external Python script, then you would get  22,000 fast cases per second on the very same device.  So this is the speed difference that the JSON serialization, the serialization and the USB  in between make.  So I did a few more measurements.  And sadly on the iPhone 8, there is a bug that prevents me from collecting coverage.  But what you can see is so the first part here is if you have just a bit flipper in a loop  that calls the target function, you can get 17,000 fast cases per second on an iPhone  7.  As soon as you start collecting basic block coverage, not processing it, just collecting,  you drop to 250 fast cases per second.  So you need to ask yourself if your puzzle gets really that much better from collecting  coverage.  And another thing is that's this line above.  So if you just print the packet that you first or injected and print this via Python  to your laptop, you also have a huge slowdown, which is not as large as the coverage slowdown.  But still you can see every print and every sending of a message in between a Python script  and JavaScript takes a lot of time.  Now if you have this remote SMS injection that I had before, then you drop to 400 fast  cases per second.  There's a blind injection without any coverage.  If you collect coverage but don't process coverage, then you are down to 100 fast cases  per second.  So for the initial toothpick design, this would be the optimum.  But because the radams and mutators very slow and because you also need to process the coverage  information, et cetera, that's down to 20 fast cases per second.  So this is the comparison here and now you can imagine why collecting coverage probably  isn't always useful and why also having your laptop calculating better mutation because  it's easier to write a mutator there than directly in JavaScript is not always the best  idea.  So let's watch one last demo video.  What you can see here is when you try to delete SMS after all of the fuzzing, it really doesn't  break neither by the settings nor via the SMS app.  So you really need to reset your iPhone after fuzzing it for too long.  No other chance than this to delete the messages.  With this, we are already at the end of this talk.  But of course, there will be a Q&A session and if you missed the Q&A session, you can  also ask me on Twitter or write me an email.  Thanks for watching.  We'll see you in the next video.
rc3-mcr-11358-eng-deu-Fuzzing_the_phone_in_the_iPhone_hd_0_59.png rc3-mcr-11358-eng-deu-Fuzzing_the_phone_in_the_iPhone_hd_60_167.png rc3-mcr-11358-eng-deu-Fuzzing_the_phone_in_the_iPhone_hd_168_215.png rc3-mcr-11358-eng-deu-Fuzzing_the_phone_in_the_iPhone_hd_216_263.png rc3-mcr-11358-eng-deu-Fuzzing_the_phone_in_the_iPhone_hd_264_1223.png rc3-mcr-11358-eng-deu-Fuzzing_the_phone_in_the_iPhone_hd_1224_1307.png rc3-mcr-11358-eng-deu-Fuzzing_the_phone_in_the_iPhone_hd_1308_1403.png rc3-mcr-11358-eng-deu-Fuzzing_the_phone_in_the_iPhone_hd_1404_1511.png rc3-mcr-11358-eng-deu-Fuzzing_the_phone_in_the_iPhone_hd_1512_1907.png rc3-mcr-11358-eng-deu-Fuzzing_the_phone_in_the_iPhone_hd_1908_2027.png rc3-mcr-11358-eng-deu-Fuzzing_the_phone_in_the_iPhone_hd_2028_2303.png rc3-mcr-11358-eng-deu-Fuzzing_the_phone_in_the_iPhone_hd_2304_2387.png rc3-mcr-11358-eng-deu-Fuzzing_the_phone_in_the_iPhone_hd_2388_7823.png rc3-mcr-11358-eng-deu-Fuzzing_the_phone_in_the_iPhone_hd_7824_9671.png rc3-mcr-11358-eng-deu-Fuzzing_the_phone_in_the_iPhone_hd_9672_9959.png rc3-mcr-11358-eng-deu-Fuzzing_the_phone_in_the_iPhone_hd_9960_11579.png rc3-mcr-11358-eng-deu-Fuzzing_the_phone_in_the_iPhone_hd_11580_17135.png rc3-mcr-11358-eng-deu-Fuzzing_the_phone_in_the_iPhone_hd_17136_18527.png rc3-mcr-11358-eng-deu-Fuzzing_the_phone_in_the_iPhone_hd_18528_22907.png rc3-mcr-11358-eng-deu-Fuzzing_the_phone_in_the_iPhone_hd_22908_29315.png rc3-mcr-11358-eng-deu-Fuzzing_the_phone_in_the_iPhone_hd_29316_29399.png rc3-mcr-11358-eng-deu-Fuzzing_the_phone_in_the_iPhone_hd_29400_29447.png rc3-mcr-11358-eng-deu-Fuzzing_the_phone_in_the_iPhone_hd_29448_29699.png rc3-mcr-11358-eng-deu-Fuzzing_the_phone_in_the_iPhone_hd_29700_30059.png rc3-mcr-11358-eng-deu-Fuzzing_the_phone_in_the_iPhone_hd_30060_30215.png rc3-mcr-11358-eng-deu-Fuzzing_the_phone_in_the_iPhone_hd_30216_30515.png rc3-mcr-11358-eng-deu-Fuzzing_the_phone_in_the_iPhone_hd_30516_32423.png rc3-mcr-11358-eng-deu-Fuzzing_the_phone_in_the_iPhone_hd_32424_34079.png rc3-mcr-11358-eng-deu-Fuzzing_the_phone_in_the_iPhone_hd_34080_37115.png rc3-mcr-11358-eng-deu-Fuzzing_the_phone_in_the_iPhone_hd_37116_42131.png rc3-mcr-11358-eng-deu-Fuzzing_the_phone_in_the_iPhone_hd_42132_42563.png rc3-mcr-11358-eng-deu-Fuzzing_the_phone_in_the_iPhone_hd_42564_46475.png rc3-mcr-11358-eng-deu-Fuzzing_the_phone_in_the_iPhone_hd_46476_46631.png rc3-mcr-11358-eng-deu-Fuzzing_the_phone_in_the_iPhone_hd_46632_46751.png rc3-mcr-11358-eng-deu-Fuzzing_the_phone_in_the_iPhone_hd_46752_46799.png rc3-mcr-11358-eng-deu-Fuzzing_the_phone_in_the_iPhone_hd_46800_46847.png rc3-mcr-11358-eng-deu-Fuzzing_the_phone_in_the_iPhone_hd_46848_46919.png rc3-mcr-11358-eng-deu-Fuzzing_the_phone_in_the_iPhone_hd_46920_47063.png rc3-mcr-11358-eng-deu-Fuzzing_the_phone_in_the_iPhone_hd_47064_47123.png rc3-mcr-11358-eng-deu-Fuzzing_the_phone_in_the_iPhone_hd_47124_51527.png rc3-mcr-11358-eng-deu-Fuzzing_the_phone_in_the_iPhone_hd_51528_51575.png rc3-mcr-11358-eng-deu-Fuzzing_the_phone_in_the_iPhone_hd_51576_51899.png rc3-mcr-11358-eng-deu-Fuzzing_the_phone_in_the_iPhone_hd_51900_51959.png rc3-mcr-11358-eng-deu-Fuzzing_the_phone_in_the_iPhone_hd_51960_52247.png rc3-mcr-11358-eng-deu-Fuzzing_the_phone_in_the_iPhone_hd_52248_52295.png rc3-mcr-11358-eng-deu-Fuzzing_the_phone_in_the_iPhone_hd_52296_52643.png rc3-mcr-11358-eng-deu-Fuzzing_the_phone_in_the_iPhone_hd_52644_52691.png rc3-mcr-11358-eng-deu-Fuzzing_the_phone_in_the_iPhone_hd_52692_52823.png rc3-mcr-11358-eng-deu-Fuzzing_the_phone_in_the_iPhone_hd_52824_53075.png rc3-mcr-11358-eng-deu-Fuzzing_the_phone_in_the_iPhone_hd_53076_57515.png rc3-mcr-11358-eng-deu-Fuzzing_the_phone_in_the_iPhone_hd_57516_61739.png rc3-mcr-11358-eng-deu-Fuzzing_the_phone_in_the_iPhone_hd_61740_62423.png rc3-mcr-11358-eng-deu-Fuzzing_the_phone_in_the_iPhone_hd_62424_64619.png rc3-mcr-11358-eng-deu-Fuzzing_the_phone_in_the_iPhone_hd_64620_64859.png rc3-mcr-11358-eng-deu-Fuzzing_the_phone_in_the_iPhone_hd_64860_71051.png rc3-mcr-11358-eng-deu-Fuzzing_the_phone_in_the_iPhone_hd_71052_71711.png rc3-mcr-11358-eng-deu-Fuzzing_the_phone_in_the_iPhone_hd_71712_75839.png rc3-mcr-11358-eng-deu-Fuzzing_the_phone_in_the_iPhone_hd_75840_80747.png rc3-mcr-11358-eng-deu-Fuzzing_the_phone_in_the_iPhone_hd_80748_80819.png rc3-mcr-11358-eng-deu-Fuzzing_the_phone_in_the_iPhone_hd_80820_80915.png rc3-mcr-11358-eng-deu-Fuzzing_the_phone_in_the_iPhone_hd_80916_81131.png rc3-mcr-11358-eng-deu-Fuzzing_the_phone_in_the_iPhone_hd_81132_81203.png rc3-mcr-11358-eng-deu-Fuzzing_the_phone_in_the_iPhone_hd_81204_81575.png rc3-mcr-11358-eng-deu-Fuzzing_the_phone_in_the_iPhone_hd_81576_82019.png rc3-mcr-11358-eng-deu-Fuzzing_the_phone_in_the_iPhone_hd_82020_82511.png rc3-mcr-11358-eng-deu-Fuzzing_the_phone_in_the_iPhone_hd_82512_82559.png rc3-mcr-11358-eng-deu-Fuzzing_the_phone_in_the_iPhone_hd_82560_82763.png rc3-mcr-11358-eng-deu-Fuzzing_the_phone_in_the_iPhone_hd_82764_82779.png