Can strace make you fail?
CC Attribution 2.0 Belgium:
You are free to use, adapt and copy, distribute and transmit the work or content in adapted or unchanged form for any legal purpose as long as the work is attributed to the author in the manner specified by the author or licensor.
Levin, Dmitry
strace is a diagnostic, debugging and instructional utility for Linux. It isused to monitor interactions between processes and the Linux kernel, whichinclude system calls, signal deliveries, and changes of process state. In2016, strace has been extended to tamper with tracees using syscall faultinjection, making strace also a syscall fault injection tool, the onlyunprivileged userspace command-line runtime tool of this kind for Linux. The last summer as a part of strace GSoC 2016 project Nahim El Atmani hasimplemented a prototype of syscall fault injection - a software testingtechnique used for improving test coverage of error handling code paths thatmight otherwise rarely be followed by introducing faults. Now the feature hasfinally been implemented and is a part of strace 4.15 released in December2016. In this talk the maintainer of strace will demonstrate the new feature,describe its implementation, and show examples of real bugs found with itshelp.
 Hello everybody. I am Dmitry Reryan, the maintainer of stress. Today I will talk about a new stress feature called Fault Injection. So, does anybody know what stress is? Nice. So, as most of you know, stress is a traditional diagnostic debugging and instructional user space utility for Linux. How traditional is 25 years old, so quite traditional. And traditionally, it's used to monitor interactions between user space process and Linux kernel. The most known system calls and also signal deliveries, changes of process state. Stress has traditional, everything is traditional with stress. It has traditional common line interface and multiple filtering capabilities. So, because the interface is traditional, it's easy to use for people who are used to this for 20-something years. And it's quite powerful because of these filtering capabilities. But last year, stress has been extended to do something very untraditional. That is, to tamper with traces by injection faults. It's called fault injection. The current implementation is based on the work made by GSOC student of last year, Nahime Mane. I'm not sure how to pronounce his name, so I'm sorry if it's wrong. He will someday correct me. So, what is a fault injection? Just a brief correction. It's a software testing technique used for improvement test coverage. Most of error handling paths that might otherwise fairly be followed by introducing faults. This is a nice definition from Wikipedia. Thanks to Wikipedia for this. So, where do we place stress among other fault injection tools? It's obviously software. It's runtime. Most of the instrumentation tools are compile time. This is runtime. It doesn't work by means of CISCOL interposition. It uses space, unlike many current techniques we have. It's unprivileged, and as I said, it has a traditional common line interface. So, now I'll show you a series of examples. But hopefully, I will get you an idea what this is called fault injection syntax is and what you could do with it. And I'll start with a simple program, cat from Corvutals, which is linked dynamically with NLIP-C, and we'll see what could be done. So, in the top box, you see a traditional output, and in the bottom box, something with fault injection. So, let's filter all open CISCOLs of this trivial command. And let's filter them all. What you can see here is that this ends in dynamic linger. It tries to open its cache, it fails, it tries to open Lipsy in predefined locations. On this architecture, there are four locations in this build of Lipsy, and it fails. Nothing very... It's quite predictable. Let's do something different. Let's change error code that dynamic linker will get from default one, which is in a CISC function of the pre-empted, to something that has a different meaning for dynamic linker. As you see, it tries twice as many locations when it sees error code in the end. The error code of open CISCOL. You can inject fault not on all invocations, but just in the first one or whatever else. So, if you inject a fault just to the first invocation, you can see that dynamic linker just does all the same. It tries Lipsy in different locations, finds it, and everything works. But if you fail second and all subsequent invocations, what happens? Dynamic linker opens its cache and tries to open Lipsy according to the cache, as you see, it fails, and then it tries all these predefined locations. And you can notice that it tries to open Lipsy at the same location it tried before. Why dynamic linker does this? Because there are different code paths, and they don't know about each other. In the first path, it tries to open, and in another code path, it also tries to open, and it doesn't care, but it's already tried this. It's not a bug, just a funny thing. Let's dynamic linker do its work and have a look at the cache itself. So, the third open CISCOL is the CISCOL made by cache itself, and it looks like what you would expect from a cache. It fails, and it exits with every code. And another way you can specify faults is to say how often you would to inject them. For example, you can inject them starting with third one, with third, and then every second one. So, you see that cat handles the situation outside property. So, it reports error on every case when there is an error, and processes everything that's opened, and exits with error code. So, looks like cat works fine. And now, let's have a look at something more interesting. Let's combine fault injection with path filtering. So, in the top box, you see a sequence of CISCOLs that are related to the files passed to cat, in this case. And in the bottom, I will just fail each of the CISCOLs. I will see how cat handles this. So, the first one we will see, and the second one is f-start, and for some reason, cat considers this as an error. It's quite an unusual thing that f-start fails. So, probably it's the same thing to do is to fail anything. From another side, the f-advice is called its advice. So, cat is also quite right that it ignores this error. It's just an advice to kernel that cat is going to do some sequential reads. More or less the same with read. If you fail a read with a hard error, it's quite right to report it and fail. But what do you think cat will do if it sees a temporary error? Would it restart read or would it fail? It would restart. Cat is a good program after all, and it knows that in traffic system call is a temporary error that should be restarted. So, and this is a peculiar thing. This is a, then you fail close to scol that has no importance. What cat does? It just opens file, processes it and close it. It opens it read only. And what's the use of reporting this as a hard error when you can't close a file open read only? I don't know. I would say it's a minor barking cat, very minor to complain and exit with error code. Because the file was successfully processed from the beginning to the end. So why does here you can see in brief that you can specify different fold injection expressions that would work in the top box. And the second box you see that a trace actually can follow descriptors. So even if cat has no idea what the file it's working with, a trace does know and can apply its filters. So it's like what you can see in the bottom box, it's a primitive access control using this trace. And let's have a look at real box. This is a more or less famous bug in Python 3.5. It was found by the student who did this GSOC project. Python on every invocation needs some randomness for instantization. And when it fails to obtain this randomness, it's a fatal error, which is fine probably, but it's not fine that it throws a segmentation fold at hexadecimal address 50. It's not good. It's a bug. Why it does this ridiculous thing? Because it addresses a method of object which was not allocated because of lack of randomness. I saw hexadecimal address is just offset with an inertial table like this. Fortunately, this bug was, seems to be at least fixed in Python 3.6. First they were working on it by using get randomness call. But I actually tried to fold get random and found out that it no longer sick folds. That's funny thing anyway. And another real bug was found with dynamic linker itself. So if you fail and protect, you can see that dynamic linker from libc ignores error from the first and protect error. But it treats all subsequent errors as fatal. It's quite natural to do so with all subsequent, but why it ignores the first error? It's because there are different code paths. And the only code path that exists in current's libcs dynamic linker is the one that's early in its running and it just ignores the error. And this error actually can happen, for example, because of fragmentation. And the call that's failed is the one that tries to remove access. It's pro-none. So actually, some pages are remains to be accessible, which I suppose not to be. So it's like it would be a minor security issue if it's not a management problem. Because if first and protect call fails, it's very likely that all subsequent also fail and they are probably, but still it's a bug. Now I'll try to explain you what's going on under the hood and how it's all implemented. So when traces invoke the syscall, kernel puts it in so-called syscall interstop state. So it's completely stopped. At the same time as trace awakens, it fetches syscall number, its arguments, applies filters you've seen on all other kinds of filters as trace supports. It decides and may skip this syscall or print something and then it tells the kernel to go on with trace, then kernel executes the syscall and before passing control to user space, it puts traces again in a stop state. It's a slightly different state. It's called exit stop. At this time, a trace awakens again. It may fetch syscall return code and arguments. It depends whether the call is filtered or not. Also, it prints at this time. This return code is necessary and tells kernel to let the trace go on and all this cycle repeats itself until something happens with trace, whether it's exits or whatever. So taking these two parts together, you can see the sequence and in this sequence, there are two places where a trace actually can't tamper with syscall. With syscall number, we have syscall arguments with syscall return code and this is exactly the way how syscall fault injection is implemented. So on an engine syscall, a trace replaces syscall number with an invalid one minus one and then kernel sees this invalid syscall. It's just invalid syscall. So it returns an error for an invalid syscall which is in a system most architectures but not on all of them. So you shouldn't rely on this error code. And then on exit syscall, if instructed to replace error code, it replaces this error code with the one that was specified. So it's actually pretty simple in sight. But this is not the only thing, as you can see from this slide, it's not the only thing that could be done not just syscall fault injection but other kinds of injections and tamperings. For example, this is some recent development, not in the erased trace version yet. You can inject a signal at any of these points and current limitation, it's injected on an exit anything? Yeah, an exit syscall. So what would you use this signal injection for? For example, if you want to terminate program, you would probably use a different signal for this. Or if you want to dump it for the analysis by another tool like gdb, and another way you can tamper with is to replace return code of syscall with something that doesn't look like an error actually. So you can pretend that syscall completed successfully, but actually skip it. So by replacing syscall number on entering to an invalid one, this way you skip the syscall. And on exiting, you can replace return code with like zero, which for syscalls like unlink means that everything is fine. So the traceer will suppose that unlink is complete successfully and the file is gone, but it's actually... This is not as easy as it looks. For more complicated syscalls, they have some semantics that has to be followed somehow. For example, many syscalls, when they succeed, some portions of memory are filled with some useful information by the kernel. So for them, this is not as easy. But for simple syscalls like unlink, it's work from current master. Probably I forgot to tell that this API as trace is using is called ptrace. It's the same API used by... It's a traditional interface as I think about trace. So it's the same that's used by gdb. More or less, that all I would like to tell about fault injection, but I can answer your questions because some things are better answered in questions. So by the way, the title of this talk was probably invented by the student who made this GSOC project. You can visit his GSOC page and maybe you'll get an idea what was planned in the beginning and how it's changed and compare these things. You can grab the source. Syscalls fault injection is part of the recent release which was in December I think. So it's... You can use it. Do you have any... I'm kind of curious if are there testing tools you have seen that are built on top of the fault injection in S trace? So this fault injection makes a trace a testing tool, but it's quite a modern thing. So I'm not sure it's already widely used as a testing tool. I'm thinking of using S trace itself for a stress dispute because it's not easy to reproduce some faults related to P trace system call, for example. So just a new way of fault injection, a new way of testing something that otherwise would require a compile time instrumentation. It's probably the only runtime unprivileged user space fault injection tool for Linux we have now. So I think it's time to ask questions. So which one of slides? So the question was how was able to complain about how was able to report an error if linker was unable to load libc. It's not a cut. The error diagnostics you see comes from dynamic linker. It's still a dynamic linker at this point. So you can see the prefix is cat, but it's not a cat. It's dynamic linker for complaints that it can't load libc. Any more questions? Yes please. First of all I've been told so the question is why the current implementation uses an invalid syscall number for fault injection instead of using some harmless syscall that has no side effects so that a monitoring system like a clinux wouldn't or something like this audit with it wouldn't complain about something wrong is going on. First of all it's not easy to find a harmless syscall. I've been told here recently that getPidMyFail. In current Linux channels getPidMyFail. It's not allowed. Okay so in theory it's possible to invoke a getPid I think. It's not a technical issue but from optimization point of view it's less probably less work. You will need less syscalling vacations to do this. On exiting syscall you'll have to replace like two registers on some architectures and in case of invalid syscall you will not have to replace at least the register that contains the indication that it's an error. So it's slightly I think it's slightly faster to use an invalid syscall but I think it's quite a valid point. Thank you. Any more questions? No? Really? Okay, yes please. Is there already something there probably external to S-trace to do random like fuzzing stuff that randomly some syscall fails? So the question as I heard it was is there anything probably external from S-trace that do some random what's a random injection random what and so in the initial GSOC design there was an interface to do some random injection but later I decided that it's better done by a fuzzy driver that's Ranses Trace. So my idea is that if you want randomness you just do it in a driver that runs a trace and not in a trace itself. So currently this injection is predictable. There is no random injection but it might change if somebody would suggest a plausible case how this would be used. Currently I think it's better done in a fuzzy driver outside. The command line will get very long if you have a real life application and the command line will have 10,000 of these real to fail or they're not to fail. Is it the problem that the command line is long? Okay. Does anybody have any more questions? Yes please. I'm not sure I hear anything but I will have to repeat what you're saying so please make it audible at least for me. Okay so Steve said that it's a well it probably is you know it's like just a good words about this. I'm not going to repeat this. Thank you. So coming up next in this room we have a quick talk on testing web applications.
 Hello everybody. I am Dmitry Reryan, the maintainer of stress. Today I will talk about  a new stress feature called Fault Injection. So, does anybody know what stress is? Nice.  So, as most of you know, stress is a traditional diagnostic debugging and instructional user  space utility for Linux. How traditional is 25 years old, so quite traditional. And traditionally,  it's used to monitor interactions between user space process and Linux kernel. The most known  system calls and also signal deliveries, changes of process state.  Stress has traditional, everything is traditional with stress. It has traditional  common line interface and multiple filtering capabilities. So, because the interface is  traditional, it's easy to use for people who are used to this for 20-something years. And  it's quite powerful because of these filtering capabilities. But last year, stress has been  extended to do something very untraditional. That is, to tamper with traces by injection faults.  It's called fault injection. The current implementation is based on the work made by GSOC  student of last year, Nahime Mane. I'm not sure how to pronounce his name, so I'm sorry if it's wrong.  He will someday correct me. So, what is a fault injection? Just a brief correction.  It's a software testing technique used for improvement test coverage.  Most of error handling paths that might otherwise fairly be followed by introducing faults.  This is a nice definition from Wikipedia. Thanks to Wikipedia for this.  So, where do we place stress among other fault injection tools? It's obviously software.  It's runtime. Most of the instrumentation tools are compile time. This is runtime.  It doesn't work by means of CISCOL interposition. It uses space, unlike many current techniques we have.  It's unprivileged, and as I said, it has a traditional common line interface.  So, now I'll show you a series of examples. But hopefully, I will get you an idea what this is  called fault injection syntax is and what you could do with it. And I'll start with a simple program,  cat from Corvutals, which is linked dynamically with NLIP-C, and we'll see what could be done.  So, in the top box, you see a traditional output, and in the bottom box, something with fault injection.  So, let's filter all open CISCOLs of this trivial command. And let's  filter them all. What you can see here is that this ends in dynamic linger. It tries to open its cache,  it fails, it tries to open Lipsy in predefined locations. On this architecture, there are four  locations in this build of Lipsy, and it fails. Nothing very... It's quite predictable. Let's do  something different. Let's change error code that dynamic linker will get from default one,  which is in a CISC function of the pre-empted, to something that has a different meaning for  dynamic linker. As you see, it tries twice as many locations when it sees error code in the end.  The error code of open CISCOL. You can inject fault not on all invocations, but just in the first  one or whatever else. So, if you inject a fault just to the first invocation, you can see that  dynamic linker just does all the same. It tries Lipsy in different locations, finds it, and everything  works. But if you fail second and all subsequent invocations, what happens? Dynamic linker  opens its cache and tries to open Lipsy according to the cache, as you see, it fails, and then it  tries all these predefined locations. And you can notice that it tries to open Lipsy at the same  location it tried before. Why dynamic linker does this? Because there are different code paths,  and they don't know about each other. In the first path, it tries to open, and in another  code path, it also tries to open, and it doesn't care, but it's already tried this. It's not a bug,  just a funny thing. Let's dynamic linker do its work and have a look at the cache itself.  So, the third open CISCOL is the CISCOL made by cache itself, and it looks like  what you would expect from a cache. It fails, and it exits with every code.  And another way you can specify faults is to say how often you would to inject them.  For example, you can inject them starting with third one, with third, and then every second one.  So, you see that cat handles the situation outside property. So, it reports error on every  case when there is an error, and processes everything that's opened, and exits with error code. So,  looks like cat works fine. And now, let's have a look at something more interesting. Let's combine  fault injection with path filtering. So, in the top box, you see a sequence of CISCOLs that are  related to the files passed to cat, in this case. And in the bottom, I will just fail each of the  CISCOLs. I will see how cat handles this. So, the first one we will see, and the second one is f-start,  and for some reason, cat considers this as an error. It's quite an unusual thing that f-start fails.  So, probably it's the same thing to do is to fail anything. From another side, the f-advice is called  its advice. So, cat is also quite right that it ignores this error. It's just an advice to kernel  that cat is going to do some sequential reads. More or less the same with read. If you fail a read  with a hard error, it's quite right to report it and fail. But what do you think cat will do  if it sees a temporary error? Would it restart read or would it fail? It would restart. Cat is a  good program after all, and it knows that in traffic system call is a temporary error that  should be restarted. So, and this is a peculiar thing. This is a, then you fail close to scol  that has no importance. What cat does? It just opens file, processes it and close it. It opens it  read only. And what's the use of reporting this as a hard error when you can't close a file open  read only? I don't know. I would say it's a minor barking cat, very minor to complain and exit with  error code. Because the file was successfully processed from the beginning to the end. So why does  here you can see in brief that you can specify different fold injection expressions that would  work in the top box. And the second box you see that a trace actually can follow descriptors.  So even if cat has no idea what the file it's working with, a trace does know and can apply  its filters. So it's like what you can see in the bottom box, it's a primitive access control  using this trace. And let's have a look at real box. This is a more or less famous bug in Python 3.5.  It was found by the student who did this GSOC project. Python on every invocation needs some  randomness for instantization. And when it fails to obtain this randomness, it's a fatal error,  which is fine probably, but it's not fine that it throws a segmentation fold at hexadecimal address  50. It's not good. It's a bug. Why it does this ridiculous thing? Because it addresses a method  of object which was not allocated because of lack of randomness. I saw hexadecimal address is just  offset with an inertial table like this. Fortunately, this bug was, seems to be at least fixed in  Python 3.6. First they were working on it by using get randomness call. But I actually tried to  fold get random and found out that it no longer sick folds. That's funny thing anyway. And another  real bug was found with dynamic linker itself. So if you fail and protect,  you can see that dynamic linker from libc ignores error from the first and protect error. But  it treats all subsequent errors as fatal. It's quite natural to do so with all subsequent,  but why it ignores the first error? It's because there are different code paths. And the only  code path that exists in current's libcs dynamic linker is the one that's early in its running  and it just ignores the error. And this error actually can happen, for example, because of  fragmentation. And the call that's failed is the one that tries to remove access. It's  pro-none. So actually, some pages are remains to be accessible, which I suppose not to be.  So it's like it would be a minor security issue if it's not a management problem. Because if  first and protect call fails, it's very likely that all subsequent also fail and they are  probably, but still it's a bug. Now I'll try to explain you what's going on under the hood and  how it's all implemented. So when traces invoke the syscall, kernel puts it in so-called syscall  interstop state. So it's completely stopped. At the same time as trace awakens, it fetches syscall  number, its arguments, applies filters you've seen on all other kinds of filters as trace supports.  It decides and may skip this syscall or print something and then it tells the kernel to go on  with trace, then kernel executes the syscall and before passing control to user space,  it puts traces again in a stop state. It's a slightly different state. It's called exit stop.  At this time, a trace awakens again. It may fetch syscall return code  and arguments. It depends whether the call is filtered or not. Also, it prints at this time.  This return code is necessary and tells kernel to let the trace go on and all this cycle  repeats itself until something happens with trace, whether it's exits or whatever.  So taking these two parts together, you can see the sequence and in this sequence,  there are two places where a trace actually can't tamper with syscall. With syscall number,  we have syscall arguments with syscall return code and this is exactly the way how syscall fault  injection is implemented. So on an engine syscall, a trace replaces syscall number with an invalid one  minus one and then kernel sees this invalid syscall. It's just invalid syscall. So it returns an error  for an invalid syscall which is in a system most architectures but not on all of them. So  you shouldn't rely on this error code. And then on exit syscall, if instructed to replace error  code, it replaces this error code with the one that was specified. So it's actually pretty simple  in sight. But this is not the only thing, as you can see from this slide, it's not the only thing  that could be done not just syscall fault injection but other kinds of injections and tamperings.  For example, this is some recent development, not in the erased trace version yet. You can inject a  signal at any of these points and current limitation, it's injected on  an exit anything? Yeah, an exit syscall. So what would you use this signal injection for?  For example, if you want to terminate program, you would probably use a different signal for this.  Or if you want to dump it for the analysis by another tool like gdb,  and another way you can tamper with is to replace return code of syscall with something  that doesn't look like an error actually. So you can pretend that syscall completed successfully,  but actually skip it. So by replacing syscall number on entering to an invalid one,  this way you skip the syscall. And on exiting, you can replace return code with like zero,  which for syscalls like unlink means that everything is fine. So the traceer will  suppose that unlink is complete successfully and the file is gone, but it's actually...  This is not as easy as it looks. For more complicated syscalls, they have some semantics  that has to be followed somehow. For example, many syscalls, when they succeed,  some portions of memory are filled with some useful information by the kernel. So for them,  this is not as easy. But for simple syscalls like unlink, it's work from current master.  Probably I forgot to tell that this API as trace is using is called ptrace. It's the same API used  by... It's a traditional interface as I think about trace. So it's the same that's used by gdb.  More or less, that all I would like to tell about fault injection, but I can answer your  questions because some things are better answered in questions. So by the way, the title of this  talk was probably invented by the student who made this GSOC project. You can visit his GSOC page  and maybe you'll get an idea what was planned in the beginning and how it's changed and  compare these things. You can grab the source. Syscalls fault injection is part of the  recent release which was in December I think. So it's... You can use it.  Do you have any...  I'm kind of curious if are there testing tools you have seen that are built on top of  the fault injection in S trace? So this fault injection makes a trace a testing tool,  but it's quite a modern thing. So I'm not sure it's already widely used as a testing tool.  I'm thinking of using S trace itself for a stress dispute because it's not easy to reproduce some  faults related to P trace system call, for example. So just a new way of fault injection,  a new way of testing something that otherwise would require a compile time instrumentation.  It's probably the only runtime unprivileged user space  fault injection tool for Linux we have now.  So I think it's time to ask questions.  So which one of slides?  So the question was how was able to complain about  how was able to report an error if linker was unable to load libc. It's not a cut.  The error diagnostics you see comes from dynamic linker. It's still a dynamic linker at this point.  So you can see the prefix is cat, but it's not a cat. It's dynamic linker for complaints  that it can't load libc. Any more questions? Yes please.  First of all I've been told so the question is why the current implementation uses an  invalid syscall number for fault injection instead of using some harmless syscall that has no side  effects so that a monitoring system like a clinux wouldn't or something like this  audit with it wouldn't complain about something wrong is going on. First of all it's not easy to  find a harmless syscall. I've been told here recently that getPidMyFail. In current Linux  channels getPidMyFail. It's not allowed. Okay so in theory it's possible to invoke a getPid  I think. It's not a technical issue but from optimization point of view it's less probably  less work. You will need less syscalling vacations to do this. On exiting syscall you'll have to  replace like two registers on some architectures and in case of invalid syscall you will not have  to replace at least the register that contains the indication that it's an error. So it's slightly  I think it's slightly faster to use an invalid syscall but I think it's quite a valid point.  Thank you. Any more questions? No? Really? Okay, yes please.  Is there already something there probably external to S-trace to do random like fuzzing stuff that randomly some syscall fails?  So the question as I heard it was is there anything probably external from S-trace that do some  random what's a random injection random what and so in the initial GSOC design there was  an interface to do some random injection but later I decided that it's better done by a fuzzy driver  that's Ranses Trace. So my idea is that if you want randomness you just do it in a  driver that runs a trace and not in a trace itself. So currently this injection is predictable.  There is no random injection but it might change if somebody would suggest a plausible case how this  would be used. Currently I think it's better done in a fuzzy driver outside.  The command line will get very long if you have a real life application and the command line will have  10,000 of these real to fail or they're not to fail. Is it the problem that the command line  is long? Okay.  Does anybody have any more questions?  Yes please.  I'm not sure I hear anything but I will have to repeat what you're saying so please make it  audible at least for me. Okay so Steve said that it's a well it probably is you know it's like  just a good words about this. I'm not going to repeat this. Thank you.  So coming up next in this room we have a quick talk on testing web applications.
Can_strace_make_you_fail_strace_syscall_fault_injection_0_59.png Can_strace_make_you_fail_strace_syscall_fault_injection_60_131.png Can_strace_make_you_fail_strace_syscall_fault_injection_132_935.png Can_strace_make_you_fail_strace_syscall_fault_injection_936_3227.png Can_strace_make_you_fail_strace_syscall_fault_injection_3228_5075.png Can_strace_make_you_fail_strace_syscall_fault_injection_5076_7151.png Can_strace_make_you_fail_strace_syscall_fault_injection_7152_9023.png Can_strace_make_you_fail_strace_syscall_fault_injection_9024_9623.png Can_strace_make_you_fail_strace_syscall_fault_injection_9624_10619.png Can_strace_make_you_fail_strace_syscall_fault_injection_10620_11843.png Can_strace_make_you_fail_strace_syscall_fault_injection_11844_12407.png Can_strace_make_you_fail_strace_syscall_fault_injection_12408_14495.png Can_strace_make_you_fail_strace_syscall_fault_injection_14496_15719.png Can_strace_make_you_fail_strace_syscall_fault_injection_15720_17975.png Can_strace_make_you_fail_strace_syscall_fault_injection_17976_20351.png Can_strace_make_you_fail_strace_syscall_fault_injection_20352_21443.png Can_strace_make_you_fail_strace_syscall_fault_injection_21444_22571.png Can_strace_make_you_fail_strace_syscall_fault_injection_22572_28763.png Can_strace_make_you_fail_strace_syscall_fault_injection_28764_32723.png Can_strace_make_you_fail_strace_syscall_fault_injection_32724_32771.png Can_strace_make_you_fail_strace_syscall_fault_injection_32772_32831.png Can_strace_make_you_fail_strace_syscall_fault_injection_32832_32951.png Can_strace_make_you_fail_strace_syscall_fault_injection_32952_32999.png Can_strace_make_you_fail_strace_syscall_fault_injection_33000_33071.png Can_strace_make_you_fail_strace_syscall_fault_injection_33072_33167.png Can_strace_make_you_fail_strace_syscall_fault_injection_33168_42263.png Can_strace_make_you_fail_strace_syscall_fault_injection_42264_42827.png Can_strace_make_you_fail_strace_syscall_fault_injection_42828_42947.png Can_strace_make_you_fail_strace_syscall_fault_injection_42948_42995.png Can_strace_make_you_fail_strace_syscall_fault_injection_42996_43127.png Can_strace_make_you_fail_strace_syscall_fault_injection_43128_43199.png Can_strace_make_you_fail_strace_syscall_fault_injection_43200_43403.png Can_strace_make_you_fail_strace_syscall_fault_injection_43404_43451.png Can_strace_make_you_fail_strace_syscall_fault_injection_43452_43631.png Can_strace_make_you_fail_strace_syscall_fault_injection_43632_43739.png Can_strace_make_you_fail_strace_syscall_fault_injection_43740_43823.png Can_strace_make_you_fail_strace_syscall_fault_injection_43824_44099.png Can_strace_make_you_fail_strace_syscall_fault_injection_44100_44207.png Can_strace_make_you_fail_strace_syscall_fault_injection_44208_44399.png Can_strace_make_you_fail_strace_syscall_fault_injection_44400_44663.png Can_strace_make_you_fail_strace_syscall_fault_injection_44664_44735.png Can_strace_make_you_fail_strace_syscall_fault_injection_44736_44927.png Can_strace_make_you_fail_strace_syscall_fault_injection_44928_45035.png Can_strace_make_you_fail_strace_syscall_fault_injection_45036_45083.png Can_strace_make_you_fail_strace_syscall_fault_injection_45084_45231.png