Strace

strace is a diagnostic, debugging and instructional userspace utility for Linux. It is used to monitor and tamper with interactions between processes and the Linux kernel, which include system calls, signal deliveries, and changes of process state. The operation of strace is made possible by the kernel feature known as ptrace.
Some Unix-like systems provide other diagnostic tools similar to strace, such as truss.
History.
Strace was originally written for SunOS by Paul Kranenburg in 1991, according to its copyright notice, and published early in 1992, in the volume three of comp.sources.sun. The initial README file contained the following:
 is a system call tracer for Sun(tm) systems much like the Sun supplied program . is a useful utility to sort of debug programs for which no source is available which unfortunately includes almost all of the Sun supplied system software.
Later, Branko Lankester ported this version to Linux, releasing his version in November 1992 with the second release following in 1993. Richard Sladkey combined these separate versions of strace in 1993, and ported the program to SVR4 and Solaris in 1994, resulting in strace 3.0 that was announced in comp.sources.misc in mid-1994.
Beginning in 1996, strace was maintained by Wichert Akkerman. During his tenure, strace development migrated to CVS; ports to FreeBSD and many architectures on Linux (including ARM, IA-64, MIPS, PA-RISC, PowerPC, s390, SPARC) were introduced. In 2002, the burden of strace maintainership was transferred to Roland McGrath. Since then, strace gained support for several new Linux architectures (AMD64, s390x, SuperH), bi-architecture support for some of them, and received numerous additions and improvements in syscalls decoders on Linux; strace development migrated to git during that period. Since 2009, strace is actively maintained by Dmitry Levin. strace gained support for AArch64, ARC, AVR32, Blackfin, Meta, Nios II, OpenSISC 1000, RISC-V, Tile/TileGx, Xtensa architectures since that time.
The last version of strace that had some (evidently dead) code for non-Linux operating systems was 4.6, released in March 2011. In strace version 4.7, released in May 2012, all non-Linux code had been removed; since strace 4.13, the project follows Linux kernel's release schedule, and with the version 5.0, it follows Linux's versioning scheme as well.
In 2012 strace also gained support for path tracing and file descriptor path decoding. In August 2014, strace 4.9 was released, where support for stack traces printing was added. In December 2016, syscall fault injection feature was implemented.
Usage and features.
The most common use is to start a program using strace, which prints a list of system calls made by the program. This is useful if the program continually crashes, or does not behave as expected; for example using strace may reveal that the program is attempting to access a file which does not exist or cannot be read.
An alternative application is to use the flag to attach to a running process. This is useful if a process has stopped responding, and might reveal, for example, that the process is blocking whilst attempting to make a network connection.
Among other features, strace allows the following:
strace supports decoding of arguments of some classes of ioctl commands, such as BTRFS_*, V4L2_*, DM_*, NSFS_*, MEM*, EVIO*, KVM_*, and several others.
As strace only details system calls, it cannot be used to detect as many problems as a code debugger such as GNU Debugger (gdb). It is, however, easier to use than a code debugger, and is a very useful tool for system administrators. It is also used by researchers to generate system call traces for later "system call replay".
Examples.
The following is an example of typical output of the codice_16 command:
user@server:~$ strace ls
open(".", O_RDONLY|O_NONBLOCK|O_LARGEFILE|O_DIRECTORY|O_CLOEXEC) = 3
fstat64(3, {st_mode=S_IFDIR|0755, st_size=4096, ...}) = 0
fcntl64(3, F_GETFD) = 0x1 (flags FD_CLOEXEC)
getdents64(3, /* 18 entries */, 4096) = 496
getdents64(3, /* 0 entries */, 4096) = 0
close(3) = 0
fstat64(1, {st_mode=S_IFIFO|0600, st_size=0, ...}) = 0
mmap2(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0xb7f2c000
write(1, "autofs\nbackups\ncache\nflexlm\ngames"..., 86autofsA
The above fragment is only a small part of the output of strace when run on the 'ls' command. It shows that the current working directory is opened, inspected and its contents retrieved. The resulting list of file names is written to standard output.
Similar tools.
Different operating systems feature other similar or related instrumentation tools, offering similar or more advanced features; some of the tools (although using the same or a similar name) may use completely different work mechanisms, resulting in different feature sets or results. Such tools include the following: