Q:

Linux kernel 3+ system call table modification detection

Is there any available tool which can check kernel 3+ system call table for modifications?
Also I want to know, is there any automated general tool to check Linux for unknown rootkits? Not like rkhunter etc. which is mainly signature based, I want something like GMER for Linux, check for code modifications, kernel hooks, function hooks, etc.

A:

It is impossible to detect kernel modifications from inside the system. Whatever rootkit detector you use, the rootkit can be programmed to lie to it. If someone controls your kernel, then they control your system, not you, end of story. GMER suffers from that flaw too, it is intrinsic in the operating principle: it only finds rootkits that aren't good at hiding.
It is possible to detect kernel modifications from the outside. A common technology for that purpose is a TPM, which is a piece of hardware. It's a bit more complicated than “check the system call table for modifications” because the system call table is just a small part of the kernel, a modification anywhere else can have similar-reaching consequences. TPM works by taking a snapshot of the system from time to time, and comparing these snapshots with a reference snapshot. The TPM stores only hashes, not the whole memory image, so the snapshots have to match bit-for-bit. Loading modules changes the kernel, so you need to have a fixed set of modules as well as a fixed kernel version.
Detecting a kernel modification is useful only if there is some place to report it to. The TPM can't tell the user, because it has no user interface. The way to leverage a TPM is when you use your computer to authenticate to a remote service. The TPM sends a measurement of your PC, and the remote service compares that with a reference value and decides whether your PC is in a known good state or not. An unknown state could be a version upgrade or a compromise, the remote service has no way to know, so usefully using a TPM requires more infrastructure to manage upgrades.
Instead of a hardware TPM, you can implement similar functionality in a hypervisor. This requires no special hardware, but of course implies that your system runs in a virtual machine.

