A Survey of Techniques for Improving Security of GPUs <s> Malware attack <s> Mobile phones are the most intimate computing devices of our time. We use them for private and business purposes. At the same time lax update habits of manufacturers make them accumulate disclosed vulnerabilities. That is why smartphones have become very attractive targets for attackers. Until today Graphics Processing Units (GPU) were not considered an interesting mean of payload delivery in mobile devices. However, in this paper, we present how the Direct Memory Access (DMA) capabilities of a mobile GPU can be abused for a privilege escalation attack. We describe a successful and real-world GPU-based attack, discuss problems that the GPU’s different programming model poses, and techniques that lead to a successful attack. We also show a proof-of-concept exploit against a very popular smartphone line. We conclude that DMA-based malware is a serious threat to mobile devices. <s> BIB001 </s> A Survey of Techniques for Improving Security of GPUs <s> Malware attack <s> Protecting the confidentiality of cryptographic keys in the event of partial or full system compromise is crucial for containing the impact of attacks. The Heartbleed vulnerability of April 2014, which allowed the remote leakage of secret keys from HTTPS web servers, is an indicative example. In this paper we present PixelVault, a system for keeping cryptographic keys and carrying out cryptographic operations exclusively on the GPU, which allows it to protect secret keys from leakage even in the event of full system compromise. This is possible by exposing secret keys only in GPU registers, keeping PixelVault's critical code in the GPU instruction cache, and preventing any access to both of them from the host. Due to the non-preemptive execution mode of the GPU, an adversary that has full control of the host cannot tamper with PixelVault's GPU code, but only terminate it, in which case all sensitive data is lost. We have implemented a PixelVault-enabled version of the OpenSSL library that allows the protection of existing applications with minimal modifications. Based on the results of our evaluation, PixelVault not only provides secure key storage using commodity hardware, but also significantly speeds up the processing throughput of cryptographic operations for server applications. <s> BIB002 </s> A Survey of Techniques for Improving Security of GPUs <s> Malware attack <s> GPUs have become an integral part of modern systems, but their implications for system security are not yet clear. This paper demonstrates both that discrete GPUs cannot be used as secure co-processors and that GPUs provide a stealthy platform for malware. First, we examine a recent proposal to use discrete GPUs as secure co-processors and show that the security guarantees of the proposed system do not hold on the GPUs we investigate. Second, we demonstrate that (under certain circumstances) it is possible to bypass IOMMU protections and create stealthy, long-lived GPU-based malware. We demonstrate a novel attack that compromises the in-kernel GPU driver and one that compromises GPU microcode to gain full access to CPU physical memory. In general, we find that the highly sophisticated, but poorly documented GPU hardware architecture, hidden behind obscure close-source device drivers and vendor-specific APIs, not only make GPUs a poor choice for applications requiring strong security, but also make GPUs into a security threat. <s> BIB003
Zhu et al. BIB003 demonstrate that discrete GPUs cannot be trusted as secure co-processors, rather, they may host stealthy malware. They study the limitations of PixelVault BIB002 technique which assumes GPU hardware registers to be secure storage. PixelVault presents a GPU-accelerator for AES and RSA encryption. PixelVault assumes that after the kernel is launched, the attacker can control entire system and run the program with any privilege. Their work is based on assumptions shown in column 2 of Table 7 since these assumptions ensure certain properties (column 3). However, Zhu et al. refute these assumptions (column 4). BIB002 assumptions and their security benefits and why they don't hold BIB003 No. Assumptions Benefits from assumption Why assumption does not hold 1 If the code of a running kernel is stored entirely in I-cache, it cannot be replaced. It ensures that an attacker cannot substitute PixelVault GPU code without stopping it and hence, without losing the master key stored in GPU registers. The MMIO registers on NVIDIA GPUs can flush I-cache and thus, replace code of running kernels with malicious code 2 GPU register values cannot be obtained after the kernel terminates. This prohibits an attacker from obtaining a master key by stopping a running kernel. On launching two kernels on different CUDA streams, cuda-gdb can leak keys stored in registers of the terminated kernel as long as the other kernel is running. 3 Unless compiled with debug support, a running kernel cannot be stopped and debugged. It ensures that attacker cannot read registers by attaching a GPU debugger to the running PixelVault kernel. Latest CUDA runtime allows attaching a debugger to a running kernel using root privileges. They show that violation of these assumptions and unrestricted DMA access from GPU to CPU memory allow subverting the protection of IOMMU. Using this, an adversary can run stealthy malware on GPU and thus, systems based on PixelVault's approach are not secure. Note that IOMMU monitors devices' DMA to system memory for protecting them from illegitimate accesses. The IOMMU prohibits a device from accessing CPU pages which are not mentioned in its I/O page table. They further show two attacks. The first attack is on the in-kernel NVIDIA GPU driver. When the driver is being loaded and used by OS kernel, they binary-patch it. The patched driver maps sensitive CPU memory into address-space of an unprivileged GPU application. The second attack combines the microcode running on a microprocessor with malicious code to access any portion of CPU memory. Balzarotti et al.  divide the GPU malware into three categories based on the OS privileges required by them and the amount of knowledge they possess about internal details of the GPU driver. (1) User-space malware have only normal privileges. They perform regular computations using standard GPU commands, without relying on any software bug. (2) Super-user malware need elevated privileges and hence, can execute additional tasks which are not possible through user space. However, the adversary has no knowledge of the GPU driver/card. (3) Kernel-level malware not only have elevated privileges but also know the internals of the graphic driver. Hence, they can tamper the data structures in the kernel. They further describe four strategies which the malware can adopt to avoid detection by forensic tools. (1): "Unlimited code execution": due to non-preemptive nature of GPU, a single task can occupy the GPU completely. To avoid this, GPU driver uses a timeout scheme (e.g., hangcheck function), by which a longlasting process can be killed. However, the malware can disable this to occupy the GPU indefinitely. (2) "Process-less code execution": a GPU kernel is usually always controlled by a host process. However, the malware may run a kernel without any controlling process on the host by killing the host process right after the GPU kernel starts. (3) "Context-less code execution": The "process-less execution" still leaves traces in the GPU driver, e.g., the buffer objects and hardware context of the GPU kernel may still be present in the driver's memory. In "context-less" execution, the hardware context of GPU kernel is removed completely from the records of the driver. Thus, the kernel can hide its presence completely. Both process-less and context-less executions require that the kernel has super-user privileges and has already achieved "unlimited code execution". However, the latter additionally requires knowledge about driver internals. (4) "Inconsistent memory mapping": Generally, the list of accessible physical pages is kept both in the OS and GPU memories. However, the information kept in these two-page tables can be made to differ. Then, the OS page table points to the correct page, but the GPU address points to a random memory location. Table 8 summarizes the characteristics of these strategies along with the privileges and information needed by them. The last three columns show the results of memory forensic, whether it can find (1) the processes which are using GPU, (2) the code executed by them and (3) the memory regions accessible to these kernels. The sign shows that the forensic goal can be realized by inspecting only system memory. If so, the analysis may need information of only the OS data-structures or driver internals, and this is indicated as (OS) or (driver), respectively. The sign shows that the corresponding forensic analysis is not possible. They also recommend that checking the graphic page tables, hangcheck flag, list of buffer objects, contexts and register files can provide an insight into the type of malware present. Danisevskis et al. BIB001 demonstrate an attack whereby an adversary can exploit DMA to bypass GPU memory protection schemes and gain access to privileged regions of the memory. They demonstrate these attacks on an Android smart-phone with sound memory isolation and security mechanisms. The adversary tricks a benign user into installing a malicious application which initially requests only normal access rights. This application seeks to escalate its privilege by patching the text section of the kernel. For this, the application patches the sys_reboot system-call which is called only on power-down, and thus, patching of this is likely to go unnoticed. Then, the physical address for patching is determined and the above system-call is attached to the session address space of the mobile-GPU. Then, they configure the GPU to change the caller's user ID to zero (or root) and write this to text section of the kernel. Based on this, they prepare a rendering job and submit it to the kernel driver. Their work confirms that DMA-based malware can pose security threats for mobile GPUs. Keyloggers are malware that stealthily log keyboard activity for stealing sensitive data. Ladakis et al.  implement a keylogger malware on GPU which tracks the keyboard buffer directly from GPU through DMA without modifying kernel's code or data structures except for that page table. Their keylogger has two modules. A CPU-based module runs only once at the time of booting. It finds the address of keyboard buffer in physical memory, since this address may randomly change on every system boot. They implement a loadable kernel module which scans entire host memory to find this address, although its limitation is that the search time increases with increasing physical memory size. Then, a GPU-based module continuously tracks changes in this keyboard buffer. For this, the GPU needs to have access to the buffer. Since NVIDIA GPUs share an address space with the host process, the buffer must be mapped in the virtual address space of the host process. To achieve this, the pages where buffer resides can be included in the page table of the host process. A keystroke monitoring interval of 100ms incurs small overhead while still allowing the capture of keystrokes of even fast typists. Their keylogger can track all keystrokes, store this data in the large GPU memory and process it in real-time using GPU's massive compute-power.