A Survey of Techniques for Improving Security of GPUs <s> Fig. 3. Illustration of (a) normal execution (b) EoC attack and (c) EoK attack [17] <s> This paper adopts a method to retrieve graphic data stored in the global memory of an NVIDIA GPU. Experimentation shows that a 24-bit TIFF formatted graphic can be retrieved from the GPU in a forensically sound manner. However, like other types of Random Access Memory, acquired data cannot be verified due to the volatile nature of the GPU memory. In this work a Color Pattern Map Test is proposed to reveal the relationship between a graphic and its GPU memory organization. The mapping arrays derived from such testing can be used to visually restore graphics stored in the GPU memory. Described ‘photo tests’ and ‘redo tests’ demonstrate that it is possible to visually restore a graphic from the data stored in GPU memory. While initial results are promising, more work is still needed to determine if such methods of data acquisition within GPU memory can be considered forensically sound. <s> BIB001 </s> A Survey of Techniques for Improving Security of GPUs <s> Fig. 3. Illustration of (a) normal execution (b) EoC attack and (c) EoK attack [17] <s> Abstract According to previous reports, information could be leaked from GPU memory; however, the security implications of such a threat were mostly over-looked, because only limited information could be indirectly extracted through side-channel attacks. In this paper, we propose a novel algorithm for recovering raw data directly from the GPU memory residues of many popular applications such as Google Chrome and Adobe PDF reader. Our algorithm enables harvesting highly sensitive information including credit card numbers and email contents from GPU memory residues. Evaluation results also indicate that nearly all GPU-accelerated applications are vulnerable to such attacks, and adversaries can launch attacks without requiring any special privileges both on traditional multi-user operating systems, and emerging cloud computing scenarios. <s> BIB002
In EoK attack, shown in Figure 3 (c), the registers (termed as "private memory" in OpenCL  ) and shared memory of victim's kernel are stolen after termination of the victim's kernel. Since GPUs do not allow preemptive scheduling, long GPU programs are run using multiple same/different kernels. These kernels produce and consume intermediate results which can be leaked by EoK. In NVIDIA GPUs where the L1 cache and shared memory can exchange space  , EoK attack can also obtain the data from the L1 data cache. In EoC attack, the adversary waits till the victim program's GPU context ends and gets the final data from the global memory, if it has not been cleared by the victim program. The adversary dumps the data to the host memory only after the GPU context ends. In EoK attack, the adversary targets to get the intermediate data between kernels of the victim program. The adversary runs a loop to dump data to the host memory after each kernel ends. The loop ends when the GPU context ends. When multiple users are running their kernels on the GPU, the data of only the last kernel can be attacked by EoK, however, the EoC attack can succeed even when multiple victims run kernels sequentially. On AMD GPUs, the available memory size cannot be measured at runtime, and hence, they use the change in kernel execution time for detecting when a victim is using the GPU. The EoC attack is not required in AMD GPUs since an adversary can obtain the whole global memory of an AMD GPU due to its memory management features. They demonstrate their attacks using Firefox and Chromium web-browsers. These browsers use GPUs for accelerating webpage rendering and leave rendered webpage textures in the global memory. The textures are stored in a rearranged form using proprietary schemes. Also, due to the use of virtual and page-based memory and presence of non-color data, reconstructing the original texture is infeasible. Hence, they instead seek to infer the visited webpages based on the rearranged textures. The adversary collects memory dumps of certain webpages on the same GPU as the victim. Then, he compares them with memory dumps of an unknown webpage. This attack can accurately guess most of the webpages. Further, if the adversary collects image snapshots of certain webpages on any GPU, and compares them with memory dumps of unknown webpages, he can still guess nearly half of the webpages. The limitation of their technique is that it uses only side-channel data and cannot obtain raw images from GPU memories. Also, it can match the target against the websites in a pre-decided list only. Zhou et al. BIB002 discuss an attack on GPU where a nonprivileged adversary can leak sensitive information from the remanent raw data in GPU memory. They first detect data partitions which have a high likelihood of being parts of images. The challenge in this is that the metadata of images are generally stripped. Hence, the accurate metadata such as image dimensions cannot be easily measured. Second, GPUs also store and process non-image data which makes it difficult to distinguish the image data. To resolve these issues, they use many features of image data. Initially, the adversary overwrites entire memory that it can access with a fixed value, e.g., 0xFF. Then, the malicious program executes in the background and monitors the free GPU memory space. A sudden increase in available GPU memory indicates that a victim has released GPU memory portion. The malicious program copies this and since GPU does not erase the memory before reallocation, the adversary can analyze the memory. Then, they split the memory dumps into fixed-size partitions and merge those partitions which are contiguous in memory space and are expected to be pixels. The partition size needs to be smaller than a single image but large enough that any large white region does not break-down the image. For example, they find that a partition size of 4K balances these tradeoffs. The partitions with all 0xFF are removed since they are unlikely to have been modified after initialization. Also, the partitions with all 0x00 are removed since these partitions have been zeroed by the OS/application. The remaining contiguous blocks are merged to form a tile. Then, remanent partitions of the victim program are extracted. Although skipping partitions with all 0xFF may leave out white spaces of an image, the remaining image is still meaningful and this justifies the choice of 0xFF as the canary. To identify partitions storing graphical data, they note that pixels are stored as four 8b channels which show red, green, blue and alpha components, respectively. The alpha component, which is 0x00 or 0xFF, shows pixel transparency. Thus, by detecting 0x00 or 0xFF, one can ascertain an opaque pixel or an unused channel. Thus, by examining the alpha value, a graphical partition can be detected. Finally, heading/trailing elements having 0x00/0xFF values are removed and middle values are retained. Generally, a tile has only single image, however, some tiles may have portions from multiple images if they are at nearby locations in the memory. Then, they detect image area in the tiles with imprecise boundaries, based on insights that (1) successive rows/columns of an image show high similarities which can be detected in the frequency domain (2) before loading into GPU, the image is decompressed and its similarities are preserved. Finally, they rearrange recovered images in proper order. They show that their technique can recover images even under high noise and after performing image transformations such as changing the contrast/brightness. They demonstrate the recovery of the opened tabs, address bar and page-body from Google chrome, figure-portions and text-lines from recently-opened Adobe Reader documents, whole or portions of images from MATLAB. Similarly, Zhang et al. BIB001 study the relationship between a graphic and its organization in GPU memory. Based on this, they present a scheme for visually recovering a graphic from the data stored in GPU memory.