A Review of Spatial Sound for Virtual Environments and Games with Graphics Processing Units <s> Composition Stage <s> Wearable, camera based, head-tracking systems use spatial image registration algorithms to align images taken as the wearer gazes around their environment. This allows for computer-generated information to appear to the user as though it was anchored in the real world. Often, these algorithms require creation of a multiscale Gaussian pyramid or repetitive re-projection of the images. Such operations, however can be computationally expensive, and such head-tracking algorithms are desired to run in real-time on a body borne computer In this paper we present a method of using the 3D computer graphics hardware that is available in a typical wearable computer to accelerate the repetitive image projections required in many computer vision algorithms. We apply this "graphics for vision" technique to a wearable camera based head-tracking algorithm, implemented on a wearable computer with 3D graphics hardware. We perform an analysis of the acceleration achieved by applying graphics hardware to computer vision to create a Mediated Reality. <s> BIB001 </s> A Review of Spatial Sound for Virtual Environments and Games with Graphics Processing Units <s> Composition Stage <s> We present a novel use of commodity graphics hardware to perform real-time image segmentation and image morphology operations. Our preliminary results show a performance increase of over 30% using an nVidia GeForce4 when compared to an implementation using Intel MMX optimized code on a 2.2 Ghz Intel P4 CPU. <s> BIB002 </s> A Review of Spatial Sound for Virtual Environments and Games with Graphics Processing Units <s> Composition Stage <s> The latest real-time graphics architectures include programmable floating-point vertex and fragment processors, with support for data-dependent control flow in the vertex processor. We present a programming language and a supporting system that are designed for programming these stream processors. The language follows the philosophy of C, in that it is a hardware-oriented, general-purpose language, rather than an application-specific shading language. The language includes a variety of facilities designed to support the key architectural features of programmable graphics processors, and is designed to support multiple generations of graphics architectures with different levels of functionality. The system supports both of the major 3D graphics APIs: OpenGL and Direct3D. This paper identifies many of the choices that we faced as we designed the system, and explains why we made the decisions that we did. <s> BIB003 </s> A Review of Spatial Sound for Virtual Environments and Games with Graphics Processing Units <s> Composition Stage <s> In this paper, we present Brook for GPUs, a system for general-purpose computation on programmable graphics hardware. Brook extends C to include simple data-parallel constructs, enabling the use of the GPU as a streaming co-processor. We present a compiler and runtime system that abstracts and virtualizes many aspects of graphics hardware. In addition, we present an analysis of the effectiveness of the GPU as a compute engine compared to the CPU, to determine when the GPU can outperform the CPU for a particular algorithm. We evaluate our system with five applications, the SAXPY and SGEMV BLAS operators, image segmentation, FFT, and ray tracing. For these applications, we demonstrate that our Brook implementations perform comparably to hand-written GPU code and up to seven times faster than their CPU counterparts. <s> BIB004 </s> A Review of Spatial Sound for Virtual Environments and Games with Graphics Processing Units <s> Composition Stage <s> We present new algorithms for performing fast computation of several common database operations on commodity graphics processors. Specifically, we consider operations such as conjunctive selections, aggregations, and semi-linear queries, which are essential computational components of typical database, data warehousing, and data mining applications. While graphics processing units (GPUs) have been designed for fast display of geometric primitives, we utilize the inherent pipelining and parallelism, single instruction and multiple data (SIMD) capabilities, and vector processing functionality of GPUs, for evaluating boolean predicate combinations and semi-linear queries on attributes and executing database operations efficiently. Our algorithms take into account some of the limitations of the programming model of current GPUs and perform no data rearrangements. Our algorithms have been implemented on a programmable GPU (e.g. NVIDIA's GeForce FX 5900) and applied to databases consisting of up to a million records. We have compared their performance with an optimized implementation of CPU-based algorithms. Our experiments indicate that the graphics processor available on commodity computer systems is an effective co-processor for performing database operations. <s> BIB005 </s> A Review of Spatial Sound for Virtual Environments and Games with Graphics Processing Units <s> Composition Stage <s> In this paper, we present a method for real-time visual simulation of diverse dynamic phenomena using programmable graphics hardware. The simulations we implement use an extension of cellular automata known as the coupled map lattice (CML). CML represents the state of a dynamic system as continuous values on a discrete lattice. In our implementation we store the lattice values in a texture, and use pixel-level programming to implement simple next-state computations on lattice nodes and their neighbors. We apply these computations successively to produce interactive visual simulations of convection, reaction-diffusion, and boiling. We have built an interactive framework for building and experimenting with CML simulations running on graphics hardware, and have integrated them into interactive 3D graphics applications. <s> BIB006 </s> A Review of Spatial Sound for Virtual Environments and Games with Graphics Processing Units <s> Composition Stage <s> We present algorithms for fast quantile and frequency estimation in large data streams using graphics processors (GPUs). We exploit the high computation power and memory bandwidth of graphics processors and present a new sorting algorithm that performs rasterization operations on the GPUs. We use sorting as the main computational component for histogram approximation and construction of e-approximate quantile and frequency summaries. Our algorithms for numerical statistics computation on data streams are deterministic, applicable to fixed or variable-sized sliding windows and use a limited memory footprint. We use GPU as a co-processor and minimize the data transmission between the CPU and GPU by taking into account the low bus bandwidth. We implemented our algorithms on a PC with a NVIDIA GeForce FX 6800 Ultra GPU and a 3.4 GHz Pentium IV CPU and applied them to large data streams consisting of more than 100 million values. We also compared the performance of our GPU-based algorithms with optimized implementations of prior CPU-based algorithms. Overall, our results demonstrate that the graphics processors available on a commodity computer system are efficient stream-processor and useful co-processors for mining data streams. <s> BIB007 </s> A Review of Spatial Sound for Virtual Environments and Games with Graphics Processing Units <s> Composition Stage <s> The rapid increase in the performance of graphics hardware, coupled with recent improvements in its programmability, have made graphics hardware a compelling platform for computationally demanding tasks in a wide variety of application domains. In this report, we describe, summarize, and analyze the latest research in mapping general-purpose computation to graphics hardware. We begin with the technical motivations that underlie general-purpose computation on graphics processors (GPGPU) and describe the hardware and software developments that have led to the recent interest in this field. We then aim the main body of this report at two separate audiences. First, we describe the techniques used in mapping general-purpose computation to graphics hardware. We believe these techniques will be generally useful for researchers who plan to develop the next generation of GPGPU algorithms and techniques. Second, we survey and categorize the latest developments in general-purpose application development on graphics hardware. This survey should be of particular interest to researchers who are interested in using the latest GPGPU applications in their systems of interest. <s> BIB008
Pixel values are determined from the fragments. In contrast to the "traditional" fixed-function pipelines with "modern" (programmable) GPUs, both the vertex and fragment stages are user-programmable. Programs written to control the vertex stage are known as vertex programs or vertex shaders while programs written to control the fragment stage are known as fragment programs or fragment shaders. Early on, these programs were written in assembly language. However, higher level languages have been introduced, including Microsoft's high level shading language (HLSL), OpenGL shading language (GLSL)  , NVIDIA's compute unified device architecture (CUDA), and NVIDIA's Cg BIB003 . Generally, the input to both of these programmable stages is four-element vectors where each element represents a 32-bit floating point number. The vertex stage will output a limited number of 32-bit, four element vectors while the fragment stage will output a maxi mum of four floating point, four element vectors that typically represent color. The fragment stage is capable of fetching data from texture memory (i.e., perform memory gather) but cannot alter the address of its output which is determined before processing of the fragment begins (i.e., incapable of memory scatter). In contrast, within the vertex stage, the position of input vertices can be altered, thus a ecting where the image pixels will be drawn (i.e., the vertex stage supports both memory gather and memory scatter) BIB008 . In addition to vertex and fragment Shader Model 4.0 currently supported by Direct3D 10 and OpenGL 3.0 defines a new type of shader, the geometry shader. A geometry shader receives input from the vertex shader, can be used to create new geometry and is capable of operating on entire primitives  . In order to take advantage of the power inherent in GPUs in addition to their relatively low cost, recently, a number of e orts have investigated the use of GPUs to a variety of noncomputer graphics applications. Collectively, this e ort is known as "general purpose computing on the GPU" (GPGPU) and given the flexibility of GPUs, has led to a number of GPU-based applications, outside the scope for which GPUs were originally designed for BIB008 . Examples include solving di erential equations and general linear algebra problems BIB008 , applications in computer vision BIB001 , image processing BIB002 , implementation of fast Fourier transform BIB004 , the simulation of dynamic phenomena described by partial di erential equations (e.g., boiling, convection, and chemical reaction di usion) BIB006 , database and data mining BIB005 BIB007 , and audio processing. That being said, currently GPUs do not support integers and associated operations including bit-wise logical operations making them ill-suited for a operations requiring such features (e.g., cryptography). A thorough review including a detailed summary of GPGPU-based applications is provided by Owens et al. BIB008 and will therefore not be provided here.