A Review of Spatial Sound for Virtual Environments and Games with Graphics Processing Units <s> INTRODUCTION <s> 1. Auditory Space.- 2. The Physical and Psychophysical Basis of Sound Localization.- 3. Digital Signal Processing for the Auditory Scientist: A Tutorial Introduction.- 4. Generation and Validation of Virtual Auditory Space.- 5. Implementation of Virtual Acoustic Space for Neurophysiological Studies of Directional Hearing.- 6. Recent Developments in Virtual Auditory Space. <s> BIB001 </s> A Review of Spatial Sound for Virtual Environments and Games with Graphics Processing Units <s> INTRODUCTION <s> It is well known that the effectiveness of loudspeaker-based 3D audio systems is critically dependent on the listener being in a known position, the so-called "sweet spot". In this paper we propose a new system that provides increased robustness to perturbations such as head movement, reverberation, and different head shapes. The system is based on using a particular geometry that offers a combination of symmetric and asymmetric loudspeaker positions. This combination increases the bandwidth over which the proposed system is robust, when compared with existing 3D audio systems. <s> BIB002 </s> A Review of Spatial Sound for Virtual Environments and Games with Graphics Processing Units <s> INTRODUCTION <s> In this paper, we present Brook for GPUs, a system for general-purpose computation on programmable graphics hardware. Brook extends C to include simple data-parallel constructs, enabling the use of the GPU as a streaming co-processor. We present a compiler and runtime system that abstracts and virtualizes many aspects of graphics hardware. In addition, we present an analysis of the effectiveness of the GPU as a compute engine compared to the CPU, to determine when the GPU can outperform the CPU for a particular algorithm. We evaluate our system with five applications, the SAXPY and SGEMV BLAS operators, image segmentation, FFT, and ray tracing. For these applications, we demonstrate that our Brook implementations perform comparably to hand-written GPU code and up to seven times faster than their CPU counterparts. <s> BIB003 </s> A Review of Spatial Sound for Virtual Environments and Games with Graphics Processing Units <s> INTRODUCTION <s> It is a common belief that computer performance growth is over 50% annually, or that performance doubles every 18-20 months. By analyzing publicly available results from the SPEC integer (CINT) benchmark suites, we conclude that this was true between 1985 and 1996 -- the early years of the RISC paradigm.During the last 7.5 years (1996-2004), however, performance growth has slowed down to 41%, with signs of a continuing decline. Meanwhile, clock frequency has improved with about 29% annually. The improvement in clock frequency was enabled both by an annual device speed scaling of 20% as well as by longer pipelines with a lower gate-depth in each stage. This paper takes a fresh look at -- and tries to remove the confusion about -- performance scaling that exists in the computer architecture community. <s> BIB004 </s> A Review of Spatial Sound for Virtual Environments and Games with Graphics Processing Units <s> INTRODUCTION <s> For several years, graphics processing units' performance has been increasing faster than the pace predicted by Moore's law. This has occurred because GPUs must meet the demands of increasingly complex visual effects in games and entertainment applications. The performance and functionality have made GPUs potentially attractive as coprocessors for general-purpose computation. Researchers have also developed new algorithms and applications that exploit GPUs' parallelism and vector-processing capabilities, in which one operation can process an entire vector of numbers. <s> BIB005 </s> A Review of Spatial Sound for Virtual Environments and Games with Graphics Processing Units <s> INTRODUCTION <s> GPUs have moved away from the traditional fixed-function 3D graphics pipeline toward a flexible general-purpose computational engine. Today, GPUs can implement many parallel algorithms directly using graphics hardware. Well-suited algorithms that leverage all the underlying computational horsepower often achieve tremendous speedups. Truly, the GPU is the first widely deployed commodity desktop parallel computer <s> BIB006 </s> A Review of Spatial Sound for Virtual Environments and Games with Graphics Processing Units <s> INTRODUCTION <s> The rapid increase in the performance of graphics hardware, coupled with recent improvements in its programmability, have made graphics hardware a compelling platform for computationally demanding tasks in a wide variety of application domains. In this report, we describe, summarize, and analyze the latest research in mapping general-purpose computation to graphics hardware. We begin with the technical motivations that underlie general-purpose computation on graphics processors (GPGPU) and describe the hardware and software developments that have led to the recent interest in this field. We then aim the main body of this report at two separate audiences. First, we describe the techniques used in mapping general-purpose computation to graphics hardware. We believe these techniques will be generally useful for researchers who plan to develop the next generation of GPGPU algorithms and techniques. Second, we survey and categorize the latest developments in general-purpose application development on graphics hardware. This survey should be of particular interest to researchers who are interested in using the latest GPGPU applications in their systems of interest. <s> BIB007 </s> A Review of Spatial Sound for Virtual Environments and Games with Graphics Processing Units <s> INTRODUCTION <s> To be immersed in a virtual environment, the user must be presented with plausible sensory input including auditory cues. A virtual (three-dimensional) audio display aims to allow the user to perceive the position of a sound source at an arbitrary position in three-dimensional space despite the fact that the generated sound may be emanating from a fixed number of loudspeakers at fixed positions in space or a pair of headphones. The foundation of virtual audio rests on the development of technology to present auditory signals to the listener's ears so that these signals are perceptually equivalent to those the listener would receive in the environment being simulated. This paper reviews the human perceptual and technical literature relevant to the modeling and generation of accurate audio displays for virtual environments. Approaches to acoustical environment simulation are summarized and the advantages and disadvantages of the various approaches are presented. <s> BIB008
A virtual (or three-dimensional (3D), or spatial) audio system (or audio display) allows a listener to perceive the position of a sound source(s), emanating from a static number of stationary loudspeakers or a pair of headphones, as coming from arbitrary locations in three-dimensional space. Spatial sound technology goes far beyond traditional stereo and surround sound techniques by allowing a virtual sound source to have such attributes as left-right, back-forth, and up-down  . Incorporating spatialized auditory information in an immersive virtual environment and video games is beneficial for a variety of reasons. Spatial auditory cues can add a better sense of "presence" or "immersion", compensate for poor visual cues (graphics), and at the very least, add a "pleasing quality" to the simulation   . Despite these benefits and despite the fact that spatial sound is a critical cue to the perception of our environment, it is often overlooked in immersive virtual environments and video games where, historically, emphasis has been placed on the visual senses BIB001  . That being said, the generation of spatial sound for dynamic, and interactive virtual environments using traditional software-based methods and techniques is computationally very expensive except for trivial environments which are typically of little use. Driven by the gaming industry, consumer computer graphics hardware has greatly advanced in recent years, out *Address correspondence to this author at the Faculty of Business and Information Technology, University of Ontario Institute of Technology, 2000 Simcoe Street North, Oshawa, Ontario, Canada, L1H 7K4; Email: bill.kapralos@uoit.ca performing the computational capacity of central processing units (CPUs). A graphics processing unit (GPU) is a dedicated graphics rendering device whose purpose is to provide a high performance, visually rich, interactive 3D experience by exploiting the inherent parallelism in the feed-forward graphics pipeline BIB006 . In contrast to the processors on-board consumer-grade audio cards, the GPUs available on all modern video cards have moved away from the traditional fixedfunction 3D graphics pipeline towards a flexible generalpurpose computational engine that can currently implement many parallel algorithms directly using graphics hardware. This results in tremendous computational speed-ups. Due to a number of reasons including the explosion of the consumer video game market and advances in manufacturing technology, GPUs are, on a dollar-per-dollar basis, the most powerful computational hardware, providing "tremendous memory bandwidth and computational horsepower" BIB007 . GPUs are also becoming faster and more powerful very quickly, far exceeding Moore's Law applied to traditional microprocessors BIB004 . In fact, instead of doubling every 18 months as with CPUs, GPU performance increases by a factor of five every 18 months or doubles every eight months BIB005 . In contrast to older GPUs that contained a fixed-function pipeline with output limited to 8-bits-per-color-channel, current GPUs include fully programmable processing units which support vectorized floating point operations BIB007 . As a result, a number of high level languages have been introduced to allow for the control of vertex and pixel pipelines BIB003 . Given the typically large computational requirements associated with spatial sound generation and audio processing in general, the GPU is an economical and computationally feasible alternative to traditional software-based meth-ods and techniques. With respect to the potential computational e ciencies that GPUs o er and their applicability to audio processing, this paper reviews the research e orts that have examined the application of the GPU to the generation of spatial sound and audio processing for virtual environments and video games. Various approaches will be summarized and in the process of doing so, advantages, disadvantages, limitations, drawbacks, and trade-o s will be presented. Being an overview, this paper does not introduce any new research results. Rather, it presents a general review of GPU-based spatial sound and audio processing compiling the relevant information available from a variety of sources, providing the reader with a summary of the technological literature relevant to the creation of spatial sound using the GPU. The foundation of spatial sound rests on the ability to control the auditory signals arriving at the listener's ears such that these signals are perceptually equivalent to the signals the listener would receive in the environment being simulated BIB002 . However, a review of human auditory perception is beyond the scope of this work (an excellent overview of human auditory perception is available in  ). Similarly, a complete overview of spatial sound will not be described here but a recent review is available in BIB008 .