A Review of Spatial Sound for Virtual Environments and Games with Graphics Processing Units <s> GPU-based Sound Processing <s> In this work, the emphasis is on the development of strategies to realize techniques of numerical computing on the graphics chip. In particular, the focus is on the acceleration of techniques for solving sets of algebraic equations as they occur in numerical simulation. We introduce a framework for the implementation of linear algebra operators on programmable graphics processors (GPUs), thus providing the building blocks for the design of more complex numerical algorithms. In particular, we propose a stream model for arithmetic operations on vectors and matrices that exploits the intrinsic parallelism and efficient communication on modern GPUs. Besides performance gains due to improved numerical computations, graphics algorithms benefit from this model in that the transfer of computation results to the graphics processor for display is avoided. We demonstrate the effectiveness of our approach by implementing direct solvers for sparse matrices, and by applying these solvers to multi-dimensional finite difference equations, i.e. the 2D wave equation and the incompressible Navier-Stokes equations. <s> BIB001 </s> A Review of Spatial Sound for Virtual Environments and Games with Graphics Processing Units <s> GPU-based Sound Processing <s> Modal synthesis is a physically-motivated sound modeling method. It has been successfully used in many applications. However, if large number of modes are involved in a simulated scene, it becomes an overwhelming task to synthesize sounds in real time without special hardware support. An implementation based on commodity graphics hardware is proposed as an alternative solution by using the parallelism and programmability in graphics pipeline. <s> BIB002 </s> A Review of Spatial Sound for Virtual Environments and Games with Graphics Processing Units <s> GPU-based Sound Processing <s> A technology for automatically creating and adding the sound to interactive CG animations of spark discharges in real time has been developed. In the procedure proposed in this paper, the user inputs the electric charge distribution, boundary conditions and other parameters affecting the initiation of electric discharges in virtual space. The animation of the discharge is then created by generating the shape of the discharge pattern and rendering it, and the sound synchronized with the animation is automatically generated in real time. The noises from spark discharges are shock waves, which exhibit complicated behavior; but, in this study, an empirical shape for a shock wave is employed to efficiently generate the acoustic waveform. Effective procedures for expressing lightning discharges and continuous discharges are also proposed. <s> BIB003 </s> A Review of Spatial Sound for Virtual Environments and Games with Graphics Processing Units <s> GPU-based Sound Processing <s> We present a virtual drum with a large range of interaction modes: The user can change the shape of the drum while playing it through a controller that offers hitting the virtual drum skin at 16 different points as well as simultaneously damping it and pushing it in. The real-time, low-latency simulation leverages the computing power of a GPU, which allows the resolution of the mesh to be increased greatly. This and the interactive control of shape and local tension are differences to the work of Jones and Schloss [2007]. <s> BIB004
GPUs have also been applied to a wide variety of audiobased applications. Von Tycowicz and Loviscach BIB004 describe the implementation of a flexible virtual drum that is simulated in real-time and with low latency on the GPU. The drum is modeled using a 64 64 mesh where each point is connected to neighbor points with springs and dampers. Using the GPU has allowed the resolution of the mesh to be increased. The user can modify the shape of the drum in realtime. A MIDI controller with 16 pressure points is used for pressure recognition and a finite di erence method is employed to synthesize sound based on location and pressure information. Matsuyama et al. BIB003 , describe a method for the automatic generation of real-time sound for graphics-based animation of sparks to simulate thunder and lighting e ects. The implementation also makes use of GPU-based numerical methods introduced by Kruger and Westermann BIB001 . There have been a number of e orts to utilize the GPU for the implementation of a variety of digital signal processing methods and techniques motivated by the fact that most DSP functions are suitable for GPU-based processing (i.e., they are parallelizable, are highly arithmetic intense, have limited data dependency, and make use of multiply-add (MADD) calculation units that are included on GPU architecture). Using the Cg shading language, Whalen  implements seven common audio functions: chorus, compress, delay, high-pass filter, low-pass filter, noise-gate and normalization. A performance comparison was made between the GPU and corresponding CPU implementation using a Pentium IV (3.0 GHz CPU) and an NVIDIA GeForce FX 5200 video card. The GPU showed better performance for several of the functions (compress and chorus with speedups of up to a factor of four). However, the CPU implementation was better for other functions (high-pass and low-pass filters). It was suggested that GPU performance was poorer for some algorithms given the implementation of these algorithms was not suitable for GPU implementation given that they required (computationally expensive) texture access. With more modern video cards, texture access has been improved and this will undoubtedly lead to greater improvements in these methods. Trebien and Oliveira  propose a GPU-based method for real-time sound generation and multichannel audio processing to be used in live music performances. The motivation behind the approach comes from the observation that many DSP units include several independent units that work in parallel. They mapped a network model of virtually interconnected software modules to the GPU graphics pipeline. In their design, audio blocks are stored in texture memory and are passed between modules as texture coordinates. An audio sample corresponds to a fragment with its amplitude stored as a luminance value. Since GPU memory access is restricted, they use multi-pass access to implement a "ping-pong" model whereby a register is write-only in one pass and becomes read-only in the next pass. Using this approach, several common audio algorithms such as additive synthesis (used to generate new sound waves from sinusoids), sawtooth, square or triangular waves, feedback delay, gain, and envelope shaping (a variant of amplitude modulation that scales each sample by a fast varying factor) were implemented. The system was tested on a computer with an AMD 2.21GHz CPU and an NVIDIA GeForce 8800 GTX video card. The GPU showed speed-ups of up to four orders of magnitude (17,641 ) over a CPU implementation. It is suggested that in the future, additional algorithms that implement various filters and frequency modulation can be developed using this approach. Gallo and Tsingos  considered the application of GPUs to variable delay-line (delaying the signal of each sound source by the propagation time of the sound wave) and filtering (filtering the audio signal to simulate directivity functions, occlusions, and interaction with the medium of propagation). Variable delay line and filtering are two common spatial audio processing operations  . Delaying the signal involved re-sampling the signal at non-integer index values and was performed on the GPU using texture resampling. Filtering was performed using a four-band equalizer and implemented on the GPU using a four-component dot-product. Sound signals were stored as RGBA textures where each of the components held a band-pass copy of the original signal. Experimental results indicated a performance increase associated with the GPU-based implementation when compared to optimized software implementations on a standard CPU. Despite the promising results, their work also showed that there are still a number of shortcomings that limit the e cient use of GPU processors for "mainstream" audio processing. In particular, long 1D textures cannot be accessed easily, and infinite impulse response filters (commonly used in audio processing) cannot be implemented e ciently. The scheme was implemented on a Pentium IV 1.8 GHz CPU and an ATI Radeon 5700 graphics card. Each sound event was a three sub-band monaural signal at 44.1 kHz and was processed in blocks of 1024 samples. A scene with approximately 70,000 polygons was rendered concurrently with audio. One shortcoming of the implementation is that since each cluster's signals are premixed, bus tra c is increased. Also, because the GPU only supported 8-bit mixing as opposed to the CPU's 32-bit support, the quality of the GPU rendered signal was not as good as the signal processed on the CPU. However, the processing required 38% of the CPU's processing time. Without the use of the GPU, the scheme can render 50 to 60 sources while using the GPU allows for rendering of more than 160 sources possible. Modal synthesis is a physically-based audio synthesis method to model sounds made by objects through a bank of damped harmonic oscillators that are excited by an external stimulus  . Zhang et al. BIB002 present a two-phase GPUbased modal synthesis method based on a bank of damped harmonic oscillators that are excited by an external stimulus to model a vibrating object. Two sets of factors a ect the system: i) static factors, which are independent of interaction and include geometry and material properties, and ii) dynamic factors, which depend on the interaction and include contact location and external force. The first set of factors are taken into account during a pre-computation stage and the second set are incorporated during run-time. Each of the modes is precalculated in parallel and stored in 2D textures for runtime retrieval. 10-30 sampling contact locations are employed for common objects. In the first step, the response for each individual mode for all sounding objects is calculated and is implemented as a dot product between two vectors (performed e ciently on the GPU). The second step involves summarizing the responses from the objects; it is essentially a reduction operation that is implemented as a multi-pass "ping-pong" process. For a texture with N N resolution, log 2 2N rendering passes are performed until the final sum is obtained in a single pixel. Some experimental results are presented but the experiments were restricted to a maximum of 5000 modes due to hardware restrictions (Pentium IV 2.8 GHz CPU and an NVIDIA GeForce 6800 GT video card). Although performance is not ideal, it is suggested that this is due to the slow AGP memory bus of the 6800 GT video card and results will improve by employing a video card that employs a PCI-Express bus.