Pedestrian Protection Systems: Issues, Survey, and Challenges <s> A. Sensor Modalities <s> This book, a groundbreaking resource, offers professionals a comprehensive overview of cutting-edge intelligent vehicle (IV) systems aimed at providing enhanced safety, greater productivity, and less stress for drivers. Rather than bogging readers down with difficult technical discourse, this easy to understand book presents a conceptual and realistic view of how IV systems work and the issues involved with their introduction into road vehicles. Offering a thorough understanding of how electronics and electronic systems must work within automobiles, heavy trucks and buses, the book examines: (1) real world IV products, along with practical issues, including cost, market aspects, driver interface, and user acceptance; (2) current systems such as adaptive cruise control, lane departure warning, and forward collision mitigation; (3) the next wave of driver assist systems, including pedestrian detection, lane-keeping assistance, and seamless information flow between road vehicle and the road infrastructure; (4) traffic assist systems, in which intelligent vehicles automatically coordinate their movements to improve traffic flow; and (5) a view of the future of this rapidly evolving technological area. <s> BIB001 </s> Pedestrian Protection Systems: Issues, Survey, and Challenges <s> A. Sensor Modalities <s> This paper presents a 3D-camera system and appropriate algorithms for the image processing to provide pedestrian recognition. According to international legislative proposals the automotive industry is forced to take action in the area of protecting vulnerable road users. The presented photonic mixer device (PMD) sensor system is able to fulfil specific requirements of a pedestrian protection assistant. It consists of a sensor with a resolution of 64 times 16 pixel, lightsources, and the image processing unit. The light emitting diodes (LEDs) of the lightsources emit a modulated signal in the infrared (IR) spectrum. The sensor calculates the object distance by means of the phase of the reflected signal. The range of the system is approximately 15 m for pedestrians and up to 30 m for objects with high reflectivity, e.g. cars with number plates. The horizontal field-of-view is 55deg. The presented image processing unit consists of two main steps. In the first step both a robust and efficient segmentation method is performed to create reliable detections of the objects and a useful description of their projection in the image plane. It uses the linking pyramid method regarding the partition of the distance image and a contour-based grouping algorithm, where the objects are described using the chain code representation. In the second step a classification of the resulting objects is carried out. Depending on the distance of the objects which have to be classified a shape- or motion-based verification is applied adaptively. Both approaches discussed in this paper deliver very good results at the corresponding distance and represent a solid foundation of further works <s> BIB002
Various types of sensors have been employed for vehicle as well as infrastructure-based pedestrian detection systems. Commonly used sensors for detecting pedestrians are imaging sensors in various configurations using visible light and infrared (IR) radiation, as well as the "time-of-flight" sensors such as RADARs and LASER scanners. Imaging sensors can capture a high-resolution perspective view of the scene, but extracting information involves substantial amount of processing. On the other hand, time-of-flight scanners directly give accurate information about object distance, but resolution is often limited. In this sense, these two types of sensors are complementary, and their fusion is expected to result in more robust detection. Table II gives a comparison of various types of sensors that are used for pedestrian detection. It is a marvel that the human visual system can process vast amount of data from the scene and extract information in real time that enables driving. Video sensors would therefore be a natural choice for intelligent driver support systems. Various approaches for detecting pedestrians in visible light images are shown in Table III . The video camera technology is mature and cost effective. However, processing of video data to extract useful information is a complex task. In particular, separating objects from background clutter is a difficult problem in computer vision. Furthermore, visible light becomes less effective during dark conditions. Thermal IR sensors are sensitive to the radiation emitted by the human body and, hence, are very effective for detection of pedestrians, particularly at night. Although these sensors are expensive at present, the costs have been decreasing, and they have been of considerable interest for night vision in vehicles. Luxury cars have already started offering systems that increase the range of sight in the car by displaying a thermal IR image. However, thermal imaging is less effective in hot daytime conditions where there is less temperature difference between the pedestrians and the background. Another type of sensor that can be useful for night TABLE IV  PEDESTRIAN DETECTION USING NONVISIBLE LIGHT AND TIME-TO-FLIGHT SENSORS vision is a near-IR sensor accompanied by an illuminator. As of now, such systems are less expensive than thermal IR sensors, and they have been used for surveillance applications. In fact, an ordinary black-and-white charge-coupled device (CCD) camera is sensitive to the near-IR spectrum and can be used in these systems. Unlike thermal IR, these sensors produce images that resemble visible light images. Hence, image processing techniques that are developed for monochrome visible light images can be easily modified for analysis with these images. They also have higher resolution for comparable field of view (FOV) due to larger number of pixels in the CCD compared to thermal cameras. Table IV shows various systems that use thermal or near IR for performing pedestrian detection. The imaging sensors give a 2-D perspective projection of the 3-D scene, losing the depth information in the process. Although binocular stereo can recover depth information, it needs the solution of the difficult and often ambiguous correspondence problem of matching significant features between the images. The time-of-flight sensors, such as RADARs and LASER scanners, directly give accurate depth information by measuring the time it takes for the emitted rays to return to the sensor. Pedestrian detection using time-of-flight sensors is shown in Table V . RADAR sensors use electromagnetic energy in the microwave region in order to measure the distance to the objects. In order to localize objects in azimuthal dimension, multibeam RADARs are often employed in vehicular applications. RADAR-based adaptive cruise control that maintains a distance to the lead vehicle has been introduced in luxury vehicles over the past few years. LASER scanners have recently invoked considerable interest in the IV community for detection, tracking, and classification of road users. A LASER scanner consists of a pulsed LASER transmitter, a receiver, and a mirror rotating around the vertical axis. The mirror reflects the LASER pulse in order to direct it at any azimuth angle in the horizontal plane. The scanner therefore generates a range map of a horizontal section of the scene called scanning plane, providing as outputs the range and reflectivity data at a number of azimuth angle samples. However, in the case of vehicles, pitching and road curvature would not always keep the scanning plane parallel to the road, therefore missing actual obstacles or detecting the road as an obstacle. Multilayer laser scanners have been designed to solve this problem. For example, the Automotive Multi-Layer LASER scanner ALASCA designed by IBEO uses four beams in order to cover the road scene. Currently, LASER scanner technology is very expensive, but mass production could make it economically viable for vehicles. According to IBEO, it may be possible to introduce initial application of LASER scanner as early as 2008 BIB001 . In BIB002 , a novel 3-D sensor system using photonic mixer device is proposed. This system contains a 64 × 16 CCD camera with LED light sources emitting a modulated IR signal. The sensor calculates the object distance by measuring the phase difference between the transmitted and the reflected signals. Thus, a distance and amplitude images are created, which are used by the image processing step to identify the pedestrians. Every sensor has its advantages and limitations. In order to enhance the advantages and overcome the limitations, one can use a combination of multiple sensors that give complementary information. For example, the day-time capabilities of visible light sensors could be combined with night-time capabilities of IR sensors. Higher resolution of imaging sensors could be combined with range information at low angular resolution from time-of-flight sensors to obtain 3-D information at higher resolution. Systems using multiple sensors are also shown in Table V .