Pedestrian Protection Systems: Issues, Survey, and Challenges <s> C. Video Cues for Pedestrian Detection <s> In this paper, we present a real-time pedestrian detection system that uses a pair of moving cameras to detect both stationary and moving pedestrians in crowded environments. This is achieved through stereo-based segmentation and neural network-based recognition. Stereo-based segmentation allows us to extract objects from a changing background; neural network-based recognition allows us to identify pedestrians in various poses, shapes, sizes, clothing, occlusion status. The experiments on a large number of urban street scenes demonstrate the feasibility of the approach in terms of pedestrian detection rate and frame processing rate. <s> BIB001 </s> Pedestrian Protection Systems: Issues, Survey, and Challenges <s> C. Video Cues for Pedestrian Detection <s> Current car safety systems are passive systems. Modern car assistance systems are based only on vehicle data. Future safety systems will also include object recognition in the near frontal area of the vehicle to detect dangerous situations. Therefore, special sensors and algorithms are needed. The paper discusses a system using a laserscanner and a video camera. <s> BIB002 </s> Pedestrian Protection Systems: Issues, Survey, and Challenges <s> C. Video Cues for Pedestrian Detection <s> This paper presents a method for pedestrian detection and tracking using a night vision video camera installed on the vehicle. To deal with the nonrigid nature of human appearance on the road, a two-step detection/tracking method is proposed. The detection phase is performed by a support vector machine (SVM) with size-normalized pedestrian candidates and the tracking phase is a combination of Kalman filter prediction and mean shift tracking. The detection phase is further strengthened by information obtained by a road detection module that provides key information for pedestrian validation. Experimental comparisons have been carried out on gray-scale SVM recognition vs. binary SVM recognition and entire body detection vs. upper body detection. <s> BIB003 </s> Pedestrian Protection Systems: Issues, Survey, and Challenges <s> C. Video Cues for Pedestrian Detection <s> Vehicle-mounted laser scanners are able to observe the vehicles environment in order to detect, track and classify the surrounding objects and thus providing data for active safety systems. The latest development of IBEO combines several innovations. The receiver diodes are arranged in an array, which enables simultaneous measurements in 4 horizontal planes, e.g. to compensate pitching of the vehicle. In addition a multi target capability is integrated. This technique enables the detection of two distances with a single measurement, thus enhancing the robustness against rain. This paper introduces improved high speed object detection and high performance object tracking algorithms for real-time data processing. Additionally a classification of the road users is possible. A system architecture for detection and modelling of dynamic traffic scenes is introduced in order to provide a general idea of the different tasks necessary to reach the aim of a complete environmental model using a sensor for a wide range of applications. <s> BIB004 </s> Pedestrian Protection Systems: Issues, Survey, and Challenges <s> C. Video Cues for Pedestrian Detection <s> This paper describes a pedestrian detection system that integrates image intensity information with motion information. We use a detection style algorithm that scans a detector over two consecutive frames of a video sequence. The detector is trained (using AdaBoost) to take advantage of both motion and appearance information to detect a walking person. Past approaches have built detectors based on appearance information, but ours is the first to combine both sources of information in a single detector. The implementation described runs at about 4 frames/second, detects pedestrians at very small scales (as small as 20/spl times/15 pixels), and has a very low false positive rate. Our approach builds on the detection work of Viola and Jones. Novel contributions of this paper include: i) development of a representation of image motion which is extremely efficient, and ii) implementation of a state of the art pedestrian detection system which operates on low resolution images under difficult conditions (such as rain and snow). <s> BIB005 </s> Pedestrian Protection Systems: Issues, Survey, and Challenges <s> C. Video Cues for Pedestrian Detection <s> This paper presents a method for pedestrian detection using a stereo night vision system installed on the vehicle. Motion information extracted from an image sequence is used to locate possible pedestrians, as moving people have motions that are not consistent with the movement of the background. Experimental results are shown to validate our approach and comparisons have been carried out between our approach and frame-by-frame based pattern recognition approaches. <s> BIB006 </s> Pedestrian Protection Systems: Issues, Survey, and Challenges <s> C. Video Cues for Pedestrian Detection <s> This paper presents a method for active background subtraction in sequential images taken from a moving camera on the vehicle. The active subtraction is carried out by estimating camera motion using a gyrosensor. Applying the proposed method to contour matching, we present a fast and robust system for detecting moving pedestrians. <s> BIB007 </s> Pedestrian Protection Systems: Issues, Survey, and Challenges <s> C. Video Cues for Pedestrian Detection <s> This paper presents the improvements of a system for pedestrian detection in infrared images. The system is based on a multi-resolution localization of warm symmetrical objects with specific size and aspect ratio; the multi-resolution approach allows to detect both close and far pedestrians. A match against a set of 3D models encoding human shape's morphological and thermal characteristics is used as a validation process to remove false positives. No temporal correlation, nor motion cues are used for the processing that is based on the analysis of single frames only. <s> BIB008 </s> Pedestrian Protection Systems: Issues, Survey, and Challenges <s> C. Video Cues for Pedestrian Detection <s> This paper presents a general, trainable system for object detection in unconstrained, cluttered scenes. The system derives much of its power from a representation that describes an object class in terms of an overcomplete dictionary of local, oriented, multiscale intensity differences between adjacent regions, efficiently computable as a Haar wavelet transform. This example-based learning approach implicitly derives a model of an object class by training a support vector machine classifier using a large set of positive and negative examples. We present results on face, people, and car detection tasks using the same architecture. In addition, we quantify how the representation affects detection performance by considering several alternate representations including pixels and principal components. We also describe a real-time application of our person detection system as part of a driver assistance system. <s> BIB009 </s> Pedestrian Protection Systems: Issues, Survey, and Challenges <s> C. Video Cues for Pedestrian Detection <s> Nighttime driving is more dangerous than daytime driving-particularly for senior drivers. Three to four times as many driving-related deaths occur at night than in the daytime. To improve the safety of night driving, automatic pedestrian detection based on infrared images has drawn increased attention because pedestrians tend to stand out more against the background in infrared images than they do in visible light images. Nevertheless, pedestrian detection in infrared images is by no means trivial-many of the known difficulties carry over from visible light images, such as image variability occasioned by pedestrians being in different poses. Typically, several different pedestrian templates have to be used in order to deal with a range of poses. Furthermore, pedestrian detection is difficult because of poor infrared image quality (low resolution, low contrast, few distinguishable feature points, little texture information, etc.) and misleading signals. To address these problems, this paper introduces a shape-independent pedestrian-detection method. Our segmentation algorithm first estimates pedestrians' horizontal locations through projection-based horizontal segmentation and then determines pedestrians' vertical locations through brightness/bodyline-based vertical segmentation. Our classification method defines multidimensional histogram-, inertia-, and contrast-based classification features. The features are shape-independent, complementary to one another, and capture the statistical similarities of image patches containing pedestrians with different poses. Thus, our pedestrian-detection system needs only one pedestrian template-corresponding to a generic walking pose-and avoids brute-force searching for pedestrians throughout whole images, which typically involves brightness-similarity comparisons between candidate image patches and a multiplicity of pedestrian templates. Our pedestrian-detection system is neither based on tracking nor does it depend on camera calibration to determine the relationship between an object's height and its vertical image locations. Thus, it is less restricted in applicability. Even if much work is still needed to bridge the gap between present pedestrian-detection performance and the high reliability required for real-world applications, our pedestrian-detection system is straightforward and provides encouraging results in improving speed, reliability, and simplicity. <s> BIB010 </s> Pedestrian Protection Systems: Issues, Survey, and Challenges <s> C. Video Cues for Pedestrian Detection <s> We describe the functional and architectural breakdown of a monocular pedestrian detection system. We describe in detail our approach for single-frame classification based on a novel scheme of breaking down the class variability by repeatedly training a set of relatively simple classifiers on clusters of the training set. Single-frame classification performance results and system level performance figures for daytime conditions are presented with a discussion about the remaining gap to meet a daytime normal weather condition production system. <s> BIB011 </s> Pedestrian Protection Systems: Issues, Survey, and Challenges <s> C. Video Cues for Pedestrian Detection <s> This paper presents the results of the first large-scale field tests on vision-based pedestrian protection from a moving vehicle. Our PROTECTOR system combines pedestrian detection, trajectory estimation, risk assessment and driver warning. The paper pursues a "system approach" related to the detection component. An optimization scheme models the system as a succession of individual modules and finds a good overall parameter setting by combining individual ROCs using a convex-hull technique. On the experimental side, we present a methodology for the validation of the pedestrian detection performance in an actual vehicle setting. We hope this test methodology to contribute towards the establishment of benchmark testing, enabling this application to mature. We validate the PROTECTOR system using the proposed methodology and present interesting quantitative results based on tens of thousands of images from hours of driving. Although results are promising, more research is needed before such systems can be placed at the hands of ordinary vehicle drivers. <s> BIB012 </s> Pedestrian Protection Systems: Issues, Survey, and Challenges <s> C. Video Cues for Pedestrian Detection <s> This paper presents a novel pedestrian detection method based on the use of a convolutional neural network (CNN) classifier. Our method achieves high accuracy by automatically optimizing the feature representation to the detection task and regularizing the neural network. We evaluate the proposed method on a difficult database containing pedestrians in a city environment with no restrictions on pose, action, background and lighting conditions. The false positive rate (FPR) of the proposed CNN classifier is less than 1/5-th of the FPR of a support vector machine (SVM) classifier using Haar-wavelet features when the detection rate is 90%. The accuracy of the SVM classifier using the features learnt by the CNN is equivalent to the accuracy of the CNN, confirming the importance of automatically optimized features. The computational demand of the CNN classifier is, however, more than an order of magnitude lower than that of the SVM, irrespective of the type of features used. <s> BIB013 </s> Pedestrian Protection Systems: Issues, Survey, and Challenges <s> C. Video Cues for Pedestrian Detection <s> Vehicle warning and control systems are the key component of ITS. Pedestrian detection is an important research content of vehicle active safety. The central idea behind such pedestrian safety systems is to protect the pedestrian from injuries. In this paper, we address the problem of pedestrian represent and detection where the motion cue is not used. Inspired by the work proposed by Zehang Sun [2004], we proposed a pedestrian feature representation approach based on sparse Gabor filters (SGF) learning from examples. In the phase of pedestrian detection, we used support vector machine to detect the pedestrian. Promising results demonstrate the potential of the proposed framework. <s> BIB014 </s> Pedestrian Protection Systems: Issues, Survey, and Challenges <s> C. Video Cues for Pedestrian Detection <s> Recently, the focus of safety systems for intelligent vehicles has been on researching and developing Advanced Driver Assistance Systems (ADAS). Most efforts have been concentrated at the driver, not taking into account the protection of the most vulnerable road users. This paper describes a pedestrian detection algorithm based on stereo vision. The use of visual information is a promising approach to cope with the different appearances of pedestrians and changes of illumination in cluttered environments. Active contour models are used to detect and track people from the images taken by an on-board vision system, performing contour extraction in sequential frames. <s> BIB015 </s> Pedestrian Protection Systems: Issues, Survey, and Challenges <s> C. Video Cues for Pedestrian Detection <s> Thispaperpresents acomplete methodforpedes- trian detection applied toinfrared images. First, westudy an imagedescriptor basedonhistograms oforiented gradients (HOG),associated witha Support Vector Machine(SVM) classifier andevaluate itsefficiency. Afterhaving tunedthe HOG descriptor andtheclassifier, weinclude this methodin acomplete system, whichdeals withstereo infrared images. Thisapproach gives goodresults forwindowclassification, andapreliminary testapplied onavideosequence proves thatthis approach isverypromising. I.INTRODUCTION Since thelast fewyears now,thedevelopment ofdriving assistance systems hasbeenveryactive inorder toin- crease thevehicle anditsenvironment safety. Atthe present time, themainobjective inthisdomainisto provide thedrivers withsomeinformation concerning its environment andanypotential hazard. Oneamongall useful information isthedetection andlocalization ofa pedestrian infront ofavehicle. Thisproblem ofdetecting pedestrians isaverydifficult problemthathasessentially beenaddressed using vision sensors, imageprocessing andpattern recognition techniques. Inparticular, detecting pedestrians thanks to images isacomplex challenge duetotheir appearance andposevariability. Inthecontext ofdaylight vision, several approaches havebeenproposed andarebased on different imageprocessing techniques ormachine learning (9), (5), (12). Recently, owingtothedevelopment oflow-cost infrared cameras, night vision systems havegained moreand moreinterest, thusincreasing theneedofautomatic detection ofpedestrians atnight. Thisproblem of detecting pedestrians frominfrared images hasbeen investigated byvarious research teamsinthelast years. Themainmethodology isbasedon extracting cues (symmetry, shape-independent <s> BIB016 </s> Pedestrian Protection Systems: Issues, Survey, and Challenges <s> C. Video Cues for Pedestrian Detection <s> Automatic detection of moving objects at distance and in all weather conditions is a critical task in many visionbased safety applications such as video surveillance and vehicle forewarn collision warning. In such applications, prior knowledge about the object class (vehicle, pedestrian, tree, etc.) and imaging conditions (shadow, depth) is unavailable. What makes the task even more challenging is when the camera is non-stationary, e.g., mounted on a moving vehicle. The essential problem in this case lies in distinguishing between camera-induced motion and independent motion. This paper proposes a robust algorithm for automatic moving object detection at distance. The camera is mounted on a moving vehicle and operates in both day and night time. Through the utilization of the focus of expansion (FOE) and its associated residual map, the proposed method is able to detect and separate independently moving objects (IMOs) from the "moving" background caused by the camera motion. Experimentations on numerous realworld driving videos have shown the effectiveness of the proposed technique. Moving objects such as pedestrians and vehicles up to 40 meters away from the camera have been reliably detected at 10 frames per second on a 1.8GHz PC. <s> BIB017
In the case of imaging sensors, the shape and appearance of the pedestrians can be used to separate them from the background. For this purpose, characteristic features are extracted from images, and a trained classifier is used to separate pedestrian from the background and other objects. Some of the features used for appearance-based detection are raw subimages BIB013 , size and aspect ratio of bounding boxes BIB008 , Haar wavelets BIB009 , Gabor filter outputs BIB014 , symmetry BIB008 ,  , intensity gradients BIB001 and their histograms BIB016 , and active contours BIB015 . In BIB005 , texture information is extracted using simple masks, and classification is performed based on integrating the weak classifiers obtained from these masks. In thermal IR images, pedestrians that are warmer than the background form hot spots, which are used for detection, as in BIB003 . In BIB010 , features based on histogram, inertia, and contrast are used to distinguish pedestrians. Motion is also an important cue in detecting pedestrians. In the case of stationary infrastructure-based cameras, background subtraction is used to separate moving objects from static background. However, in the case of moving platforms, the background undergoes ego-motion that depends on the camera motion as well as the scene structure. For laterally moving pedestrians, it is usually feasible to separate the pedestrian motion from ego-motion. However, for longitudinally moving pedestrians, the image motion is parallel to the ego-motion and, therefore, difficult to separate. The vehicle ego-motion can be split into rotation and translation. Rotational motion in video does not depend on the distance of the scene feature and is sometimes neglected BIB006 , BIB017 or compensated for using gyrosensors BIB007 . The translational motion is inversely proportional to the distance to the scene and, hence, can be used in determining the scene structure. In the absence of rotational motion, the image motion vectors converge at a single point in the image called the focus of expansion (FOE). In BIB017 , ego-motion estimation is performed using sparse optical flow at corner-like features. Motion of outliers corresponding to independently moving objects do not pass through FOE and are clustered using region-growing segmentation on the residual image. In BIB006 , a two-stage stereo correspondence and motion-detection procedure is developed to distinguish an object motion that is inconsistent with the background. This procedure does not need explicit ego-motion computation. Motion information can also be combined with texture information, as in BIB005 . An extremely efficient representation of image motion is developed based on five types of shifted image differences. Features characteristic to periodicity of human body motion which is within a frequency range are useful in detecting pedestrians and separating them from other moving and stationary objects. Spatial motion distribution represented by moment features  , power spectral distribution of the motion time series  , symmetry characteristics of the legs  , and gait BIB011 are some of the cues used to detect and verify pedestrian candidates. Stereo cameras as well as time-of-flight sensors return information from which the distance of the object from the camera can be computed. This information is very useful for disambiguating pedestrians from background, handling occlusion between pedestrians, and eliminating extraneous features based on the image size. Disparity discontinuities can be used to aid segmentation, as in BIB012 and BIB001 , to divide the image of the scene into layers. In BIB015 , stereo is used to guide the active contour model for pedestrians. LASER scanners output radial distance at discrete azimuth angles in the scanning plane. These data are clustered into objects based on range discontinuities BIB002 and grouping measurements near each other in the 3-D space BIB004 ,  . The objects are tracked and classified into number of classes using models of object outlines and their dynamic behavior. The system also warns the driver or activates automatic braking in case of imminent collision.