Pedestrian Protection Systems: Issues, Survey, and Challenges <s> IV. INFRASTRUCTURE-BASED SOLUTIONS FOR PEDESTRIAN DETECTION <s> We present an algorithm for segmentation of traffic scenes that distinguishes moving objects from their moving cast shadows. A fading memory estimator calculates mean and variance of all three color components for each background pixel. Given the statistics for a background pixel, simple rules for calculating its statistics when covered by a shadow are used. Then, MAP classification decisions are made for each pixel. In addition to the color features, we examine the use of neighborhood information to produce smoother classification. We also propose the use of temporal information by modifying class a priori probabilities based on predictions from the previous frame. <s> BIB001 </s> Pedestrian Protection Systems: Issues, Survey, and Challenges <s> IV. INFRASTRUCTURE-BASED SOLUTIONS FOR PEDESTRIAN DETECTION <s> This paper presents various issues related to the development of an integrated software architecture for a traffic incident monitoring, mitigation, and analysis system. The novel concept of using a set of distributed databases, having many different functions and types, is proposed for distributed coordination of sensing, control and analysis algorithms. This coordination paradigm using databases makes the whole architecture robust by providing means to efficiently manage current and past states of the monitored environment and the monitoring system. Use of a semantic event/activity database in the integrated architecture also provides high level abstractions through its query language to model traffic incidents and traffic behaviors. We also present experimental results of the use of a concrete database-centered architecture and algorithms in identifying important traffic flow events (such as tail-gating, exit from a ramp, etc.). <s> BIB002 </s> Pedestrian Protection Systems: Issues, Survey, and Challenges <s> IV. INFRASTRUCTURE-BASED SOLUTIONS FOR PEDESTRIAN DETECTION <s> We describe a novel architecture for developing distributed video networks for incident detection and management. The networks utilize both rectilinear and omnidirectional cameras. It is recognized that robust and reliable segmentation of automobiles and shadows is critical in our application. We describe new segmentation procedure and present experimental results to support the basic feasibility and utility of the algorithms. <s> BIB003 </s> Pedestrian Protection Systems: Issues, Survey, and Challenges <s> IV. INFRASTRUCTURE-BASED SOLUTIONS FOR PEDESTRIAN DETECTION <s> Moving shadows need careful consideration in the development of robust dynamic scene analysis systems. Moving shadow detection is critical for accurate object detection in video streams since shadow points are often misclassified as object points, causing errors in segmentation and tracking. Many algorithms have been proposed in the literature that deal with shadows. However, a comparative evaluation of the existing approaches is still lacking. In this paper, we present a comprehensive survey of moving shadow detection approaches. We organize contributions reported in the literature in four classes two of them are statistical and two are deterministic. We also present a comparative empirical evaluation of representative algorithms selected from these four classes. Novel quantitative (detection and discrimination rate) and qualitative metrics (scene and object independence, flexibility to shadow situations, and robustness to noise) are proposed to evaluate these classes of algorithms on a benchmark suite of indoor and outdoor video sequences. These video sequences and associated "ground-truth" data are made available at http://cvrr.ucsd.edu/aton/shadow to allow for others in the community to experiment with new algorithms and metrics. <s> BIB004 </s> Pedestrian Protection Systems: Issues, Survey, and Challenges <s> IV. INFRASTRUCTURE-BASED SOLUTIONS FOR PEDESTRIAN DETECTION <s> The goal of this project is to monitor activities at traffic intersections for detecting/predicting situations that may lead to accidents. Some of the key elements for robust intersection monitoring are camera calibration, motion tracking, incident detection, etc. In this paper, we consider the motion-tracking problem. A multilevel tracking approach using Kalman filter is presented for tracking vehicles and pedestrians at intersections. The approach combines low-level image-based blob tracking with high-level Kalman filtering for position and shape estimation. An intermediate occlusion-reasoning module serves the purpose of detecting occlusions and filtering relevant measurements. Motion segmentation is performed by using a mixture of Gaussian models which helps us achieve fairly reliable tracking in a variety of complex outdoor scenes. A visualization module is also presented. This module is very useful for visualizing the results of the tracker and serves as a platform for the incident detection module. <s> BIB005 </s> Pedestrian Protection Systems: Issues, Survey, and Challenges <s> IV. INFRASTRUCTURE-BASED SOLUTIONS FOR PEDESTRIAN DETECTION <s> Video surveillance activity has dramatically increased over the past few years. Earlier work dealt mostly with single stationary cameras, but the recent trend is toward active multicamera systems. Such systems offer several advantages over single camera systems - multiple overlapping views for obtaining 3D information and handling occlusions, multiple nonoverlapping cameras for covering wide areas, and active pan-tilt-zoom (PTZ) cameras for observing object details. To address these issues, we have developed a multicamera video surveillance approach, called distributed interactive video array. The DIVA framework provides multiple levels of semantically meaningful information ("situational" awareness) to match the needs of multiple remote observers. We have designed DIVA-based systems that can track and identify vehicles and people, monitor perimeters and bridges, and analyze activities. A new video surveillance approach employing a large-scale cluster of video sensors demonstrates the promise of multicamera arrays for homeland security. <s> BIB006 </s> Pedestrian Protection Systems: Issues, Survey, and Challenges <s> IV. INFRASTRUCTURE-BASED SOLUTIONS FOR PEDESTRIAN DETECTION <s> In this paper, we address the problem of detecting pedestrians in crowded real-world scenes with severe overlaps. Our basic premise is that this problem is too difficult for any type of model or feature alone. Instead, we present an algorithm that integrates evidence in multiple iterations and from different sources. The core part of our method is the combination of local and global cues via probabilistic top-down segmentation. Altogether, this approach allows examining and comparing object hypotheses with high precision down to the pixel level. Qualitative and quantitative results on a large data set confirm that our method is able to reliably detect pedestrians in crowded scenes, even when they overlap and partially occlude each other. In addition, the flexible nature of our approach allows it to operate on very small training sets. <s> BIB007 </s> Pedestrian Protection Systems: Issues, Survey, and Challenges <s> IV. INFRASTRUCTURE-BASED SOLUTIONS FOR PEDESTRIAN DETECTION <s> Video-based surveillance systems have a wide range of applications for traffic monitoring, as they provide more information as compared to other sensors. In this paper, we present a rule-based framework for behavior and activity detection in traffic videos obtained from stationary video cameras. Moving targets are segmented from the images and tracked in real time. These are classified into different categories using a novel Bayesian network approach, which makes use of image features and image-sequence-based tracking results for robust classification. Tracking and classification results are used in a programmed context to analyze behavior. For behavior recognition, two types of interactions have mainly been considered. One is interaction between two or more mobile targets in the field of view (FoV) of the camera. The other is interaction between targets and stationary objects in the environment. The framework is based on two types of a priori information: 1) the contextual information of the camera's FoV, in terms of the different stationary objects in the scene and 2) sets of predefined behavior scenarios, which need to be analyzed in different contexts. The system can recognize behavior from videos and give a lexical output of the detected behavior. It also is capable of handling uncertainties that arise due to errors in visual signal processing. We demonstrate successful behavior recognition results for pedestrian-vehicle interaction and vehicle-checkpost interactions. <s> BIB008 </s> Pedestrian Protection Systems: Issues, Survey, and Challenges <s> IV. INFRASTRUCTURE-BASED SOLUTIONS FOR PEDESTRIAN DETECTION <s> This paper presents an efficient and robust paradigm for analysis and query of moving-object interactions in planar homography domain.People and vehicle activities/interactions are analyzed for situational awareness by using a multi-perspective approach.Planar homography constraints are exploited to extract view-invariant object features including footage area and velocity of objects on the ground plane. Spatio-temporal relationships between person-and vehicle-tracks are represented by a semantic event grammar. Semantic-level information of the situation is achieved with the anticipation of possible directions of near-future tracks using piecewise velocity history. An efficient query paradigm is proposed by histogram-based approximation of probability density functions of objects and by quad-tree indexing. Experimental data show promising results.Our framework can be applied to applications for enhanced situational awareness such as disaster prevention,human interactions in structured environments,and crowd movement analysis in wide-view areas. <s> BIB009
Infrastructure-mounted cameras have been extensively studied for video surveillance as well as traffic analysis BIB006 . Detection and tracking of objects from these cameras is easier and more reliable due to the absence of camera motion. Background subtraction, which is one of the standard methods to extract moving objects from stationary background, is often employed, followed by classification of objects and activities. Shadow removal is important in getting a robust performance of object detection in outdoor scenes. A detailed comparative evaluation of various shadow detection algorithms in the literature is performed in BIB004 . In BIB001 , a shadow analysis algorithm, which is based on color and brightness changes due to cast shadows, is proposed. Here, we briefly describe the research on detecting and analyzing people from stationary cameras. Ideas used in many of these systems would also be useful for pedestrian protection. In BIB005 , vehicles and pedestrians are tracked by combining lowlevel blob analysis using a mixture of Gaussian background models with high-level Kalman filtering to determine position and shape. Occlusions between objects as well as between the object and the background are handled based on detecting the shape changes. In crowded scenes, there are severe occlusions between people due to which the silhouettes of the persons are not well separated. In such cases, it is useful to combine global Fig. 6 . Estimation of interaction patterns of moving objects with different velocities using the homography-mapped footage regions in the projection planes. Green track corresponds to the walking person, and red track corresponds to the car BIB009 . models with local models at body part level. An approach for detecting pedestrians in crowded scenes is proposed in BIB007 . The first stage uses local features to generate hypotheses about potential object locations using the scale invariant extension of implicit shape model. The hypotheses are verified using a probabilistic top-down segmentation that is based on minimum description length principle. A comprehensive video surveillance system for traffic-monitoring application is described in BIB008 . The system detects and tracks moving vehicles and pedestrians, classifies these objects, and analyzes their behaviors. A Bayesian network classifier is proposed, which uses the target attributes such as size, position, velocity, type, and track to deduce the target class in probabilistic terms. Our laboratory has been actively involved in research based on multicamera systems for traffic analysis. The research project Autonomous Agents for On-Scene Networked Incident Management (ATON) aimed at making contributions to the realization of powerful and integrated traffic-incident detection, monitoring, and recovery system to reduce congestion on highways and to make travel safer, smoother, and economical, and to reduce pollution. In BIB002 , hierarchical database architecture is proposed for detecting, storing, and querying of traffic incidents. The semantic abstractions are arranged hierarchically as events (atomic), activities (spatio-temporal composition of events), behaviors (set of related activities), and incidents (higher level abstraction from events, activities, and behaviors). A prototype of such a database was designed, developed, and used for detecting complex traffic activities such as exiting the freeway, tailgating, and simultaneous entering of a lane while turning. The details of sensor network using rectilinear and omnidirectional cameras, with algorithms for detecting vehicles and removing shadows, are described in BIB003 . Although this research was conducted for freeway traffic, many of the concepts developed in this research would also be useful for pedestrian incident detection. Activities and interactions of people and vehicle are analyzed in BIB009 , as shown in Fig. 6 . For detection, wide baseline stereo cameras are used, with planar homography for robustly generating tracks on the ground plane. A semantic event grammar is designed for the representation of the spatio-temporal relationship between people and vehicle tracks to identify events. The future prediction of tracks is performed using piecewise velocity history. A quad-tree-based indexing scheme is used for efficient querying of events that are stored in the database. The systems described above perform detection and analysis of humans and vehicles, including their motion and behavior from stationary cameras that are mounted on the infrastructure. Wireless communication would enable these systems to communicate with the vehicles, which can then generate appropriate warnings for the driver or activate the autonomous protection systems.