Pedestrians and vehicles often share the road in complex inner city traffic. This leads to interactions between the vehicle and pedestrians, with each affecting the other's motion. In order to create robust methods to reason about pedestrian behavior and to design interfaces of communication between self-driving cars and pedestrians we need to better understand such interactions. In this paper, we present a data-driven approach to implicitly model pedestrians' interactions with vehicles, to better predict pedestrian behavior. We propose a LSTM model that takes as input the past trajectories of the pedestrian and ego-vehicle, and pedestrian head orientation, and predicts the future positions of the pedestrian. Our experiments based on a real-world, inner city dataset captured with vehicle mounted cameras, show that the usage of such cues improve pedestrian prediction when compared to a baseline that purely uses the past trajectory of the pedestrian.
More closely related to our work are approaches that use vehicle mounted cameras and use information of pedestrian dynamics coupled with the pedestrian's awareness @cite_29 @cite_2 @cite_15 @cite_3 @cite_0 @cite_18 . However they are usually focused on specific scenarios as signalized crosswalks @cite_3 and intersections @cite_0 , or does not have a high diversity in number of scenarios and pedestrians @cite_15 @cite_19 @cite_29 @cite_32 . Our approach is not restricted to a specific scenario and we reason about pedestrian future positions by learning such cues from realistic data.
@cite_18 @cite_29 @cite_32 @cite_3 @cite_0 @cite_19 @cite_2 @cite_15
2004641798 2789309368 2291379690 2101415982
Future vehicle systems for active pedestrian safety will not only require a high recognition performance but also an accurate analysis of the developing traffic situation. In this paper, we present a study on pedestrian path prediction and action classification at short subsecond time intervals. We consider four representative approaches: two novel approaches (based on Gaussian process dynamical models and probabilistic hierarchical trajectory matching) that use augmented features derived from dense optical flow and two approaches as baseline that use positional information only (a Kalman filter and its extension to interacting multiple models). In experiments using stereo vision data obtained from a vehicle, we investigate the accuracy of path prediction and action classification at various time horizons, the effect of various errors (image localization, vehicle egomotion estimation), and the benefit of the proposed approaches. The scenario of interest is that of a crossing pedestrian, who might stop or continue walking at the road curbside. Results indicate similar performance of the four approaches on walking motion, with near-linear dynamics. During stopping, however, the two newly proposed approaches, with nonlinear and or higher order models and augmented motion features, achieve a more accurate position prediction of 10-50 cm at a time horizon of 0-0.77 s around the stopping event. We consider scenarios where we have zero instances of real pedestrian data (e.g., a newly installed surveillance system in a novel location in which no labeled real data or unsupervised real data exists yet) and a pedestrian detector must be developed prior to any observations of pedestrians. Given a single image and auxiliary scene information in the form of camera parameters and geometric layout of the scene, our approach infers and generates a large variety of geometrically and photometrically accurate potential images of synthetic pedestrians along with purely accurate ground-truth labels through the use of computer graphics rendering engine. We first present an efficient discriminative learning method that takes these synthetic renders and generates a unique spatially-varying and geometry-preserving pedestrian appearance classifier customized for every possible location in the scene. In order to extend our approach to multi-task learning for further analysis (i.e., estimating pose and segmentation of pedestrians besides detection), we build a more generalized model employing a fully convolutional neural network architecture for multi-task learning leveraging the “free" ground-truth annotations that can be obtained from our pedestrian synthesizer. We demonstrate that when real human annotated data is scarce or non-existent, our data generation strategy can provide an excellent solution for an array of tasks for human activity analysis including detection, pose estimation and segmentation. Experimental results show that our approach (1) outperforms classical models and hybrid synthetic-real models, (2) outperforms various combinations of off-the-shelf state-of-the-art pedestrian detectors and pose estimators that are trained on real data, and (3) surprisingly, our method using purely synthetic data is able to outperform models trained on real scene-specific data when data is limited. With the rapid development of the techniques for autonomous driving and ADAS in the last decade, more advanced methods to understand pedestrian behavior are required. Crosswalks at intersections are the one of most hazardous where many accidents between turning-vehicles and pedestrians occur. In this paper, we present a method for estimating the pedestrian’s intention to cross a signalized crosswalk or stop in front of it. The intention is crucial to not only the collision avoidance but also smooth traffic in the context of autonomous driving by reducing unnecessary risk margins. Regarding the behavioral flow of pedestrian: assessment, decision-making and physical movement, as a stochastic process, we construct a probabilistic model with the Dynamic Bayesian Network. It takes account of not only pedestrian physical states but also contextual information and integrates the relationship among them. By employing the particle filter as a Bayesian filtering framework, the model estimates the pedestrian state from signal information and pedestrian position measurements. Evaluation using experimental data collected in real traffic scene shows that the proposed model has an ability to detect the pedestrian intention to cross a crosswalk even when he she is far from it. We present a novel Dynamic Bayesian Network for pedestrian path prediction in the intelligent vehicle domain. The model incorporates the pedestrian situational awareness, situation criticality and spatial layout of the environment as latent states on top of a Switching Linear Dynamical System (SLDS) to anticipate changes in the pedestrian dynamics. Using computer vision, situational awareness is assessed by the pedestrian head orientation, situation criticality by the distance between vehicle and pedestrian at the expected point of closest approach, and spatial layout by the distance of the pedestrian to the curbside. Our particular scenario is that of a crossing pedestrian, who might stop or continue walking at the curb. In experiments using stereo vision data obtained from a vehicle, we demonstrate that the proposed approach results in more accurate path prediction than only SLDS, at the relevant short time horizon (1 s), and slightly outperforms a computationally more demanding state-of-the-art method.