Pedestrians and vehicles often share the road in complex inner city traffic. This leads to interactions between the vehicle and pedestrians, with each affecting the other's motion. In order to create robust methods to reason about pedestrian behavior and to design interfaces of communication between self-driving cars and pedestrians we need to better understand such interactions. In this paper, we present a data-driven approach to implicitly model pedestrians' interactions with vehicles, to better predict pedestrian behavior. We propose a LSTM model that takes as input the past trajectories of the pedestrian and ego-vehicle, and pedestrian head orientation, and predicts the future positions of the pedestrian. Our experiments based on a real-world, inner city dataset captured with vehicle mounted cameras, show that the usage of such cues improve pedestrian prediction when compared to a baseline that purely uses the past trajectory of the pedestrian.
Interactions among traffic agents have been explored in prior work by means of explicit communication between pedestrians and drivers @cite_41 , pedestrians and cars @cite_28 @cite_26 @cite_31 , implicit communication @cite_13 , crosswalks scenarios @cite_24 @cite_4 , through the usage of gestures @cite_42 , and by analyzing interactions with objects that may interfere with pedestrian awareness of the environment @cite_22 . Recent work presented by Rasouli and Tsotsos @cite_34 summarized pedestrian behavior studies, discussing interactions between pedestrians and autonomous vehicles, and also presented factors that pedestrians take into consideration when crossing streets. @cite_9 used VR to insert pedestrians in virtual environments. The focus of the study was to estimate the pedestrians' acceptance of autonomous cars. Rothenb " u @cite_28 used a Wizard of Oz approach to analyze the interactions between pedestrians and autonomous cars, where the driver camouflaged themselves to blend in with the seat, simulating the appearance of autonomous vehicles.
@cite_26 @cite_4 @cite_22 @cite_41 @cite_28 @cite_9 @cite_42 @cite_24 @cite_31 @cite_34 @cite_13
2951834965 2507800322 2551641937 2903870308 2291379690 2082585576 2734962429 2561106390 2890092150 2963315052 2888662204 2757876960 2010256323 2004641798 2963888093 1951211040 2758542544 2889428601 2949153416 2079567201
Interactions between vehicles and pedestrians have always been a major problem in traffic safety. Experienced human drivers are able to analyze the environment and choose driving strategies that will help them avoid crashes. What is not yet clear, however, is how automated vehicles will interact with pedestrians. This paper proposes a new method for evaluating the safety and feasibility of the driving strategy of automated vehicles when encountering unsignalized crossings. MobilEye sensors installed on buses in Ann Arbor, Michigan, collected data on 2,973 valid crossing events. A stochastic interaction model was then created using a multivariate Gaussian mixture model. This model allowed us to simulate the movements of pedestrians reacting to an oncoming vehicle when approaching unsignalized crossings, and to evaluate the passing strategies of automated vehicles. A simulation was then conducted to demonstrate the evaluation procedure. In urban environments traffic safety is primarily determined by the successful interaction between road users. In order to define behavioral requirements for future autonomous vehicles, an understanding of these interactions is needed. Therefore this paper analyzes the interaction between drivers and pedestrians at crosswalks from both the driver's and the pedestrian's perspective. A study with driving experiments in real-life traffic was conducted. The study not only investigates how drivers respond to pedestrians who intend to use the crosswalk, but also their response to pedestrians who waive their right-of-way. An analysis of recorded driving and video data identified typical behavior patterns of drivers approaching a crosswalk, as well as of pedestrians crossing on it. Supplemented by questionnaires filled in by drivers and pedestrians, we determined factors which have a relevant impact on successful driver-pedestrian interaction. We confirmed that with higher initial speeds, drivers tend to exhibit more anticipatory driving behavior. Contrary to previous studies, the presented results show a significant influence of the vehicle's speed on the pedestrian's decision process. The gained insights into the driver-pedestrian interaction process provide an opportunity to develop autonomous driving functions with a cooperative driving behavior that conforms to the expectations of both pedestrians and vehicle occupants. How will pedestrians and bicyclists interact with autonomous vehicles when there is no human driver? In this paper, we outline a novel method for performing observational field experiments to investigate interactions with driverless cars. We provide a proof-of-concept study (N=67), conducted at a crosswalk and a traffic circle, which applies this method. In the study, participants encountered a vehicle that appeared to have no driver, but which in fact was driven by a human confederate hidden inside. We constructed a car seat costume to conceal the driver, who was specially trained to emulate an autonomous system. Data included video recordings and participant responses to post-interaction questionnaires. Pedestrians who encountered the car reported that they saw no driver, yet they managed interactions smoothly, except when the car misbehaved by moving into the crosswalk just as they were about to cross. This method is the first of its kind, and we believe that it contributes a valuable technique for safely acquiring empirical data and insights about driverless vehicle interactions. These insights can then be used to design vehicle behaviors well in advance of the broad deployment of autonomous technology. Cooperative interaction in traffic is vital for resolving a wide range of ambiguities arising from road users' actions. Autonomous vehicles are no exception and require the ability to understand the intention of road users and communicate with them in order to ensure their safety and maintain traffic flow. In this paper, we address the problem of traffic interaction by analyzing a large sample of pedestrians communicating with drivers. We highlight the ways pedestrians communicate and use a logistic regression model to identify what factors influence communication patterns of pedestrians and how. We also discuss practical challenges regarding the recognizing and understanding of pedestrians' intention and how our theoretical findings can help to solve them. Our analysis suggests that pedestrians predominantly rely on implicit communication cues such as stepping onto the road to transmit their intention of crossing. In addition, we found that the presence of traffic signal, street width, and pedestrian group size can influence the frequency and type of pedestrian communication, while factors such as pedestrians' age and gender did not show any significant impact. With the rapid development of the techniques for autonomous driving and ADAS in the last decade, more advanced methods to understand pedestrian behavior are required. Crosswalks at intersections are the one of most hazardous where many accidents between turning-vehicles and pedestrians occur. In this paper, we present a method for estimating the pedestrian’s intention to cross a signalized crosswalk or stop in front of it. The intention is crucial to not only the collision avoidance but also smooth traffic in the context of autonomous driving by reducing unnecessary risk margins. Regarding the behavioral flow of pedestrian: assessment, decision-making and physical movement, as a stochastic process, we construct a probabilistic model with the Dynamic Bayesian Network. It takes account of not only pedestrian physical states but also contextual information and integrates the relationship among them. By employing the particle filter as a Bayesian filtering framework, the model estimates the pedestrian state from signal information and pedestrian position measurements. Evaluation using experimental data collected in real traffic scene shows that the proposed model has an ability to detect the pedestrian intention to cross a crosswalk even when he she is far from it. In this paper, we study the safe navigation of a mobile robot through crowds of dynamic agents with uncertain trajectories. Existing algorithms suffer from the “freezing robot” problem: once the environment surpasses a certain level of complexity, the planner decides that all forward paths are unsafe, and the robot freezes in place (or performs unnecessary maneuvers) to avoid collisions. Since a feasible path typically exists, this behavior is suboptimal. Existing approaches have focused on reducing the predictive uncertainty for individual agents by employing more informed models or heuristically limiting the predictive covariance to prevent this overcautious behavior. In this work, we demonstrate that both the individual prediction and the predictive uncertainty have little to do with the frozen robot problem. Our key insight is that dynamic agents solve the frozen robot problem by engaging in “joint collision avoidance”: They cooperatively make room to create feasible trajectories. We develop IGP, a nonparametric statistical model based on dependent output Gaussian processes that can estimate crowd interaction from data. Our model naturally captures the non-Markov nature of agent trajectories, as well as their goal-driven navigation. We then show how planning in this model can be efficiently implemented using particle based inference. Lastly, we evaluate our model on a dataset of pedestrians entering and leaving a building, first comparing the model with actual pedestrians, and find that the algorithm either outperforms human pedestrians or performs very similarly to the pedestrians. We also present an experiment where a covariance reduction method results in highly overcautious behavior, while our model performs desirably. The goal of this research is to use agent simulation in realizing human-machine interaction, so that we can perform different aspects of experiments. The domain that we have chosen is the pedestrian-vehicle interaction in street crossing. The participants of the simulation include pedestrians, vehicles, and traffic lights, which are all represented by agents. The autonomous nature along with individual traits of agents enable the simulator to show the dynamic interaction among the agents. The attributes of agents like age, gender, and aggressiveness are considered along with different reaction time. A prototype system uses Jade as the platform in implementation. The experiments show a variety of results based on interaction among various agents. Over the last decade, there have been many studies that focus on modeling driver behavior, and in particular detecting and overcoming driver distraction in an effort to reduce accidents caused by driver negligence. Such studies assume that the entire onus of avoiding accidents are on the driver alone. In this study, we adopt a different stance and study the behavior of pedestrians instead. In particular, we focus on detecting pedestrians who are engaged in secondary activities involving their cellphones and similar hand-held multimedia devices from a purely vision-based standpoint. To achieve this objective, we propose a pipeline incorporating articulated human pose estimation, and the use gradient based image features to detect the presence absence of a device in either hand of a pedestrian. Information from different streams and their dependencies on one another are encoded by a belief network. This network is then used to predict a probability score suggesting the involvement of a subject with his her device. When driving in urban environments, an autonomous vehicle must account for the interaction with other traffic participants. It must reason about their future behavior, how its actions affect their future behavior, and potentially consider multiple motion hypothesis. In this paper we introduce a method for joint behavior estimation and trajectory planning that models interaction and multi-policy decision-making. The method leverages Partially Observable Markov Decision Processes to estimate the behavior of other traffic participants given the planned trajectory for the ego-vehicle, and Receding-Horizon Control for generating safe trajectories for the ego-vehicle. To achieve safe navigation we introduce chance constraints over multiple motion policies in the receding-horizon planner. These constraints account for uncertainty over the behavior of other traffic participants. The method is capable of running in real-time and we show its performance and good scalability in simulated multi-vehicle intersection scenarios. In this paper, we consider the problem of pedestrian detection in natural scenes. Intuitively, instances of pedestrians with different spatial scales may exhibit dramatically different features. Thus, large variance in instance scales, which results in undesirable large intracategory variance in features, may severely hurt the performance of modern object instance detection methods. We argue that this issue can be substantially alleviated by the divide-and-conquer philosophy. Taking pedestrian detection as an example, we illustrate how we can leverage this philosophy to develop a Scale-Aware Fast R-CNN (SAF R-CNN) framework. The model introduces multiple built-in subnetworks which detect pedestrians with scales from disjoint ranges. Outputs from all of the subnetworks are then adaptively combined to generate the final detection results that are shown to be robust to large variance in instance scales, via a gate function defined over the sizes of object proposals. Extensive evaluations on several challenging pedestrian detection datasets well demonstrate the effectiveness of the proposed SAF R-CNN. Particularly, our method achieves state-of-the-art performance on Caltech [P. Dollar, C. Wojek, B. Schiele, and P. Perona, “Pedestrian detection: An evaluation of the state of the art,” IEEE Trans. Pattern Anal. Mach. Intell. , vol. 34, no. 4, pp. 743–761, Apr. 2012], and obtains competitive results on INRIA [N. Dalal and B. Triggs, “Histograms of oriented gradients for human detection,” in Proc. IEEE Conf. Comput. Vis. Pattern Recognit. , 2005, pp. 886–893], ETH [A. Ess, B. Leibe, and L. V. Gool, “Depth and appearance for mobile scene analysis,” in Proc. Int. Conf. Comput. Vis ., 2007, pp. 1–8], and KITTI [A. Geiger, P. Lenz, and R. Urtasun, “Are we ready for autonomous driving? The KITTI vision benchmark suite,” in Proc. IEEE Conf. Comput. Vis. Pattern Recognit ., 2012, pp. 3354–3361]. For mobile robots navigating on sidewalks, it is essential to be able to safely cross street intersections. Most existing approaches rely on the recognition of the traffic light signal to make an informed crossing decision. Although these approaches have been crucial enablers for urban navigation, the capabilities of robots employing such approaches are still limited to navigating only on streets containing signalized intersections. In this paper, we address this challenge and propose a multimodal convolutional neural network framework to predict the safety of a street intersection for crossing. Our architecture consists of two subnetworks; an interaction-aware trajectory estimation stream IA-TCNN, that predicts the future states of all observed traffic participants in the scene, and a traffic light recognition stream AtteNet. Our IA-TCNN utilizes dilated causal convolutions to model the behavior of the observable dynamic agents in the scene without explicitly assigning priorities to the interactions among them. While AtteNet utilizes Squeeze-Excitation blocks to learn a content-aware mechanism for selecting the relevant features from the data, thereby improving the noise robustness. Learned representations from the traffic light recognition stream are fused with the estimated trajectories from the motion prediction stream to learn the crossing decision. Furthermore, we extend our previously introduced Freiburg Street Crossing dataset with sequences captured at different types of intersections, demonstrating complex interactions among the traffic participants. Extensive experimental evaluations on public benchmark datasets and our proposed dataset demonstrate that our network achieves state-of-the-art performance for each of the subtasks, as well as for the crossing safety prediction. Self-driving technologies have been increasingly developed and tested in recent years (e.g., Volvo's and Google's self-driving cars). However, only a limited number of investigations have so far been conducted into communication between self-driving cars and pedestrians. For example, when a pedestrian is about to cross a street, that pedestrian needs to know the intension of the approaching self-driving car. In the present study, we designed a novel interface known as "Eyes on a Car" to address this problem. We added eyes onto a car so as to establish eye contact communication between that car and pedestrians. The car looks at the pedestrian in order to indicate its intention to stop. This novel interface design was evaluated via a virtual reality (VR) simulated environment featuring a street-crossing scenario. The evaluation results show that pedestrians can make the correct street-crossing decision more quickly if the approaching car has the novel interface "eyes" than in the case of normal cars. In addition, the results show that pedestrians feel safer with regard to crossing a street if the approaching car has eyes and if the eyes look at them. We present an active pedestrian protection system that performs an autonomous lane-keeping evasive maneuver in urban traffic scenarios when collision avoidance by braking is no longer possible. The system focuses on pedestrians standing at the curb and intending to cross the street despite an approaching car. It is demonstrated that the evasive maneuver of the car can be initiated before the pedestrian's foot hits the lane, by means of video-based motion contour histograms of oriented gradients and stationary detection. Using clothoid-based real-time trajectory planning and a lateral control of the car, combining feedforward and feedback control, the difference between the driven and the calculated trajectories is kept below 10 cm at maximum lateral accelerations of 4 ms-2 and -5 ms-2. We present the technical realization of the system and its precision with respect to intention recognition and driven trajectories. A case study showed that the system reacted faster than human drivers in five out of 11 cases, with an average time gain of 214 ms, even though the drivers were able to pay the utmost attention to the behavior of the crossing pedestrian. Future vehicle systems for active pedestrian safety will not only require a high recognition performance but also an accurate analysis of the developing traffic situation. In this paper, we present a study on pedestrian path prediction and action classification at short subsecond time intervals. We consider four representative approaches: two novel approaches (based on Gaussian process dynamical models and probabilistic hierarchical trajectory matching) that use augmented features derived from dense optical flow and two approaches as baseline that use positional information only (a Kalman filter and its extension to interacting multiple models). In experiments using stereo vision data obtained from a vehicle, we investigate the accuracy of path prediction and action classification at various time horizons, the effect of various errors (image localization, vehicle egomotion estimation), and the benefit of the proposed approaches. The scenario of interest is that of a crossing pedestrian, who might stop or continue walking at the road curbside. Results indicate similar performance of the four approaches on walking motion, with near-linear dynamics. During stopping, however, the two newly proposed approaches, with nonlinear and or higher order models and augmented motion features, achieve a more accurate position prediction of 10-50 cm at a time horizon of 0-0.77 s around the stopping event. We develop predictive models of pedestrian dynamics by encoding the coupled nature of multi-pedestrian interaction using game theory and deep learning-based visual analysis to estimate person-specific behavior parameters. We focus on predictive models since they are important for developing interactive autonomous systems (e.g., autonomous cars, home robots, smart homes) that can understand different human behavior and pre-emptively respond to future human actions. Building predictive models for multi-pedestrian interactions however, is very challenging due to two reasons: (1) the dynamics of interaction are complex interdependent processes, where the decision of one person can affect others, and (2) dynamics are variable, where each person may behave differently (e.g., an older person may walk slowly while the younger person may walk faster). To address these challenges, we utilize concepts from game theory to model the intertwined decision making process of multiple pedestrians and use visual classifiers to learn a mapping from pedestrian appearance to behavior parameters. We evaluate our proposed model on several public multiple pedestrian interaction video datasets. Results show that our strategic planning model predicts and explains human interactions 25 better when compared to a state-of-the-art activity forecasting method. Active safety systems which assess highly dynamic traffic situations including pedestrians are required with growing demands in autonomous driving and ADAS. In this paper, we focus on one of the most hazardous traffic situations: the possible collision between a pedestrian and a turning vehicle at intersections. This paper presents a probabilistic model of pedestrian behavior to signalized crosswalks. For this purpose, we take not only pedestrian physical states but also contextual information into account. We propose a model based on the Dynamic Bayesian Network (DBN) which integrates relations among the intersection context information and the pedestrian behavior in the same way as human. Afterwards, the model jointly estimates their states by the particle filter. Experimental evaluation using real traffic data shows that this model is able to recognize the pedestrian crossing decision in advance from the traffic signal and pedestrian position information. This paper reports on a data-driven, interaction-aware motion prediction approach for pedestrians in environments cluttered with static obstacles. When navigating in such workspaces shared with humans, robots need accurate motion predictions of the surrounding pedestrians. Human navigation behavior is mostly influenced by their surrounding pedestrians and by the static obstacles in their vicinity. In this paper we introduce a new model based on Long-Short Term Memory (LSTM) neural networks, which is able to learn human motion behavior from demonstrated data. To the best of our knowledge, this is the first approach using LSTMs, that incorporates both static obstacles and surrounding pedestrians for trajectory forecasting. As part of the model, we introduce a new way of encoding surrounding pedestrians based on a 1d-grid in polar angle space. We evaluate the benefit of interaction-aware motion prediction and the added value of incorporating static obstacles on both simulation and real-world datasets by comparing with state-of-the-art approaches. The results show, that our new approach outperforms the other approaches while being very computationally efficient and that taking into account static obstacles for motion predictions significantly improves the prediction accuracy, especially in cluttered environments. This paper presents a study performed in virtual reality on the effect of gaze interception during collision avoidance between two walkers. In such a situation, mutual gaze can be considered as a form of nonverbal communication. Additionally, gaze is believed to detail future path intentions and to be part of the nonverbal negotiation to achieve avoidance collaboratively. We considered an avoidance task between a real subject and a virtual human character and studied the influence of the character's gaze direction on the avoidance behaviour of the participant. Virtual reality provided an accurate control of the situation: seventeen participants were immersed in a virtual environment, instructed to navigate across a virtual space using a joystick and to avoid a virtual character that would appear from either side. The character would either gaze or not towards the participant. Further, the character would either perform or not a reciprocal adaptation of its trajectory to avoid a potential collision with the participant. The findings of this paper were that during an orthogonal collision avoidance task, gaze behaviour did not influence the collision avoidance behaviour of the participants. Further, the addition of reciprocal collision avoidance with gaze did not modify the collision behaviour of participants. These results suggest that for the duration of interaction in such a task, body motion cues were sufficient for coordination and regulation. We discuss the possible exploitation of these results to improve the design of virtual characters for populated virtual environments and user interaction. As humans we possess an intuitive ability for navigation which we master through years of practice; however existing approaches to model this trait for diverse tasks including monitoring pedestrian flow and detecting abnormal events have been limited by using a variety of hand-crafted features. Recent research in the area of deep-learning has demonstrated the power of learning features directly from the data; and related research in recurrent neural networks has shown exemplary results in sequence-to-sequence problems such as neural machine translation and neural image caption generation. Motivated by these approaches, we propose a novel method to predict the future motion of a pedestrian given a short history of their, and their neighbours, past behaviour. The novelty of the proposed method is the combined attention model which utilises both "soft attention" as well as "hard-wired" attention in order to map the trajectory information from the local neighbourhood to the future positions of the pedestrian of interest. We illustrate how a simple approximation of attention weights (i.e hard-wired) can be merged together with soft attention weights in order to make our model applicable for challenging real world scenarios with hundreds of neighbours. The navigational capability of the proposed method is tested on two challenging publicly available surveillance databases where our model outperforms the current-state-of-the-art methods. Additionally, we illustrate how the proposed architecture can be directly applied for the task of abnormal event detection without handcrafting the features. Due to the anticipated future, extensive use of robots, human beings will probably share common spaces with them. The relationships between robots and humans will be conducted at close distances. Predicting people's future positions helps robots understand human behavior and react safely and naturally. In this paper, we propose a method for predicting people's positions in crossing behaviors, i.e. different trajectories people follow when they are crossing each other. We conducted a field experiment to gather various crossing behaviors of pedestrians in a shopping mall environment and analyzed them by focusing on “hot areas” spaces where people modify their trajectories for crossing. We clustered typical crossing behaviors in hot areas and modeled them using Hidden Markov Models for predictions. Our algorithm more accurately predicts the future positions of pedestrians by considering moving direction and speed.