Anticipating possible behaviors of traffic participants is an essential capability of autonomous vehicles. Many behavior detection and maneuver recognition methods only have a very limited prediction horizon that leaves inadequate time and space for planning. To avoid unsatisfactory reactive decisions, it is essential to count long-term future rewards in planning, which requires extending the prediction horizon. In this paper, we uncover that clues to vehicle behaviors over an extended horizon can be found in vehicle interaction, which makes it possible to anticipate the likelihood of a certain behavior, even in the absence of any clear maneuver pattern. We adopt a recurrent neural network (RNN) for observation encoding, and based on that, we propose a novel vehicle behavior interaction network (VBIN) to capture the vehicle interaction from the hidden states and connection feature of each interaction pair. The output of our method is a probabilistic likelihood of multiple behavior classes, which matches the multimodal and uncertain nature of the distant future. A systematic comparison of our method against two state-of-the-art methods and another two baseline methods on a publicly available real highway dataset is provided, showing that our method has superior accuracy and advanced capability for interaction modeling.
Detection-based methods, such as maneuver recognition approaches, have been studied extensively. Houenou @cite_8 identify maneuver patterns such as LK and LC based on a handcrafted model which evaluates the similarity between the vehicle trajectory and lane center line. However, user-specified parameters and thresholds are required. Mandalia @cite_13 classify the LC maneuver using support vector machine (SVM), while Schlechitriemen @cite_3 use Gaussian mixture models to estimate driving intentions. Woo @cite_1 also adopt an SVM, and suggest using trajectory prediction to reject false alarms from the intention estimations. These methods detect LCs based on the target's own maneuver pattern and only have a limited prediction horizon. In contrast, our method captures the driver intention, even in the absence of any clear maneuver pattern, by making use of the vehicle behavior interactions.
@cite_1 @cite_13 @cite_3 @cite_8
2582443945 2162966954 2138997468 2055556996
Predicting other traffic participants trajectories is a crucial task for an autonomous vehicle, in order to avoid collisions on its planned trajectory. It is also necessary for many Advanced Driver Assistance Systems, where the ego-vehicle's trajectory has to be predicted too. Even if trajectory prediction is not a deterministic task, it is possible to point out the most likely trajectory. This paper presents a new trajectory prediction method which combines a trajectory prediction based on Constant Yaw Rate and Acceleration motion model and a trajectory prediction based on maneuver recognition. It takes benefit on the accuracy of both predictions respectively a short-term and long-term. The defined Maneuver Recognition Module selects the current maneuver from a predefined set by comparing the center lines of the road's lanes to a local curvilinear model of the path of the vehicle. The overall approach was tested on prerecorded human real driving data and results show that the Maneuver Recognition Module has a high success rate and that the final trajectory prediction has a better accuracy. In order to determine a cooperative driving strategy, it is beneficial for an autonomous vehicle to incorporate the intended motion of surrounding vehicles within its own motion planning. However, as intentions cannot be measured directly and the motion of multiple vehicles often are highly interdependent, this incorporation has proven challenging. In this paper, the problem of maneuver estimation is addressed, focusing on situations with close interaction between traffic participants. Therefore, we define collective maneuvers based on trajectory homotopy, describing the relative motion of multiple vehicles in a scene. Representing maneuvers by sample trajectories, maneuver-dependent prediction models of the vehicle states can be defined. This allows for a Bayesian estimation of maneuver probabilities given observations of the real motion. The approach is evaluated by simulation in overtaking scenarios with oncoming traffic and merging scenarios at an intersection. The authors propose a Bayesian trajectory prediction and criticality assessment system that allows to reason about imminent collisions of a vehicle several seconds in advance. The authors first infer a distribution of high-level, abstract driving maneuvers such as lane changes, turns, road followings, etc. of all vehicles within the driving scene by modeling the domain in a Bayesian network with both causal and diagnostic evidences. This is followed by maneuver-based, long-term trajectory predictions, which themselves contain random components due to the immanent uncertainty of how drivers execute specific maneuvers. Taking all uncertain predictions of all maneuvers of every vehicle into account, the probability of the ego vehicle colliding at least once within a time span is evaluated via Monte-Carlo simulations and given as a function of the prediction horizon. This serves as the basis for calculating a novel criticality measure, the Time-To-Critical-Collision-Probability (TTCCP) - a generalization of the common Time-To-Collision (TTC) in arbitrary, uncertain, multi-object driving environments and valid for longer prediction horizons. The system is applicable from highly-structured to completely non-structured environments and additionally allows the prediction of vehicles not behaving according to a specific maneuver class. Accurately predicting the possible behaviors of traffic participants is an essential capability for future autonomous vehicles. The majority of current researches fix the number of driving intentions by considering only a specific scenario. However, distinct driving environments usually contain various possible driving maneuvers. Therefore, a intention prediction method that can adapt to different traffic scenarios is needed. To further improve the overall vehicle prediction performance, motion information is usually incorporated with classified intentions. As suggested in some literature, the methods that directly predict possible goal locations can achieve better performance for long-term motion prediction than other approaches due to their automatic incorporation of environment constraints. Moreover, by obtaining the temporal information of the predicted destinations, the optimal trajectories for predicted vehicles as well as the desirable path for ego autonomous vehicle could be easily generated. In this paper, we propose a Semantic-based Intention and Motion Prediction (SIMP) method, which can be adapted to any driving scenarios by using semantic-defined vehicle behaviors. It utilizes a probabilistic framework based on deep neural network to estimate the intentions, final locations, and the corresponding time information for surrounding vehicles. An exemplar real-world scenario was used to implement and examine the proposed method. Risk estimation for the current traffic situation is crucial for safe autonomous driving systems. One part of the uncertainty in risk estimation is the behavior of the surrounding traffic participants. In this paper we focus on highway scenarios, where possible behaviors consist of a change in acceleration and lane change maneuvers. We present a novel approach for the recognition of lane change intentions of traffic participants. Our novel approach is an extension of the Naive Bayesian approach and results in a generative model. It builds on the relations to the directly surrounding vehicles and to the static traffic environment. We obtain the conditional probabilities of all relevant features using Gaussian mixtures with a flexible number of components. We systematically reduce the number of features by selecting the most powerful ones. Furthermore we investigate the predictive power of each feature with respect to the time before a lane change event. In a large scale experiment on real world data with over 160.781 samples collected on a test drive of 1100km we trained and validated our intention prediction model and achieved a significant improvement in the recognition performance of lane change intentions compared to current state of the art methods. A method for detecting drivers’ intentions is essential to facilitate operating mode transitions between driver and driver assistance systems. We propose a driver behavior recognition method using Hidden Markov Models (HMMs) to characterize and detect driving maneuvers and place it in the framework of a cognitive model of human behavior. HMM-based steering behavior models for emergency and normal lane changes as well as for lane keeping were developed using a moving base driving simulator. Analysis of these models after training and recognition tests showed that driver behavior modeling and recognition of different types of lane changes is possible using HMMs. Maneuver recognition and trajectory prediction of moving vehicles are two important and challenging tasks of advanced driver assistance systems (ADAS) at urban intersections. This paper presents a continuing work to handle these two problems in a consistent framework using non-parametric regression models. We provide a feature normalization scheme and present a strategy for constructing three-dimensional Gaussian process regression models from two-dimensional trajectory patterns. These models can capture spatio-temporal characteristics of traffic situations. Given a new, partially observed and unlabeled trajectory, the maneuver can be recognized online by comparing the likelihoods of the observation data for each individual regression model. Furthermore, we take advantage of our representation for trajectory prediction. Because predicting possible trajectories at urban intersection involves obvious multimodalities and non-linearities, we employ the Monte Carlo method to handle these difficulties. This approach allows the incremental prediction of possible trajectories in situations where unimodal estimators such as Kalman Filters would not work well. The proposed framework is evaluated experimentally in urban intersection scenarios using real-world data. Reliable prediction of surround vehicle motion is a critical requirement for path planning for autonomous vehicles. In this paper, we propose a unified framework for surround vehicle maneuver classification and motion prediction that exploits multiple cues, namely, the estimated motion of vehicles, an understanding of typical motion patterns of freeway traffic and intervehicle interaction. We report our results in terms of maneuver classification accuracy and mean and median absolute error of predicted trajectories against the ground truth for real traffic data collected using vehicle mounted sensors on freeways. An ablative analysis is performed to analyze the relative importance of each cue for trajectory prediction. Additionally, an analysis of execution time for the components of the framework is presented. Finally, we present multiple case studies analyzing the outputs of our model for complex traffic scenarios. We present an active pedestrian protection system that performs an autonomous lane-keeping evasive maneuver in urban traffic scenarios when collision avoidance by braking is no longer possible. The system focuses on pedestrians standing at the curb and intending to cross the street despite an approaching car. It is demonstrated that the evasive maneuver of the car can be initiated before the pedestrian's foot hits the lane, by means of video-based motion contour histograms of oriented gradients and stationary detection. Using clothoid-based real-time trajectory planning and a lateral control of the car, combining feedforward and feedback control, the difference between the driven and the calculated trajectories is kept below 10 cm at maximum lateral accelerations of 4 ms-2 and -5 ms-2. We present the technical realization of the system and its precision with respect to intention recognition and driven trajectories. A case study showed that the system reacted faster than human drivers in five out of 11 cases, with an average time gain of 214 ms, even though the drivers were able to pay the utmost attention to the behavior of the crossing pedestrian. Driving is a complex task that requires constant attention, and intelligent transportation systems that support drivers in this task must continually infer driver intentions to produce reasonable, safe responses. In this paper we describe a technique for inferring driver intentions, specifically the intention to change lanes, using support vector machines (SVMs). The technique was applied to experimental data from an instrumented vehicle that included both behavioral data and environmental data. Comparing these results to recent results using a novel “mind-tracking” technique, we found that SVMs outperformed earlier algorithms and proved especially effective in early detection of driver lane changes. Accurate maneuver prediction for surrounding vehicles enables intelligent vehicles to make safe and socially compliant decisions in advance, thus improving the safety and comfort of the driving. The main contribution of this paper is proposing a practical, high-performance, and low-cost maneuver-prediction approach for intelligent vehicles. Our approach is based on a dynamic Bayesian network, which exploits multiple predictive features, namely, historical states of predicting vehicles, road structures, as well as traffic interactions for inferring the probability of each maneuver. The paper also presents algorithms of feature extraction for the network. Our approach is verified on real traffic data in large-scale publicly available datasets. The results show that our approach can recognize the lane-change maneuvers with an F1 score of 80 and an advanced prediction time of 3.75 s, which greatly improves the performance on prediction compared to other baseline approaches. By predicting a driver's maneuvers before they occur, a driver-assistance system can prepare for or avoid dangerous situations. This article describes a real-time, on-road lane-change-intent detector that can enhance driver safety. Predicting driver behavior is a key component for Advanced Driver Assistance Systems (ADAS). In this paper, a novel approach based on Support Vector Machine and Bayesian filtering is proposed for online lane change intention prediction. The approach uses the multiclass probabilistic outputs of the Support Vector Machine as an input to the Bayesian filter, and the output of the Bayesian filter is used for the final prediction of lane changes. A lane tracker integrated in a passenger vehicle is used for real-world data collection for the purpose of training and testing. Data from different drivers on different highways were used to evaluate the robustness of the approach. The results demonstrate that the proposed approach is able to predict driver intention to change lanes on average 1.3 seconds in advance, with a maximum prediction horizon of 3.29 seconds. Cooperative driving behavior is essential for driving in traffic, especially for ramp merging, lane changing or navigating intersections. Autonomous vehicles should also manage these situations by behaving cooperatively and naturally. The challenge of cooperative driving is estimating other vehicles' intentions. In this paper, we present a novel method to estimate other human-driven vehicles' intentions with the aim of achieving a natural and amenable cooperative driving behavior, without using wireless communication. The new approach allows the autonomous vehicle to cooperate with multiple observable merging vehicles on the ramp with a leading vehicle ahead of the autonomous vehicle in the same lane. To avoid calculating trajectories, simplify computation, and take advantage of mature Level-3 components, the new method reacts to merging cars by determining a following target for an off-the-shelf distance keeping module (ACC) which governs speed control of the autonomous vehicle. We train and evaluate the proposed model using real traffic data. Results show that the new approach has a lower collision rate than previous methods and generates more human driver-like behaviors in terms of trajectory similarity and time-to-collision to leading vehicles. Advanced Driver Assistance Systems (ADAS) have made driving safer over the last decade. They prepare vehicles for unsafe road conditions and alert drivers if they perform a dangerous maneuver. However, many accidents are unavoidable because by the time drivers are alerted, it is already too late. Anticipating maneuvers beforehand can alert drivers before they perform the maneuver and also give ADAS more time to avoid or prepare for the danger. In this work we anticipate driving maneuvers a few seconds before they occur. For this purpose we equip a car with cameras and a computing device to capture the driving context from both inside and outside of the car. We propose an Autoregressive Input-Output HMM to model the contextual information alongwith the maneuvers. We evaluate our approach on a diverse data set with 1180 miles of natural freeway and city driving and show that we can anticipate maneuvers 3.5 seconds before they occur with over 80 F1-score in real-time. Avoiding vehicle-to-pedestrian crashes is a critical requirement for nowadays advanced driver assistant systems (ADAS) and future self-driving vehicles. Accordingly, detecting pedestrians from raw sensor data has a history of more than 15 years of research, with vision playing a central role. During the last years, deep learning has boosted the accuracy of image-based pedestrian detectors. However, detection is just the first step towards answering the core question, namely is the vehicle going to crash with a pedestrian provided preventive actions are not taken? Therefore, knowing as soon as possible if a detected pedestrian has the intention of crossing the road ahead of the vehicle is essential for performing safe and comfortable maneuvers that prevent a crash. However, compared to pedestrian detection, there is relatively little literature on detecting pedestrian intentions. This paper aims to contribute along this line by presenting a new vision-based approach which analyzes the pose of a pedestrian along several frames to determine if he or she is going to enter the road or not. We present experiments showing 750 ms of anticipation for pedestrians crossing the road, which at a typical urban driving speed of 50 km h can provide 15 additional meters (compared to a pure pedestrian detector) for vehicle automatic reactions or to warn the driver. Moreover, in contrast with state-of-the-art methods, our approach is monocular, neither requiring stereo nor optical flow information. Accurate and robust tracking of surrounding road participants plays an important role in autonomous driving. However, there is usually no prior knowledge of the number of tracking targets due to object emergence, object disappearance and false alarms. To overcome this challenge, we propose a generic vehicle tracking framework based on modified mixture particle filter, which can make the number of tracking targets adaptive to real-time observations and track all the vehicles within sensor range simultaneously in a uniform architecture without explicit data association. Each object corresponds to a mixture component whose distribution is non-parametric and approximated by particle hypotheses. Most tracking approaches employ vehicle kinematic models as the prediction model. However, it is hard for these models to make proper predictions when sensor measurements are lost or become low quality due to partial or complete occlusions. Moreover, these models are incapable of forecasting sudden maneuvers. To address these problems, we propose to incorporate learning-based behavioral models instead of pure vehicle kinematic models to realize prediction in the prior update of recursive Bayesian state estimation. Two typical driving scenarios including lane keeping and lane change are demonstrated to verify the effectiveness and accuracy of the proposed framework as well as the advantages of employing learning-based models. Misunderstanding of driver correction behaviors is the primary reason for false warnings of lane-departure-prediction systems. We proposed a learning-based approach to predict unintended lane-departure behaviors and chances of drivers to bring vehicles back to the lane. First, a personalized driver model for lane-departure and lane-keeping behavior is established by combining the Gaussian mixture model and the hidden Markov model. Second, based on this model, we developed an online model-based prediction algorithm to predict the forthcoming vehicle trajectory and judge whether the driver will act a lane departure behavior or correction behavior. We also develop a warning strategy based on the model-based prediction algorithm that allows the lane-departure warning system to be acceptable for drivers according to the predicted trajectory. In addition, the naturalistic driving data of ten drivers were collected to train the personalized driver model and validate this approach. We compared the proposed method with a basic time-to-lane-crossing (TLC) method and a TLC-directional sequence of piecewise lateral slopes (TLC-DSPLS) method. Experimental results show that the proposed approach can reduce the false-warning rate to 3.13 on average at 1-s prediction time. This paper presents a novel cooperative-driving prediction and planning framework for dynamic environments based on the methods of game theory. The proposed algorithm can be used for highly automated driving on highways or as a sophisticated prediction module for advanced driver-assistance systems with no need for intervehicle communication. The main contribution of this paper is a model-based interaction-aware motion prediction of all vehicles in a scene. In contrast to other state-of-the-art approaches, the system also models the replanning capabilities of all drivers. With that, the driving strategy is able to capture complex interactions between vehicles, thus planning maneuver sequences over longer time horizons. It also enables an accurate prediction of traffic for the next immediate time step. The prediction model is supported by an interpretation of what other drivers intend to do, how they interact with traffic, and the ongoing observation. As part of the prediction loop, the proposed planning strategy incorporates the expected reactions of all traffic participants, offering cooperative and robust driving decisions. By means of experimental results under simulated highway scenarios, the validity of the proposed concept and its real-time capability is demonstrated. Advanced Driver Assistance Systems (ADAS) have made driving safer over the last decade. They prepare vehicles for unsafe road conditions and alert drivers if they perform a dangerous maneuver. However, many accidents are unavoidable because by the time drivers are alerted, it is already too late. Anticipating maneuvers beforehand can alert drivers before they perform the maneuver and also give ADAS more time to avoid or prepare for the danger. In this work we propose a vehicular sensor-rich platform and learning algorithms for maneuver anticipation. For this purpose we equip a car with cameras, Global Positioning System (GPS), and a computing device to capture the driving context from both inside and outside of the car. In order to anticipate maneuvers, we propose a sensory-fusion deep learning architecture which jointly learns to anticipate and fuse multiple sensory streams. Our architecture consists of Recurrent Neural Networks (RNNs) that use Long Short-Term Memory (LSTM) units to capture long temporal dependencies. We propose a novel training procedure which allows the network to predict the future given only a partial temporal context. We introduce a diverse data set with 1180 miles of natural freeway and city driving, and show that we can anticipate maneuvers 3.5 seconds before they occur in real-time with a precision and recall of 90.5 and 87.4 respectively.