A Review of Research on Traffic Conflicts Based on Intelligent Vehicles <s> A. RESEARCH ON TRAFFIC CONFLICTS BASED ON COMPUTER VISION <s> A new algorithm for background estimation and removal in video sequences obtained with stereo cameras is presented. Per-pixel Gaussian mixtures are used to model recent scene observations in the combined space of depth and luminance-invariant color. These mixture models adapt over time, and are used to build a new model of the background at each time step. This combination in itself is novel, but we also introduce the idea of modulating the learning rate of the background model according to the scene activity level on a per-pixel basis, so that dynamic foreground objects are incorporated into the background more slowly than are static scene changes. Our results show much greater robustness than prior state-of-the-art methods to challenging phenomena such as video displays, non-static background objects, areas of high foreground traffic, and similar color of foreground and background. Our method is also well-suited for use in real-time systems. <s> BIB001 </s> A Review of Research on Traffic Conflicts Based on Intelligent Vehicles <s> A. RESEARCH ON TRAFFIC CONFLICTS BASED ON COMPUTER VISION <s> Monitoring traffic intersections in real time and predicting possible collisions is an important first step towards building an early collision-warning system. We present a vision-based system addressing this problem and describe the practical adaptations necessary to achieve real-time performance. Innovative low-overhead collision-prediction algorithms (such as the one using the time-as-axis paradigm) are presented. The proposed system was able to perform successfully in real time on videos of quarter-video graphics array (VGA) (320 /spl times/ 240) resolution under various weather conditions. The errors in target position and dimension estimates in a test video sequence are quantified and several experimental results are presented. <s> BIB002 </s> A Review of Research on Traffic Conflicts Based on Intelligent Vehicles <s> A. RESEARCH ON TRAFFIC CONFLICTS BASED ON COMPUTER VISION <s> This work aims at addressing the many problems that have hindered the development of vision-based systems for automated road safety analysis. The approach relies on traffic conflicts used as surrogates for collision data. Traffic conflicts are identified by computing the collision probability for any two road users in an interaction. A complete system is implemented to process traffic video data, detect and track road users, and analyze their interactions. Motion patterns are needed to predict road users' movements and determine their probability of being involved in a collision. An original incremental algorithm for the learning of prototype trajectories as motion patterns is presented. The system is tested on real world traffic data, including a few traffic conflict instances. Traffic patterns are successfully learnt on two datasets, and used for collision probability computation and traffic conflict detection. <s> BIB003 </s> A Review of Research on Traffic Conflicts Based on Intelligent Vehicles <s> A. RESEARCH ON TRAFFIC CONFLICTS BASED ON COMPUTER VISION <s> The successful planning, design and management of a traffic system is impossible without knowledge of how the traffic environment affects the behaviour of road users and how the behaviour is related to the main qualities of the traffic system (e.g. safety, efficiency). Automated video analysis is a promising tool for traffic behaviour research in that it enables collection of micro-level behaviour data for large populations of road users and provides a detailed description of their motion. This thesis describes the tests done with an automated video analysis system developed at Lund University. The system was used in two large scale studies with the main task of detecting the presence of road users of a particular type. Accuracy of position and speed estimates were tested in a study specially designed for that purpose. The thesis also elaborates on the problem of relating the behaviour of road users to safety and proposes organising all the elementary events in traffic (defined here as encounters between two road users) into a severity hierarchy. The process of an encounter is described with a set of continuous safety indicators that can handle the various approach angles and transfer between being and not being on a collision course. When an objective measure for an encounter severity is found, the severity hierarchies may be used not only for describing safety but also for studying the balance between safety and other qualities valued by road users. <s> BIB004 </s> A Review of Research on Traffic Conflicts Based on Intelligent Vehicles <s> A. RESEARCH ON TRAFFIC CONFLICTS BASED ON COMPUTER VISION <s> Road collisions are a worldwide pandemic that can be addressed through the improvement of existing tools for safety analysis. A refined probabilistic framework is presented for the analysis of road-user interactions. In particular, the identification of potential collision points is used to estimate collision probabilities, and their spatial distribution can be visualized. A probabilistic time to collision is introduced, and interactions are grouped into four categories: head-on, rear-end, side, and parallel. The framework is applied to a large data set of video recordings collected in Kentucky that contains more than 300 severe interactions and collisions. The results demonstrate the usefulness of the approach for studying road-user behavior and mechanisms that may lead to collisions. <s> BIB005 </s> A Review of Research on Traffic Conflicts Based on Intelligent Vehicles <s> A. RESEARCH ON TRAFFIC CONFLICTS BASED ON COMPUTER VISION <s> Traffic violation is the main cause of traffic accidents. To reduce the incidence of traffic accidents, the common practice at present is to strength the penalties for traffic violation. However, little attention has been paid to issue warning for dangerous driving behaviors, especially for the case where two vehicles have a good chance of collision. In this paper, a framework for collision risk estimation using RGB-D camera is proposed for vehicles running on the urban road, where the depth information is fused with the video information for accurate calculation of the position and speed of the vehicles, two essential parameters for motion trajectory estimation. Considering that the motion trajectory or its differences can be considered as a steady signal, a method based on autoregressive integrated moving average (ARIMA) models is presented to predict vehicle trajectory. Then, the collision risk is estimated based on the predicted trajectory. The experiments are carried out on the data from the real vehicles. The result shows that the accuracy of position and speed estimation can be guaranteed within urban road and the error of trajectory prediction is very minor which is unlikely to have a significant impact on calculating the probability of collision in most situations, so the proposed framework is effective in collision risk estimation. <s> BIB006 </s> A Review of Research on Traffic Conflicts Based on Intelligent Vehicles <s> A. RESEARCH ON TRAFFIC CONFLICTS BASED ON COMPUTER VISION <s> In this paper we present a stereovision-based approach for tracking multiple objects in crowded environments where, typically, the road lane markings are not visible and the surrounding infrastructure is not known. The proposed technique relies on measurement data provided by an intermediate occupancy grid derived from processing a stereovision-based elevation map and on free-form object delimiters extracted from this grid. Unlike other existing methods that track rigid objects using also rigid representations, we present a particle filter-based solution for tracking visual appearance-based free-form obstacle representations. At each step, the particle state is described by two components, i.e., the object's dynamic parameters and its estimated geometry. In order to solve the high-dimensionality state–space problem, a Rao–Blackwellized particle filter is used. By accurately modeling the object geometry using the polygonal lines instead of a 3-D box and, at the same time, separating the position and speed tracking from the geometry tracking at the estimator level, the proposed solution combines the efficiency of the rigid model with the benefits of a flexible object model. <s> BIB007 </s> A Review of Research on Traffic Conflicts Based on Intelligent Vehicles <s> A. RESEARCH ON TRAFFIC CONFLICTS BASED ON COMPUTER VISION <s> Extensive research interest has been focused on protecting vulnerable road users in recent years, particularly pedestrians and cyclists, due to their attributes of vulnerability. However, comparatively little effort has been spent on detecting pedestrian and cyclist together, particularly when it concerns quantitative performance analysis on large datasets. In this paper, we present a unified framework for concurrent pedestrian and cyclist detection, which includes a novel detection proposal method (termed UB-MPR) to output a set of object candidates, a discriminative deep model based on Fast R-CNN for classification and localization, and a specific postprocessing step to further improve detection performance. Experiments are performed on a new pedestrian and cyclist dataset containing 30 490 annotated pedestrian and 26 771 cyclist instances in over 50 000 images, recorded from a moving vehicle in the urban traffic of Beijing. Experimental results indicate that the proposed method outperforms other state-of-the-art methods significantly. <s> BIB008 </s> A Review of Research on Traffic Conflicts Based on Intelligent Vehicles <s> A. RESEARCH ON TRAFFIC CONFLICTS BASED ON COMPUTER VISION <s> Accurate collision prediction algorithms are important to provide drivers reliable warning messages. This study introduces a novel approach for collision prediction at intersections. The algorithm involves the use of an index called `minimal future distance (MFD)', which is defined to be a future distance (FD) between the subject vehicle and the primary other vehicle, and a two-level dynamic threshold for performing the collision prediction task. Real-time vehicle motion information and surrounding road geometry are utilised to forecast FDs and identify MFD within an upcoming time horizon. The dynamic threshold in both emergency warning and normal warning situations consists of two parts, a vehicle heading direction-related part and a speed value-related part. Potential collisions are determined by the comparison of MFD with different dynamic thresholds in different driving scenarios. The combined use of vehicle real-time state and road geometry in the algorithm significantly increased the collision prediction accuracy. Furthermore, the use of dynamic thresholds ensured the promptness and robustness of the collision warning system. Simulation results show that the false positive rate and false negative rate of severe collisions at intersections can be robustly eliminated and those of marginal collisions can be kept low at 2.4 and 3.6%, respectively. <s> BIB009 </s> A Review of Research on Traffic Conflicts Based on Intelligent Vehicles <s> A. RESEARCH ON TRAFFIC CONFLICTS BASED ON COMPUTER VISION <s> The Internet of Things (IoT) has been growing in recent years with the improvements in several different applications in the military, marine, intelligent transportation, smart health, smart grid, smart home and smart city domains. Although IoT brings significant advantages over traditional information and communication (ICT) technologies for Intelligent Transportation Systems (ITS), these applications are still very rare. Although there is a continuous improvement in road and vehicle safety, as well as improvements in IoT, the road traffic accidents have been increasing over the last decades. Therefore, it is necessary to find an effective way to reduce the frequency and severity of traffic accidents. Hence, this paper presents an intelligent traffic accident detection system in which vehicles exchange their microscopic vehicle variables with each other. The proposed system uses simulated data collected from vehicular ad-hoc networks (VANETs) based on the speeds and coordinates of the vehicles and then, it sends traffic alerts to the drivers. Furthermore, it shows how machine learning methods can be exploited to detect accidents on freeways in ITS. It is shown that if position and velocity values of every vehicle are given, vehicles' behavior could be analyzed and accidents can be detected easily. Supervised machine learning algorithms such as Artificial Neural Networks (ANN), Support Vector Machine (SVM), and Random Forests (RF) are implemented on traffic data to develop a model to distinguish accident cases from normal cases. The performance of RF algorithm, in terms of its accuracy, was found superior to ANN and SVM algorithms. RF algorithm has showed better performance with 91.56% accuracy than SVM with 88.71% and ANN with 90.02% accuracy. <s> BIB010 </s> A Review of Research on Traffic Conflicts Based on Intelligent Vehicles <s> A. RESEARCH ON TRAFFIC CONFLICTS BASED ON COMPUTER VISION <s> Abstract Robust and efficient vehicle detection in monocular vision is an important task in Intelligent Transportation Systems. With the development of computer vision techniques and consequent accessibility of video image data, new applications have been enabled to on-road vehicle detection algorithms. This paper provides a review of the literature in vehicle detection under varying environments. Due to the variability of on-road driving environments, vehicle detection may face different problems and challenges. Therefore, many approaches have been proposed, and can be categorized as appearance-based methods and motion-based methods. In addition, special illumination, weather and driving scenarios are discussed in terms of methodology and quantitative evaluation. In the future, efforts should be focused on robust vehicle detection approaches for various on-road conditions. <s> BIB011 </s> A Review of Research on Traffic Conflicts Based on Intelligent Vehicles <s> A. RESEARCH ON TRAFFIC CONFLICTS BASED ON COMPUTER VISION <s> Curb detection, a significant area of autonomous driving, plays an important role in road detection and obstacle avoidance, etc. However, curb detection is challenging due to the problems like occlusions, shadows and the small size of the target. In this paper, we propose a curb detection paradigm for road and sidewalk detection for mobile robots using stereo vision in the urban residential region. First, the flat area is estimated based on the disparity and v-disparity maps generated from stereo matching. In the estimated flat area, to distinguish curbs from the road and obstacles, we propose an efficient 16-dimensional descriptor based on the appearance, geometry, and disparity characteristics of curbs. The curb points can be extracted by an SVM classifier with the obtained descriptors. Second, the curb points are exploited by a vanishing point constrained Dijkstra road model to find the road region, where two shortest paths, generated from the vanishing point to the last row of the cost map, are searched and regarded as the road borders. Following the curb and road region detection results, the sidewalk area is detected by a region-growing-like method based on the geometry characteristic of the sidewalk area and the layout of road scenes. The proposed method has been tested for road and sidewalk detection tasks on the KITTI dataset. The experimental results of the curb, road, and sidewalk detection demonstrate the accuracy and robustness of this method. <s> BIB012
At this stage, the researchers around the world have gradually realized that the human-observed conflicts have problems such as high labor costs, strong subjectivity, and low accuracy of observation results. With the application of computer vision technology and video image processing technology in the field of intelligent transportation, it is possible to solve the above problems  . The processes of traffic conflicts detection based on vision perception of intelligent vehicle are shown in Fig. 5 . The traffic conflicts detection based on computer vision technology has become a hot spot. For example, Wang et al proposed a detection scheme for UB-MPR vulnerable road users based on visual perception and big data analysis BIB008 . Dogru et al. utilized the random forest classifier to detect the traffic accident BIB010 . Pin and Ching-Yao BIB009 used the realtime vehicle motion information and the surrounding road geometry to predict the distance between the ego vehicle and the surrounding vehicles. Paul St-Aubin et al.  used computer vision technology to extract the vehicle trajectory and collisions at the roundabouts, which achieves a detailed analysis of the driving behaviors. Vatavu et al. BIB007 proposed a stereo vision-based approach that uses particle filters to process stereoscopic elevation maps for tracking multiple objects in a crowded environment. Yang and Pun-Cheng BIB011 discussed the vision-based road vehicle detection method to achieve traffic monitoring, such as vehicle counting, traffic accident detection and so on. RGB-D cameras provide a solution for vehicle overtaking detection, and RGB-D camera depth data can also identify vehicle motion changes BIB001 . Zhenyu et al. BIB006 proposed an urban vehicle collision risk assessment framework based on RGB-D camera. Shi  proposed a three-dimensional reconstruction and obstacle detection algorithm for binocular vision road scenes and intelligent vehicle trajectories based on visual navigation. Cheng et al. BIB012 proposed a road and sidewalk detection model based on stereo vision system. According to the geometric structure of the road and the overall layout of the road, the area of the sidewalk can be calculated and the road boundary and road area can be detected. At present, the more popular application is the automatic conflict resolution technology based on video image analysis BIB004 . For example, Hu et al.  used video image processing technology to analyze the pedestrian-car conflicts at intersections. Based on the video image processing technology, Saunier et al. BIB005 proposed an automatic conflict resolution model for traffic collisions where there are multiple vehicles at the intersections. Ismail et al.  used video data to automatically analyze the conflicts between pedestrians and vehicles. Atev et al. BIB002 used the video system to monitor intersections in real-time and predict potential conflicts. Reference BIB003 proposed a complete traffic video data processing system, which identifies traffic conflicts by calculating the collision probability of any two road users in the interaction, and realizes the detection and tracking analysis of road users. Zhang  extracted the microscopic traffic parameters such as motion trajectory, velocity, and acceleration of the conflicting vehicle by processing the video data. According to the kinematics of the vehicle, the automatic detection of motor vehicle conflicts under simple traffic conditions was preliminarily realized. Qu et al.  proposed a method for automatically discriminating traffic conflicts at the unsignalized intersections based on video processing. The GM(1,1) gray model is used to predict the trajectory of the moving targets; After obtaining the predicted trajectories of possible conflicts, the calculation is performed. The relative distance of the center of gravity on the time series determines the probability of collision. Pan  used video image processing to study traffic conflict detection at intersections, and proposed a traffic conflict discrimination model based on vehicle collision field. For the effective identification of descendants and cyclists in complex driving environments, a research team from Tsinghua University established a joint recognition method for pedestrians and cyclists based on vehicle images  . The distance information provided by the monocular vision sensor is of low accuracy, which makes it difficult to realize the recognition and tracking of the obstacles in the surrounding environment. At the same time, considering that the stereoscopic vision can measure the distance information of the obstacles, its huge computation load impedes the real-time application. The traffic conflicts observation methods based on video processing technology still need to be improved in several core aspects: target detection, tracking accuracy, real-time computer processing speed, conflict discrimination and severity classification. However, it can make up for the shortcomings of traditional observation methods and improve the speed and accuracy of traffic conflicts observation.