Traffic congestion is a widespread problem. Dynamic traffic routing systems and congestion pricing are getting importance in recent research. Lane prediction and vehicle density estimation is an important component of such systems. We introduce a novel problem of vehicle self-positioning which involves predicting the number of lanes on the road and vehicle's position in those lanes using videos captured by a dashboard camera. We propose an integrated closed-loop approach where we use the presence of vehicles to aid the task of self-positioning and vice-versa. To incorporate multiple factors and high-level semantic knowledge into the solution, we formulate this problem as a Bayesian framework. In the framework, the number of lanes, the vehicle's position in those lanes and the presence of other vehicles are considered as parameters. We also propose a bounding box selection scheme to reduce the number of false detections and increase the computational efficiency. We show that the number of box proposals decreases by a factor of 6 using the selection approach. It also results in large reduction in the number of false detections. The entire approach is tested on real-world videos and is found to give acceptable results.
A similar problem to self-positioning has been handled in @cite_16 using spatial RAY features. It tries to predict the position of the vehicle on the road from an input video stream. However, they predict at most 3 lanes at a time either on the left or right hand side, whereas we have upto 6 lanes in our database with upto 3 lanes being on each side. In addition, there experiments are spread over only 2 days with similar road conditions. Our experiments are more rigorous, spreading over 5 days and the data includes 6 different road conditions. They have low traffic density whereas we have moderate-to-high traffic density. Thus direct comparison between these two approaches may not be possible.
@cite_16
2019933614
Driver-assistance systems that monitor driver intent, warn drivers of lane departures, or assist in vehicle guidance are all being actively considered. It is therefore important to take a critical look at key aspects of these systems, one of which is lane-position tracking. It is for these driver-assistance objectives that motivate the development of the novel "video-based lane estimation and tracking" (VioLET) system. The system is designed using steerable filters for robust and accurate lane-marking detection. Steerable filters provide an efficient method for detecting circular-reflector markings, solid-line markings, and segmented-line markings under varying lighting and road conditions. They help in providing robustness to complex shadowing, lighting changes from overpasses and tunnels, and road-surface variations. They are efficient for lane-marking extraction because by computing only three separable convolutions, we can extract a wide variety of lane markings. Curvature detection is made more robust by incorporating both visual cues (lane markings and lane texture) and vehicle-state information. The experiment design and evaluation of the VioLET system is shown using multiple quantitative metrics over a wide variety of test conditions on a large test path using a unique instrumented vehicle. A justification for the choice of metrics based on a previous study with human-factors applications as well as extensive ground-truth testing from different times of day, road conditions, weather, and driving scenarios is also presented. In order to design the VioLET system, an up-to-date and comprehensive analysis of the current state of the art in lane-detection research was first performed. In doing so, a comparison of a wide variety of methods, pointing out the similarities and differences between methods as well as when and where various methods are most useful, is presented Assigning the ego-vehicle to a lane is not only beneficial for navigation but will be an essential element in future Advanced Driver Assistance Systems. This paper describes an approach for ego-lane index estimation using only a monocular camera and no additional sensing equipment like, e.g., the typically employed GPS and Inertial Measurement Unit. Key aspect of the approach are SPatial RAY (SPRAY) features which represent the spatial layout of the road in the visual scene. The proposed method perceives a variety of local visual properties of the scene by means of base classifiers operating on patches extracted from camera images. The spatial arrangement of these local visual properties are captured using SPRAY features. With a boosting classifier trained on these features the ego-lane index is obtained. The system is evaluated on low traffic density and complementary to an object-based approach suitable for heavy traffic. In the conducted experiments, the proposed approach reaches recognition rates of 93 to 97 on individual highway images without applying any kind of temporal filtering. In this paper, we propose a unified end-to-end trainable multi-task network that jointly handles lane and road marking detection and recognition that is guided by a vanishing point under adverse weather conditions. We tackle rainy and low illumination conditions, which have not been extensively studied until now due to clear challenges. For example, images taken under rainy days are subject to low illumination, while wet roads cause light reflection and distort the appearance of lane and road markings. At night, color distortion occurs under limited illumination. As a result, no benchmark dataset exists and only a few developed algorithms work under poor weather conditions. To address this shortcoming, we build up a lane and road marking benchmark which consists of about 20,000 images with 17 lane and road marking classes under four different scenarios: no rain, rain, heavy rain, and night. We train and evaluate several versions of the proposed multi-task network and validate the importance of each task. The resulting approach, VPGNet, can detect and classify lanes and road markings, and predict a vanishing point with a single forward pass. Experimental results show that our approach achieves high accuracy and robustness under various conditions in real-time (20 fps). The benchmark and the VPGNet model will be publicly available. This paper describes a system for detecting and estimating the properties of multiple travel lanes in an urban road network from calibrated video imagery and laser range data acquired by a moving vehicle. The system operates in real-time in several stages on multiple processors, fusing detected road markings, obstacles, and curbs into a stable non-parametric estimate of nearby travel lanes. The system incorporates elements of a provided piecewise-linear road network as a weak prior. Our method is notable in several respects: it detects and estimates multiple travel lanes; it fuses asynchronous, heterogeneous sensor streams; it handles high-curvature roads; and it makes no assumption about the position or orientation of the vehicle with respect to the road. We analyze the system's performance in the context of the 2007 DARPA Urban Challenge. With five cameras and thirteen lidars, our method was incorporated into a closed-loop controller to successfully guide an autonomous vehicle through a 90 km urban course at speeds up to 40 km h amidst moving traffic. In this paper, we propose a unified end-to-end trainable multi-task network that jointly handles lane and road marking detection and recognition that is guided by a vanishing point under adverse weather conditions. We tackle rainy and low illumination conditions, which have not been extensively studied until now due to clear challenges. For example, images taken under rainy days are subject to low illumination, while wet roads cause light reflection and distort the appearance of lane and road markings. At night, color distortion occurs under limited illumination. As a result, no benchmark dataset exists and only a few developed algorithms work under poor weather conditions. To address this shortcoming, we build up a lane and road marking benchmark which consists of about 20,000 images with 17 lane and road marking classes under four different scenarios: no rain, rain, heavy rain, and night. We train and evaluate several versions of the proposed multi-task network and validate the importance of each task. The resulting approach, VPGNet, can detect and classify lanes and road markings, and predict a vanishing point with a single forward pass. Experimental results show that our approach achieves high accuracy and robustness under various conditions in realtime (20 fps). The benchmark and the VPGNet model will be publicly available Classifying and counting vehicles in road traffic has numerous applications in the transportation engineering domain. However, the wide variety of vehicles (two-wheelers, three-wheelers, cars, buses, trucks etc.) plying on roads of developing regions without any lane discipline, makes vehicle classification and counting a hard problem to automate. In this paper, we use state of the art Convolutional Neural Network (CNN) based object detection models and train them for multiple vehicle classes using data from Delhi roads. We get upto 75 MAP on an 80-20 train-test split using 5562 video frames from four different locations. As robust network connectivity is scarce in developing regions for continuous video transmissions from the road to cloud servers, we also evaluate the latency, energy and hardware cost of embedded implementations of our CNN model based inferences. This paper describes the generic obstacle and lane detection system (GOLD), a stereo vision-based hardware and software architecture to be used on moving vehicles to increment road safety. Based on a full-custom massively parallel hardware, it allows to detect both generic obstacles (without constraints on symmetry or shape) and the lane position in a structured environment (with painted lane markings) at a rate of 10 Hz. Thanks to a geometrical transform supported by a specific hardware module, the perspective effect is removed from both left and right stereo images; the left is used to detect lane markings with a series of morphological filters, while both remapped stereo images are used for the detection of free-space in front of the vehicle. The output of the processing is displayed on both an on-board monitor and a control-panel to give visual feedbacks to the driver. The system was tested on the mobile laboratory (MOB-LAB) experimental land vehicle, which was driven for more than 3000 km along extra-urban roads and freeways at speeds up to 80 km h, and demonstrated its robustness with respect to shadows and changing illumination conditions, different road textures, and vehicle movement. In this paper, a technique for the identification of the unwanted lane departure of a traveling vehicle on a road is proposed. A piecewise linear stretching function (PLSF) is used to improve the contrast level of the region of interest (ROI). Lane markings on the road are detected by dividing the ROI into two subregions and applying the Hough transform in each subregion independently. This segmentation approach improves the computational time required for lane detection. For lane departure identification, a distance-based departure measure is computed at each frame, and a necessary warning message is issued to the driver when such measure exceeds a threshold. The novelty of the proposed algorithm is the identification of the lane departure only using three lane-related parameters based on the Euclidean distance transform to estimate the departure measure. The use of the Euclidean distance transform in combination with the PLSF keeps the false alarm around 3 and the lane detection rate above 97 under various lighting conditions. Experimental results indicate that the proposed system can detect lane boundaries in the presence of several image artifacts, such as lighting changes, poor lane markings, and occlusions by a vehicle, and it issues an accurate lane departure warning in a short time interval. The proposed technique shows the efficiency with some real video sequences. Accurate maneuver prediction for surrounding vehicles enables intelligent vehicles to make safe and socially compliant decisions in advance, thus improving the safety and comfort of the driving. The main contribution of this paper is proposing a practical, high-performance, and low-cost maneuver-prediction approach for intelligent vehicles. Our approach is based on a dynamic Bayesian network, which exploits multiple predictive features, namely, historical states of predicting vehicles, road structures, as well as traffic interactions for inferring the probability of each maneuver. The paper also presents algorithms of feature extraction for the network. Our approach is verified on real traffic data in large-scale publicly available datasets. The results show that our approach can recognize the lane-change maneuvers with an F1 score of 80 and an advanced prediction time of 3.75 s, which greatly improves the performance on prediction compared to other baseline approaches. Lane relative vehicle navigation and control requires accurate lane-relative positioning of the vehicle. This relative position can be computed by comparing the vehicle absolute position with analytic roadway maps, which requires both high-accuracy positioning of the vehicle and high-accuracy lane maps. Carrier Phase Differential GPS (CPDGPS) aided INS or CPDGPS aided encoders is capable of estimating vehicle absolute position (relative to earth center) with centimeter level accuracy; however, to the best of the author's knowledge, the accuracy of lane level maps is currently not sufficient. In this paper, we first consider the structure of lane level maps that are compatible with standard practices of GIS road modeling. Then, various analytic lane definition are discussed. We also present a method of building lane level maps from high-accuracy positioning data along the lane center. The data is segmented according to road intersections. Shape points (vertices) as a function of arclength are located based on changes in estimated curvature. For each segment, the parameters are estimated by least-square criteria and can be refined as new datasets become available. This process is shown by an example. Modern cars are incorporating an increasing number of driver assist features, among which automatic lane keeping. The latter allows the car to properly position itself within the road lanes, which is also crucial for any subsequent lane departure or trajectory planning decision in fully autonomous cars. Traditional lane detection methods rely on a combination of highly-specialized, hand-crafted features and heuristics, usually followed by post-processing techniques, that are computationally expensive and prone to scalability due to road scene variations. More recent approaches leverage deep learning models, trained for pixel-wise lane segmentation, even when no markings are present in the image due to their big receptive field. Despite their advantages, these methods are limited to detecting a pre-defined, fixed number of lanes, e.g. ego-lanes, and can not cope with lane changes. In this paper, we go beyond the aforementioned limitations and propose to cast the lane detection problem as an instance segmentation problem - in which each lane forms its own instance - that can be trained end-to-end. To parametrize the segmented lane instances before fitting the lane, we further propose to apply a learned perspective transformation, conditioned on the image, in contrast to a fixed ”bird’s-eye view” transformation. By doing so, we ensure a lane fitting which is robust against road plane changes, unlike existing approaches that rely on a fixed, predefined transformation. In summary, we propose a fast lane detection algorithm, running at 50 fps, which can handle a variable number of lanes and cope with lane changes. We verify our method on the tuSimple dataset and achieve competitive results. Modern cars are incorporating an increasing number of driver assist features, among which automatic lane keeping. The latter allows the car to properly position itself within the road lanes, which is also crucial for any subsequent lane departure or trajectory planning decision in fully autonomous cars. Traditional lane detection methods rely on a combination of highly-specialized, hand-crafted features and heuristics, usually followed by post-processing techniques, that are computationally expensive and prone to scalability due to road scene variations. More recent approaches leverage deep learning models, trained for pixel-wise lane segmentation, even when no markings are present in the image due to their big receptive field. Despite their advantages, these methods are limited to detecting a pre-defined, fixed number of lanes, e.g. ego-lanes, and can not cope with lane changes. In this paper, we go beyond the aforementioned limitations and propose to cast the lane detection problem as an instance segmentation problem - in which each lane forms its own instance - that can be trained end-to-end. To parametrize the segmented lane instances before fitting the lane, we further propose to apply a learned perspective transformation, conditioned on the image, in contrast to a fixed "bird's-eye view" transformation. By doing so, we ensure a lane fitting which is robust against road plane changes, unlike existing approaches that rely on a fixed, pre-defined transformation. In summary, we propose a fast lane detection algorithm, running at 50 fps, which can handle a variable number of lanes and cope with lane changes. We verify our method on the tuSimple dataset and achieve competitive results. In order to support driver assistance systems in unconstrained environments, we propose to extend local appearance-based road classification with a spatial feature generation and classification. Therefore, a hierarchical approach consisting of multiple low level base classifiers, the novel spatial feature generation, as well as a final road terrain classification, is used. The system perceives a variety of local properties of the environment by means of base classifiers operating on patches extracted from monocular camera images, each represented in a metric confidence map. The core of the proposed approach is the computation of spatial ray features (SPRAY) from these confidence maps. With this, the road-terrain classifier can decide based on local visual properties and their spatial layout in the scene. In order to show the feasibility of the approach, the extraction and evaluation of the metric ego-lane driving corridor on an inner city stream is demonstrated. This is a challenging task because on a local appearance level, ego-lane is not distinguishable from other asphalt parts on the road. However, by incorporating the proposed SPRAY features the distinction is possible without requiring an explicit lane model. Due to the parallel structure of this bottom-up approach, the implemented system operates in real-time with approximately 25 Hz on a GPU. This paper proposes a method for achieving improved ego-vehicle global localization with respect to an approaching intersection, which is based on the alignment of visual landmarks perceived by the on-board visual system, with the information from a proposed extended digital map (EDM). The visual system relies on a stereovision system that provides a detailed 3-D description of the environment, including road landmark information (lateral lane delimiters, painted traffic signs, curbs, and stop lines) and dynamic environment information (other vehicles). An EDM is proposed, which enriches the standard map information with a detailed description of the intersection required for current lane identification, landmark alignment, and ego-vehicle accurate global localization. A novel approach for lane-delimiter classification, which is necessary for the lane identification, is also presented. An original solution for identifying the current lane, combining visual and map information with the help of a Bayesian network (BN), is proposed. Extensive experiments have been performed, and the results are evaluated with a Global Navigation Satellite System of high accuracy (2 cm). The achieved global localization accuracy is of submeter level, depending on the performance of the stereovision system. For safe and reliable autonomous driving systems, prediction of surrounding vehicles' future behavior and potential risks are critical. The state-of-the-art prediction algorithms tend to show limited performance on long-term predictions due to their deterministic nature. In this paper, a probabilistic lateral motion prediction algorithm is proposed based on multilayer perceptron (MLP) approach. The MLP model consists of two parts; target lane and trajectory models. In order to develop an intuitive and accurate prediction algorithm, a lane-based trajectory prediction model is introduced based on the fact that vehicles drive within a lane except for during lane changes. More specifically, a set of three representative trajectories with different levels of lane-change positions are generated for each target lane, and real-world traffic data is categorized by each trajectory for MLP training. These target lane and trajectory models enable the stochastic MLP modeling and training. The proposed MLP model outputs probabilities of how likely a vehicle will follow each trajectory and each lane for a given input of vehicle position history including current position. For training the MLP model, Next Generation Simulation traffic data are used. Simulation results show that the proposed algorithm detects lane-changes one to one and a half second earlier than existing methods and three seconds before lane crossing with about ninety percentages accuracy. Detecting the road area and ego-lane ahead of a vehicle is central to modern driver assistance systems. While lane-detection on well-marked roads is already available in modern vehicles, finding the boundaries of unmarked or weakly marked roads and lanes as they appear in inner-city and rural environments remains an unsolved problem due to the high variability in scene layout and illumination conditions, amongst others. While recent years have witnessed great interest in this subject, to date no commonly agreed upon benchmark exists, rendering a fair comparison amongst methods difficult. In this paper, we introduce a novel open-access dataset and benchmark for road area and ego-lane detection. Our dataset comprises 600 annotated training and test images of high variability from the KITTI autonomous driving project, capturing a broad spectrum of urban road scenes. For evaluation, we propose to use the 2D Bird's Eye View (BEV) space as vehicle control usually happens in this 2D world, requiring detection results to be represented in this very same space. Furthermore, we propose a novel, behavior-based metric which judges the utility of the extracted ego-lane area for driver assistance applications by fitting a driving corridor to the road detection results in the BEV. We believe this to be important for a meaningful evaluation as pixel-level performance is of limited value for vehicle control. State-of-the-art road detection algorithms are used to demonstrate results using classical pixel-level metrics in perspective and BEV space as well as the novel behavior-based performance measure. All data and annotations are made publicly available on the KITTI online evaluation website in order to serve as a common benchmark for road terrain detection algorithms. High-end vehicles are already equipped with safety systems, such as assistive braking and automatic lane following, enhancing vehicle safety. Yet, these current solutions can only help in low-complexity driving situations. In this paper, we introduce a parallel autonomy , or shared control, framework that computes safe trajectories for an automated vehicle, based on human inputs. We minimize the deviation from the human inputs while ensuring safety via a set of collision avoidance constraints. Our method achieves safe motion even in complex driving scenarios, such as those commonly encountered in an urban setting. We introduce a receding horizon planner formulated as nonlinear model predictive control (NMPC), which includes the analytic descriptions of road boundaries and the configuration and future uncertainties of other road participants. The NMPC operates over both steering and acceleration simultaneously. We introduce a nonslip model suitable for handling complex environments with dynamic obstacles, and a nonlinear combined slip vehicle model including normal load transfer capable of handling static environments. We validate the proposed approach in two complex driving scenarios. First, in an urban environment that includes a left-turn across traffic and passing on a busy street. And second, under snow conditions on a race track with sharp turns and under complex dynamic constraints. We evaluate the performance of the method with various human driving styles. We consequently observe that the method successfully avoids collisions and generates motions with minimal intervention for parallel autonomy. We note that the method can also be applied to generate safe motion for fully autonomous vehicles. The detection of multiple curved lane markings is still a challenge for advanced driver assistance systems today, due to interference such as road markings and shadows cast by roadside structures and vehicles. The vanishing point @math contains the global information of the road image. Hence, @math -based lane detection algorithms are quite insensitive to interference. When curved lanes are assumed, @math shifts with respect to the rows of the image. In this paper, a @math for each individual row of the image is estimated by first extracting a @math (vertical position of the @math ) for each individual row of the image from the v-disparity. Then, based on the estimated @math 's, a 2-D @math (horizontal position of the @math ) accumulator is efficiently formed. Thus, by globally optimizing this 2-D @math accumulator, globally optimum @math s for the road image are extracted. Then, estimated @math s are utilized for multiple curved lane marking detection on nonflat road surfaces. The resultant system achieves a detection rate of 99 in 1862 frames of six stereo vision test sequences. In this work, a new inverse perspective mapping (IPM) technique is proposed based on a robust estimation of the vanishing point, which provide bird-view images of the road, so that facilitating the tasks of road modeling and vehicle detection and tracking. This new approach has been design to cope with the instability that cameras mounted on a moving vehicle suffer. The estimation of the vanishing point relies on a novel and efficient feature extraction strategy, which segmentates the lane markings of the images by combining a histogram-based segmentation with temporal and frequency filtering. Then, the vanishing point of each image is stabilized by means of a temporal filtering along the estimates of previous images. In a last step, the IPM image is computed based on the stabilized vanishing point. Tests have been carried out on several long video sequences captured from cameras inside a vehicle being driven along highways and local roads, with different illumination and weather conditions, presence of shadows, occluding vehicles, and slope changes. Results have shown a significant improvement in terms of lane width constancy and parallelism between lane markings over non-stabilized IPM algorithms. Risk estimation for the current traffic situation is crucial for safe autonomous driving systems. One part of the uncertainty in risk estimation is the behavior of the surrounding traffic participants. In this paper we focus on highway scenarios, where possible behaviors consist of a change in acceleration and lane change maneuvers. We present a novel approach for the recognition of lane change intentions of traffic participants. Our novel approach is an extension of the Naive Bayesian approach and results in a generative model. It builds on the relations to the directly surrounding vehicles and to the static traffic environment. We obtain the conditional probabilities of all relevant features using Gaussian mixtures with a flexible number of components. We systematically reduce the number of features by selecting the most powerful ones. Furthermore we investigate the predictive power of each feature with respect to the time before a lane change event. In a large scale experiment on real world data with over 160.781 samples collected on a test drive of 1100km we trained and validated our intention prediction model and achieved a significant improvement in the recognition performance of lane change intentions compared to current state of the art methods.