TECHNICAL FIELD

The present disclosure relates generally navigating an autonomous vehicle through traffic and more specifically to routing of an autonomous vehicle through traffic based on modulation of a world model representing traffic entities.

BACKGROUND

An autonomous vehicle uses different types of sensors to receive input describing the surroundings (or environment) of the autonomous vehicle while driving through traffic. For example, an autonomous vehicle may perceive the surroundings using camera images and lidar scans. The autonomous vehicle determines whether an object in the surroundings is stationary, for example, buildings or trees or the object is non-stationary, for example, a pedestrian, a vehicle, and so on. The autonomous vehicle system predicts the motion of non-stationary objects to make sure that the autonomous vehicle is able to navigate through non-stationary obstacles in the traffic.

Conventional systems predict motion of pedestrians and other vehicles to determine whether they are likely to come in the path of the autonomous vehicle. The autonomous vehicle navigates through traffic so as to avoid collisions with any pedestrians or other vehicles. However, conventional techniques fail to accurately predict motion of certain non-stationary objects for example, pedestrians, bicyclists, and so on. For example, if the autonomous vehicle detects a pedestrian standing in a street corner, the motion of the pedestrian does not help predict whether the pedestrian will cross the street or whether the pedestrian will remain standing on a street corner. Similarly, if the autonomous vehicle detects a bicyclist in a lane, the current motion of the bicycle does not help the autonomous vehicle predict whether the bicycle will change lanes.

Failure of autonomous vehicles to accurately predict motion of non-stationary traffic objects results in unnatural movement of the autonomous vehicle, for example, as a result of the autonomous vehicle suddenly stopping due to a pedestrian moving in the road or the autonomous vehicle continuing to wait for a person to cross a street even if the person never intends to cross the street.

SUMMARY

Embodiments of the invention predict hidden context attributes associated with traffic entities that determine behavior of these traffic entities in the traffic. The traffic entities represent non-stationary objects in the traffic in which the autonomous vehicle is driving, for example, a pedestrian, a bicycle, a vehicle, a delivery robot, and so on. Hidden context includes factors that affect the behavior of such traffic entities, for example, a state of mind of a user represented by a traffic entity such as a pedestrian. A hidden context may represent a task that a user represented by the traffic entity is planning on accomplishing, for example, crossing the street or stepping on to the street to pick up some object. The hidden context may represent a degree of awareness of the autonomous vehicle by a user represented by the traffic entity. An example of a degree of awareness of the existence of the autonomous vehicle in the vicinity is whether a bicyclist is aware of the existence of the autonomous vehicle in the proximity of the bicyclist. The bicyclist may navigate the bicycle differently depending on the degree of awareness of the existence of the autonomous vehicle in the vicinity. The system uses the hidden context to predict behavior of people near a vehicle in a way that more closely resembles how human drivers would judge the behavior. The hidden context may represent a goal of a user represented by the traffic entity, wherein the user expects to achieve the goal within a threshold time interval. For example, the goal may represent crossing the street to reach the other side of the street within the next few seconds.

In one embodiment, a group of users (or human observers) view sample images of traffic entities near streets and/or vehicles and indicate or are measured for their understanding of how they believe the people will behave. These indicators or measurements are then used as a component for training a machine learning based model that predicts how people will behave in a real-world context. The machine learning based model is trained based on the reactions of human observers to sample images in a training environment. The trained machine learning based model predicts behavior of traffic entities in a real-world environment, for example, actual pedestrian behavior in traffic as a vehicle navigates through the traffic.

According to an embodiment, an autonomous vehicle identifies traffic entities, for example, based on the sensor data. For each traffic entity, motion parameters describing movement of the traffic entity are determined. A hidden context is determined based on a machine learning based model. The machine learning based model is trained based on feedback received from users responsive to presenting images or videos showing traffic scenarios to the users. The autonomous vehicle is navigated based on the motion parameters and the hidden context of each of the traffic entities.

According to an embodiment, an autonomous vehicle receives sensor data from sensors mounted on the autonomous vehicle. Traffic entities from the traffic are identified based on the sensor data. For each of traffic entity, a hidden context is determined based on a machine learning based model. The machine learning based model is trained based on feedback received from users presented with images or videos showing traffic scenarios. The output of the machine learning based model comprises a measure of statistical distribution of the hidden context. An activation threshold value is determined for a braking system of the autonomous vehicle based on the measure of statistical distribution of the hidden context. A prediction is made that the autonomous vehicle is likely to reach within the activation threshold of a particular traffic entity within a threshold time interval. Responsive to determining that the autonomous vehicle is likely to reach within the activation threshold of the particular traffic entity within the threshold time interval, the braking system of the autonomous vehicle is activated.

In an embodiment, the activation threshold value for the braking system of the autonomous vehicle has a value inversely proportionate to the degree of uncertainty in determining the hidden context.

In an embodiment, the activation threshold value for braking system of the autonomous vehicle is determined based on the distribution of the hidden context. Accordingly, a measure of kurtosis of the output distribution of the hidden context is determined. The activation threshold value for braking system of the autonomous vehicle is determined to be a value inversely proportionate to the measure of kurtosis of the output distribution of the hidden context.

According to another embodiment, an autonomous vehicle modifies a world model based on the hidden context predicted by a machine learning based model. A point cloud representation of the surroundings of an autonomous vehicle is generated, for example, based on sensor data obtained by sensors of the autonomous vehicle. Traffic entities are identified based on the sensor data. For each traffic entity, motion parameters describing movement of the traffic entity are determined. A hidden context is predicted using a machine learning based model trained based on feedback received from users presented with images or videos showing traffic scenarios. A region of the point cloud where the traffic entity is expected to reach within a threshold time interval is determined. The region is modified based on the hidden context. The autonomous vehicle is navigated so that the autonomous vehicle stays at least a threshold distance away from the modified region of each traffic entity.

In an embodiment, the region is in the direction determined based on a motion vector of the traffic entity. The region is modified based on the hidden context as follows. If a determination is made that the hidden context indicates that the user represented by the traffic entity is likely to move in the direction having a component along the motion vector, the region is extended along the direction of the motion vector. If a determination is made that the hidden context indicates that the user represented by the traffic entity is likely to move in a direction having a component opposite to direction of the motion vector, the size of the region is decreased along the direction of the motion vector.

In an embodiment, the autonomous vehicle is navigated by generating signals for controlling the autonomous vehicle based on the motion parameters and the hidden context of each of the traffic entities. The generated signals are sent to controls of the autonomous vehicle.

The sensor data may represent images captured by cameras mounted on the autonomous vehicle or lidar scans captured by a lidar mounted on the autonomous vehicle.

