TECHNICAL FIELD

The present disclosure relates generally to vehicles equipped with sensor damage detection system useful for driving assistance or autonomous driving, and in particular, some embodiments relate to vehicle navigation in response to broken or uncalibrated sensors.

BACKGROUND

On-board sensors in vehicles, such as autonomous vehicles (AVs), supplement and bolster the vehicle's field of vision by providing accurate sensor data. Sensor data is used, for example, in applications of blind spot detection, lane change assisting, read end radar for collision warning or collision avoidance, park assisting, cross-traffic monitoring, brake assisting, emergency braking, and/or automatic distance controlling. Examples of on-board sensors include, for example, passive sensors and active sensors. On-board sensors include camera, Lidar, radar, GPS, sonar, ultrasonic, IMU (inertial measurement unit), accelerometers, gyroscopes, magnetometers, and FIR (far infrared) sensors. Sensor data may include image data, reflected laser data, and/or the like. Often, images captured by the on-board sensors utilize a coordinate system to determine the distance and angle of the contents and objects captured in the image. Such real-time space information may be acquired near the vehicle using various on-board sensors located throughout the vehicle, which may then be processed to calculate and to determine the safe driving operation of the vehicle. On-board sensors may be subject to spectral distribution, ambient light, specular reflections, and other optical phenomena. Sensors may also not be able to capture data and may be broken or uncalibrated. If left uncalibrated, the accuracy of information obtained from the sensors may be compromised. If the sensor is determined to be broken, the vehicle may be unsafe to operate. Therefore, it is imperative that the vehicle is capable of detecting uncalibrated or damaged sensors on the vehicle, and undergoing preventive measures to avoid danger, accident, or catastrophe in an event of an uncalibrated or broken sensor.

Currently, as an example, calibration panels are installed in select areas (e.g., garage) to calibrate the on-board sensors. However, not every uncalibrated sensor can be recalibrated by the calibration panels. Additionally, the calibration panels cannot fix broken sensors. As a result, the on-board sensors may be uncalibrated or broken for a period of time, thereby causing a concern of safety of vehicle navigation for a vehicle such as an AV. For example, if the sensor is uncalibrated, the vehicle may not be able to capture all the data from the field of view captured by the sensor. The shortfalls are addressed by the present disclosures, which provides the vehicle with capability to not only detect broken or uncalibrated sensors, but also preventive measures when a sensor is broken or uncalibrated, including limiting a range of motions of the vehicle (e.g., a limited number of driving options), restricting the vehicle from using the broken or uncalibrated sensors, and allowing the vehicle to execute a safe driving action.

SUMMARY

Described herein are methods and systems for determining whether sensors, such as sensors disposed on a side of a vehicle, are broken or uncalibrated, determining whether the an uncalibrated sensor can be calibrated, determining that the uncalibrated sensor is broken if the uncalibrated sensor cannot be calibrated, and limiting a range of motion of the vehicle when the side sensor is determined to be broken or uncalibrated. Various embodiments of the present disclosure provide a vehicle system comprising a sensor configured to capture data, one or more processors, and a memory storing instructions. The memory storing instructions, when executed by the one or more processors, may cause the system to perform determining whether the sensor is broken or uncalibrated, based on the data, and limiting a range of motion of the vehicle when the sensor is broken or uncalibrated.

In some embodiments, the limiting the range of motion may comprise limiting a number of driving options for the vehicle when the sensor is determined to be broken or uncalibrated.

In some embodiments, the limiting the range of motion comprises, in response to the sensor being broken or uncalibrated, restricting the vehicle from making a turn beyond a threshold radius of curvature or angle in a direction corresponding to a side of the broken sensor. The threshold radius of curvature or angle is determined based on an extent of damage or miscalibration of the sensor based on an extent of damage of the sensor.

In some embodiments, the instructions further cause the system to perform, in response to determining that the sensor is broken or uncalibrated, obtaining missing data from another vehicle, and in response to obtaining the missing data from the another vehicle, restoring a range of motion of the vehicle.

In some embodiments, the determining whether the sensor is broken or uncalibrated comprises: determining a current parameter of the sensor from the data and a previous parameter of the sensor from historical data; determining a difference between the current parameter and the previous parameter; and determining, in response to the difference exceeding a threshold, that the sensor is broken or uncalibrated.

In some embodiments, the data comprises an image or one or more features extracted from the image at a specific location, and the historical data comprises a second image or one or more features extracted from the second image at the specific location.

In some embodiments, the sensor is located at a first side of the vehicle, and the limiting the range of motion comprises, in response to the difference exceeding the threshold, preventing the vehicle from turning in a direction of the first side.

In some embodiments, the system comprises a working sensor configured to receive second data, and wherein the determining whether the sensor is broken or uncalibrated comprises: determining a parameter of the working sensor from the second data; determining whether the sensor is broken or uncalibrated based on a difference between a parameter of the sensor and the parameter of the working sensor; and in response to the difference exceeding a threshold, determining that the sensor is broken or uncalibrated.

In some embodiments, the limiting the range of motion of the vehicle comprises: limiting, in response to the difference between the parameter of the sensor and the parameter of the second sensor exceeding the threshold, the range of motion of the vehicle to a number of driving options; and restricting the vehicle from turning in a direction of the first side.

In some embodiments, the data comprises an image or one or more features extracted from the image at a specific location, and the second data comprises a second image or one or more second features from the second image at the specific location.

In some embodiments, the image or one or more features extracted from the image at the specific location include images of a traffic light at a traffic intersection, a stationary object at a traffic intersection, a pedestrian, and/or a vehicle.

In some embodiments, the determining whether the sensor is broken or uncalibrated comprises: determining that the sensor is broken in response to the sensor not being able to capture the data, and wherein the limiting the range of motion of the vehicle comprises restricting the vehicle from moving in a direction of a field of view captured by the sensor.

In some embodiments, the determining whether the sensor is broken or uncalibrated comprises: determining whether the sensor can be calibrated based on a working sensor; in response to determining that the sensor can be calibrated, determining that the sensor is uncalibrated and calibrating the sensor; in response to determining that the sensor cannot be calibrated, determining that the sensor is broken; and in response to determining that the sensor is broken, restricting the vehicle from moving in a direction of a field of view captured by the sensor.

In some embodiments, the limiting the range of motion of the vehicle comprises: determining an amount of the range of motion by which the vehicle is restricted based on a speed of the vehicle, a traffic density, a road condition, or a weather condition.

In some embodiments, the limiting the range of motion of the vehicle comprises: determining an amount of the range by which the vehicle is restricted based on a whether the sensor is broken.

Various embodiments of the present disclosure provide a method of determining a safe driving action of a vehicle, comprising capturing data from a sensor, transmitting the data to a sensor damage detector, determining, using the sensor damage detector, whether the sensor is broken or uncalibrated; in response to determining that the sensor is broken or uncalibrated, transmitting information to an error handler that the sensor is broken or uncalibrated; and limiting, using the error handler, a range of motion of the vehicle.

In some embodiments, the capturing the data from the sensor comprises: capturing the data from the sensor located at a first side of the vehicle; and the limiting the range of motion of the vehicle comprises limiting the range of motion of the vehicle to a number of driving options of the vehicle.

In some embodiments, the determining whether the sensor is broken or uncalibrated comprises: determining a current parameter of the sensor from the data and a previous parameter of the sensor from historical data; determining a difference between the current parameter and the previous parameter; and determining, in response to the difference exceeding a threshold, that the sensor is broken or uncalibrated.

In some embodiments, the method further comprises: capturing second data from a working sensor; determining a parameter of the sensor from the data and a parameter of the working sensor from the second data; determining a difference between the parameter of the sensor from the data and the parameter of the working sensor from the second data; and determining, in response to the difference exceeding a threshold, that the sensor is broken or uncalibrated.

In some embodiments, the data comprises an image or one or more features extracted from the image at a specific location, and the second data comprises a second image or one or more features extracted from the second image at the specific location.

These and other features of the systems, methods, and non-transitory computer readable media disclosed herein, as well as the methods of operation and functions of the related elements of structure and the combination of parts and economies of manufacture, will become more apparent upon consideration of the following description and the appended claims with reference to the accompanying drawings, all of which form a part of this specification, wherein like reference numerals designate corresponding parts in the various figures. It is to be expressly understood, however, that the drawings are for purposes of illustration and description only and are not intended as a definition of the limits of the invention.

