CROSS-REFERENCE TO RELATED APPLICATION

This patent application is a U.S. National Phase Application under 35 U.S.C. § 371 of International Application No. PCT/CN2019/073970, filed Jan. 30, 2019, entitled “A RGB POINT CLOUDS BASED MAP GENERATION SYSTEM FOR AUTONOMOUS VEHICLES,” which is incorporated by reference herein by its entirety.

TECHNICAL FIELD

Embodiments of the present disclosure relate generally to operating autonomous vehicles. More particularly, embodiments of the disclosure relate to a RGB (red green blue) point clouds based map generation system for autonomous driving vehicles (ADVs).

BACKGROUND

Vehicles operating in an autonomous mode (e.g., driverless) can relieve occupants, especially the driver, from some driving-related responsibilities. When operating in an autonomous mode, the vehicle can navigate to various locations using onboard sensors, allowing the vehicle to travel with minimal human interaction or in some cases without any passengers.

Motion planning and control are critical operations in autonomous driving. However, conventional motion planning operations estimate the difficulty of completing a given path mainly from its curvature and speed, without considering the differences in features for different types of vehicles. Same motion planning and control is applied to all types of vehicles, which may not be accurate and smooth under some circumstances.

High-definition (HD) three dimensional (3D) point cloud map is important for autonomous vehicles, and a point cloud registration system is central to construction of a HD 3D point cloud map from point clouds data. A HD 3D point cloud map with high accuracy is key to motion planning for an autonomous driving vehicle (ADV).

However, several problems exist which influence both the performance and accuracy of map construction. First, current point cloud registration algorithms are highly dependent on a GPS signal for localization for the map construction, which can have a margin of errors in the orders of meters, or register a signal with GPS bounces, for example, in city streets lined by tall buildings or dense forest. Or the GPS may fail to register a signal altogether, for example, in tunnels or garages. Secondly, the registration of point clouds may be computational complex, e.g., challenging and time-consuming for large map areas.

Furthermore, an accuracy of a registered HD 3D point clouds map may depend on the quality of the point clouds or the splicing of the point clouds. If the point cloud map has a ghost image caused by, for example, splicing of the point clouds, accuracy of the HD 3D point cloud map may suffer. In order to ensure high accuracy, it is desirable to evaluate the point cloud map for ghosting effects.

SUMMARY

In a first aspect, embodiments of the disclosure provide a computer-implemented method to register point clouds to a three-dimensional (3D) point cloud map for an autonomous driving vehicle (ADV), the method including: receiving a plurality of point clouds captured by one or more LIDAR sensors of an ADV and corresponding poses; receiving a plurality of RGB images from one or more image capturing sensors of the ADV; synchronizing the RGB images with the point clouds to obtain RGB point clouds; extracting features from the RGB point clouds, the features including contextual and spatial information of the RGB point clouds; registering the RGB point clouds based on the extracted features; and generating a point cloud map based on the registration of the RGB point clouds.

In a second aspect, embodiments of the disclosure provide a non-transitory machine-readable medium having instructions stored therein, which when executed by a processor, cause the processor to perform operations, the operations including: receiving a plurality of point clouds captured by one or more LIDAR sensors of an ADV and corresponding poses; receiving a plurality of RGB images from one or more image capturing sensors of the ADV; synchronizing the RGB images with the point clouds to obtain RGB point clouds; extracting features from the RGB point clouds, the features including contextual and spatial information of the RGB point clouds; registering the RGB point clouds based on the extracted features; and generating a point cloud map based on the registration of the RGB point clouds.

In a third aspect, embodiments of the disclosure provide a data processing system, including: a processor; and a memory coupled to the processor to store instructions, which when executed by the processor, cause the processor to perform operations, the operations including: receiving a plurality of point clouds captured by one or more LIDAR sensors of an ADV and corresponding poses; receiving a plurality of RGB images from one or more image capturing sensors of the ADV; synchronizing the RGB images with the point clouds to obtain RGB point clouds; extracting features from the RGB point clouds, the features including contextual and spatial information of the RGB point clouds; registering the RGB point clouds based on the extracted features; and generating a point cloud map based on the registration of the RGB point clouds.

