RELATED APPLICATION

Under 35 U.S.C. 119, this application claims the benefit and right of priority of Chinese Patent Application No. 201811014381.3, filed on 31 Aug. 2018.

BACKGROUND

Field

This disclosure is generally related to the technical field of artificial intelligence and machine learning. More specifically, this disclosure is related to a method and system for facilitating recognition of vehicle parts based on a neural network.

Related Art

In conventional vehicle damage assessment approaches, a vehicle insurance company can send a professional (such as an insurance adjuster) to conduct a manual on-site survey and determine a repair plan for a damaged vehicle. After the professional captures images of the damaged vehicle, these “captured images” can be compared against similar images in a library or database of images. The library can include images of damaged vehicle parts, which were previously captured from other images of other damaged vehicles (“library images”). In a conventional vehicle damage assessment approach, a system can determine the damage assessment result of the damaged vehicle based on the comparison of the captured images with the library images.

However, conducting a manual survey and determining a repair plan based on the comparison of the captured images against library images can result in a long processing time, and can incur a significant cost in manpower and training of professionals. Current approaches can use image-based artificial intelligence and machine learning in vehicle damages assessment approaches that may reduce both processing time and labor costs. For example, these approaches can enable automatic identification of damaged parts and the degree of the damage based on pictures of a vehicle taken by users at the site of an accident or other incident. In addition, the on-site pictures of the vehicle can be used by computer-vision image identification technology with artificial intelligence to provide a repair solution. Therefore, by automating the survey and damage assessment approaches, the labor costs incurred by a vehicle insurance company can be reduced and the vehicle insurance claiming experience of a user can be improved.

While using the automated survey and damage assessment approach can result in some benefits, some challenges still remain in providing accurate identification or recognition of vehicle parts and the degree of damage of those parts.

SUMMARY

One embodiment facilitates a computer system for recognizing parts of a vehicle. During operation, the system generates a convolution feature map of an image of the vehicle. The system determines, based on the convolution feature map, one or more proposed regions, wherein a respective proposed region corresponds to a target of a respective vehicle part. The system determines a class and a bounding box of a first vehicle part corresponding to a first proposed region based on a feature of the first proposed region. The system optimizes classes and bounding boxes of the vehicle parts based on correlated features of the corresponding proposed regions. The system generates a result which indicates a list including an insurance claim item and corresponding damages based on the optimized classes and bounding boxes of the vehicle parts.

In some embodiments, determining the one or more proposed regions is performed by a fully convolutional network. Determining the one or more proposed regions comprises: performing, for a plurality of convolutional mapping positions, a convolutional operation for a respective convolutional mapping position with a sliding window in the convolution feature map; obtaining a feature vector of the respective convolutional mapping position; predicting whether the respective convolutional mapping position comprises a foreground target with respect to a plurality of predetermined anchors based on the feature vector of the respective convolutional mapping position; and predicting a border of a proposed region in the respective convolutional mapping position that corresponds to each anchor.

In some embodiments, the correlated features of the corresponding proposed regions include one or more of: a size of a respective proposed region; a position relationship between the proposed regions; a distance between the proposed regions; and an intersection-over-union ratio of the proposed regions.

In some embodiments, prior to optimizing the classes and the bounding boxes of the vehicle parts based on the correlated features of the corresponding proposed regions: the system obtains the classes and the bounding boxes of the vehicle parts corresponding to the proposed regions; and the system extracts the correlated features of the proposed regions.

In some embodiments, the system determines an energy function and a corresponding probability function of the conditional random field. The system solves the energy function while minimizing the probability function. The energy function comprises: a data term which is based on a probability of each proposed region belonging to each class; and a smoothing term which is based on the correlated features of the proposed regions.

In some embodiments, optimizing the classes and bounding boxes of the vehicle parts is performed by a recurrent neural network. The system solves the energy function while minimizing the probability function by performing multiple iterative operations through the recurrent neural network to obtain an approximation of the probability function. An iterative operation comprises updating the probability of each proposed region belonging to each class based on a pre-trained compatibility matrix. The pre-trained compatibility matrix indicates a probability of compatibility between the classes of the vehicle.

In some embodiments, parameters used to perform the method are jointly trained from end to end based on training samples.

In some embodiments, the parameters which are jointly trained from end to end are trained by the following operations. The system inputs the training samples into a convolution module, wherein the training samples indicate corresponding classes and bounding boxes. The system obtains a prediction result based on the classes and the bounding boxes optimized by the conditional random field module, wherein the prediction result comprises predicted classes and predicted bounding boxes of a plurality of target regions. The system determines a prediction error for the target regions based on the prediction result and the corresponding classes and bounding boxes indicated by the training samples. The system determines a loss function based on the prediction error, wherein the loss function comprises a cross term of the prediction error for the target regions. The system counter-propagates the prediction error based on the loss function by counter-propagating the prediction error of a first target region to a set of target regions correlated with the first target region.

One embodiment facilitates a computer system for recognizing parts of a vehicle. The system comprises a convolution module, a region proposal module, a classification module, a conditional random field module, and a reporting module. The convolution module is configured to generate a convolution feature map of an image of the vehicle. The region proposal module is configured to determine, based on the convolution feature map, one or more proposed regions, wherein a respective proposed region corresponds to a target of a respective vehicle part. The classification module is configured to determine a class and a bounding box of a first vehicle part corresponding to a first proposed region based on a feature of the first proposed region. The conditional random field module is configured to: optimize classes and bounding boxes of the vehicle parts based on correlated features of the corresponding proposed regions; and output on a display screen the image with annotations indicating the optimized classes and bounding boxes of the vehicle parts. The reporting module is configured to generate, based on the image with annotations, a report which indicates a degree of damage to a respective vehicle part.

