INTRODUCTION

The information provided in this section is for the purpose of generally presenting the context of the disclosure. Work of the presently named inventor, to the extent it is described in this section, as well as aspects of the description that may not otherwise qualify as prior art at the time of filing, are neither expressly nor impliedly admitted as prior art against the present disclosure.

The present disclosure relates to transforming sensor data from a given sensor configuration into arbitrary other frames of reference to train models used with different sensor configurations.

In many applications, models (e.g., machine learning-based models) are trained using data collected by sensors. In use, a trained model receives data from the sensors and outputs data that the model is trained to output based on the data received from the sensors. For example, in automotive applications (e.g., autonomous driving applications), models are trained using data collected by various sensors (e.g., cameras) mounted on a vehicle. The sensors collect data while the vehicle is being driven on roads. The collected data is used to train a model. The trained model is deployed in the vehicles. In use, the trained model receives data from the sensors and outputs data that the model is trained to output.

SUMMARY

A system comprises a processor and a memory storing instructions which when executed by the processor configure the processor to receive first data from a first set of sensors arranged in a first configuration. The instructions configure the processor to transform the first data to a second data to train a model to recognize third data captured by a second set of sensors arranged in a second configuration. The second configuration is different than the first configuration. The instructions configure the processor to train the model based on the second set of sensors sensing the second data to recognize the third data captured by the second set of sensors arranged in the second configuration.

In another feature, the trained model recognizes the third data captured by the second set of sensors arranged in the second configuration.

In another feature, at least one of the second set of sensors is different than at least one of the first set of sensors.

In other features, the instructions configure the processor to detect one or more objects in the first data and to separate the objects from background in the first data.

In other features, the instructions configure the processor to transform perspectives of the objects from 2D to 3D using a machine learning based model and to transform a perspective of the background from 2D to 3D using a computer graphics technique.

In another feature, the instructions configure the processor to combine the transformed perspectives of the objects and the transformed perspective of the background to generate a 3D scene representing the first data.

In another feature, the instructions configure the processor to train the model based on the second set of sensors sensing the 3D scene representing the first data.

In other features, the instructions configure the processor to generate 2D representations of the 3D perspectives of the objects sensed by the second set of sensors and to generate a 2D representation of the 3D perspective of the background sensed by the second set of sensors.

In another feature, the instructions configure the processor to combine the 2D representations of the 3D perspectives of the objects and the 2D representation of the 3D perspective of the background sensed by the second set of sensors.

In another feature, the instructions configure the processor to train the model based on the combination of the 2D representations of the 3D perspectives of the objects and the 2D representation of the 3D perspective of the background sensed by the second set of sensors.

In still other features, a method comprises receiving first data from a first set of sensors arranged in a first configuration. The method comprises transforming the first data to a second data that reflects the first data as perceived by a second set of sensors arranged in a second configuration. The second configuration is different than the first configuration. The method comprises training a model by sensing the second data using the second set of sensors to recognize third data captured by the second set of sensors arranged in the second configuration.

In another feature, the method further comprises recognizing, using the trained model, the third data captured by the second set of sensors arranged in the second configuration.

In another feature, at least one of the second set of sensors is different than at least one of the first set of sensors.

In other features, the method further comprises detecting one or more objects in the first data and separating the objects from background in the first data.

In other features, the method further comprises transforming perspectives of the objects from 2D to 3D using a machine learning based model and transforming a perspective of the background from 2D to 3D using a computer graphics technique.

In another feature, the method further comprises combining the transformed perspectives of the objects and the transformed perspective of the background to generate a 3D scene representing the first data.

In another feature, the method further comprises training the model based on the second set of sensors sensing the 3D scene representing the first data.

In other features, the method further comprises generating 2D representations of the 3D perspectives of the objects sensed by the second set of sensors and generating a 2D representation of the 3D perspective of the background sensed by the second set of sensors.

In another feature, the method further comprises combining the 2D representations of the 3D perspectives of the objects and the 2D representation of the 3D perspective of the background sensed by the second set of sensors.

In another feature, the method further comprises training the model based on the combination of the 2D representations of the 3D perspectives of the objects and the 2D representation of the 3D perspective of the background sensed by the second set of sensors.

Further areas of applicability of the present disclosure will become apparent from the detailed description, the claims and the drawings. The detailed description and specific examples are intended for purposes of illustration only and are not intended to limit the scope of the disclosure.

