TECHNICAL FIELD

This disclosure relates to the field of machine learning.

BACKGROUND

Humans are capable of performing complex physical tasks. Robots must be trained to perform these types of tasks. Reinforcement techniques that simulate performance of tasks by robots have been used to train robots. However, multiple types of errors (e.g., modelling errors) cause the conditions in a simulation to differ from real-world conditions. This can cause models that are successful in performing tasks in a simulation to fail when they are used to control real-world robotic systems, in what is commonly referred to as the sim-to-real gap.

SUMMARY

One aspect of the disclosure is a method that includes determining motion imitation information for causing a system to imitate a physical task using a first machine learning model that is trained using motion information that represents a performance of the physical task, determining a predicted correction based on the motion information and a current state from the system using a second machine learning model that is trained using the motion information; determining an action to be performed by the system based on the motion imitation information and the predicted correction; and controlling motion of the system in accordance with the action.

In some implementations, the motion information is based on a video that shows a demonstration of the physical task by an actor. The motion information may describe positions of parts of the actor at each of multiple time steps.

In some implementations, determining the predicted correction based on the motion information and the current state from the system comprises determining a motion representation code that is provided to the second machine learning model as an input. The motion representation code may be a fixed-length value that represents motions that are present in a portion of the motion information.

In some implementations, determining the predicted correction based on the motion information and the current state from the system comprises determining a tracking error that represents a difference between the current state and the motion information, wherein the tracking error is provided to the second machine learning model as an input.

In some implementations, the system is a simulated system and an error value for the simulated system is used as a supervision signal for reinforcement learning. In some implementations, the system is a robotic system.

Another aspect of the disclosure is a method that includes determining motion information that represents performance of a physical task, training a first machine learning model to determine motion imitation information for causing a system to imitate the physical task using the motion information, training a second machine learning model to determine a motion representation code that represents the motion information, training a third machine learning model according to a reinforcement learning technique based on a supervision signal to determine a predicted correction using the motion representation code and a current state of a simulated system, determining an action to be performed by the simulated system based on the motion imitation information and the predicted correction, and simulating performance of the action in the simulated system, wherein the supervision signal is determined in accordance with performance of the action in the simulated system.

The motion information may be based on a video that shows a demonstration of the physical task by an actor, the motion information may describe positions of parts of the actor at each of multiple time steps, and the parts of the actor may be joints. The motion representation code may be a fixed-length value that represents motions that are present in a portion of the motion information. The supervision signal may represent an error of the performance of the action in the simulated system relative to the performance of the physical task.

Another aspect of the disclosure is a method that includes determining a first distribution for a motion dynamics parameter according to a first uncertainty value, and performing iterations of a first training operation for a machine learning model using a simulator and the first distribution for the motion dynamics parameter, wherein each iteration of the first training operation includes determining a first sampled value from the first distribution and controlling a simulation using the first sampled value. The method also includes determining a second distribution for the motion dynamics parameter according to a second uncertainty value, wherein the second uncertainty value represents a higher level of uncertainty than the first uncertainty value, and performing iterations of a second training operation for the machine learning model using a simulator and the first distribution for the motion dynamics parameter, wherein each iteration of the second training operation includes determining a second sampled value from the second distribution and controlling the simulation using the second sampled value.

In some implementations of the method, each iteration of the first training operation comprises causing a simulated actor to perform a task in the simulation according to the first sampled value from the first distribution and each iteration of the second training operation comprises causing the simulated actor to perform the task in the simulator according to the second sampled value.

In some implementations of the method, each iteration of the first training operation comprises determining a first training signal based on the simulation and modifying the machine learning model according to the first training signal, and each iteration of the second training operation comprises determining a second training signal based on the simulation and modifying the machine learning model according to the second training signal.

Some implementations of the method also include initially training the machine learning model using a demonstration of a physical task. The demonstration of the physical task may include a video that depicts a human actor performing the physical task.

Some implementations of the method may also include controlling a robotic system using the machine learning model.

