FIELD OF THE INVENTION

The present disclosure generally relates to tools for generating computer-generated imagery. The disclosure relates more particularly to apparatus and techniques for building a machine learning model that learns muscle-to-skin transformations for creating computer-generated imagery of a character.

BACKGROUND

Many industries generate or use computer-generated imagery, such as images or video sequences. The computer-generated imagery might include computer-animated characters that are based on live actors. For example, a feature film creator might want to generate a computer-animated character having facial actions, movements, behaviors, etc. that is the same or substantially similar to a live actor, human or otherwise. Existing animation systems may recreate, in detail, a skin surface of the computer-animated character that closely resembles a live actor. Simulating the movements and/or facial action of the computer-animated character that may appear to be similar to the live actor remains challenging, as a large number of variables can be involved in the simulation process. For example, there are more than 40 muscles controlled by seven nerves in a human face, and a facial action can be decomposed into different combinations of changes in the movements of the muscles.

Some existing animation systems largely rely on capturing facial scans of a live actor and volume of facial scans to be processed can often be tremendous to experiment to obtain different facial actions, which is often a tedious and sometimes impractical operation.

SUMMARY

In an embodiment, a computer-implemented method or apparatus provides for computer-implemented method for processing data derived from scans of live actors. The method includes obtaining scan data corresponding to a performance of a live actor performing a facial movement over a data bundle time period, deriving a data bundle from the scan data, wherein the data bundle comprises control values for a muscle model of the live actor for application to the muscle model and rendering as an animation, to approximate the performance, the data bundle comprising a first cache of data representing movement of a skin surface over the data bundle time period and a second cache of data representing changes to muscle strains in a muscle strain vector over the data bundle time period. The method further includes obtaining an artist movement adjustment for the skin surface relative to movement corresponding to the data bundle, and generating a modified data bundle having a modified second cache, wherein the modified second cache, when applied to the muscle model, causes the muscle model to move in variance with the data bundle to approximate a modified first cache represented by the first cache adjusted according to the artist movement adjustment.

In one implementation, the performance begins at a neutral pose and ends at the neutral pose.

In one implementation, the data bundle comprises the first cache, the second cache, a third cache of data representing jaw movements over the data bundle time period, and a fourth cache of data representing eye movements over the data bundle time period.

In one implementation, the method further comprises determining an action locus for the artist movement adjustment, determining a range of action for the artist movement adjustment, determining a subset of the muscle strain vector applicable to the range of action, and limiting changes of the modified first cache relative to the first cache to changes in the subset of the muscle strain vector.

In one implementation, the method further comprises limiting a search for values for the second cache to a manifold of allowed values for the second cache of data.

In one implementation, the manifold of allowed values for the second cache of data correspond to known feasible expressions of the live actor.

In one implementation, the facial movement is one or more of a facial action, a dialog, and/or a depiction of an emotion.

