Muscle-based systems have the potential to provide both anatomical accuracy and semantic interpretability as compared to blendshape models; however, a lack of expressivity and differentiability has limited their impact. Thus, we propose modifying a recently developed rather expressive muscle-based system in order to make it fully-differentiable; in fact, our proposed modifications allow this physically robust and anatomically accurate muscle model to conveniently be driven by an underlying blendshape basis. Our formulation is intuitive, natural, as well as monolithically and fully coupled such that one can differentiate the model from end to end, which makes it viable for both optimization and learning-based approaches for a variety of applications. We illustrate this with a number of examples including both shape matching of three-dimensional geometry as as well as the automatic determination of a three-dimensional facial pose from a single two-dimensional RGB image without using markers or depth information.
@cite_0 was one of the first to utilize quasistatic simulations to drive the deformation of a 3D face, especially for motion capture. There has also been interest in using quasistative simulations to drive muscle deformations in the body @cite_22 @cite_8 @cite_28 . However, in general, facial muscle simulations tend to be less expressive than their artist-driven blendshape counterparts. More recently, significant work has been done to make muscle simulations more expressive @cite_35 @cite_2 . While these methods can be used to target data in the form of geometry, it is unclear how to cleanly transfer these methods to target non-geometry data such as two-dimensional RGB images. Other work has been done to try to introduce physical simulations into the blendshape models themselves @cite_39 @cite_12 @cite_47 @cite_4 ; however, these works do not focus on the inverse problem.
@cite_35 @cite_4 @cite_22 @cite_8 @cite_28 @cite_39 @cite_0 @cite_2 @cite_47 @cite_12
2022890264 2114106563 2474891805 2460468435
We built an anatomically accurate model of facial musculature, passive tissue and underlying skelet al structure using volumetric data acquired from a living male subject. The tissues are endowed with a highly nonlinear constitutive model including controllable anisotropic muscle activations based on fiber directions. Detailed models of this sort can be difficult to animate requiring complex coordinated stimulation of the underlying musculature. We propose a solution to this problem automatically determining muscle activations that track a sparse set of surface landmarks, e.g. acquired from motion capture marker data. Since the resulting animation is obtained via a three dimensional nonlinear finite element method, we obtain visually plausible and anatomically correct deformations with spatial and temporal coherence that provides robustness against outliers in the motion capture data. Moreover, the obtained muscle activations can be used in a robust simulation framework including contact and collision of the face with external objects. Simulation of the musculoskelet al system has important applications in biomechanics, biomedical engineering, surgery simulation, and computer graphics. The accuracy of the muscle, bone, and tendon geometry as well as the accuracy of muscle and tendon dynamic deformation are of paramount importance in all these applications. We present a framework for extracting and simulating high resolution musculoskelet al geometry from the segmented visible human data set. We simulate 30 contact collision coupled muscles in the upper limb and describe a computationally tractable implementation using an embedded mesh framework. Muscle geometry is embedded in a nonmanifold, connectivity preserving simulation mesh molded out of a lower resolution BCC lattice containing identical, well-shaped elements, leading to a relaxed time step restriction for stability and, thus, reduced computational cost. The muscles are endowed with a transversely isotropic, quasiincompressible constitutive model that incorporates muscle fiber fields as well as passive and active components. The simulation takes advantage of a new robust finite element technique that handles both degenerate and inverted tetrahedra. We present a new anatomically-constrained local face model and fitting approach for tracking 3D faces from 2D motion data in very high quality. In contrast to traditional global face models, often built from a large set of blendshapes, we propose a local deformation model composed of many small subspaces spatially distributed over the face. Our local model offers far more flexibility and expressiveness than global blendshape models, even with a much smaller model size. This flexibility would typically come at the cost of reduced robustness, in particular during the under-constrained task of monocular reconstruction. However, a key contribution of this work is that we consider the face anatomy and introduce subspace skin thickness constraints into our model, which constrain the face to only valid expressions and helps counteract depth ambiguities in monocular tracking. Given our new model, we present a novel fitting optimization that allows 3D facial performance reconstruction from a single view at extremely high quality, far beyond previous fitting approaches. Our model is flexible, and can be applied also when only sparse motion data is available, for example with marker-based motion capture or even face posing from artistic sketches. Furthermore, by incorporating anatomical constraints we can automatically estimate the rigid motion of the skull, obtaining a rigid stabilization of the performance for free. We demonstrate our model and single-view fitting method on a number of examples, including, for the first time, extreme local skin deformation caused by external forces such as wind, captured from a single high-speed camera. We propose a new framework for the simulation of facial muscle and flesh that so significantly improves the technique that it allows for immediate mainstream use of anatomically and biomechanically accurate muscle models as a bread and butter technique in a high-end production quality pipeline. The key idea is to create a blendshape system for the muscles that gives the precise directability and controllability required in a high-end production environment. The blendshape muscles are used to drive the underlying anatomically and biomechanically motivated simulation in a way that is unbound by the typical restrictions of a simulation system while still retaining the desirable degree of freedom richness that leads to high quality results. We show that we are able to target production quality facial shapes, whether from scans or an animation system, and illustrate that the resulting nonlinear simulation in-betweens are of higher quality than those obtained from traditional linear blendshapes. We also demonstrate the ability to selectively improve areas on a given blendshape using the results of a simulation, as well as the ability to edit muscle shapes and paths in order to produce directability for animator control. Then, we show how these techniques can be used to transition from one blendshape to another or even track and selectively modify an entire performance. The efficacy of our system is further demonstrated by using it to retarget animation onto new creature models given only a single static rest pose as input.