Muscle-based systems have the potential to provide both anatomical accuracy and semantic interpretability as compared to blendshape models; however, a lack of expressivity and differentiability has limited their impact. Thus, we propose modifying a recently developed rather expressive muscle-based system in order to make it fully-differentiable; in fact, our proposed modifications allow this physically robust and anatomically accurate muscle model to conveniently be driven by an underlying blendshape basis. Our formulation is intuitive, natural, as well as monolithically and fully coupled such that one can differentiate the model from end to end, which makes it viable for both optimization and learning-based approaches for a variety of applications. We illustrate this with a number of examples including both shape matching of three-dimensional geometry as as well as the automatic determination of a three-dimensional facial pose from a single two-dimensional RGB image without using markers or depth information.
@cite_0 was one of the first to utilize quasistatic simulations to drive the deformation of a 3D face, especially for motion capture. There has also been interest in using quasistative simulations to drive muscle deformations in the body @cite_22 @cite_8 @cite_28 . However, in general, facial muscle simulations tend to be less expressive than their artist-driven blendshape counterparts. More recently, significant work has been done to make muscle simulations more expressive @cite_35 @cite_2 . While these methods can be used to target data in the form of geometry, it is unclear how to cleanly transfer these methods to target non-geometry data such as two-dimensional RGB images. Other work has been done to try to introduce physical simulations into the blendshape models themselves @cite_39 @cite_12 @cite_47 @cite_4 ; however, these works do not focus on the inverse problem.
@cite_35 @cite_4 @cite_22 @cite_8 @cite_28 @cite_39 @cite_0 @cite_2 @cite_47 @cite_12
2460468435 2617644139 1980757084 2143345877 2114106563 2807843367 2022890264 2401751489
We propose a new framework for the simulation of facial muscle and flesh that so significantly improves the technique that it allows for immediate mainstream use of anatomically and biomechanically accurate muscle models as a bread and butter technique in a high-end production quality pipeline. The key idea is to create a blendshape system for the muscles that gives the precise directability and controllability required in a high-end production environment. The blendshape muscles are used to drive the underlying anatomically and biomechanically motivated simulation in a way that is unbound by the typical restrictions of a simulation system while still retaining the desirable degree of freedom richness that leads to high quality results. We show that we are able to target production quality facial shapes, whether from scans or an animation system, and illustrate that the resulting nonlinear simulation in-betweens are of higher quality than those obtained from traditional linear blendshapes. We also demonstrate the ability to selectively improve areas on a given blendshape using the results of a simulation, as well as the ability to edit muscle shapes and paths in order to produce directability for animator control. Then, we show how these techniques can be used to transition from one blendshape to another or even track and selectively modify an entire performance. The efficacy of our system is further demonstrated by using it to retarget animation onto new creature models given only a single static rest pose as input. Oftentimes facial animation is created separately from overall body motion. Since convincing facial animation is challenging enough in itself, artists tend to create and edit the face motion in isolation. Or if the face animation is derived from motion capture, this is typically performed in a mo-cap booth while sitting relatively still. In either case, recombining the isolated face animation with body and head motion is non-trivial and often results in an uncanny result if the body dynamics are not properly reflected on the face e.g. the bouncing of facial tissue when running. We present an algorithm for the finite element simulation of elastoplastic solids which is capable of robustly and efficiently handling arbitrarily large deformation. In fact, our model remains valid even when large parts of the mesh are inverted. The algorithm is straightforward to implement and can be used with any material constitutive model, and for both volumetric solids and thin shells such as cloth. We also provide a mechanism for controlling plastic deformation, which allows a deformable object to be guided towards a desired final shape without sacrificing realistic behavior. Finally, we present an improved method for rigid body collision handling in the context of mixed explicit implicit time-stepping. Since it relies on a geometrical rather than a variational framework, many find the finite volume method (FVM) more intuitive than the finite element method (FEM). We show that the FVM allows one to interpret the stress inside a tetrahedron as a simple "multidimensional force" pushing on each face. Moreover, this interpretation leads to a heuristic method for calculating the force on each node, which is as simple to implement and comprehend as masses and springs. In the finite volume spirit, we also present a geometric rather than interpolating function definition of strain. We use the FVM and a quasi-incompressible, transversely isotropic, hyperelastic constitutive model to simulate contracting muscle tissue. B-spline solids are used to model fiber directions, and the muscle activation levels are derived from key frame animations. Simulation of the musculoskelet al system has important applications in biomechanics, biomedical engineering, surgery simulation, and computer graphics. The accuracy of the muscle, bone, and tendon geometry as well as the accuracy of muscle and tendon dynamic deformation are of paramount importance in all these applications. We present a framework for extracting and simulating high resolution musculoskelet al geometry from the segmented visible human data set. We simulate 30 contact collision coupled muscles in the upper limb and describe a computationally tractable implementation using an embedded mesh framework. Muscle geometry is embedded in a nonmanifold, connectivity preserving simulation mesh molded out of a lower resolution BCC lattice containing identical, well-shaped elements, leading to a relaxed time step restriction for stability and, thus, reduced computational cost. The muscles are endowed with a transversely isotropic, quasiincompressible constitutive model that incorporates muscle fiber fields as well as passive and active components. The simulation takes advantage of a new robust finite element technique that handles both degenerate and inverted tetrahedra. We built an anatomically accurate model of facial musculature, passive tissue and underlying skelet al structure using volumetric data acquired from a living male subject. The tissues are endowed with a highly nonlinear constitutive model including controllable anisotropic muscle activations based on fiber directions. Detailed models of this sort can be difficult to animate requiring complex coordinated stimulation of the underlying musculature. We propose a solution to this problem automatically determining muscle activations that track a sparse set of surface landmarks, e.g. acquired from motion capture marker data. Since the resulting animation is obtained via a three dimensional nonlinear finite element method, we obtain visually plausible and anatomically correct deformations with spatial and temporal coherence that provides robustness against outliers in the motion capture data. Moreover, the obtained muscle activations can be used in a robust simulation framework including contact and collision of the face with external objects. In this paper we present a new paradigm for the generation and retargeting of facial animation. Like a vast majority of the approaches that have adressed these topics, our formalism is built on blendshapes. However, where prior works have generally encoded facial geometry using a low dimensional basis of these blendshapes, we propose to encode facial dynamics by looking at blendshapes as a basis of forces rather than a basis of shapes. We develop this idea into a dynamic model that naturally combines the blendshapes paradigm with physics-based techniques for the simulation of deforming meshes. Because it escapes the linear span of the shape basis through time-integration and physics-inspired simulation, this approach has a wider expressive range than previous blendshape-based methods. Its inherent physically-based formulation also enables the simulation of more advanced physical interactions, such as collision responses on lip contacts.