New Video Manipulation System Can Realistically Reproduce Anyone's Facial Expressions
A new paper reveals new technology for re-animating video portraits using only video input. Studies have concluded the results are "hard to detect."
By Loukia Papadopoulos
Researchers from the Max Planck Institute for Informatics, Germany, Technicolor, France, Technical University of Munich, Germany, University of Bath, United Kingdom and Stanford University, United States, have combined forces for a rather alarming new project. The work, published in a study last month, reveals a new method for using deep-learning in video creation that results in extremely realistic recreations of people's complete facial expressions and movements.
"We present a novel approach that enables photo-realistic re-animation of portrait videos using only an input video. In contrast to existing approaches that are restricted to manipulations of facial expressions only, we are the first to transfer the full 3D head position, head rotation, face expression, eye gaze, and eye blinking from a source actor to a portrait video of a target actor," states the paper.
Deep Video Portraits
The paper is entitled Deep Video Portraits and its results are set to be demonstrated this August at SIGGRAPH, the ACM special interest group on Computer Graphics and Interactive Techniques. A video of the technology put to use illustrates how the researchers employ a video of a chosen subject, in this case, former President Barack Obama, to extrapolate and analyze how his/her face moves naturally.
The landmarks that constitute authentic facial expressions are then tracked in order to be recreated using the researchers' own expressions as sources of visual input information. The result is an eerily similar reproduction of nearly every discreet nuance of human expression belonging to the target source.
To ensure their technology was as accurate as they suspected, the team ran their simulations through various tests. "To demonstrate the high quality of our output, we conduct an extensive series of experiments and evaluations, where for instance a user study shows that our video edits are hard to detect," states the paper.
Ethical implications abound
The project, of course, comes with many ethical implications and potential dangers. In fact, the issues surrounding the use of recreation AIs for criminal and nefarious purposes has been a concern for a while now.
The US military's research arm, the Defense Advanced Research Projects Agency (DARPA), has recently funded the media forensics (MediFor) platform in order to "attempt to level the digital imagery playing field, which currently favors the manipulator." The project seeks to develop technologies for the "automated assessment of the integrity of an image or video" in order to determine the validity of questionable sources and avoid their use in propaganda and misinformation campaign.
This Robot Can Read and Interpret Human Facial Expressions
The paper does not currently address any ethical concerns surrounding the field. However, its authors do explain that their work is a door to increasing the technology's capabilities.
"We see our approach as a step towards highly realistic synthesis of full-frame
video content under control of meaningful parameters. We hope that it will inspire future research in this very challenging field," concludes the paper.
Via: Arxiv.org
Stay on top of the latest engineering news
Just enter your email and we'll take care of the rest:
By subscribing, you agree to our Terms of Use and Privacy Policy. You may unsubscribe at any time.
A New Humanoid Robot Has the Most Advanced and Realistic Facial Expressions Yet
Ameya Paleja
Scientists Successfully Built Tiny Robots That Can Reproduce on Their Own
A New Drone System Can Clean Hard-to-Reach Solar Panels
Loukia Papadopoulos
A New DIY Laser-Guided Drone System Can Drop Metal Darts for $1,000
Derya Ozdemir
© Copyright 2021 | Interesting Engineering, Inc. | All Rights Reserved
We use cookies to ensure you get the best experience on our website. Please review our Privacy Policy for more information and/or to change your cookie settings at any time. If you continue to use this site, you consent to our use of cookies.
Stay on top of