@inproceedings{
wang2024dancecamanimator,
title={DanceCamAnimator: Keyframe-Based Controllable 3D Dance Camera Synthesis},
author={Zixuan Wang and Jiayi Li and Xiaoyu Qin and Shikun Sun and Songtao Zhou and Jia Jia and Jiebo Luo},
booktitle={ACM Multimedia 2024},
year={2024},
url={https://openreview.net/forum?id=8xzXrQCMD7}
}
Synthesizing camera movement from music and dance is highly challenging due to the contradictions and complexities of dance cinematography. Unlike human movement, which is always continuous,  dance camera movements involve continuous sequences of varying lengths and sudden drastic changes to simulate the switching of multiple cameras. However, in previous works, every camera frame is equally treated which results in jittering and unavoidable smoothing post-processing. To solve this problem, in this paper, we propose to formalize animator dance cinematography knowledge by formulating this problem as a three-stage process: keyframe detection, keyframe synthesis, and tween curve prediction. Following this formulation, we design a novel end-to-end dance camera synthesis framework \textbf{DanceCamAnimator}, which first imitates human animation procedure and shows powerful keyframe-based controllability with variable length. Extensive experiments on the DCM dataset demonstrate that our method surpasses previous works quantitatively and qualitatively. We will make our code publicly available to promote future research.
DanceCamAnimator: Keyframe-Based Controllable 3D Dance Camera Synthesis
Zixuan Wang Jiayi Li Xiaoyu Qin Shikun Sun Songtao Zhou Jia Jia Jiebo Luo