@inproceedings{
zhu2024perfrdiff,
title={Per{FRD}iff: Personalised Weight Editing for Multiple Appropriate Facial Reaction Generation},
author={Hengde Zhu and Xiangyu Kong and Weicheng Xie and Xin Huang and Linlin Shen and Lu Liu and Hatice Gunes and Siyang Song},
booktitle={ACM Multimedia 2024},
year={2024},
url={https://openreview.net/forum?id=KQVjmulG2I}
}
Human facial reactions play crucial roles in dyadic human-human interactions, where individuals (i.e., listeners) with varying cognitive process styles may display different but appropriate facial reactions in response to an identical behaviour expressed by their conversational partners. While several existing facial reaction generation approaches are capable of generating multiple appropriate facial reactions (AFRs) in response to each given human behaviour, they fail to take human's personalised cognitive process in AFRs' generation. In this paper, we propose the first online personalised multiple appropriate facial reaction generation (MAFRG) approach which learns a unique personalised cognitive style from the target human listener's previous facial behaviours and represents it as a set of network weight shifts. These personalised weight shifts are then applied to edit the weights of a pre-trained generic MAFRG model, allowing the obtained personalised model to naturally mimic the target human listener's cognitive process in its reasoning for multiple AFRs generations. Experimental results show that our approach not only largely outperformed all existing approaches in generating more appropriate and diverse generic AFRs, but also serves as the first reliable personalised MAFRG solution. Our code is provided in the Supplementary Material.
PerFRDiff: Personalised Weight Editing for Multiple Appropriate Facial Reaction Generation
Hengde Zhu Xiangyu Kong Weicheng Xie Xin Huang Linlin Shen Lu Liu Hatice Gunes Siyang Song