Emotional artificial intelligence (emotion AI) is using machine learning methods to enable computers to understand human emotions. It plays a vital role in human-computer interaction since emotions are on all the time, presented in all kinds of human activities, thinking, and decision makings. According to psychological studies, body language is an essential part for understanding human emotions. Every day, we respond to thousands of such nonverbal behaviors including facial expressions, eye movements or gaze, tone of voices, gestures, touches, and the use of space. Body language-based emotion understanding has attracted extensive attention in the communities of computer vision and affective computing, and a considerable number of datasets have been proposed, e.g., posed facial expressions [1]}, [2]}, [3]}, [4]}, [5]}, [6]}, spontaneous facial behaviors [7]}, [8]}, [9]}, [10]}, [11]}, [12]}, micro-expressions [13]}, [14]}, [15]}, voice/speech [16]}, [17]}, [10]}, [19]}, social signals [20]}, [21]}, and multi-modal datasets with facial expressions and physiological signals [22]}, [23]}, [10]}, [19]}. Although computational methodologies were proposed correspondingly and consecutively to improve the performance on these datasets, there are still significant gaps between current studies and the needs of real applications. Major limitations include:
<FIGURE>