Do visual cues intensify the emotional responses evoked by musical performance? A psychophysiological investigation.
Music is capable of conveying emotional meaning and eliciting emotional responses. In musical performance, emotional expression is communicated through auditory cues such as articulation, timbre, vibrato, dynamics, and timing (Gabrielsson, 1999; Juslin Timmers, 2010; Palmer, 1997) as well as by means of visual cues such as body posture, gestures, and facial expressions (Dahl Friberg, 2007; Davidson, 1993, 1994; Thompson, Russo, Quinto, 2008) Research conducted over the past two decades has unequivocally established that visual cues contribute significantly to observers' perceptions of the expressivity (Davidson, 1993, 1994; Vuoskoski, Thompson, Clarke, Spence, 2014) and emotional expression (Dahl Friberg, 2007; Krahe, Hahn, Whitney, 2015; Thompson et al. 2008) of a musical performance, but less is known about the role of visual cues in the emotional responses that are elicited. Although some previous studies have attempted to investigate the effect of visual performance cues on observers' experienced emotions (Krahe et al. 2015; Vines, Krumhansl, Wanderley, Dalca, Levitin, 2011) the challenges involved in distinguishing experienced emotion from perceived emotion using self report (cf. Konecni, 2008) render the implications of the findings unclear.The psychological mechanisms of music induced emotion have chiefly been considered from a unisensory, auditory perspective (Juslin Vastfjall, 2008; Scherer Zentner, 2001) Despite the fact that Juslin and Vastfjall actually proposed visual imagery as one of the potential mechanisms by which music can induce emotional responses in listeners, they discuss this mechanism in terms of nonmusical visual imagery that is conjured up by the music (such as images of nature, for example) rather than performance related imagery. However, some theories of musicinduced emotion directly predict that the observation of a musical performance through multiple sensory modalities auditory and visual ought to lead to an enhanced, or more intense, emotional response (Livingstone Thompson, 2009; Molnar Szakacs Overy, 2006) Molnar Szakacs and Overy proposed that through the coupling of perception and action a kind of "simulation" of the actions and affective cues present in musical performance the mirror neuron system might account for some of the emotional responses that are elicited by music. Mirror neurons first discovered in the macaque monkey fire both when we perform a goal directed action, and when we observe (see or hear) another agent performing the same action (di Pellegrino, Fadiga, Fogassi, Gallese, Rizzolatti, 1992; Gallese, 2006; Kohler et al. 2002) In the context of musical performance, the mirror neuron system may respond to sound producing actions as well as to the emotional expression conveyed by the performance (i.e. resonating with those auditory and gestural features in the music that resemble vocal and motor expressions of emotion; Overy MolnarSzakacs, 2009) Although we are able to hear and mirror these actions and affective cues by means of auditory information alone (Kohler et al. 2002; Molnar Szakacs Overy, 2006) it has been hypothesized that the activation of multiple channels auditory and visual may result in a heightened emotional response due to the broader range of actions and cues available for mirroring (Livingstone Thompson, 2009).This view, proposed by Livingstone and Thompson (2009) subsequently received empirical support: A recent meta analysis revealed that compared with audio only presentations audiovisual presentations consistently enhance observers' appreciation (defined as "liking," perceived "expressiveness," "overall quality," and "overall impression" of a musical performance (Platz Kopiez, 2012) Moreover, Chapados and Levitin (2008) reported that observers' emotional reactivity to clarinet performances measured using psychophysiological indices of emotional arousal was heightened in the audiovisual presentation condition (as compared with the audio only and video only conditions)