A Survey on the Affordances of “Hearables” <s> Additional Bio-Signals <s> To realize ubiquitous eating habits monitoring, we proposed the use of chewing sound sensed by an in-ear placed wireless wearable microphone. A prototype of wireless wearable in-ear microphone was developed by utilizing a common Bluetooth headset. We proposed robust chewing number counting algorithm which consists of two recognition stages: ldquochew-likerdquo signal detection and chewing verification stage. As a result, average chewing number counting error rate of 1.93% was achieved. Lastly, chewing sound mapping was proposed to provide an additional intuitive feedback for users to be able to infer the eating habits in their daily life context. <s> BIB001 </s> A Survey on the Affordances of “Hearables” <s> Additional Bio-Signals <s> Emotions are often felt in the body, and somatosensory feedback has been proposed to trigger conscious emotional experiences. Here we reveal maps of bodily sensations associated with different emotions using a unique topographical self-report method. In five experiments, participants (n = 701) were shown two silhouettes of bodies alongside emotional words, stories, movies, or facial expressions. They were asked to color the bodily regions whose activity they felt increasing or decreasing while viewing each stimulus. Different emotions were consistently associated with statistically separable bodily sensation maps across experiments. These maps were concordant across West European and East Asian samples. Statistical classifiers distinguished emotion-specific activation maps accurately, confirming independence of topographies across emotions. We propose that emotions are represented in the somatosensory system as culturally universal categorical somatotopic maps. Perception of these emotion-triggered bodily changes may play a key role in generating consciously felt emotions. <s> BIB002 </s> A Survey on the Affordances of “Hearables” <s> Additional Bio-Signals <s> Functional infrared thermal imaging (fITI) is considered an upcoming, promising methodology in the emotional arena. Driven by sympathetic nerves, observations of affective nature derive from muscular activity subcutaneous blood flow as well as perspiration patterns in specific body parts. A review of 23 experimental procedures that employed fITI for investigations of affective nature is provided, along with the adopted experimental protocol and the thermal changes that took place on selected regions of interest in human and nonhuman subjects. Discussion is provided regarding the selection of an appropriate baseline, the autonomic nature of the thermal print, the experimental setup, methodological issues, limitations, and considerations, as well as future directions. <s> BIB003 </s> A Survey on the Affordances of “Hearables” <s> Additional Bio-Signals <s> Wearable glasses are on the rising edge of development with great user popularity. However, user data stored on these devices bring privacy risks to the owner. To better protect the owner's privacy, a continuous authentication system is needed. In this paper, we propose a continuous and noninvasive authentication system for wearable glasses, named GlassGuard. GlassGuard discriminates the owner and an impostor with behavioral biometrics from six types of touch gestures (single-tap, swipe forward, swipe backward, swipe down, two-finger swipe forward, and two-finger swipe backward) and voice commands, which are all available during normal user interactions. With data collected from 32 users on Google Glass, we show that GlassGuard achieves 99% detection rate and 0.5% false alarm rate after 3.5 user events on average when all types of user events are available with equal probability. Under five typical usage scenarios, the system has a detection rate above 93% and a false alarm rate below 3% after less than five user events. <s> BIB004 </s> A Survey on the Affordances of “Hearables” <s> Additional Bio-Signals <s> Future health systems require the means to assess and track the neural and physiological function of a user over long periods of time, and in the community. Human body responses are manifested through multiple, interacting modalities – the mechanical, electrical and chemical; yet, current physiological monitors (e.g. actigraphy, heart rate) largely lack in cross-modal ability, are inconvenient and/or stigmatizing. We address these challenges through an inconspicuous earpiece, which benefits from the relatively stable position of the ear canal with respect to vital organs. Equipped with miniature multimodal sensors, it robustly measures the brain, cardiac and respiratory functions. Comprehensive experiments validate each modality within the proposed earpiece, while its potential in wearable health monitoring is illustrated through case studies spanning these three functions. We further demonstrate how combining data from multiple sensors within such an integrated wearable device improves both the accuracy of measurements and the ability to deal with artifacts in real-world scenarios. <s> BIB005 </s> A Survey on the Affordances of “Hearables” <s> Additional Bio-Signals <s> We observe the advent of body-area networks of pervasive wearable devices, whether for health monitoring, personal assistance, entertainment, or home automation. For many devices, it is critical to identify the wearer, allowing sensor data to be properly labeled or personalized behavior to be properly achieved. In this paper we propose the use of vocal resonance, that is, the sound of the person's voice as it travels through the person's body -- a method we anticipate would be suitable for devices worn on the head, neck, or chest. In this regard, we go well beyond the simple challenge of speaker recognition: we want to know who is wearing the device. We explore two machine-learning approaches that analyze voice samples from a small throat-mounted microphone and allow the device to determine whether (a) the speaker is indeed the expected person, and (b) the microphone-enabled device is physically on the speaker's body. We collected data from 29 subjects, demonstrate the feasibility of a prototype, and show that our DNN method achieved balanced accuracy 0.914 for identification and 0.961 for verification by using an LSTM-based deep-learning model, while our efficient GMM method achieved balanced accuracy 0.875 for identification and 0.942 for verification. <s> BIB006
Beyond electrical measurements, research indicates that bio-acoustic sound recordings taken from within the ear using tiny microphones might offer unique insights into the user's affective state. Bio-acoustic recordings may also be useful within applications that use audio solely as an input device, such as user authentication, which is especially important for wearable devices that might be shared by different users BIB006 BIB004 . Bio-acoustic recordings have been used to monitor cardiac activity and respiration rate BIB005 . The affordances of monitoring user respiration rate include stress/exertion levels, triggers based on punctuated exhaling (such as laughter or sobbing), the detection of sleep apnea, etc. While reports indicate that respiration rate may be obtained from recordings of the inner ear, this does not mean that the ear is an ideal location for this measurement. As another example of a hearable related bio-acoustic signal, a team of researchers has investigated the use of bio-acoustics to capture "tooth click" gestures [43] . Much like ear-wiggles and eye blink triggers, it is possible to differentiate tooth clicks from different quadrants of the mouth, and therefore employ these clicks for hands-free computer interaction. Further, bio-acoustics could also be used in eating behavior monitoring applications BIB001 . One last type of signal that will be discussed is body temperature. The inner ear is a common source for measuring body temperature, and temperature sensors can be easily embedded within hearables. One particular application of in-ear temperature monitoring includes Yono's device for fertility prediction  . Temperature sensors embedded within hearables could potentially also play a role in understanding bodily maps of emotion, as research suggests that certain types of emotion are associated with distinct temperature change patterns BIB002 BIB003 . Summary: Bio signals afford the ability to infer the affective and cognitive state of the user, thus offering interesting applications within context-and affect-aware computing. Table 1 presents a brief summary of the principle affordances offered by each feature category. We use the term "principle affordance" in a general sense to summarize the primary benefit associated with each feature, as well as to highlight how ear-mounted devices have evolved over time. While these features and their associated primary affordances hold across other wearable devices, the proposed summary is specifically aimed towards understanding the evolution of hearables. Further, Table 2 presents a summarization of the principle affordances offered by each of the biosensors discussed in the section above. 