TITLE: Multi Visual Modality Fall Detection Dataset 

ABSTRACT: Falls are one of the leading cause of injury-related deaths among the elderly
worldwide. Effective detection of falls can reduce the risk of complications
and injuries. Fall detection can be performed using wearable devices or ambient
sensors; these methods may struggle with user compliance issues or false
alarms. Video cameras provide a passive alternative; however, regular RGB
cameras are impacted by changing lighting conditions and privacy concerns. From
a machine learning perspective, developing an effective fall detection system
is challenging because of the rarity and variability of falls. Many existing
fall detection datasets lack important real-world considerations, such as
varied lighting, continuous activities of daily living (ADLs), and camera
placement. The lack of these considerations makes it difficult to develop
predictive models that can operate effectively in the real world. To address
these limitations, we introduce a novel multi-modality dataset (MUVIM) that
contains four visual modalities: infra-red, depth, RGB and thermal cameras.
These modalities offer benefits such as obfuscated facial features and improved
performance in low-light conditions. We formulated fall detection as an anomaly
detection problem, in which a customized spatio-temporal convolutional
autoencoder was trained only on ADLs so that a fall would increase the
reconstruction error. Our results showed that infra-red cameras provided the
highest level of performance (AUC ROC=0.94), followed by thermal (AUC
ROC=0.87), depth (AUC ROC=0.86) and RGB (AUC ROC=0.83). This research provides
a unique opportunity to analyze the utility of camera modalities in detecting
falls in a home setting while balancing performance, passiveness, and privacy.
Falls are one of the leading cause of injury-related deaths among the elderly
worldwide. Effective detection of falls can reduce the risk of complications
and injuries. Fall detection can be performed using wearable devices or ambient
sensors; these methods may struggle with user compliance issues or false
alarms. Video cameras provide a passive alternative; however, regular RGB
cameras are impacted by changing lighting conditions and privacy concerns. From
a machine learning perspective, developing an effective fall detection system
is challenging because of the rarity and variability of falls. Many existing
fall detection datasets lack important real-world considerations, such as
varied lighting, continuous activities of daily living (ADLs), and camera
placement. The lack of these considerations makes it difficult to develop
predictive models that can operate effectively in the real world. To address
these limitations, we introduce a novel multi-modality dataset (MUVIM) that
contains four visual modalities: infra-red, depth, RGB and thermal cameras.
These modalities offer benefits such as obfuscated facial features and improved
performance in low-light conditions. We formulated fall detection as an anomaly
detection problem, in which a customized spatio-temporal convolutional
autoencoder was trained only on ADLs so that a fall would increase the
reconstruction error. Our results showed that infra-red cameras provided the
highest level of performance (AUC ROC=0.94), followed by thermal (AUC
ROC=0.87), depth (AUC ROC=0.86) and RGB (AUC ROC=0.83). This research provides
a unique opportunity to analyze the utility of camera modalities in detecting
falls in a home setting while balancing performance, passiveness, and privacy.
Multi Visual Modality Fall Detection Dataset
http://arxiv.org/abs/2206.12740v1