A Multimodal Anomaly Detector for Robot Assisted Feeding Using an LSTM Based Variational Autoencoder
The detection of anomalous executions is valuable for reducing potential hazards in assistive manipulation. Multimodal sensory signals can be helpful for detecting a wide range of anomalies. However, the fusion of high dimensional and heterogeneous modalities is a challenging problem for model based anomaly detection. We introduce a long short term memory based variational autoencoder (LSTM VAE) that fuses signals and reconstructs their expected distribution by introducing a progress based varying prior. Our LSTM VAE based detector reports an anomaly when a reconstruction based anomaly score is higher than a state based threshold. For evaluations with 1555 robot assisted feeding executions, including 12 representative types of anomalies, our detector had a higher area under the receiver operating characteristic curve of 0.8710 than 5 other baseline detectors from the literature. We also show the variational autoencoding and state based thresholding are effective in detecting anomalies from 17 raw sensory signals without significant feature engineering effort.
IEEE Robotics and Automation Letters