A framework for the fusion of visual and tactile modalities for improving robot perception
Robots should ideally perceive objects using human like multi modal sensing such as vision, tactile feedback, smell, and hearing. However, the features presentations are different for each modal sensor. Moreover, the extracted feature methods for each modal are not the same. Some modal features such as vision, which presents a spatial property, are static while features such as tactile feedback, which presents temporal pattern, are dynamic. It is difficult to fuse these data at the feature level for robot perception. In this study, we propose a framework for the fusion of visual and tactile modal features, which includes the extraction of features, feature vector normalization and generation based on bag of system (BoS) and coding by robust multi modal joint sparse representation (RM JSR) and classification, thereby enabling robot perception to solve the problem of diverse modal data fusion at the feature level. Finally, comparative experiments are carried out to demonstrate the performance of this framework.Chuang Xin Dian Ti Chu Liao Yi Chong Shi Hong Jue Xin Xi Rong He Kuang Jia He Lu Bang Duo Mo Tai Lian He Xi Shu Biao Shi Bian Ma Fang Fa Jie Jue You Yu Ji Qi Ren Gan Zhi De Shi (Jing Tai Hong Jue (Dong Tai )Kua Mo Tai Xin Xi Te Zheng Kong Jian Wei Du Bu Tong Er Dai Lai De Te Zheng Ceng Rong He Nan Ti Ju Ti Bao Gua :Shi Hong Jue Te Zheng Ti Qu Yong "Ci Dai "Suan Fa Gui Yi Hua Wei Du Bu Tong De Te Zheng Xiang Liang Lu Bang Duo Mo Tai Lian He Xi Shu Biao Shi Bian Ma Tong Guo Shi Hong Jue Rong He Suan Fa Jin Xing Fen Lei
Science China Information Sciences