On the Role of Spatial, Spectral, and Temporal Processing for DNN-based Non-linear Multi-channel Speech Enhancement
Employing deep neural networks (DNNs) to directly learn filters for
multi-channel speech enhancement has potentially two key advantages over a
traditional approach combining a linear spatial filter with an independent
tempo-spectral post-filter: 1) non-linear spatial filtering allows to overcome
potential restrictions originating from a linear processing model and 2) joint
processing of spatial and tempo-spectral information allows to exploit
interdependencies between different sources of information. A variety of
DNN-based non-linear filters have been proposed recently, for which good
enhancement performance is reported. However, little is known about the
internal mechanisms which turns network architecture design into a game of
chance. Therefore, in this paper, we perform experiments to better understand
the internal processing of spatial, spectral and temporal information by
DNN-based non-linear filters. On the one hand, our experiments in a difficult
speech extraction scenario confirm the importance of non-linear spatial
filtering, which outperforms an oracle linear spatial filter by 0.24 POLQA
score. On the other hand, we demonstrate that joint processing results in a
large performance gap of 0.4 POLQA score between network architectures
exploiting spectral versus temporal information besides spatial information.