Multi-speaker speech recognition can gain substantial improvement by deploying other sources of information such as speaker identity in addition to acoustic features [1]}, [2]}. However, designing an efficient method to fuse a combination of multiple sources (i.e., acoustic features and speaker embeddings) to obtain higher quality and improved information is still a challenging task. Additionally, capturing complex interactions between multiple sources should maintain a balanced compromise between model/network computational cost and performance. A popular method to address this problem is to use a feature-wise transformation [3]} which can model the complex relation between speaker-specific characteristics and acoustic features in a multi-speaker speech scenario to identify and recognize the desired speaker in the mixed speech recording. This transformation can be performed in several manners such as conditional biasing, conditional scaling, and conditional affine transformation. In this section, we focus on a conditional affine transformation which is a more general approach. The aforementioned conditional biasing and scaling are two specific examples of conditional affine transformation.
