Multi-talker speech recognition is focused on recognizing individual speech sources from overlap speech, and is one main challenge for current ASR systems [1]}, [2]}, [3]}, [4]}, [5]}, [6]}, [7]}, [8]}.
Current solutions for multi-speaker speech recognition can be categorized into two main approaches: (i) performing front-end speech processing based on separation on the overlap speech, then applying ASR to the separated speech signals [9]}, [10]}, [11]}, [12]}, [13]}, [14]}, [15]}; or (ii) skipping the explicit separation step and developing a multi-speaker speech recognition system directly using either hybrid [16]}, [17]},  or end-to-end [18]}, [19]} ASR frameworks. Recently, an end-to-end multi-speaker speech recognition system was proposed based on Transformers [1]}. This approach achieved considerable improvement at the expense of more computational cost for a reasonable temporal resolution. In another study [21]}, overlap speech was considered as a mismatch condition of the clean speech recognition scenario, and teacher-student training was employed for transfer learning from clean to overlap speech. The main drawback of this approach is requiring training sets with parallel clean and overlapped speech, which is difficult to collect in real-world applications [22]}.
Recently, several studies [16]}, [24]}, [25]} have used speaker-specific embeddings to learn a frame-level mask for the target speaker which suppresses interfering speech. Although these approaches use the additional speaker-specific information to guide the ASR system, their main limitation is that they assume only one speaker is active in each Time-Frequency bin.
