Since PESQ and STOI are calculated using discontinuous functions, they are unsuited to be used as loss functions directly. Several approaches have been proposed to bridge this gap. One popular method is to use a GAN [1]}. The idea is that the discriminator is able to learn features of realistic speech and transfer this knowledge to the generator, which serves as the speech enhancement network. Another procedure is to augment the MSE loss function with symmetric and asymmetric loudness disturbance terms as a way to approximate the PESQ function [2]}. In [3]} the authors train a neural network to estimate PESQ values. Its weights are then frozen and used in the second training stage to optimise for PESQ indirectly.
