In addition to the low-power requirement, KWS models often face accuracy degradation due to the accent and pronunciation of different users in different regions. To recover accuracy, model personalization or customization is a popular technique for applications with data that vary significantly from person to person, such as KWS [1]}, human activity recognition [2]}, and handwriting recognition [3]}, which are demanded for edge AI devices. Model customization can be executed on either the chip or server side. For KWS, on-chip model customization is preferred over the server-side one due to the privacy concern to retrain the entire model or fine-tune a pre-trained model with a small amount of local data. However, edge devices usually use low-precision fixed-point hardware that poses a big problem for high-precision training. This situation is getting worse for fine-tuning a pre-trained model since the errors and gradients are quite small in such a case, and their quantization will lead to zero error and gradient. Thus, fine-tuning will lead to catastrophic failure.
