For hybrid models, our preliminary experiments show that <cite>transformers</cite> with multiple self-attention layers alone is hard to train without time restriction, and it can easily diverge after a few epochs.