Papers Explained 37: FastBERT
FastBERT is a novel speed-tunable language transformer with adaptive inference time. FastBERT consists of backbone and branches. The backbone is built upon 12-layers Transformer encoder with an additional teacher-classifier, while the branches include student-classifiers which are appended to each Transformer output to enable early outputs. The backbone consists of three…
Papers Explained 37: FastBERT FastBERT is a novel speed-tunable language transformer with adaptive inference time. FastBERT consists of backbone and branches. The backbone is built upon 12-layers Transformer encoder with an additional teacher-classifier, while the branches include student-classifiers which are appended to each Transformer output to enable early outputs. The backbone consists of three… deep-learning
Papers Explained 37 FastBERT FastBERT novel speedtunable language transformer adaptive inference time FastBERT consists backbone branch backbone built upon 12layers Transformer encoder additional teacherclassifier branch include studentclassifiers appended Transformer output enable early output backbone consists three… deeplearning