In this paper, we investigate the acceleration of BERT on resource-constraint FPGAs through algorithm/hardware co-design. Specifically, to reduce memory footprint, we adopt quantization to compress the original model. Different from Q8BERT[1]} and Q-BERT[2]}, which only quantize part of parameters, we propose a fully quantized BERT (FQ-BERT). FQ-BERT represents weights, activations, scale factors, softmax, layer normalization, and all the other intermediate results with integer or fixed-point format. Then we present an accelerator tailored for the proposed FQ-BERT and evaluate on two separate FPGAs. Experiments demonstrate that the proposed accelerator outperforms CPU and GPU by a large margin in terms of performance-per-watt with less than 1% accuracy degradation on SST-2 dataset. The main contributions of this paper can be summarized as follows:
