Open-source FPGA-ML codesign for the MLPerf Tiny Benchmark
We present our development experience and recent results for the MLPerf Tiny
Inference Benchmark on field-programmable gate array (FPGA) platforms. We use
the open-source hls4ml and FINN workflows, which aim to democratize AI-hardware
codesign of optimized neural networks on FPGAs. We present the design and
implementation process for the keyword spotting, anomaly detection, and image
classification benchmark tasks. The resulting hardware implementations are
quantized, configurable, spatial dataflow architectures tailored for speed and
efficiency and introduce new generic optimizations and common workflows
developed as a part of this work. The full workflow is presented from
quantization-aware training to FPGA implementation. The solutions are deployed
on system-on-chip (Pynq-Z2) and pure FPGA (Arty A7-100T) platforms. The
resulting submissions achieve latencies as low as 20 $\mu$s and energy
consumption as low as 30 $\mu$J per inference. We demonstrate how emerging ML
benchmarks on heterogeneous hardware platforms can catalyze collaboration and
the development of new techniques and more accessible tools.