Since the inception of AlexNet in 2012 [1]}, deep neural networks have been extensively used for visual understanding, such as image classification, object detection, handwriting recognition, visual wake words detection, and semantic segmentation [2]}[3]}. The trend has trickled into the TinyML community as well, evident in Table REF  and Table REF . Image recognition on the CIFAR-10 dataset and person detection on the Visual Wake Words dataset are two inference benchmarks in the MLPerf Tiny v0.5 [4]}. Among the techniques shown in Table REF , NAS on residual convolutional architectures (e.g., MCUNetV2 [5]}, \(\mu \) NAS [6]}, and SpArSe [7]}), rapid downsampling (e.g., RNNPool [8]}), sparse projection (e.g., Bonsai [9]} and ProtoNN [10]}) and deep compression (e.g., Compressed LeNet [11]} and SqueezeNet [12]}) are the most common. Models that operate on multiclass datasets are suitable for Cortex M class architectures, while models that operate on binary datasets have been shown to be deployable on AVR RISC microcontrollers. In Table REF , MCUNetv2 [5]} and AttendNets [14]} achieved the state-of-the-art top 1% accuracy (72-73%) on ImageNet for microcontrollers. MCUNetv2 uses once-for-all NAS on convolutional operators, combined with patch-by-patch inference and receptive field redistribution during runtime [5]}. AttendNets uses a standalone visual attention condenser, which improve spatial-channel selective attention [14]}. MCUNetv2 [5]} (with a YOLOv3 backbone) and AttendSeg [18]} (with attention condensers) achieved state-of-the-art performance for semantic segmentation on the Pascal VOC dataset and CamVid datasets, respectively. \(\mu \) NAS CNN achieved the state-of-the-art performance on CIFAR-10 and MNIST [6]}. Two interesting applications that deviate from traditional machine vision datasets include American sign language prediction [20]} and detecting face masks in light of COVID-19 [21]}.
