Hardware/Software Co-Exploration: Much of the development in TinyML has been software-driven, with the hardware platform being static. As a result, while IoT platforms hosting microcontrollers are shrinking due to Moore's Law, the workload and the complexity of neural networks have skyrocketed [1]}[2]}. Proposed hardware innovations include the use of a systolic array, stochastic computing, in-memory computing, near-data processing, spiking neural hardware, and non-von Neumann architectures [1]}[2]}[5]}. However, such architecture innovations are largely disjoint from the TinyML software communities. Developments in TinyML software need to be performed hand-in-hand with attention-directed hardware design, with the platform and model being optimized jointly [6]}[7]}.
