BACKGROUND

A hardware-based machine learning (ML) system typically includes multiple cores/subsystems (blocks and tiles), each having its own processing units and on-chip memory (OCM). The ML system needs to transmit and route data packets from a memory unit (a source), e.g., a DDR memory, to the processing tiles (destinations) so that the data can be processed by the processing units for various ML operations. Currently, most data routing systems optimize data packets routing based on one source—one destination pair, which is not suited to handle one source many destination (multicast) and/or one source—all destination (broadcast) problem encountered in the ML system wherein data packets often need to be transferred from the same/single memory unit to multiple or all OCMs of the processing tiles on a chip.

In some data routing systems, every router utilizes one or more first-in-first-out (FIFO) queues to preserve source/destination ordering of routed data packets. If a data packet is to be routed to multiple destinations, the data packet is replicated for each of the destinations and routed to these destinations in order. Such data packet replication, however, may result in unnecessary bandwidth usage throughout the system especially when the number of destinations is large.

