Reinforcement Learning Approach for Mapping Applications to Dataflow-Based Coarse-Grained Reconfigurable Array
The Streaming Engine (SE) is a Coarse-Grained Reconfigurable Array which
provides programming flexibility and high-performance with energy efficiency.
An application program to be executed on the SE is represented as a combination
of Synchronous Data Flow (SDF) graphs, where every instruction is represented
as a node. Each node needs to be mapped to the right slot and array in the SE
to ensure the correct execution of the program. This creates an optimization
problem with a vast and sparse search space for which finding a mapping
manually is impractical because it requires expertise and knowledge of the SE
micro-architecture. In this work we propose a Reinforcement Learning framework
with Global Graph Attention (GGA) module and output masking of invalid
placements to find and optimize instruction schedules. We use Proximal Policy
Optimization in order to train a model which places operations into the SE
tiles based on a reward function that models the SE device and its constraints.
The GGA module consists of a graph neural network and an attention module. The
graph neural network creates embeddings of the SDFs and the attention block is
used to model sequential operation placement. We show results on how certain
workloads are mapped to the SE and the factors affecting mapping quality. We
find that the addition of GGA, on average, finds 10% better instruction
schedules in terms of total clock cycles taken and masking improves reward
obtained by 20%.