Our attention mechanism of choice was Criss-cross Attention Module (CCAM) [1]}, specifically because of its fast, computationally efficient (\(N\sqrt{N}\) ), and low GPU memory usage. These characteristics are crucial in order to introduce as little time-processing overhead as possible into the model and preserve real-time performance.
<FIGURE>