To deal with the time-varying behavior of request patterns, Recurrent Neural Networks (RNNs), such as Long Short Term Memory (LSTM) [1]}, [2]}, are introduced to use historical information of the content. To extract both spatial and temporal features of CP data, Ale et al. [3]} used a combination of LSTM and CNN models. LSTM-based caching frameworks, however, suffer from long-term dependencies, computation complexity, and unsuitability for parallel computing. To address challenges associated with RNN architectures, the Transformer neural network [4]} has been designed to handle sequential input data, which is purely reliant on attention mechanisms with no recurrence or convolutions. One of the most important advantages of Transformers over RNN models is the attention mechanism, which eliminates the need to analyze data in the same order. Consequently, Transformers have higher parallelization capabilities than RNNs, implying reduced training time.
