Some recent work has focused on improving upon the original PEs proposed by [1]}. Transformers use PEs to embed sequence/time information in the model input so that Transformers can process sequential data all at once, rather than one at a time like an RNN, LSTM, or a GRU. Embedding time (seconds, minutes, hours, weeks, years, etc.) into the input allows the model to better analyze time-series data and leverage computational benefits offered by modern hardware including, GPUs, TPUs, and others. Recently, timestamp embedding [2]}, temporal encoding [3]} and other methods have been proposed for creating Transformers which can be trained more efficiently [4]}.
