Temporal aggregator:  \(f_\text{agg}\)  uses a transformer model with 3 multi-headed self-attention blocks and a 1-layer MLP. Group normalization and skip-connections are applied for each block, and we set the hidden dimension to be 128. To compute the time-encoding, we use positional-encoding [1]} with 6 exponentially increasing frequencies.
