Neural language models for tracking semantic changes over time typically distinguish between two different training protocols-continuous training of models (Kim et al., 2014) where the model for each time span is initialized with the embeddings of its predecessor, and, alternatively, independent training with a mapping between models for different points in time<cite> (Kulkarni et al., 2015)</cite> . A comparison between these two protocols, such as the one proposed in this paper, has not been carried out before.