Neural language models, originating from the word2vec algorithm (Mikolov et al., 2013a; Mikolov et al., 2013b; Mikolov et al., 2013c) , are currently considered as state-of-the-art solutions for implementing this assumption (Schnabel et al., 2015) . Within this approach, changes in similarity relations between lexical items at two different points of time are interpreted as a signal for meaning shift. Accordingly, lexical items which are very similar to the lexical item under scrutiny can be considered as approximating its meaning at a given point in time. Both techniques were already combined in prior work to show, e.g., the increasing association of the lexical item "gay" with the meaning dimension of "homosexuality" (Kim et al., 2014;<cite> Kulkarni et al., 2015)</cite> . We here investigate the accuracy and reliability of such similarity judgments derived from different training protocols dependent on word frequency, word ambiguity and the number of training epochs (i.e., iterations over all training material).