After normalizing a word embedding matrix, aren't the values that the neural network actually processes biased towards the most common words? Doesn't it hurt the effect of normalization?
valid question