Word2vec was derived based on the distributional hypothesis of Harris [1]}, which states that words in similar contexts have similar meanings. Therefore, assuming the existence of the true context distribution for given words \(P^\ast (\cdot |w)\) , it is reasonable to choose the dimensionality that has the ability to learn the context distribution most similar to the true distribution. The MDL principle [2]} is a powerful solution for model selection, and is considered for the dimensionality selection as per our interest.
