Presenting a different use for the previous work done by the field of codebooks compression based source coding, known as product quantization (PQ) [21] and additive quantization [22] , they show that by minimizing the squared distance between both distributions (baseline and composed embeddings), and using a direct learning approach for the codes in an end-to-end neural network, with a Gumbel-softmax layer [23] to encourage the discreteness <cite>[15]</cite> , it is possible to construct the word embeddings radically reducing the number of parameters without hurting performance. These word codebooks are learned using a local WTA rule in the hidden layer of an autoencoder neural network.