Traditionally, the probability of a sentence is derived as the product of conditional probabilities by using a word-level language model (LM), which can be trained much easier than sentence-level probabilistic models.
For a given sentence \(s\) , a word-level LM can compute the probability \(p(s)=p(w_1, w_2, ..., w_n)=p(w_1)\prod _{i=2}^{n}{p(w_i|w_1, w_2, ..., w_{i-1})}\) , where \(s\)  consists of word sequence \(w_1, w_2, ..., w_n\) .
A deep learning approach that trains neural LMs by minimizing a word-level cross-entropy (CE) loss between the model predictions and target words has made a remarkable progress in learning word-level LMs mainly due to its decent model capacity and computational efficiency [1]}.
