Latent Dirichlet Allocation (LDA) [1]} is indisputably the most frequently used topic modelling approach and, despite its simplicity, it has established itself as the state of the art Probabilistic Graphical Model in numerous applied research fields.
Most criticism to LDA has been addressing some of its statistical limitations, such as the the lack of a unambiguous criteria for choosing the number of topics, the inability to capture correlations between topics, or its static nature. Many solutions to these shortcomings have been proposed in further developments of the basic probabilistic approach to topic modelling.
Recent research about topic models has focused on extending or modifying LDA to account for syntax [2]}, correlations between topics [3]}, semantic data [4]}, and metadata [5]}, mainly to overcome problems related to the simplifying assumptions of the LDA model.
However, less attention has been devoted to address the limitations of LDA and related models deriving from their “bag-of-words" approach that neglects word order. LDA returns a (weighted) list of words for each topic, disregarding the short-distance semantic information contained in the sequences in which words are arranged.
Even when extended to consider n-grams, that is sequences of n consecutive words, rather than words as basic units of analysis, it returns a weighted list of such entities, without considering their interrelations.
Attempts to include and model syntagmatic information (i.e. information concerning sequential relations between words) in topic models have already been investigated in [6]}, [7]}, [8]}, [9]}, [10]}.
