page_title: Latent Dirichlet allocation
section_title: Model
hierarchical_section_title: Latent Dirichlet allocation / Model
context_page_description: In natural language processing, the latent Dirichlet allocation is a generative statistical model that allows sets of observations to be explained by unobserved groups that explain why some parts of the data are similar. For example, if observations are words collected into documents, it posits that each document is a mixture of a small number of topics and that each word's presence is attributable to one of the document's topics. LDA is an example of a topic model and belongs to the machine learning toolbox and in wider sense to the artificial intelligence toolbox.
context_section_description: With plate notation, which is often used to represent probabilistic graphical models (PGMs), the dependencies among the many variables can be captured concisely. The boxes are "plates" representing replicates, which are repeated entities. The outer plate represents documents, while the inner plate represents the repeated word positions in a given document; each position is associated with a choice of topic and word. The variable names are defined as follows: 
  M denotes the number of documents
  N is number of words in a given document (document i has  words)
  α is the parameter of the Dirichlet prior on the per-document topic distributions
  β is the parameter of the Dirichlet prior on the per-topic word distribution
   is the topic distribution for document i
   is the word distribution for topic k
   is the topic for the j-th word in document i
   is the specific word.
  The fact that W is grayed out means that words  are the only observable variables, and the other variables are latent variables.
  As proposed in the original paper, a sparse Dirichlet prior can be used to model the topic-word distribution, following the intuition that the probability distribution over words in a topic is skewed, so that only a small set of words have high probability. The resulting model is the most widely applied variant of LDA today. The plate notation for this model is shown on the right, where  denotes the number of topics and  are -dimensional vectors storing the parameters of the Dirichlet-distributed topic-word distributions ( is the number of words in the vocabulary).
  It is helpful to think of the entities represented by  and  as matrices created by decomposing the original document-word matrix that represents the corpus of documents being modeled. In this view,  consists of rows defined by documents and columns defined by topics, while  consists of rows defined by topics and columns defined by words. Thus,   refers to a set of rows, or vectors, each of which is a distribution over words, and refers to a set of rows, each of which is a distribution over topics.
caption: Plate notation representing the LDA model.