Latent Topology Induction for Understanding Contextualized Representations
In this work, we study the representation space of contextualized embeddings
and gain insight into the hidden topology of large language models. We show
there exists a network of latent states that summarize linguistic properties of
contextualized representations. Instead of seeking alignments to existing
well-defined annotations, we infer this latent network in a fully unsupervised
way using a structured variational autoencoder. The induced states not only
serve as anchors that mark the topology (neighbors and connectivity) of the
representation manifold but also reveal the internal mechanism of encoding
sentences. With the induced network, we: (1). decompose the representation
space into a spectrum of latent states which encode fine-grained word meanings
with lexical, morphological, syntactic and semantic information; (2). show
state-state transitions encode rich phrase constructions and serve as the
backbones of the latent space. Putting the two together, we show that sentences
are represented as a traversal over the latent network where state-state
transition chains encode syntactic templates and state-word emissions fill in
the content. We demonstrate these insights with extensive experiments and
visualizations.