How Does Language Evolve? - Babbel
Languages change when speakers come into contact with new populations (think colonialism and trade), and they change when social groups adopt ...
published a study
Of course, Plotkin wasn’t the first researcher to tackle the mechanisms of linguistic drift. Researchers William Hamilton, Jure Leskovec and Dan Jurafsky <anchor>published a study</anchor> that examined some of the major tendencies of semantic change. They pointed out that nouns are more likely to change due to irregular cultural shifts (like new technology) while verbs more often change due to regular semantic drift, largely because they are more semantically mutable than nouns.
Cultural Shift or Linguistic Drift? Comparing Two Computational Measures of Semantic Change - PMC
U.S. flag

An official website of the United States government

Here's how you know

Dot gov

The .gov means it’s official.

Federal government websites often end in .gov or .mil. Before
sharing sensitive information, make sure you’re on a federal
government site.

Https

The site is secure.

The https:// ensures that you are connecting to the
official website and that any information you provide is encrypted
and transmitted securely.

Close

#### Account

Logged in as:

username

Search PMC Full-Text ArchiveSearch in PMC

###### Other Formats

###### Actions

- Cite
- Collections

Add to Collections

- Create a new collection
- Add to an existing collection

Name your collection:

Name must be less than characters

Choose a collection:

Unable to load your collection due to an error

Please try again

Add

Cancel

###### Share

- Permalink

Copy

###### RESOURCES

As a library, NLM provides access to scientific literature. Inclusion in an NLM database does not imply endorsement of, or agreement with,
the contents by NLM or the National Institutes of Health.

Logo of nihpa

Proc Conf Empir Methods Nat Lang Process. Author manuscript; available in PMC 2017 Jun 1.

Published in final edited form as:

Proc Conf Empir Methods Nat Lang Process. 2016 Nov; 2016: 2116–2121.

PMCID: PMC5452980

NIHMSID: NIHMS859814

# Cultural Shift or Linguistic Drift? Comparing Two Computational Measures of Semantic Change

Author informationCopyright and License information PMC Disclaimer

Department of Computer Science, Stanford University, Stanford CA, 94305

## Abstract

Words shift in meaning for many reasons, including cultural factors like new technologies and regular linguistic processes like subjectification. Understanding the evolution of language and culture requires disentangling these underlying causes. Here we show how two different distributional measures can be used to detect two different types of semantic change. The first measure, which has been used in many previous works, analyzes global shifts in a word’s distributional semantics; it is sensitive to changes due to regular processes of linguistic drift, such as the semantic generalization of promise (“I promise.” “It promised to be exciting.”). The second measure, which we develop here, focuses on local changes to a word’s nearest semantic neighbors; it is more sensitive to cultural shifts, such as the change in the meaning of cell (“prison cell” “cell phone”). Comparing measurements made by these two methods allows researchers to determine whether changes are more cultural or linguistic in nature, a distinction that is essential for work in the digital humanities and historical linguistics.

## 1 Introduction

We show how two computational measures can be used to distinguish between semantic changes caused by cultural shifts (e.g., technological advancements) and those caused by more regular processes of semantic change (e.g., grammaticalization or subjectification). This distinction is essential for research on linguistic and cultural evolution. Detecting cultural shifts in language use is crucial to computational studies of history and other digital humanities projects. By contrast, for advancing historical linguistics, cultural shifts amount to noise and only the more regular shifts matter.

Our work builds on two intuitions: that distributional models can highlight syntagmatic versus paradigmatic relations with neighboring words ( Schutze and Pedersen, 1993) and that nouns are more likely to undergo changes due to irregular cultural shifts while verbs more readily participate in regular processes of semantic change ( Gentner and France, 1988; Traugott and Dasher, 2001). We use this noun vs. verb mapping as a proxy to compare our two measures’ sensitivities to cultural vs. linguistic shifts. Sensitivity to nominal shifts indicates a propensity to capture irregular cultural shifts in language, such as those due to technological advancements ( Traugott and Dasher, 2001). Sensitivity to shifts in verbs (and other predicates) indicates a propensity to capture regular processes of linguistic drift ( Gentner and France, 1988; Kintsch, 2000; Traugott and Dasher, 2001).

The first measure we analyze is based upon changes to a word’s local semantic neighborhood; we show that it is more sensitive to changes in the nominal domain and captures changes due to unpredictable cultural shifts. Our second measure relies on a more traditional global notion of change; we show that it better captures changes, like those in verbs, that are the result of regular linguistic drift.

Our analysis relies on a large-scale statistical study of six historical corpora in multiple languages, along with case-studies that illustrate the fine-grained differences between the two measures.

## 2 Methods

We use the diachronic word2vec embeddings constructed in our previous work ( Hamilton et al., 2016) to measure how word meanings change between consecutive decades.1 In these representations each word wi has a vector representation w( t) ( Turney and Pantel, 2010) at each time point, which captures its co-occurrence statistics for that time period. The vectors are constructed using the skip-gram with negative sampling (SGNS) algorithm ( Mikolov et al., 2013) and post-processed to align the semantic spaces between years. Measuring the distance between word vectors for consecutive decades allows us to compute the rate at which the different words change in meaning ( Gulordava and Baroni, 2011).

We analyzed the decades from 1800 to 1990 using vectors derived from the Google N-gram datasets ( Lin et al., 2012) that have large amounts of historical text (English, French, German, and English Fiction). We also used vectors derived from the Corpus of Historical American English (COHA), which is smaller than Google N-grams but was carefully constructed to be genre balanced and contains word lemmas as well as surface forms ( Davies, 2010). We examined all decades from 1850 through 2000 using the COHA dataset and used the part-of-speech tags provided with the corpora.

### 2.1 Measuring semantic change

We examine two different ways to measure semantic change ( Figure 1).

Two different measures of semantic change

With the global measure of change, we measure how far a word has moved in semantic space between two time-periods. This measure is sensitive to subtle shifts in usage and also global effects due to the entire semantic space shifting. For example, this captures how actually underwent subjectification during the 20th century, shifting from uses in objective statements about the world (“actually did try”) to subjective statements of attitude (“I actually agree”; see Traugott and Dasher, 2001 for details). In contrast, with the local neighborhood measure of change, we measure changes in a word’s nearest neighbors, which captures drastic shifts in core meaning, such as gay’s shift in meaning over the 20th century.

#### Global measure

The first measure analyzes global shifts in a word’s vector semantics and is identical to the measure used in most previous works ( Gulordava and Baroni, 2011; Jatowt and Duh, 2014; Kim et al., 2014; Hamilton et al., 2016). We simply take a word’s vectors for two consecutive decades and measure the cosine distance between them, i.e.

dG(wi(t),wi(t+1))=cos-dist(wi(t),wi(t+1)).

(1)

#### Local neighborhood measure

The second measure is based on the intuition that only a word’s nearest semantic neighbors are relevant. For this measure, we first find word wi’s set of k nearest-neighbors (according to cosine-similarity) within each decade, which we denote by the ordered set
Nk(wi(t)). Next, to measure the change between decades t and t + 1, we compute a “second-order” similarity vector for
wi(t) from these neighbor sets with entries defined as

s(t)(j)=cos-sim(wi(t),wj(t))wjNk(wi(t))Nk(wi(t+1)),

(2)

and we compute an analogous vector for
wi(t+1). The second-order vector,
si(t), contains the cosine similarity of w i and the vectors of all wi’s nearest semantic neighbors in the the time-periods t and t + 1. Working with variants of these second-order vectors has been a popular approach in many recent works, though most of these works define these vectors against the full vocabulary and not just a word’s nearest neighbors ( del Prado Martin and Brendel. 2016; Eger and Mehler, 2016; Rodda et al., 2016).

Finally, we compute the local neighborhood change as

dL(wi(t),wi(t+1))=cos-dist(si(t),si(t+1)).

(3)

This measures the extent to which wi’s similarity with its nearest neighbors has changed.

The local neighborhood measure defined in ( 3) captures strong shifts in a word’s paradigmatic relations but is less sensitive to global shifts in syntagmatic contexts ( Schutze and Pedersen, 1993). We used k = 25 in all experiments (though we found the results to be consistent for k [10, 50]).

### 2.2 Statistical methodology

To test whether nouns or verbs change more according to our two measures of change, we build on our previous work and used a linear mixed model approach ( Hamilton et al., 2016). This approach amounts to a linear regression where the model also includes “random” effects to account for the fact that the measurements for individual words will be correlated across time ( McCulloch and Neuhaus, 2001).

We ran two regressions per datatset: one with the global dG values as the dependent variables (DVs) and one with the local neighborhood dL values. In both cases we examined the change between all consecutive decades and normalized the DVs to zero-mean and unit variance. We examined nouns/verbs within the top-10000 words by frequency rank and removed all words that occurred <500 times in the smaller COHA dataset. The independent variables are word frequency, the decade of the change (represented categorically), and variable indicating whether a word is a noun or a verb (proper nouns are excluded, as in Hamilton et al., 2016).2

## 3 Results

Our results show that the two seemingly related measures actually result in drastically different notions of semantic change.

### 3.1 Nouns vs. verbs

The local neighborhood measure assigns far higher rates of semantic change to nouns across all languages and datasets while the opposite is true for the global distance measure, which tends to assign higher rates of change to verbs ( Figure 2).

The global measure is more sensitive to semantic changes in verbs while the local neighborhood measure is more sensitive to noun changes

Examining how much nouns change relative to verbs (using coefficients from mixed-model regressions) reveals that the two measures are sensitive to different types of semantic change. Across all languages, the local neighborhood measure always assigns relatively higher rates of change to nouns (i.e., the right/green bars are lower than the left/blue bars for all pairs), though the results vary by language (e.g., French has high noun change-rates overall). 95% confidence intervals are shown.

We focused on verbs vs. nouns since they are the two major parts-of-speech and previous research has shown that verbs are more semantically mutable than nouns and thus more likely to undergo linguistic drift ( Gentner and France, 1988), while nouns are far more likely to change due to cultural shifts like new technologies ( Traugott and Dasher, 2001). However, some well-known regular linguistic shifts include rarer parts of speech like adverbs (included in our case studies below). Thus we also confirmed that the differences shown in Figure 2 also hold when adverbs and adjectives are included along with the verbs. This modified analysis showed analogous significant trends, which fits with previous research arguing that adverbial and adjectival modifiers are also often the target of regular linguistic changes ( Traugott and Dasher, 2001).

The results of this large-scale regression analysis show that the local measure is more sensitive to changes in the nominal domain, a domain in which change is known to be driven by cultural factors. In contrast, the global measure is more sensitive to changes in verbs, along with adjectives and adverbs, which are known to be the targets of many regular processes of linguistic change ( Traugott and Dasher, 2001; Hopper and Traugott, 2003)

### 3.2 Case studies

We examined six case-study words grouped into two sets. These case studies show that three examples of well-attested regular linguistic shifts (set A) changed more according to the global measure, while three well-known examples of cultural changes (set B) change more according to the local neighborhood measure. Table 2 lists these words with some representative historical contexts ( Davies, 2010).

### Table 2

Example case-studies of semantic change

The first three words are examples of regular linguistic shifts, while the latter three are examples of words that shifted due to exogenous cultural factors. Contexts are from the COHA data ( Davies. 2010).

Word1850s context1990s contextactually“…dinners which you have actually eaten.”“With that, I actually agree.”must“O, George, we must have faith.”“Which you must have heard ten years ago…”promise“I promise to pay you…’“…the day promised to be lovely.”

gay“Gay bridals and other merry-makings of men.”“…the result of gay rights demonstrations.”virus“This young man is…infected with the virus.”“…a rapidly spreading computer virus.”cell“The door of a gloomy cell…”“They really need their cell phones.”

Set A contains three words that underwent attested regular linguistic shifts detailed in Traugott and Dasher (2001): actually, must, and promise. These three words represent three different types of regular linguistic shifts: actually is a case of subjectification (detailed in Figure 1); must shifted from a deontic/obligation usage (“you must do X”) to a epistemic one (“X must be the case”), exemplifying a regular pattern of change common to many modal verbs; and promise represents the class of shifting “performative speech acts” that undergo rich changes due to their pragmatic uses and subjectification ( Traugott and Dasher, 2001). The contexts listed in Table 2 exemplify these shifts.

Set B contains three words that were selected because they underwent well-known cultural shifts over the last 150 years: gay, virus, and cell. These words gained new meanings due to uses in community-specific vernacular ( gay) or technological advances ( virus, cell). The cultural shifts underlying these changes in usage — e.g., the development of the mobile “cell phone” — were unpredictable in the sense that they were not the result of regularities in human linguistic systems.

Figure 3 shows how much the meaning of these word changed from the 1850s to the 1990s according to the two different measures on the English Google data. We see that the words in set A changed more when measurements were made using the global measure, while the opposite holds for set B.

The global measure captures classic examples of linguistic drift while the local measure captures example cultural shifts

Examining the semantic distance between the 1850s and 1990s shows that the global measure is more sensitive to regular shifts (and vice-versa for the local measure). The plot shows the difference between the measurements made by the two methods.

## 4 Discussion

Our results show that our novel local neighborhood measure of semantic change is more sensitive to changes in nouns, while the global measure is more sensitive to changes in verbs. This mapping aligns with the traditional distinction between irregular cultural shifts in nominals and more regular cases of linguistic drift ( Traugott and Dasher, 2001) and is further reinforced by our six case studies.

This finding emphasizes that researchers must develop and use measures of semantic change that are tuned to specific tasks. For example, a cultural change-point detection framework would be more successful using our local neighborhood measure, while an empirical study of grammaticalization would be better off using the traditional global distance approach. Comparing measurements made by these two approaches also allows researchers to assess the extent to which semantic changes are linguistic or cultural in nature.

​

### Table 1

Number of nouns and verbs tested in each dataset.

Dataset# Nouns# VerbsGoogle English All52992722Google English Fic.49413128German54431844French23104992COHA (Word)40771267COHA (Lemma)3389783

## Acknowledgments

The authors thank C. Manning, V. Prabhakaran, S. Kumar, and our anonymous reviewers for their helpful comments. This research has been supported in part by NSF CNS-1010921, IIS-1149837, IIS-1514268 NIH BD2K, ARO MURI, DARPA XDATA, DARPA SIMPLEX, Stanford Data Science Initiative, SAP Stanford Graduate Fellowship, NSERC PGS-D, Boeing, Lightspeed, and Volkswagen.

## Footnotes

2Frequency was included since it is known to strongly influence the distributional measures ( Hamilton et al., 2016).

## References

- Davies Mark. The Corpus of Historical American English: 400 million words, 1810–2009. 2010 http://corpus.byu.edu/coha/
- Moscoso Fermin, del Martin Prado, Brendel Christian. Case and Cause in Icelandic: Reconstructing Causal Networks of Cascaded Language Changes. Proc ACL 2016[ Google Scholar]
- Eger Steffen, Mehler Alexander. On the Linearity of Semantic Change: Investigating Meaning Variation via Dynamic Graph Models. Proc ACL 2016[ Google Scholar]
- Gentner Dedre, France Ilene M. The verb mutability effect: Studies of the combinatorial semantics of nouns and verbs. Lexical ambiguity resolution: Perspectives from psycholinguistics, neuropsychology, and artificial intelligence. 1988:343–382.[ Google Scholar]
- Gulordava Kristina, Baroni Marco. Proc GEMS 2011 Workshop on Geometrical Models of Natural Language Semantics. Association for Computational Linguistics; 2011. A distributional similarity approach to the detection of semantic change in the Google Books Ngram corpus; pp. 67–71. [ Google Scholar]
- Hamilton William L, Leskovec Jure, Jurafsky Dan. Diachronic Word Embeddings Reveal Statistical Laws of Semantic Change. Proc ACL 2016[ Google Scholar]
- Hopper Paul J, Traugott Elizabeth Closs. Grammaticalization. Cambridge University Press; Cambridge, UK: 2003. [ Google Scholar]
- Jatowt Adam, Duh Kevin. Proc 14th ACM/IEEE-CS Conf on Digital Libraries. IEEE Press; 2014. A framework for analyzing semantic change of words across time; pp. 229–238. [ Google Scholar]
- Kim Yoon, Chiu Yi-I, Hanaki Kentaro, Hegde Darshan, Petrov Slav. Temporal analysis of language through neural language models. 2014 arXiv preprint arXiv: 1405.3515. [ Google Scholar]
- Kintsch Walter. Metaphor comprehension: A computational theory. Psychon Bull Rev. 2000;7(2):257–266. [ PubMed] [ Google Scholar]
- Kulkarni Vivek, Al-Rfou Rami, Perozzi Bryan, Skiena Steven. Statistically significant detection of linguistic change. Proc 24th WWW Conf. 2014:625–635.[ Google Scholar]
- Lin Yuri, Michel Jean-Baptiste, Aiden Erez Lieberman, Orwant Jon, Brockman Will, Petrov Slav. Syntactic annotations for the google books ngram corpus. Proc ACL System Demonstrations 2012[ Google Scholar]
- McCulloch Charles E, Neuhaus John M. Generalized linear mixed models. Wiley-Interscience; Hoboken, NJ: 2001. [ Google Scholar]
- Mikolov Tomas, Sutskever Ilya, Chen Kai, Corrado Greg S, Dean Jeff. Distributed representations of words and phrases and their compositionality. NIPS 2013[ Google Scholar]
- Rodda Martina, Senaldi Marco, Lenci Alessandro. Panta rei: Tracking Semantic Change with Distributional Semantics in Ancient Greek. Italian Conference of Computational Linguistics.2016. [ Google Scholar]
- Schutze Hinrich, Pedersen Jan. A vector model for syntagmatic and paradigmatic relatedness. Proc. 9th Annu. Conf. of the UW Centre for the New OED and Text Research; Citeseer: 1993. pp. 104–113. [ Google Scholar]
- Traugott Elizabeth Closs, Dasher Richard B. Regularity in Semantic Change. Cambridge University Press; Cambridge, UK: 2001. [ Google Scholar]
- Turney Peter D, Pantel Patrick. From frequency to meaning: Vector space models of semantics. J Artif Intell Res. 2010;37(1):141–188.[ Google Scholar]
- Xu Yang, Kemp Charles. A computational evaluation of two laws of semantic change. Proc 37th Annu Conf Cogn Sci Soc 2015[ Google Scholar]

###### Other Formats

###### Actions

- Cite
- Collections

Add to Collections

- Create a new collection
- Add to an existing collection

Name your collection:

Name must be less than characters

Choose a collection:

Unable to load your collection due to an error

Please try again

Add

Cancel

###### Share

- Permalink

Copy

###### RESOURCES

- Similar articles

- Cited by other articles

- Links to NCBI Databases

[x]

Cite

Copy
Download .nbib.nbib

Format:
AMA

APA

MLA

NLM

Follow NCBI

National Library of Medicine