TITLE: Privacy Policies over Time: Curation and Analysis of a Million-Document Dataset 

ABSTRACT: Automated analysis of privacy policies has proved a fruitful research
direction, with developments such as automated policy summarization, question
answering systems, and compliance detection. Prior research has been limited to
analysis of privacy policies from a single point in time or from short spans of
time, as researchers did not have access to a large-scale, longitudinal,
curated dataset. To address this gap, we developed a crawler that discovers,
downloads, and extracts archived privacy policies from the Internet Archive's
Wayback Machine. Using the crawler and following a series of validation and
quality control steps, we curated a dataset of 1,071,488 English language
privacy policies, spanning over two decades and over 130,000 distinct websites.
  Our analyses of the data paint a troubling picture of the transparency and
accessibility of privacy policies. By comparing the occurrence of
tracking-related terminology in our dataset to prior web privacy measurements,
we find that privacy policies have consistently failed to disclose the presence
of common tracking technologies and third parties. We also find that over the
last twenty years privacy policies have become even more difficult to read,
doubling in length and increasing a full grade in the median reading level. Our
data indicate that self-regulation for first-party websites has stagnated,
while self-regulation for third parties has increased but is dominated by
online advertising trade associations. Finally, we contribute to the literature
on privacy regulation by demonstrating the historic impact of the GDPR on
privacy policies.
Automated analysis of privacy policies has proved a fruitful research
direction, with developments such as automated policy summarization, question
answering systems, and compliance detection. Prior research has been limited to
analysis of privacy policies from a single point in time or from short spans of
time, as researchers did not have access to a large-scale, longitudinal,
curated dataset. To address this gap, we developed a crawler that discovers,
downloads, and extracts archived privacy policies from the Internet Archive's
Wayback Machine. Using the crawler and following a series of validation and
quality control steps, we curated a dataset of 1,071,488 English language
privacy policies, spanning over two decades and over 130,000 distinct websites.
  Our analyses of the data paint a troubling picture of the transparency and
accessibility of privacy policies. By comparing the occurrence of
tracking-related terminology in our dataset to prior web privacy measurements,
we find that privacy policies have consistently failed to disclose the presence
of common tracking technologies and third parties. We also find that over the
last twenty years privacy policies have become even more difficult to read,
doubling in length and increasing a full grade in the median reading level. Our
data indicate that self-regulation for first-party websites has stagnated,
while self-regulation for third parties has increased but is dominated by
online advertising trade associations. Finally, we contribute to the literature
on privacy regulation by demonstrating the historic impact of the GDPR on
privacy policies.
Privacy Policies over Time: Curation and Analysis of a Million-Document Dataset
http://arxiv.org/abs/2008.09159v4