Johannes Zirngibl, Steffen Deusch, Patrick Sattler, Juliane Aulbach, Georg Carle, Mattijs Jonker &ndash; Technical University of Munich, Germany; University of Twente, The Netherlands
Domain parking typically involves leveraging advertisements to generate revenue on otherwise inactive domain names. Their content is rarely of real value to users and tends to be highly similar across parked domains. They have commonalities beyond content alone: parked domains can share hosting and DNS infrastructure. Parking rarely receives special treatment in existing studies (e.g., content analyses or infrastructure concentration studies). While the presence and possible bias introduced by parked pages is sometimes acknowledged in studies, the studies still treat parked domains as any other, either because differentiation is infeasible, or because doing so is considered out-of-scope. We argue that the impact of parked domains on analyses regarding the current state and future development of the Internet should not be overlooked. In this paper, we motivate this argument through quantification, and take steps towards helping other researchers identify parked domains. We systematically collect a list of 82 parking services and develop DNS-based indicators to help identify parked domains. We next quantify the presence of parked domains, using large-scale DNS data containing hundreds of millions of registered domain names, representative for a significant part of the global DNS namespace. Overall, we pinpoint 60 M parked domains, which is a significant percentage of all names under consideration (23 %) and identify up to 4 % of domains from top lists to be parked. These findings demonstrate that the effect of parked pages is potentially pronounced. We also break down into the various parking services and DNS zones. This helps us demonstrate and further discuss the effect that domain parking can have on research and Internet consolidation.
Common Crawl While visual identification allowed us to validate the inferences to a reasonable extent, we wanted to upscale validation. Therefore, we consider Common Crawl (CC) data [21] [C. Crawl. (2022) The Common Crawl Corpus. [Online]. Available: https://commoncrawl.org/] and calculate the similarity of pages. Common Crawl is an open repository of web crawl data, collected at monthly intervals, accounting for hundreds of millions of unique domain names, and many more URLs. We consider CC data for Jan 2022 and the ∼60 M parked domains that we identify on Jan 28th, 2022. We extract the HTML content of parked pages from CC data, only considering URLs that contain exactly the registered domain. Furthermore, we require the crawl target to have been the landing page (i.e., the path of the URL is /) and also to have resulted in a useful response (i.e., HTTP status code of 200). Given these filters, ∼1.29 M HTML rich responses can be obtained. We extract visible text and tokenize it into words, remove stop words, apply lemmatization, and create a vector for the most-frequently used words for each page.