<story><title>In search of the least viewed article on Wikipedia</title><url>http://colinmorris.github.io/blog/unpopular-wiki-articles</url></story><parent_chain></parent_chain><comment><author>onychomys</author><text>&amp;gt; there’s unfortunately no easy way to sort out the least viewed pages, short of a very slow linear search for the needle in the haystack&lt;p&gt;So this sentence made me wonder why he didn&amp;#x27;t actually just go do it, 6M pages isn&amp;#x27;t really all that big of a data set. Turns out that it&amp;#x27;s a problem of how the data is arranged. The raw files are divided by year&amp;#x2F;month&amp;#x2F;day&amp;#x2F;second[0], and then each of those seconds is a zipped file of about 500MB in size, where pages are listed like this:&lt;p&gt;en.m Alcibiades_(character) 1 0&lt;p&gt;en.m Alcibiades_DeBlanc 2 0&lt;p&gt;en.m Alcibiades_the_Schoolboy 1 0&lt;p&gt;en.m Alcide_De_Gasperi 2 0&lt;p&gt;en.m Alcide_Herveaux 1 0&lt;p&gt;en.m Alcide_Laurin 1 0&lt;p&gt;en.m Alcide_de_Gasperi 1 0&lt;p&gt;en.m Alcides_Escobar 3 0&lt;p&gt;en.m Alcimus_(mythology) 1 0&lt;p&gt;with en.m getting a separate listing from the desktop em, and all the other country codes getting their own listings too. So just collating the data would be a huge job.&lt;p&gt;The API also doesn&amp;#x27;t offer a specific list of all pages by time, so you&amp;#x27;d have to go and make a separate call for each of the 6M pages for a given year and then collate that data too.&lt;p&gt;[0]for example,  &lt;a href=&quot;https:&amp;#x2F;&amp;#x2F;dumps.wikimedia.org&amp;#x2F;other&amp;#x2F;pageviews&amp;#x2F;2019&amp;#x2F;2019-01&amp;#x2F;&quot; rel=&quot;nofollow&quot;&gt;https:&amp;#x2F;&amp;#x2F;dumps.wikimedia.org&amp;#x2F;other&amp;#x2F;pageviews&amp;#x2F;2019&amp;#x2F;2019-01&amp;#x2F;&lt;/a&gt;</text></comment>
<story><title>In search of the least viewed article on Wikipedia</title><url>http://colinmorris.github.io/blog/unpopular-wiki-articles</url></story><parent_chain></parent_chain><comment><author>dmix</author><text>I noticed moths came up a couple times, a brief guess why: Wikipedia says it&amp;#x27;s &amp;quot;one of most speciose orders&amp;quot; (besides flies and beetles). But maybe it has the most pages because they are so easy to catch with a light in your backyard, that it&amp;#x27;s far easier to name them all than something parasitoid wasps [1]?&lt;p&gt;Although this same paper says that &amp;quot;more species of beetles (&amp;gt;350,000) have been described than any other order of animal, insect or otherwise&amp;quot;&lt;p&gt;[1] &lt;a href=&quot;https:&amp;#x2F;&amp;#x2F;www.biorxiv.org&amp;#x2F;content&amp;#x2F;10.1101&amp;#x2F;274431v1.full.pdf&quot; rel=&quot;nofollow&quot;&gt;https:&amp;#x2F;&amp;#x2F;www.biorxiv.org&amp;#x2F;content&amp;#x2F;10.1101&amp;#x2F;274431v1.full.pdf&lt;/a&gt;&lt;p&gt;Just speculation...</text></comment>