Task

Sort most popular programming languages based on the number of users on Rosetta Code.

Show the languages with at least 100 users.



A method to solve the task

Users of a computer programming language   X   are those referenced in the page:

 https://rosettacode.org/wiki/Category:X_User,     or preferably: 
 https://rosettacode.org/mw/index.php?title=Category:X_User&redirect=no to avoid re-directions. 

In order to find the list of such categories,   it's possible to first parse the entries of:

 http://rosettacode.org/mw/index.php?title=Special:Categories&limit=5000. 

Then download and parse each computer language   users   category to find the number of users of that computer language.



Sample output on 18 February 2019:

Language             Users
--------------------------
C                      391
Java                   276
C++                    275
Python                 262
JavaScript             238
Perl                   171
PHP                    167
SQL                    138
UNIX Shell             131
BASIC                  120
C sharp                118
Pascal                 116
Haskell                102
A Rosetta Code user usually declares using a language with the mylang template. This template is expected to appear on the User page. However, in some cases it appears in a user Talk page. It's not necessary to take this into account. For instance, among the 373 C users in the table above, 3 are actually declared in a Talk page.




use HTTP::UserAgent;
use URI::Escape;
use JSON::Fast;
 
my $client = HTTP::UserAgent.new;
 
my $url = 'http://rosettacode.org/mw';
 
my $start-time = now;
 
say "========= Generated: { DateTime.new(time) } =========";
 
my $lang = 1;
my $rank = 0;
my $last = 0;
my $tie = ' ';
my $minimum = 25;
 
.say for
    mediawiki-query(
        $url, 'pages',
        :generator<categorymembers>,
        :gcmtitle<Category:Language users>,
        :gcmlimit<350>,
        :rawcontinue(),
        :prop<categoryinfo>
    )
 
    .map({ %( count => .<categoryinfo><pages> || 0,
              lang  => .<title>.subst(/^'Category:' (.+) ' User'/, ->$/ {$0}) ) })
 
    .sort( { -.<count>, .<lang> } )
 
    .map( { last if .<count> < $minimum; display(.<count>, .<lang>) } );
 
say "========= elapsed: {(now - $start-time).round(.01)} seconds =========";
 
sub display ($count, $which) {
    if $last != $count { $last = $count; $rank = $lang; $tie = ' ' } else { $tie = 'T' };
    sprintf "#%3d  Rank: %2d %s  with %-4s users:  %s", $lang++, $rank, $tie, $count, $which;
}
 
sub mediawiki-query ($site, $type, *%query) {
    my $url = "$site/api.php?" ~ uri-query-string(
        :action<query>, :format<json>, :formatversion<2>, |%query);
    my $continue = '';
 
    gather loop {
        my $response = $client.get("$url&$continue");
        my $data = from-json($response.content);
        take $_ for $data.<query>.{$type}.values;
        $continue = uri-query-string |($data.<query-continue>{*}».hash.hash or last);
    }
}
 
sub uri-query-string (*%fields) {
    join '&', %fields.map: { "{.key}={uri-escape .value}" }
}