Task

Sort most popular programming languages based on the number of users on Rosetta Code.

Show the languages with at least 100 users.



A method to solve the task

Users of a computer programming language   X   are those referenced in the page:

 https://rosettacode.org/wiki/Category:X_User,     or preferably: 
 https://rosettacode.org/mw/index.php?title=Category:X_User&redirect=no to avoid re-directions. 

In order to find the list of such categories,   it's possible to first parse the entries of:

 http://rosettacode.org/mw/index.php?title=Special:Categories&limit=5000. 

Then download and parse each computer language   users   category to find the number of users of that computer language.



Sample output on 18 February 2019:

Language             Users
--------------------------
C                      391
Java                   276
C++                    275
Python                 262
JavaScript             238
Perl                   171
PHP                    167
SQL                    138
UNIX Shell             131
BASIC                  120
C sharp                118
Pascal                 116
Haskell                102
A Rosetta Code user usually declares using a language with the mylang template. This template is expected to appear on the User page. However, in some cases it appears in a user Talk page. It's not necessary to take this into account. For instance, among the 373 C users in the table above, 3 are actually declared in a Talk page.




"""Rank languages by number of users. Requires Python >=3.5"""
 
import requests
 
# MediaWiki API URL.
URL = "http://rosettacode.org/mw/api.php"
 
# Query string parameters
PARAMS = {
    "action": "query",
    "format": "json",
    "formatversion": 2,
    "generator": "categorymembers",
    "gcmtitle": "Category:Language users",
    "gcmlimit": 500,
    "prop": "categoryinfo",
}
 
 
def fetch_data():
    counts = {}
    continue_ = {"continue": ""}
 
    # Keep making HTTP requests to the MediaWiki API if more records are
    # available.
    while continue_:
        resp = requests.get(URL, params={**PARAMS, **continue_})
        resp.raise_for_status()
 
        data = resp.json()
 
        # Grab the title (language) and size (count) only.
        counts.update(
            {
                p["title"]: p.get("categoryinfo", {}).get("size", 0)
                for p in data["query"]["pages"]
            }
        )
 
        continue_ = data.get("continue", {})
 
    return counts
 
 
if __name__ == "__main__":
    # Request data from the MediaWiki API.
    counts = fetch_data()
 
    # Filter out languages that have less than 100 users.
    at_least_100 = [(lang, count) for lang, count in counts.items() if count >= 100]
 
    # Sort languages by number of users
    top_languages = sorted(at_least_100, key=lambda x: x[1], reverse=True)
 
    # Pretty print
    for i, lang in enumerate(top_languages):
        print(f"{i+1:<5}{lang[0][9:][:-5]:<20}{lang[1]}")
 