<commit_before>wurstminestats
==============

Plotting the Wurstmineberg Minecraft Server

![Whitelist growth](http://i.wurstmineberg.de/wurstminestats/WhitelistGrowth.png)  
![Play Time](http://i.wurstmineberg.de/wurstminestats/playTime.png) 

## What this is
Primarily it's me playing around with [R](http://www.r-project.org).  
Besides that, this will be an addition to the [wurstmineberg](http://wurstminebger.de) player statistics page at our [stats page](http://wurstminebger.de/stats). 

Since now `Plots`and `data`are gitignored, the public directoy at [i.wurstmineberg.de/wurstminestats](http://i.wurstmineberg.de/wurstminestats/) is the place to go for up to date data (hopefully). 
For everything else, I'm currently working on a [shiny](http://shiny.rstudio.com/) version of wurstminestats, which should serve as a more convenient place to scroll through the data. In the long run, other fancy stuff might happen, but it's all very early as of now.

## Deployment
The plots are now generated by executing `genAll.R`on wurstmineberg.de via the `fcrontab` entry

    30 3,6,18,21 * * * /opt/hub/wurstmineberg/wurstminestats/genAll.R

…which, in theory, should update the plots every 3 hours at 30m past the hour, to reduce load on full hours caused by backup compression and whatnot.  

## Dependencies
Besides a recent version of R (duh), there are some things you need for this to be portable to any other Minecraft server. Let's go over the foreign data sources in this briefly.  

### Data sources
* http://api.wurstmineberg.de/server/playerstats/general.json
* http://api.wurstmineberg.de/server/playerstats/achievement.json
* http://api.wurstmineberg.de/server/playerstats/item.json
* http://api.wurstmineberg.de/server/playerstats/entity.json  

These are [Minecraft API](http://api.wurstmineberg.de) endpoints, which expose the actual player statistics we're interested in visualising. If you have this set up for your own server, you're pretty much good to go. Almost.

### General information sources
* http://wurstmineberg.de/assets/serverstatus/people.json  

This is our people database, which holds metadata about the people on our server, such as join date, nickname (ID), minecraft username and what have you. This is required to sort and classify the datasets. It is required in many dataset transformations, because I substitute minecraft usernames with their nicknames (IDs), sort them by join dates and group by their join status, so… If you don't want to set up a people.json, you'd need to get rid of these parts manually.

* http://wurstmineberg.de/static/json/achievements.json
<commit_msg>I said every 3 hours.<commit_after>wurstminestats
==============

Plotting the Wurstmineberg Minecraft Server

![Whitelist growth](http://i.wurstmineberg.de/wurstminestats/WhitelistGrowth.png)  
![Play Time](http://i.wurstmineberg.de/wurstminestats/playTime.png) 

## What this is
Primarily it's me playing around with [R](http://www.r-project.org).  
Besides that, this will be an addition to the [wurstmineberg](http://wurstminebger.de) player statistics page at our [stats page](http://wurstminebger.de/stats). 

Since now `Plots`and `data`are gitignored, the public directoy at [i.wurstmineberg.de/wurstminestats](http://i.wurstmineberg.de/wurstminestats/) is the place to go for up to date data (hopefully). 
For everything else, I'm currently working on a [shiny](http://shiny.rstudio.com/) version of wurstminestats, which should serve as a more convenient place to scroll through the data. In the long run, other fancy stuff might happen, but it's all very early as of now.

## Deployment
The plots are now generated by executing `genAll.R`on wurstmineberg.de via the `fcrontab` entry

    30 0,3,6,9,12,15,18,21 * * * /opt/hub/wurstmineberg/wurstminestats/genAll.R

…which, in theory, should update the plots every 3 hours at 30m past the hour, to reduce load on full hours caused by backup compression and whatnot.  

## Dependencies
Besides a recent version of R (duh), there are some things you need for this to be portable to any other Minecraft server. Let's go over the foreign data sources in this briefly.  

### Data sources
* http://api.wurstmineberg.de/server/playerstats/general.json
* http://api.wurstmineberg.de/server/playerstats/achievement.json
* http://api.wurstmineberg.de/server/playerstats/item.json
* http://api.wurstmineberg.de/server/playerstats/entity.json  

These are [Minecraft API](http://api.wurstmineberg.de) endpoints, which expose the actual player statistics we're interested in visualising. If you have this set up for your own server, you're pretty much good to go. Almost.

### General information sources
* http://wurstmineberg.de/assets/serverstatus/people.json  

This is our people database, which holds metadata about the people on our server, such as join date, nickname (ID), minecraft username and what have you. This is required to sort and classify the datasets. It is required in many dataset transformations, because I substitute minecraft usernames with their nicknames (IDs), sort them by join dates and group by their join status, so… If you don't want to set up a people.json, you'd need to get rid of these parts manually.

* http://wurstmineberg.de/static/json/achievements.json

{'fraction_non_alphanumeric': 0.09510250569476082, 'fraction_numerical': 0.005504935459377373, 'mean_word_length': 4.665591397849463, 'pattern_counts': {'":': 0, '<': 3, '<?xml version=': 0, '>': 3, 'https://': 0, 'lorem ipsum': 0, 'www.': 2, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '3240735', 'n_tokens_mistral': 1600, 'n_tokens_neox': 1570, 'n_words': 608}