<commit_before># ![Uxie](static/uxie_icon.png) Uxie [![Build Status](https://travis-ci.org/Protectator/Uxie.svg?branch=master)](https://travis-ci.org/Protectator/Uxie)

Uxie is a tool to build a database using Pokemon Showdown's usage stats.

## Installation

- Install [Python 2.7 or 3.5](https://www.python.org/)
- Install [pip](https://pip.pypa.io/en/stable/installing/)
- In the root directory of the project, run `pip install -r requirements.txt`

## Usage

Be sure you have an accessible instance of [MySQL](http://www.mysql.com/)

Run `uxie.py`.

    usage: uxie.py [-h] [-v] [-V] [-1] [-2] [-3] [-y YEAR [YEAR ...]]
                   [-m MONTH [MONTH ...]] [-f FORMAT [FORMAT ...]]
                   [-g GXE [GXE ...]]
                   [-t {usage,chaos,leads,metagame,moveset} [{usage,chaos,leads,metagame,moveset} ...]]
                   [-d DIRECTORY]
                   {mysql} host user password dbname
    
    Download all Pokemon Showdown's stats files, and fill a database with its stats.
    
    positional arguments:
      {mysql}               Database Management System
      host                  Database address
      user                  Database user
      password              User password
      dbname                Database name
    
    optional arguments:
      -h, --help            show this help message and exit
      -v, --verbose         be verbose
      -V, --version         show the version number and exit
      -1, --skip-download   do not download any file from the internet and only use available local files to build the database
      -2, --skip-parse      do not parse and do not store any file in a database, and only download files from the internet
      -3, --skip-index      do not create index the recommended columns in the final database
      -y YEAR [YEAR ...], --year YEAR [YEAR ...]
                            Filter : Only treat files in use these years
      -m MONTH [MONTH ...], --month MONTH [MONTH ...]
                            Filter : Only treat files in these months
      -f FORMAT [FORMAT ...], --format FORMAT [FORMAT ...]
                            Filter : Only treat files of these formats
      -g GXE [GXE ...], --gxe GXE [GXE ...]
                            Filter : Only treat files at exactly these gxe limits
      -t {usage,chaos,leads,metagame,moveset} [{usage,chaos,leads,metagame,moveset} ...], --type {usage,chaos,leads,metagame,moveset} [{usage,chaos,leads,metagame,moveset} ...]
                            Filter : Only treat files of these types
      -d DIRECTORY, --directory DIRECTORY
                            directory to use to download files into, and to parse from

## License

Uxie is distributed under the terms of the [MIT License](LICENSE).<commit_msg>Better README<commit_after># ![Uxie](static/uxie_icon.png) Uxie [![Build Status](https://travis-ci.org/Protectator/Uxie.svg?branch=master)](https://travis-ci.org/Protectator/Uxie)

**Uxie** is a tool to build a database using Pokemon Showdown's usage stats.

## Installation

- Install [Python 2.7 or 3.5](https://www.python.org/)
- Install [pip](https://pip.pypa.io/en/stable/installing/)
- In the root directory of the project, run `pip install -r requirements.txt`

## Usage

Be sure you have an accessible instance of [MySQL](http://www.mysql.com/)

Run `uxie.py` with your preferred python version while providing necessary arguments, for example :

### Simple example

Connects via `mysql` to `localhost` with user `uxie` and password `<PASSWORD>` to database `uxiedb` :

`python uxie.py mysql localhost uxie secretpw uxiedb`

### Parameters

#### Database connexion

These parameters are required, and must be in this order :
`dbms host user password dbname`

| Name | Values | Description |
| --- | --- | --- |
| dbms | `mysql` | DataBase Management System. Currently only supports `mysql` |
| host | Domain or IP | The machine on which to build the database |
| user | string | The username used to connect to the database |
| password | string | The password corresponding to the username |
| dbname | string | The name of the database to use |

#### Information

| Flag | Description |
| --- |  --- |
| `-h` or `--help`| Show the help and exits without doing anything else |
| `-V` or `--version` | Show the version number and exits without doing anything else |
| `-v` or `--verbose` | Describe actions done |

#### Behaviour

| Flag | Argument | Description |
| --- | --- | --- |
| `-1` or `--skip-download` | - | Skip the download phase |
| `-2` or `--skip-parse` | - | Skip the parse and feed the DB phase |
| `-3` or `--skip-index` | - | Skip the indexing DB phase |
| `-d` or `--directory` | `DIRECTORY` | Save and load files from `DIRECTORY`. Default directory used is `stats` |

#### Filters

| Flag | Argument(s) | Values | Description | Example |
| --- | --- | --- | --- | --- |
| `-y` | `YEAR [YEAR ...]` | `int` >= `2014` | Only treat files from the years listed by `YEAR` | `-y 2014 2015` |
| `-m` | `MONTH [MONTH ...]` | `int` between `1` and `12` | Only treat files from the months listed by `MONTH` | `-m 1 2 3 10 11 12` |
| `-f` | `FORMAT [FORMAT ...]` | `string` | Only treat files of the formats listed by `FORMAT` | `-f ou gen5ubers vgc2016` |
| `-g` | `GXE [GXE ...]` | `int` |Only treat files of the minimal GXE of the values listed by `GXE` | `-g 0 1630` |
| `-t` | `TYPE [TYPE ...]` | `` | Only treat files of the types listed by `TYPE` | `-t chaos moveset` |

## License

Uxie is distributed under the terms of the [MIT License](LICENSE).
{'fraction_non_alphanumeric': 0.13834776334776336, 'fraction_numerical': 0.0091991341991342, 'mean_word_length': 2.43343653250774, 'pattern_counts': {'":': 0, '<': 4, '<?xml version=': 0, '>': 5, 'https://': 8, 'lorem ipsum': 0, 'www.': 4, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 7, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '6140296', 'n_tokens_mistral': 1745, 'n_tokens_neox': 1674, 'n_words': 623}