<commit_before># TMLP Statistics Web Application

## Installing:
This repo includes a Vagrant configuration to make development easier.

### Clone Repo
```
$ mkdir ~/dev
$ cd ~/dev
$ git clone <repo url>
```

### Setup for Database Seeding
If you have an export of the database, the provisioning will seed the database for you. Generate a CSV export
of the database, and copy the files into `~/dev/export/`. Note, the export should be a collection of files named
after the table they contain.
You can also snap the database after provisioning using Laravel's artisan migrate command:
```
$ cd /vagrant/tmlpstats
$ php artisan migrate:refresh --seed

```

### Start Your dev VM
```
$ cd ~/dev
$ vagrant up
```
Wait for the scripts to complete. This will take a while the first time since Vagrant has to download the base image.

Your VM should now be ready to go.

You can add an entry to your hosts file for ease of testing:
```
echo '192.168.56.102  vagrant-dev.com' | sudo tee -a /etc/hosts"
```

View the application in your browser. Visit: `http://vagrant-dev.com/tmlpstats/`<commit_msg>Update README instructions<commit_after># TMLP Statistics Web Application

## Installing:
This repo includes a Vagrant configuration to make development easier.

### Clone Repo
```
$ cd ~/dev
$ git clone https://github.com/pdarg/tmlpstats
```

### Setup for Database Seeding
If you have an export of the database, the provisioning will seed the database for you. Generate a CSV export of the database, and copy the files into `~/dev/tmlpstats/export/`. Note, the export should be a collection of files named after the table they contain.

You can also snap the database after provisioning using Laravel's artisan migrate command:
```
$ vagrant ssh
<inside vagrant VM>
$ cd /vagrant/tmlpstats
$ php artisan migrate:refresh --seed
```

### Start Your dev VM
```
$ cd ~/dev/tmlpstats
$ vagrant up
```
Wait for the scripts to complete. This will take a while the first time since Vagrant has to download the base image.

Your VM should now be ready to go.

You can add an entry to your hosts file for ease of testing:
```
echo '192.168.56.102  vagrant-dev.com' | sudo tee -a /etc/hosts"
```

View the application in your browser. Visit: `http://vagrant-dev.com/tmlpstats/`
{'fraction_non_alphanumeric': 0.09296148738379814, 'fraction_numerical': 0.009738822487826471, 'mean_word_length': 4.033407572383074, 'pattern_counts': {'":': 0, '<': 5, '<?xml version=': 0, '>': 5, 'https://': 1, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 2, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 2, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '160559', 'n_tokens_mistral': 701, 'n_tokens_neox': 652, 'n_words': 321}