<commit_before># minst-dataset

The MNIST of music instrument sounds.

## Installing Dependencies

```
$ pip install -r requirements.txt
$ pip install git+git://github.com/ejhumphrey/minst-dataset.git
```

## Building the dataset from scratch

### Get the data

This project uses three different solo instrument datasets.
- [University of Iowa - MIS](http://theremin.music.uiowa.edu/MIS.html)
- [Philharmonia](http://www.philharmonia.co.uk/explore/make_music)
- [RWC - Instruments](https://staff.aist.go.jp/m.goto/RWC-MDB/rwc-mdb-i.html)

We provide "manifest" files with which one can download the first two collections. For access to the third (RWC), you should contact the kind folks at AIST.

To download the data, you can invoke the following from your cloned repository:

```
$ make download
```

(which is equivalent to:)

```
$ python scripts/download.py data/uiowa.json ~/data/uiowa
...
$ python scripts/download.py data/philharmonia.json ~/data/philharmonia
```
<commit_msg>readme fix<commit_after># minst-dataset

The MNIST of music instrument sounds.

## Installing Dependencies

```
$ pip install git+git://github.com/ejhumphrey/minst-dataset.git
$ cd minst-dataset

$ pip install -r requirements.txt
OR
$ make deps
```

## Building the dataset from scratch

### Get the data

This project uses three different solo instrument datasets.
- [University of Iowa - MIS](http://theremin.music.uiowa.edu/MIS.html)
- [Philharmonia](http://www.philharmonia.co.uk/explore/make_music)
- [RWC - Instruments](https://staff.aist.go.jp/m.goto/RWC-MDB/rwc-mdb-i.html)

We provide "manifest" files with which one can download the first two collections. For access to the third (RWC), you should contact the kind folks at AIST.

To download the data, you can invoke the following from your cloned repository:

```
$ make download
```

(which is equivalent to:)

```
$ python scripts/download.py data/uiowa.json ~/data/uiowa
...
$ python scripts/download.py data/philharmonia.json ~/data/philharmonia
```

{'fraction_non_alphanumeric': 0.1255, 'fraction_numerical': 0.0, 'mean_word_length': 4.224543080939948, 'pattern_counts': {'":': 0, '<': 3, '<?xml version=': 0, '>': 3, 'https://': 2, 'lorem ipsum': 0, 'www.': 2, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 10, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '1929320', 'n_tokens_mistral': 670, 'n_tokens_neox': 633, 'n_words': 194}