<commit_before># WoTrack: Spotify Words to Track (Poetic rewrite on non-matching)

## Description
A simple text to track generator. This application will process user input and find tracks that resemble the text pattern.
If some words or phrases are not found, then the application will generate the best poetic rewrite it can.

## How the processing works
This application combines natural language processing with dynamic programming to quickly and accurately
(still very much in its infancy) generate tracks from Spotify resembling the input text pattern. It does so
by:

1. Breaking the sentences down into trigrams and collecting the trigram collocations with
the highest PMI (Pointwise Mutual Information).

2. The trigram collocations are used as the search phrases (or queries) on Spotify's Track API.

3. The tracks are retrieved from Spotify and cached through Memcached.

4. The Levenshtein Distance between the track names and the search phrases is then determined and minimized.

5. The Levenshtein Distance determination/minimization step is done recursively until the input text is fully consumed.

*Note* Order is currently not maintained in the demo frontend page.

## Setup
This is a self contained Python/Django application. Perform the following steps
to install the applications setup the environment and to install dependencies:

1. Install Memcached by following the instructions at: <http://memcached.org/downloads>

2. Install Python 2.7x on your target platform. Python 2.7x can be downloaded and installed from: <https://www.python.org/downloads/>

3. Install pip for Python. Instructions for doing so could be found at: <https://pip.pypa.io/en/latest/installing.html>

4. It is recommended to setup a virtual environement for your Python 2.7x interpreter. Instructions to doing so could be found at <http://docs.python-guide.org/en/latest/dev/virtualenvs/>

5. install all of the applications requirements by running the following commands in the project's root:
    i. pip install -r requirements.txt
    ii. pip install -r requirements-dev.txt

6. Setup caching by running the following command from the project's root:
    i. python manage.py createcachetable

7. Run the tests to confirm everything is working properly by running the following command in the project's root:<commit_msg>Update README with more specific information<commit_after># WoTrack: Spotify Words to Track (Poetic rewrite on non-matching)

## Description
A simple text to track generator. This application will process user input and find tracks that resemble the text pattern.
If some words or phrases are not found, then the application will generate the best poetic rewrite it can.

## How the processing works
This application combines natural language processing with dynamic programming to quickly and accurately
(still very much in its infancy) generate tracks from Spotify resembling the input text pattern. It does so
by:

1. Breaking the sentences down into trigrams and collecting the trigram collocations with
the highest PMI (Pointwise Mutual Information).

2. The trigram collocations are used as the search phrases (or queries) on Spotify's Track API.

3. The tracks are retrieved from Spotify and cached through Memcached.

4. The Levenshtein Distance between the track names and the search phrases is then determined and minimized.

5. The Levenshtein Distance determination/minimization step is done recursively until the input text is fully consumed.

6. If there were no good trigram phrase matches, then the remaining trigram collocations are decomposed into digrams and unigrams. The same logic is ran on each, respectively.

*Note* Order is currently not maintained in the demo frontend page.

## Setup
This is a self contained Python/Django application. Perform the following steps
to install the applications setup the environment and to install dependencies:

1. Install Memcached by following the instructions at: <http://memcached.org/downloads>

2. Install Python 2.7x on your target platform. Python 2.7x can be downloaded and installed from: <https://www.python.org/downloads/>

3. Install pip for Python. Instructions for doing so could be found at: <https://pip.pypa.io/en/latest/installing.html>

4. It is recommended to setup a virtual environement for your Python 2.7x interpreter. Instructions to doing so could be found at <http://docs.python-guide.org/en/latest/dev/virtualenvs/>

5. install all of the applications requirements by running the following commands in the project's root:
    i. pip install -r requirements.txt
    ii. pip install -r requirements-dev.txt

6. Setup caching by running the following command from the project's root:
    i. python manage.py createcachetable

7. Run the tests to confirm everything is working properly by running the following command in the project's root:
{'fraction_non_alphanumeric': 0.04851362510322048, 'fraction_numerical': 0.00763831544178365, 'mean_word_length': 4.493197278911564, 'pattern_counts': {'":': 0, '<': 11, '<?xml version=': 0, '>': 11, 'https://': 4, 'lorem ipsum': 0, 'www.': 2, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 6, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '5743024', 'n_tokens_mistral': 1226, 'n_tokens_neox': 1145, 'n_words': 678}