<commit_before>PlotSearch
============
An information retrieval project.<commit_msg>added instructions in readme<commit_after>PlotSearch
============
An application to search for movies based on plot description. The user is provided with a text box where queries can be input in the form of sentences describing the scenes of what actually happened. Our system returns relevant movies that match the description.

### File structure

```
/
├───data
|    ├───movie_corpus.json  #main corpus file
|    ├───names.txt  #names to pronoun mapping file
|    ├───toy_corpus.json  #test corpus file
|    └───wn_s.pl  #wordnet synonyms file
├───doc
|    ├───PlotSearch.pdf
|    └───PlotSearch.tex
├───libs
|    ├───__init__.py
|    └───search_engine.py  #script for searching documents with elasticsearch
├───static
|    ├───images
|    |    └───background.jpg
|    ├───js
|    |    ├───bootstrap.min.js
|    |    ├───jquery.min.js
|    |    └───main.js
|    └───styles
|         ├───bootstrap.min.css
|         └───style.css
├───templates
|    ├───base.html
|    ├───index.html
|    ├───movie.html
|    └───results.html
├───app.py  # flask app main controller
├───create_corpus.py  #script for creating corpus
├───es_settings  # set up elasticsearch
├───README.md
└───requirements.txt
```

### Local Setup

To install dependencies:
```
pip install -r requirements
```

To create corpus from scratch:
```
python create_corpus.y
```

It will go to themoviespoiler.com and omdbapi.com, fetch and clean movie data, and save it to `data/movie_corpus.json`.

To build elasticsearch index (it copies synonym files into elasticsearch config directory, therefore root access is required):
```
sudo <virtualenv_dir>/bin/python es_settings.py
```

Then the flask application can be kicked off by running:
```
python vs_query.py
```
{'fraction_non_alphanumeric': 0.15286975717439294, 'fraction_numerical': 0.0, 'mean_word_length': 3.967123287671233, 'pattern_counts': {'":': 0, '<': 4, '<?xml version=': 0, '>': 4, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '3535732', 'n_tokens_mistral': 601, 'n_tokens_neox': 602, 'n_words': 181}