<commit_before>## Email Search 

[![Build status](https://secure.travis-ci.org/kigster/email-search.png)](http://travis-ci.org/kigster/email-search)

Email Search is a simple rails application, whose goal is to provide fast and effective search across many
email messages in the "*.emlx" format (used by Apple Mail).  Application provides a way to import these messages
from a directory to the database, and then uses Apache Solr (via Sunspot gem) to search imported messages.

## Why?

I had to dig through a whole lot (tens of thousands) of very old email files, and search for relevant items,
and using ```grep``` and ```find``` quickly became complete nuisance. I wanted to search case insensitively,
including terms, excluding terms, using full text search and natural language capabilities of modern search engines.

And then I thought, hell, maybe someone else will need to do the same sometimes.  So I decided to make it properly,
and make it open source.

This simple Rails 4 / Ruby 2.0 app allows sophisticated full-text searches powered by Solr, and offers a
convenient screen viewport for scanning through email content that match.  You can export emails back, that match
a particular search criteria, using the "Download" button, which creates a zip file of matching emails.

## Usage

1. You should have a directory containing a bunch of *.emlx files you want import.  This could be a flat directory, or a hierarchy.
2. Run ```bundle install```
3. Run ```rake db:create && rake db:migrate```
4. Run ```bundle exec foreman start``` to start the application and Solr
5. Run ```bundle exec rake emails:load:directory["/top/level/directory/with/your/files"]```
6. Connect to http://localhost:8080/
7. Search, refine your search, hover over matching emails, search again.
8. Results of any search query can be exported as a zip file and downloaded.

### Screen Shot

Below is a sample screen shot with the Rails application running.
<commit_msg>Updating README<commit_after>## Email Search 

[![Build status](https://secure.travis-ci.org/kigster/email-search.png)](http://travis-ci.org/kigster/email-search)

Email Search is a simple rails application, whose goal is to provide fast and effective search across many
email messages in the "*.emlx" format (used by Apple Mail).  Application provides a way to import these messages
from a directory to the database, and then uses Apache Solr (via Sunspot gem) to search imported messages.

## Why?

I had to dig through a whole lot (tens of thousands) of very old email files, and search for relevant items,
and using ```grep``` and ```find``` quickly became complete nuisance. I wanted to search case insensitively,
including terms, excluding terms, using full text search and natural language capabilities of modern search engines.

And then I thought, hell, maybe someone else will need to do this too.  So I decided to make it a "proper" app,
write some tests, and open source it.

The result is a rather simple but effective Rails app, that supports sophisticated full-text searches
powered by Solr (powered by Sunspot), and offers a convenient screen viewport for scanning through email
content that match search, with a UI based on Twitter Bootstrap.  You can then export the results that match
a particular search criteria using the "Download" button, which creates a zip file that includes matching email
messages, and then sends it to the browser.

## Dependencies

- Rails 4
- Ruby 2.0

## Usage

1. You should have a directory containing a bunch of *.emlx files you want import.  This could be a flat directory, or a hierarchy.
2. Run ```bundle install```
3. Run ```rake db:create && rake db:migrate```
4. Run ```bundle exec foreman start``` to start the application and Solr
5. Run ```bundle exec rake emails:load:directory["/top/level/directory/with/your/files"]```
6. Connect to http://localhost:8080/
7. Search, refine your search, hover over matching emails, search again.
8. Results of any search query can be exported as a zip file and downloaded.

### Screen Shot

Below is a sample screen shot with the Rails application running.

{'fraction_non_alphanumeric': 0.07669831994156319, 'fraction_numerical': 0.007304601899196494, 'mean_word_length': 4.462765957446808, 'pattern_counts': {'":': 0, '<': 3, '<?xml version=': 0, '>': 3, 'https://': 2, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 4, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '7089331', 'n_tokens_mistral': 1095, 'n_tokens_neox': 1046, 'n_words': 604}