<commit_before># couchmail.py

**Very** early stage email archiver for Apache CouchDB
and Cloudant.  This README assumes you'll be using a Cloudant 
account and DB to to archive and search through your mail.

## Installation

Copy `config.sample.ini` to `config.ini`.
Customize it.  You may need to use an application-specific 
password for GMail.  Here is what a config should look like


	[imap]
	host = imap.gmail.com
	user = <USERNAME><EMAIL>
	mailbox = Inbox
	password = <Password>
	
	[couch]
	server = http://<USERNAME>:<PASSWORD>@<USERNAME>.cloudant.com/
	db = mail
	; if doing multiple accounts per couch, you
	; might want to sha1() the email address to
	; be the mailbox name.

Then you'll want to make sure you have all the libraries.

    $ pip install -r requirements.txt
    $ python import.py

Hope for the best!

It *should* export the number of emails you specify,
store them by their `Message-ID` header value (or
timestamp...if `Message-ID` is missing).

Now, you'll want to install the design and index documents so you 
can search through your imported emails.

## Design Doc (aka CouchApp)

If you don't already have couchapp installed you'll have to do that first:

    $ pip install couchapp

then

    $ cd couchapp
    $ couchapp push . http://<USERNAME>:<PASSWORD>@<USERNAME>.cloudant.com/mail

Once you've pushed, you should be able to see your data and the design docs in the 'mail' database on your cloudant account.  You can now perform some searches:

#### Full-Text Search

The Design Doc includes basic MapReduce for counting things (see Futon, Fauxton, or the Cloudant Dashboard for more).

It also includes some [Cloudant](http://cloudant.com/) specific Full-Text Search indexes. These indexes let you search email addresses, subject lines, and message bodies.

Here are some examples…

Search for "fauxton":

```
/mail/_design/couchmail/_search/mail?q=fauxton
```

Count those results by `subject`, `to`, and `from`:

```
/mail/_design/couchmail/_search/mail?q=fauxton&counts=["subject", "to", "from"]
```

Now drilldown those results to just what I've posted:
<commit_msg>Update README_cloudant.md<commit_after># couchmail.py

**Very** early stage email archiver for Apache CouchDB
and Cloudant.  This README assumes you'll be using a Cloudant 
account and DB to to archive and search through your mail.

## Installation

Copy `config.sample.ini` to `config.ini`.
Customize it.  You may need to use an application-specific 
password for GMail.  Here is what a config should look like


	[imap]
	host = imap.gmail.com
	user = <USERNAME><EMAIL>
	mailbox = Inbox
	password = <Password>
	
	[couch]
	server = http://<USERNAME>:<PASSWORD>@<USERNAME>.cloudant.com/
	db = mail
	; if doing multiple accounts per couch, you
	; might want to sha1() the email address to
	; be the mailbox name.

You need to create the 'mail' db in your Cloudant account.

Then you'll want to make sure you have all the libraries.

    $ pip install -r requirements.txt
    $ python import.py

Hope for the best!

It *should* export the number of emails you specify,
store them by their `Message-ID` header value (or
timestamp...if `Message-ID` is missing).

Now, you'll want to install the design and index documents so you 
can search through your imported emails.

## Design Doc (aka CouchApp)

If you don't already have couchapp installed you'll have to do that first:

    $ pip install couchapp

If you are running iOS Mavericks it's possible you'll get a compiling error.  You can brute force your way
around this by using:

    $ export ARCHFLAGS="-Wno-error=unused-command-line-argument-hard-error-in-future"

Then

    $ cd couchapp
    $ couchapp push . http://<USERNAME>:<PASSWORD>@<USERNAME>.cloudant.com/mail

Once you've pushed, you should be able to see your data and the design docs in the 'mail' database on your cloudant account.  You can now perform some searches:

#### Full-Text Search

The Design Doc includes basic MapReduce for counting things (see Futon, Fauxton, or the Cloudant Dashboard for more).

It also includes some [Cloudant](http://cloudant.com/) specific Full-Text Search indexes. These indexes let you search email addresses, subject lines, and message bodies.

Here are some examples…

Search for "fauxton":

```
/mail/_design/couchmail/_search/mail?q=fauxton
```

Count those results by `subject`, `to`, and `from`:

```
/mail/_design/couchmail/_search/mail?q=fauxton&counts=["subject", "to", "from"]
```

Now drilldown those results to just what I've posted:

{'fraction_non_alphanumeric': 0.09270348192503881, 'fraction_numerical': 0.00044355732978487467, 'mean_word_length': 4.073115860517436, 'pattern_counts': {'":': 2, '<': 21, '<?xml version=': 0, '>': 21, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '1411033', 'n_tokens_mistral': 1392, 'n_tokens_neox': 1335, 'n_words': 583}