<commit_before># groove-to-helpscout-migration-tool

**Work-in-progress**. ETL project to migrate from GrooveHQ to HelpScout via APIs.

This ETL tool uses the Acquire -> Process -> Publish sequence of phases as suggested by http://www.seabourneinc.com/rethinking-etl-for-the-api-age/

## Requirements

- PHP 5.4+
- `allow_url_fopen` must be allowed (for attachment downloads)
- [Laravel](https://laravel.com/docs/5.1/installation) (for console commands)
- [Composer](https://getcomposer.org/download/)

### Dependencies

We leverage the following libraries via Composer:
- [helpscout/api](https://github.com/helpscout/helpscout-api-php)
- [jadb/php-groovehq](https://github.com/jadb/php-groovehq)

## Usage

Clone project and run `composer install` in the root folder of this project.

Update config/services.php to use your API keys and your default HelpScout mailbox.

Ensure mailbox names within HelpScout correspond to the same mailbox names as Groove. A check will be made before syncing tickets.

### Within HelpScout

Create all of your agent (team, user & mailbox) accounts in HelpScout first. Our tool will need to map the Groove agent 
email addresses with HelpScout user emails and Groove mailboxes to HelpScout mailboxes (manual input may be required).

### CLI Usage

In the root of the Laravel project, run: 
```
php artisan sync-customers
``` 
Customers come first, as the process of creating conversations may create a new customer.

You can also resume a previously stopped migration by passing in a `startPage` parameter:
```
php artisan sync-customers --startPage=10
```

Once that succeeds, run: 
```
php artisan sync-tickets
```
to migrate Groove tickets, messages, images, attachments and tags.

The `sync-tickets` command also accepts the `startPage` optional parameter as described above.

If, for any reason, a particular ticket fails to migrate (e.g. attachment fails), you can 
redo `sync-tickets` with just that particular ticket number:
  
```
php artisan sync-tickets [<comma-separated list of Groove ticket numbers>] 
```

e.g. `php artisan sync-tickets 1000,2000,3000` to sync over just tickets #1000, 2000, and 3000.

By default, `sync-tickets` will ensure no duplicate tickets are created (keep in mind there is a slight delay before HelpScout picks up newly-created tickets).
You can bypass the duplication check (e.g. on the initial import) by specifying:

```
php artisan sync-tickets --checkDuplicates=false 
```
<commit_msg>Update documentation<commit_after># groove-to-helpscout-migration-tool

ETL project to migrate from GrooveHQ to HelpScout via APIs.

This ETL tool uses the Acquire -> Process -> Publish sequence of phases as suggested by http://www.seabourneinc.com/rethinking-etl-for-the-api-age/

## Requirements

- PHP 5.4+
- `allow_url_fopen` must be allowed (for attachment downloads)
- [Composer](https://getcomposer.org/download/)

### Dependencies

We leverage the following libraries via Composer:
- [Laravel](https://laravel.com/docs/5.1/installation) (for console commands)
- [helpscout/api](https://github.com/helpscout/helpscout-api-php)
- [jadb/php-groovehq](https://github.com/jadb/php-groovehq)

## Usage

Clone project and run `composer install` in the root folder of this project.

Update config/services.php to use your API keys and your default HelpScout mailbox.

Ensure mailbox names within HelpScout correspond to the same mailbox names as Groove. A check will be made before syncing tickets.

### Within HelpScout

Create all of your agent (team, user & mailbox) accounts in HelpScout first. Our tool will need to map the Groove agent 
email addresses with HelpScout user emails and Groove mailboxes to HelpScout mailboxes (manual input may be required).

### CLI Usage

#### Customers

In the root of the Laravel project, run: 
```
php artisan sync-customers
``` 
Customers come first, as the process of creating conversations may create a new customer.

You can also resume a previously stopped migration by passing in a `startPage` parameter:
```
php artisan sync-customers --startPage=10
```

#### Tickets

Once the syncing of customers succeeds, run: 
```
php artisan sync-tickets
```
to migrate Groove tickets, messages, images, attachments and tags.

The `sync-tickets` command also accepts the `startPage` optional parameter as described above.

If, for any reason, a particular ticket fails to migrate (e.g. attachment fails), you can 
redo `sync-tickets` with just that particular ticket number:
  
```
php artisan sync-tickets [<comma-separated list of Groove ticket numbers>] 
```

e.g. `php artisan sync-tickets 1000,2000,3000` will sync over just tickets #1000, 2000, and 3000.

By default, `sync-tickets` will ensure no duplicate tickets are created (keep in mind there is a slight delay before HelpScout picks up newly-created tickets).
You can bypass the duplication check (e.g. on the initial import) by specifying:

```
php artisan sync-tickets --checkDuplicates=false 
```

{'fraction_non_alphanumeric': 0.09046947410840217, 'fraction_numerical': 0.012089462018940157, 'mean_word_length': 4.534002229654403, 'pattern_counts': {'":': 0, '<': 5, '<?xml version=': 0, '>': 9, 'https://': 8, 'lorem ipsum': 0, 'www.': 2, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 4, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '1578445', 'n_tokens_mistral': 1543, 'n_tokens_neox': 1434, 'n_words': 605}