<commit_before>crowdbase-hackpad-importer
==========================

small script to import your crowdbase files to hackpad<commit_msg>added readme<commit_after>crowdbase-hackpad-importer
==========================

Small script to import your crowdbase files to hackpad.

## Usage
First, export your crowdbase documents. As an admin, you can download them to your computer as a zip file. Unzip the file.

Go to your hackpad account and view your profile to get your Hackpad API clientId and secret. You will need both of these.

Run the following command, specifying your clientId, secret, and crowdbase directory:
```bash
./import.js -i CLIENTID -t SECRET -p /path/to/crowdbase/directory
```

The script will only import files with valid extensions: `.html`,`.md`, and `.txt` 

If you are using a private hackpad subdomain, you can specify it with the -s option
```bash
./import.js -i CLIENTID -t SECRET -p /path/to/crowdbase/directory -s mycompany.hackpad.com
```

If you'd like to see what files would be imported, but not actually import them yet, you can use the `--dryrun` flag
```bash
./import.js -i CLIENTID -t SECRET -p /path/to/crowdbase/directory -s mycompany.hackpad.com --dryrun
```


To see additional details of how to use this import script
```bash
./import.js --help
```
{'fraction_non_alphanumeric': 0.12723041117145073, 'fraction_numerical': 0.0, 'mean_word_length': 4.222672064777328, 'pattern_counts': {'":': 0, '<': 3, '<?xml version=': 0, '>': 3, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 1, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '2383907', 'n_tokens_mistral': 410, 'n_tokens_neox': 373, 'n_words': 170}