<commit_before>
#Description
Project cnet importer is a c++ implementation of conceptnet v 5.3 to atomspace import.

#Requirements
QT4 or higher

#Usage
1. Download the csv version of Conceptnet database from
   [here](http://conceptnet5.media.mit.edu/downloads/current/)
2. Run `./import.sh -i ` for installing dependencies
3. Run `./import.sh -c /path/to/the/downloaded/conceptnet/tar/file`. The imports
   will be found in the outputs directory.

#Notes
1. Run `./import.sh -r` for the above three steps, under `Usage`, to be
   undertaken for you.
2. It takes more than 20 minutes to run each file on an intel core i7
   3.07GHz with an SSD. So you may want want to download the ready made
   compressed version from [here](http://buildbot.opencog.org/downloads/).
3. For compressing the imports of conceptnet-5.4, run
   `tar -jcvf conceptnet5.4-scm-$(date +'%b-%d-%Y').tar.bz2 outputs/`.
4. If you are compressing for the scheme files to be hosted at the download
   site, add a `README.md` file simillar to the following.
<commit_msg>update the link for downloading conceptnet in atomese scheme files<commit_after>
#Description
Project cnet importer is a c++ implementation of conceptnet v 5.3 to atomspace import.

#Requirements
QT4 or higher

#Usage
1. Download the csv version of Conceptnet database from
   [here](http://conceptnet5.media.mit.edu/downloads/current/)
2. Run `./import.sh -i ` for installing dependencies
3. Run `./import.sh -c /path/to/the/downloaded/conceptnet/tar/file`. The imports
   will be found in the outputs directory.

#Notes
1. Run `./import.sh -r` for the above three steps, under `Usage`, to be
   undertaken for you.
2. It takes more than 20 minutes to run each file on an intel core i7
   3.07GHz with an SSD. So you may want want to download the ready made
   compressed version from [here](https://drive.google.com/file/d/0BwSBtmJFZjVXbHlEQnZ4YXdFU3M/view?usp=sharing).
3. For compressing the imports of conceptnet-5.4, run
   `tar -jcvf conceptnet5.4-scm-$(date +'%b-%d-%Y').tar.bz2 outputs/`.
4. If you are compressing for the scheme files to be hosted at the download
   site, add a `README.md` file simillar to the following.

{'fraction_non_alphanumeric': 0.09935602575896964, 'fraction_numerical': 0.021619135234590615, 'mean_word_length': 3.667381974248927, 'pattern_counts': {'":': 0, '<': 3, '<?xml version=': 0, '>': 3, 'https://': 1, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 12, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '319677', 'n_tokens_mistral': 724, 'n_tokens_neox': 689, 'n_words': 289}