<commit_before># Punc
Read a file, remove the words, see all the punctuations

![image](https://cloud.githubusercontent.com/assets/3915598/13369726/783be3d0-dc9c-11e5-846a-5f1ec6517966.png)

Ever wonder what your favorite books look like without words?

## Installation
`npm install punc`

## Usage
Please see the [wiki][WIKI].

## Notes
  - Punctuations to keep when removing words:

    `; : ' " , ! ? . ( ) -`

  - Punctuations to use when counting occurances:

    `; : ' " , ! ? .`

### Inspiration
*inspired by [this article][1] on medium which was in-turn inspired by [this article][2]*

## License
[MIT][LICENSE]

<!-- urls -->

[1]: https://medium.com/@neuroecology/punctuation-in-novels-8f316d542ec4#.6e7lvvwp8

[2]: http://www.c82.net/work/?id=347

[WIKI]: https://github.com/sgnl/punc/wiki/Usage

[LICENSE]: https://github.com/sgnl/punc/blob/master/LICENSE<commit_msg>updates copy<commit_after># Punc
Read a file, remove the words, see all the punctuations

![image](https://cloud.githubusercontent.com/assets/3915598/13369726/783be3d0-dc9c-11e5-846a-5f1ec6517966.png)

Ever wonder what your favorite books look like without words?

## Installation
`npm install punc`

## Usage
Please see the [wiki][WIKI].

## Notes
  - Punctuations to keep when removing words:

    `; : ' " , ! ? . ( ) -`

  - Punctuations to use when counting occurances:

    `; : ' " , ! ? .`

## Inspiration
*inspired by [this article][1] on medium which was in-turn inspired by [this article][2]*

## License
[MIT][LICENSE]

<!-- urls -->

[1]: https://medium.com/@neuroecology/punctuation-in-novels-8f316d542ec4#.6e7lvvwp8

[2]: http://www.c82.net/work/?id=347

[WIKI]: https://github.com/sgnl/punc/wiki/Usage

[LICENSE]: https://github.com/sgnl/punc/blob/master/LICENSE
{'fraction_non_alphanumeric': 0.1621160409556314, 'fraction_numerical': 0.06370875995449374, 'mean_word_length': 3.9134078212290504, 'pattern_counts': {'":': 0, '<': 5, '<?xml version=': 0, '>': 5, 'https://': 8, 'lorem ipsum': 0, 'www.': 2, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 2, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '3559715', 'n_tokens_mistral': 719, 'n_tokens_neox': 654, 'n_words': 140}