<commit_before># Hetchy

TODO: Write a gem description

## Installation

Add this line to your application's Gemfile:

```ruby
gem 'hetchy'
```

And then execute:

    $ bundle

Or install it yourself as:

    $ gem install hetchy

## Usage

TODO: Write usage instructions here

## Contributing

1. Fork it ( https://github.com/[my-github-username]/hetchy/fork )
2. Create your feature branch (`git checkout -b my-new-feature`)
3. Commit your changes (`git commit -am 'Add some feature'`)
4. Push to the branch (`git push origin my-new-feature`)
5. Create a new Pull Request
<commit_msg>Write initial draft of interface, flesh out README<commit_after># Hetchy

A high performance, thread-safe reservoir sampler with snapshot and percentile support.

Each reservoir can accepts an arbitrary number of samples but will consume a limited amount of memory based on the reservoir's size.

## Installation

Add this line to your application's Gemfile:

```ruby
gem 'hetchy'
```

And then execute:

    $ bundle

Or install it yourself as:

    $ gem install hetchy

## Usage


Create a reservoir and designate how big you want it to be:

```ruby
reservoir = Hetchy.new(size: 1000)
```

Add samples as they arrive or are generated:

```ruby
reservoir << 123
```

You can add sets of samples as well:

```ruby
reservoir << [45,89,124,96]
```

You can calculate a percentile at any time:

```ruby
reservoir.percentile(95)
```

Hetchy supports high resolution percentiles as well:

```ruby
reservoir.percentile(99.99)
```

NOTE: you may need to increase your reservoir size for very high resolution percentiles

For threaded applications where the reservoir is accepting samples rapidly you can increase performance by snapshotting before running calculations on the reservoir:

```ruby
snapshot = reservoir.snapshot
perc_95 = snapshot.percentile(95)
```

If you want to reset the reservoir just empty it:

```ruby
reservoir.empty
```

## Contributing

* Check out the latest master to make sure the feature hasn't been implemented or the bug hasn't been fixed yet.
* Check out the issue tracker to make sure someone already hasn't requested it and/or contributed it.
* Fork the project and submit a pull request from a feature or bugfix branch.
* Please include tests. This is important so we don't break your changes unintentionally in a future version.
* Please don't modify the gemspec, Rakefile, version, or changelog. If you do change these files, please isolate a separate commit so we can cherry-pick around it.

## Credits

Parts of Hetchy are inspired by <NAME>'s [metriks]() gem.

## Copyright

Copyright (c) 2015 <NAME>. See LICENSE for details.


{'fraction_non_alphanumeric': 0.08040770101925254, 'fraction_numerical': 0.01321253303133258, 'mean_word_length': 4.3, 'pattern_counts': {'":': 0, '<': 9, '<?xml version=': 0, '>': 5, 'https://': 1, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '388026', 'n_tokens_mistral': 825, 'n_tokens_neox': 733, 'n_words': 364}