<commit_before>[![Build Status](https://travis-ci.org/helioid/categorize.png?branch=master)](https://travis-ci.org/helioid/categorize)

# Categorize

**Categorize** is a text categorization library in Ruby.  It prioritizes
performance over accuracy and is built to run online in dynamic web services.

## Installation

**Categorize** is installable as a [Ruby Gem](https://rubygems.org/gems/categorize):
```base
$ gem install categorize
```

## Basic Usage

```ruby
documents = {
  "doc0" => ["lorem", "ipsum", "dolor"],
  "doc1" => ["sed", "perspiciatis", "unde"],
  "doc2" => ["vero", "eos", "accusamus"],
  "doc3" => ["vero", "eos", "accusamus", "iusto", "odio"]
}

Categorize.make_model('lorem', documents)
=> {
  "ipsum"            => ["doc0"],
  "sed perspiciatis" => ["doc1"],
  "vero"             => ["doc2", "doc3"]
}
```

## Ownership

Copyright (c) 2013, Helioid Inc. and <NAME>. Licensed under the 3-clause BSD License, see file LICENSE for details.<commit_msg>use single quotes consistently<commit_after>[![Build Status](https://travis-ci.org/helioid/categorize.png?branch=master)](https://travis-ci.org/helioid/categorize)

# Categorize

**Categorize** is a text categorization library in Ruby.  It prioritizes
performance over accuracy and is built to run online in dynamic web services.

## Installation

**Categorize** is installable as a [Ruby Gem](https://rubygems.org/gems/categorize):
```base
$ gem install categorize
```

## Basic Usage

```ruby
documents = {
  'doc0' => ['lorem', 'ipsum', 'dolor'],
  'doc1' => ['sed', 'perspiciatis', 'unde'],
  'doc2' => ['vero', 'eos', 'accusamus'],
  'doc3' => ['vero', 'eos', 'accusamus', 'iusto', 'odio']
}

Categorize.make_model('lorem', documents)
=> {
  'ipsum'            => ['doc0'],
  'sed perspiciatis' => ['doc1'],
  'vero'             => ['doc2', 'doc3']
}
```

## Ownership

Copyright (c) 2013, Helioid Inc. and <NAME>. Licensed under the 3-clause BSD License, see file LICENSE for details.
{'fraction_non_alphanumeric': 0.19052470708099847, 'fraction_numerical': 0.013245033112582781, 'mean_word_length': 4.395604395604396, 'pattern_counts': {'":': 0, '<': 5, '<?xml version=': 0, '>': 21, 'https://': 6, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '156547', 'n_tokens_mistral': 731, 'n_tokens_neox': 705, 'n_words': 180}