<commit_before># low_card_tables

Greatly improve scalability and maintainability of your database tables by breaking out columns containing few distinct values (e.g., booleans and other flags) into a separate table that's transparently referenced and used. Supports Rails 3.0.x, 3.1.x, 3.2.x, and 4.0.x, running on Ruby 1.8.7, 1.9.3, and 2.0.0 with MySQL, PostgreSQL, and Sqlite. (JRuby is supported, but only with MySQL, because `low_card_tables` depends on the `activerecord-import` gem, and it currently does not have JRuby support for anything but MySQL.) Adding support for other databases is trivial!

`low_card_tables` is short for "low-cardinality tables". Cardinality, when applied to a database column, is the measure of the number of distinct values that column can hold. This Gem is meant to be used for columns that hold few distinct values throughout the table &mdash; hence, they have low cardinality.

Current build status: ![Current Build Status](https://api.travis-ci.org/ageweke/low_card_tables.png?branch=master)

===
# Documentation is on [the Wiki](https://github.com/ageweke/low_card_tables/wiki)!

This file would be incredibly long if it contained all the information present there. A quickstart guide is below;
see the Wiki for everything else.

===

### Installing low_card_tables

	# Gemfile
	gem 'low_card_tables'

### Getting Started
<commit_msg>Update to README.<commit_after># low_card_tables

Greatly improve scalability and maintainability of your database tables by breaking out columns containing few distinct values (e.g., booleans and other flags) into a separate table that's transparently referenced and used. Supports Rails 3.0.x, 3.1.x, 3.2.x, and 4.0.x, running on Ruby 1.8.7, 1.9.3, and 2.0.0 with MySQL, PostgreSQL, and Sqlite. (JRuby is supported, but only with MySQL, because `low_card_tables` depends on the `activerecord-import` gem, and it currently does not have JRuby support for anything but MySQL.) Adding support for other databases is trivial!

`low_card_tables` is the successor to similar, but more primitive, systems that have been in place at very large commercial websites serving tens of millions of pages a day, and in database tables with hundreds of millions of rows. The predecessor systems were extremely successful and reliable &mdash; hence the desire to evolve this into an open-source gem.

`low_card_tables` is short for "low-cardinality tables". Cardinality, when applied to a database column, is the measure of the number of distinct values that column can hold. This Gem is meant to be used for columns that hold few distinct values throughout the table &mdash; hence, they have low cardinality.

Current build status: ![Current Build Status](https://api.travis-ci.org/ageweke/low_card_tables.png?branch=master)

===
# Documentation is on [the Wiki](https://github.com/ageweke/low_card_tables/wiki)!

This file would be incredibly long if it contained all the information present there. A quickstart guide is below;
see the Wiki for everything else.

===

### Installing low_card_tables

	# Gemfile
	gem 'low_card_tables'

### Getting Started

{'fraction_non_alphanumeric': 0.07213850593138826, 'fraction_numerical': 0.010900929785187561, 'mean_word_length': 4.853658536585366, 'pattern_counts': {'":': 0, '<': 3, '<?xml version=': 0, '>': 3, 'https://': 4, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '3327618', 'n_tokens_mistral': 900, 'n_tokens_neox': 840, 'n_words': 413}