<commit_before># cascading.cassandra

A [Cascading](http://www.cascading.org) Scheme and Tap for [Cassandra](http://cassandra.apache.org).

## Installing

The project can be built using [Maven](http://maven.apache.org) and installed into your local repository:

    mvn install

## Restrictions

This is very much an early Work-in-Progress. The Tap can only be used as a sink and only supports regular column families (it won't work with Super-Columns or Counter Columns).

## Usage

`cascading.cassandra` is heavily influenced by [cascading.hbase](https://github.com/cwensel/cascading.hbase) and uses Cassandra's built-in `ColumnFamilyOutputFormat`.

First, create a `CassandraScheme` and specify the field to be used as the row key (currently the Scheme will only support single-field keys). The second parameter is an array of Fields that represent the columns you wish to store. The column names will be serialized from the name provided and the values will come from the Tuples during the flow.

    Fields keyFields = new Fields("num");
    Fields[] valueFields = new Fields[] {new Fields("lower"), new Fields("upper")};
    CassandraScheme scheme = new CassandraScheme(keyFields, valueFields);

Finally, hook the `CassandraScheme` into a `CassandraTap` and provide the Cassandra Thrift RPC Host and Port that the `ColumnFamilyOutputFormat` should connect to, as well as the keyspace and column family names you wish to store values for.

    Tap sink = new CassandraTap(getRpcHost(), getRpcPort(), keyspaceName, columnFamilyName, scheme);

## License

Licensed under the Apache 2.0 license.

## Copyright

Copyright &copy; <NAME>, 2011.
<commit_msg>Add a note about the conjars repository<commit_after># cascading.cassandra

A [Cascading](http://www.cascading.org) Scheme and Tap for [Cassandra](http://cassandra.apache.org).

## Installing

The project can be built using [Maven](http://maven.apache.org) and installed into your local repository:

    mvn install

Alternatively, there's a snapshot available from [Conjars](http://conjars.org/cascading.pingles/cascading.cassandra).

### Leiningen

    [cascading.pingles/cascading.cassandra "0.0.1-SNAPSHOT"]
    
and add the repository into your `project.clj` with

    :repositories {"conjars" "http://conjars.org/repo"}

### Maven

    <dependency>
      <groupId>cascading.pingles</groupId>
      <artifactId>cascading.cassandra</artifactId>
      <version>0.0.1-SNAPSHOT</version>
    </dependency>

If you haven't already added it you'll need to add the Conjars repository

    <repositories>
      <repository>
        <id>conjars</id>
        <url>http://conjars.org/repo</url>
      </repository>
    </repositories>


## Restrictions

This is very much an early Work-in-Progress. The Tap can only be used as a sink and only supports regular column families (it won't work with Super-Columns or Counter Columns).

## Usage

`cascading.cassandra` is heavily influenced by [cascading.hbase](https://github.com/cwensel/cascading.hbase) and uses Cassandra's built-in `ColumnFamilyOutputFormat`.

First, create a `CassandraScheme` and specify the field to be used as the row key (currently the Scheme will only support single-field keys). The second parameter is an array of Fields that represent the columns you wish to store. The column names will be serialized from the name provided and the values will come from the Tuples during the flow.

    Fields keyFields = new Fields("num");
    Fields[] valueFields = new Fields[] {new Fields("lower"), new Fields("upper")};
    CassandraScheme scheme = new CassandraScheme(keyFields, valueFields);

Finally, hook the `CassandraScheme` into a `CassandraTap` and provide the Cassandra Thrift RPC Host and Port that the `ColumnFamilyOutputFormat` should connect to, as well as the keyspace and column family names you wish to store values for.

    Tap sink = new CassandraTap(getRpcHost(), getRpcPort(), keyspaceName, columnFamilyName, scheme);

## License

Licensed under the Apache 2.0 license.

## Copyright

Copyright &copy; <NAME>, 2011.

{'fraction_non_alphanumeric': 0.09230009871668311, 'fraction_numerical': 0.004442250740375123, 'mean_word_length': 4.454912516823688, 'pattern_counts': {'":': 0, '<': 21, '<?xml version=': 0, '>': 21, 'https://': 2, 'lorem ipsum': 0, 'www.': 2, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '2025829', 'n_tokens_mistral': 1226, 'n_tokens_neox': 1140, 'n_words': 456}