<commit_before># hbase-jruby

*hbase-jruby* is a simple JRuby binding for HBase.

*hbase-jruby* provides the followings:
- Easy, Ruby-esque interface for the fundamental HBase operations
- ActiveRecord-like method chaining for data retrieval
- Automatic Hadoop/HBase dependency resolution

## Installation

    gem install hbase-jruby

### Using hbase-jruby in HBase shell

You can use this gem in HBase shell without external JRuby installation.

First, clone this repository,

```sh
git clone https://github.com/junegunn/hbase-jruby.git
```

then start up the shell (`hbase shell`) and type in the following lines:

```ruby
$LOAD_PATH << 'hbase-jruby/lib'
require 'hbase-jruby'
```

Now, you're all set.

```ruby
# Start using it!
hbase = HBase.new

hbase.list

hbase[:my_table].create! :f
hbase[:my_table].put 100, 'f:a' => 1, 'f:b' => 'two', 'f:c' => 3.14
hbase[:my_table].get(100).double('f:c') # Returns 3.14
```

## A quick example

```ruby
require 'hbase-jruby'

# Load required JAR files from CDH distribution using Maven
HBase.resolve_dependency! 'cdh4.3.0'

# Connect to HBase on localhost
hbase = HBase.new

# Define table schema for easier data access
hbase.schema = {
  # Schema for `book` table
  book: {
    # Columns in cf1 family
    cf1: {
      title:     :string,     # String (UTF-8)
      author:    :string,
      category:  :string,
      year:      :short,      # Short integer (2-byte)
      pages:     :int,        # Integer (4-byte)
      price:     :bigdecimal, # BigDecimal
      weight:    :float,      # Double-precision floating-point number
      in_print:  :boolean,    # Boolean (true | false)
      image:     :raw         # Java byte array; no automatic type conversion
      thumbnail: :byte_array  # HBase::ByteArray
    },
    # Columns in cf2 family
    cf2: {
      summary:  :string,
      reviews:  :fixnum,     # Long integer (8-byte)
      stars:    :fixnum,
      /^comment\d+/ => :string
    }
  }
}

# Create book table with two column families
table = hbase[:book]
unless table.exists?
  table.create! cf1: { min_versions: 2 },
                cf2: { bloomfilter: :rowcol, versions: 5 }
end

# PUT
table.put 1,
  title:     'The Golden Bough: A Study of Magic and Religion',
  author:    '<NAME>',
  category:  'Occult',
  year:      1890,
  pages:     1006,
  price:     BigDecimal('21.50'),
  weight:    3.0,
  in_print:  true,
  image:     File.open('thumbnail.png', 'rb') { |f| f.read }.to_java_bytes,
  summary:   'A wide-ranging, comparative study of mythology and religion',
  reviews:   52,
  stars:     226,
  comment1:  'A must-have',
  comment2:  'Rewarding purchase'

# GET (using schema)
book     = table.get(1)
title    = book[:title]
comment2 = book[:comment2]
as_hash  = book.to_h

# GET (not using schema)
title    = book.string('cf1:title')       # cf:cq notation
year     = book.short('cf1:year')
reviews  = book.fixnum('cf2:reviews')
stars    = book.fixnum(['cf2', 'stars'])  # Array notation of [cf, cq]

# SCAN
table.range(0..100)
     .project(:cf1, :reviews, :summary)
     .filter(year:     1880...1900,
             in_print: true,
             category: ['Comics', 'Fiction', /cult/i],
             price:    { lt: BigDecimal('30.00') },
             summary:  /myth/i)
     .each do |book|

  # Update columns
  table.put book.rowkey, price: book[:price] + BigDecimal('1')

  # Atomic increment
  table.increment book.rowkey, reviews: 1, stars: 5

  # Delete two columns
  table.delete book.rowkey, :comment1, :comment2
end

# Delete row
table.delete 1
```

## Setting up

### Resolving Hadoop/HBase dependency

To be able to access HBase from JRuby, Hadoop/HBase dependency must be satisfied.
This can be done by either setting up CLASSPATH variable beforehand
or by `require`ing relevant JAR files after launching JRuby.

### `HBase.resolve_dependency!`

Well, there's an easier way.
Call `HBase.resolve_dependency!` helper method passing one of the arguments listed below.

| Argument   | Dependency               | Default version  | Required executable |
| ---------- | ------------------------ | ---------------  | ------------------- |
| cdh4.5[.*] | Cloudera CDH4.5          | cdh4.5.0         | mvn                 |
| cdh4.4[.*] | Cloudera CDH4.4          | cdh4.4.0         | mvn                 |
| cdh4.3[.*] | Cloudera CDH4.3          | cdh4.3.2         | mvn                 |
| cdh4.2[.*] | Cloudera CDH4.2          | cdh4.2.2         | mvn                 |
| cdh4.1[.*] | Cloudera CDH4.1          | cdh4.1.5         | mvn                 |
| cdh3[u*]   | Cloudera CDH3            | cdh3u6           | mvn                 |
| 0.96[.*]   | Apache HBase 0.96        | 0.96.1.1-hadoop2 | mvn                 |
| 0.94[.*]   | Apache HBase 0.94        | 0.94.16          | mvn                 |
| 0.92[.*]   | Apache HBase 0.92        | 0.92.2           | mvn                 |
| *POM PATH* | Custom Maven POM file    | -                | mvn                 |
| `:local`   | Local HBase installation | -                | hbase               |

(Default version is used when an argument prefix is given without specific patch version.
 e.g. `cdh4.2` defaults to `cdh4.2.0`)

#### Examples

```ruby
# Load JAR files from CDH4 using Maven
HBase.resolve_dependency! 'cdh4.3.0'
HBase.resolve_dependency! 'cdh4.2'

# Load JAR files of HBase 0.94.x using Maven
HBase.resolve_dependency! '0.94.7'
HBase.resolve_dependency! '0.94.2', verbose: true

# Dependency resolution with custom POM file
HBase.resolve_dependency! '/path/to/my/pom.xml'
HBase.resolve_dependency! '/path/to/my/pom.xml', profile: 'trunk'

# Load JAR files from local HBase installation
# (equivalent to: export CLASSPATH=$CLASSPATH:`hbase classpath`)
HBase.resolve_dependency! :local
```
<commit_msg>Update dependency profile table<commit_after># hbase-jruby

*hbase-jruby* is a simple JRuby binding for HBase.

*hbase-jruby* provides the followings:
- Easy, Ruby-esque interface for the fundamental HBase operations
- ActiveRecord-like method chaining for data retrieval
- Automatic Hadoop/HBase dependency resolution

## Installation

    gem install hbase-jruby

### Using hbase-jruby in HBase shell

You can use this gem in HBase shell without external JRuby installation.

First, clone this repository,

```sh
git clone https://github.com/junegunn/hbase-jruby.git
```

then start up the shell (`hbase shell`) and type in the following lines:

```ruby
$LOAD_PATH << 'hbase-jruby/lib'
require 'hbase-jruby'
```

Now, you're all set.

```ruby
# Start using it!
hbase = HBase.new

hbase.list

hbase[:my_table].create! :f
hbase[:my_table].put 100, 'f:a' => 1, 'f:b' => 'two', 'f:c' => 3.14
hbase[:my_table].get(100).double('f:c') # Returns 3.14
```

## A quick example

```ruby
require 'hbase-jruby'

# Load required JAR files from CDH distribution using Maven
HBase.resolve_dependency! 'cdh4.3.0'

# Connect to HBase on localhost
hbase = HBase.new

# Define table schema for easier data access
hbase.schema = {
  # Schema for `book` table
  book: {
    # Columns in cf1 family
    cf1: {
      title:     :string,     # String (UTF-8)
      author:    :string,
      category:  :string,
      year:      :short,      # Short integer (2-byte)
      pages:     :int,        # Integer (4-byte)
      price:     :bigdecimal, # BigDecimal
      weight:    :float,      # Double-precision floating-point number
      in_print:  :boolean,    # Boolean (true | false)
      image:     :raw         # Java byte array; no automatic type conversion
      thumbnail: :byte_array  # HBase::ByteArray
    },
    # Columns in cf2 family
    cf2: {
      summary:  :string,
      reviews:  :fixnum,     # Long integer (8-byte)
      stars:    :fixnum,
      /^comment\d+/ => :string
    }
  }
}

# Create book table with two column families
table = hbase[:book]
unless table.exists?
  table.create! cf1: { min_versions: 2 },
                cf2: { bloomfilter: :rowcol, versions: 5 }
end

# PUT
table.put 1,
  title:     'The Golden Bough: A Study of Magic and Religion',
  author:    '<NAME>',
  category:  'Occult',
  year:      1890,
  pages:     1006,
  price:     BigDecimal('21.50'),
  weight:    3.0,
  in_print:  true,
  image:     File.open('thumbnail.png', 'rb') { |f| f.read }.to_java_bytes,
  summary:   'A wide-ranging, comparative study of mythology and religion',
  reviews:   52,
  stars:     226,
  comment1:  'A must-have',
  comment2:  'Rewarding purchase'

# GET (using schema)
book     = table.get(1)
title    = book[:title]
comment2 = book[:comment2]
as_hash  = book.to_h

# GET (not using schema)
title    = book.string('cf1:title')       # cf:cq notation
year     = book.short('cf1:year')
reviews  = book.fixnum('cf2:reviews')
stars    = book.fixnum(['cf2', 'stars'])  # Array notation of [cf, cq]

# SCAN
table.range(0..100)
     .project(:cf1, :reviews, :summary)
     .filter(year:     1880...1900,
             in_print: true,
             category: ['Comics', 'Fiction', /cult/i],
             price:    { lt: BigDecimal('30.00') },
             summary:  /myth/i)
     .each do |book|

  # Update columns
  table.put book.rowkey, price: book[:price] + BigDecimal('1')

  # Atomic increment
  table.increment book.rowkey, reviews: 1, stars: 5

  # Delete two columns
  table.delete book.rowkey, :comment1, :comment2
end

# Delete row
table.delete 1
```

## Setting up

### Resolving Hadoop/HBase dependency

To be able to access HBase from JRuby, Hadoop/HBase dependency must be satisfied.
This can be done by either setting up CLASSPATH variable beforehand
or by `require`ing relevant JAR files after launching JRuby.

### `HBase.resolve_dependency!`

Well, there's an easier way.
Call `HBase.resolve_dependency!` helper method passing one of the arguments listed below.

| Argument   | Dependency               | Default version  | Required executable |
| ---------- | ------------------------ | ---------------  | ------------------- |
| cdh4.5[.*] | Cloudera CDH4.5          | cdh4.5.0         | mvn                 |
| cdh4.4[.*] | Cloudera CDH4.4          | cdh4.4.0         | mvn                 |
| cdh4.3[.*] | Cloudera CDH4.3          | cdh4.3.2         | mvn                 |
| cdh4.2[.*] | Cloudera CDH4.2          | cdh4.2.2         | mvn                 |
| cdh4.1[.*] | Cloudera CDH4.1          | cdh4.1.5         | mvn                 |
| cdh3[u*]   | Cloudera CDH3            | cdh3u6           | mvn                 |
| 0.98[.*]   | Apache HBase 0.98        | 0.98.0-hadoop2   | mvn                 |
| 0.96[.*]   | Apache HBase 0.96        | 0.96.1.1-hadoop2 | mvn                 |
| 0.94[.*]   | Apache HBase 0.94        | 0.94.16          | mvn                 |
| 0.92[.*]   | Apache HBase 0.92        | 0.92.2           | mvn                 |
| *POM PATH* | Custom Maven POM file    | -                | mvn                 |
| `:local`   | Local HBase installation | -                | hbase               |

(Default version is used when an argument prefix is given without specific patch version.
 e.g. `cdh4.2` defaults to `cdh4.2.0`)

#### Examples

```ruby
# Load JAR files from CDH4 using Maven
HBase.resolve_dependency! 'cdh4.3.0'
HBase.resolve_dependency! 'cdh4.2'

# Load JAR files of HBase 0.94.x using Maven
HBase.resolve_dependency! '0.94.7'
HBase.resolve_dependency! '0.94.2', verbose: true

# Dependency resolution with custom POM file
HBase.resolve_dependency! '/path/to/my/pom.xml'
HBase.resolve_dependency! '/path/to/my/pom.xml', profile: 'trunk'

# Load JAR files from local HBase installation
# (equivalent to: export CLASSPATH=$CLASSPATH:`hbase classpath`)
HBase.resolve_dependency! :local
```

{'fraction_non_alphanumeric': 0.13747954173486088, 'fraction_numerical': 0.030924282883969333, 'mean_word_length': 2.2778091473743647, 'pattern_counts': {'":': 0, '<': 9, '<?xml version=': 0, '>': 13, 'https://': 2, 'lorem ipsum': 0, 'www.': 0, 'xml': 4}, 'pii_count': 2, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 2, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '5718421', 'n_tokens_mistral': 4203, 'n_tokens_neox': 3915, 'n_words': 1105}