<commit_before># kdb-tree-store

k-dimensional B tree backed to a chunk store

This code is based on the [original kdb tree paper][1] and the algorithm
description in "Data Structures and Algorithms in C++, 4th edition".

For an in-memory version of this algorithm, look at the
[kdb-tree](https://npmjs.com/package/kdb-tree) package.

[1]: http://www.ccs.neu.edu/home/zhoupf/teaching/csu430/paper/kd-b-tree.pdf

# example

``` js
var kdbtree = require('kdb-tree-store')
var fdstore = require('fd-chunk-store')
var tmpdir = require('os').tmpdir()

var path = require('path')
var file = path.join(tmpdir, 'kdb-tree-' + Math.random())

var n = 5000
var kdb = kdbtree({
  types: [ 'float32', 'float32', 'float32', 'uint32' ],
  size: 1024,
  store: fdstore(1024, file)
})
var pending = n
for (var i = 0; i < n; i++) (function () {
  var x = Math.random() * 200 - 100
  var y = Math.random() * 200 - 100
  var z = Math.random() * 200 - 100
  var loc = Math.floor(Math.random() * 1000)
  kdb.insert([x,y,z], loc, function (err) {
    if (--pending === 0) check()
  })
})()

function check () {
  kdb.query([[-100,0],[0,5],[-50,-40]], function (err, pts) {
    console.log(pts)
  })
}
```

# api

``` js
var kdbtree = require('kdb-tree-store')
```

## var kdb = kdbtree(opts)

Create a new kdb tree instance `kdb` given `opts`:

* `opts.types` - array of data types for each dimension plus the payload type at
the end
* `opts.store` - [chunk store](https://npmjs.com/package/abstract-chunk-store) instance
* `opts.size` - size of the chunks in the chunk store
* `opts.available` - next free chunk index to use, set if loading a previously
saved file with data from `'available'` events

## kdb.query(q, opts={}, cb)

Query for results with `q`, an array of `[min,max]` arrays for each dimension.
The results are given as an array of points in `cb(err, results)`. Each element
in `results` has a `point` and `value` property.

Add depth information to each point in the tree by setting `opts.depth = true`.

## var stream = kdb.queryStream(q, opts={})

Return a readable `stream` of query results from the query `q`.

## kdb.insert(pt, value, cb)

Insert `value` (a 32-bit integer) at a point `pt`.

## kdb.on('available', function (n) {})

Index `n` of the next available chunk to use.

Save `n` and pass as `opts.available` to future kdb instances that load from the
same file.

### data types

These data types are provided under string aliases:

* `float` (`float32`)
* `double` (`float64`)
* `uint8`
* `uint16`<commit_msg>document opts.index<commit_after># kdb-tree-store

k-dimensional B tree backed to a chunk store

This code is based on the [original kdb tree paper][1] and the algorithm
description in "Data Structures and Algorithms in C++, 4th edition".

For an in-memory version of this algorithm, look at the
[kdb-tree](https://npmjs.com/package/kdb-tree) package.

[1]: http://www.ccs.neu.edu/home/zhoupf/teaching/csu430/paper/kd-b-tree.pdf

# example

``` js
var kdbtree = require('kdb-tree-store')
var fdstore = require('fd-chunk-store')
var tmpdir = require('os').tmpdir()

var path = require('path')
var file = path.join(tmpdir, 'kdb-tree-' + Math.random())

var n = 5000
var kdb = kdbtree({
  types: [ 'float32', 'float32', 'float32', 'uint32' ],
  size: 1024,
  store: fdstore(1024, file)
})
var pending = n
for (var i = 0; i < n; i++) (function () {
  var x = Math.random() * 200 - 100
  var y = Math.random() * 200 - 100
  var z = Math.random() * 200 - 100
  var loc = Math.floor(Math.random() * 1000)
  kdb.insert([x,y,z], loc, function (err) {
    if (--pending === 0) check()
  })
})()

function check () {
  kdb.query([[-100,0],[0,5],[-50,-40]], function (err, pts) {
    console.log(pts)
  })
}
```

# api

``` js
var kdbtree = require('kdb-tree-store')
```

## var kdb = kdbtree(opts)

Create a new kdb tree instance `kdb` given `opts`:

* `opts.types` - array of data types for each dimension plus the payload type at
the end
* `opts.store` - [chunk store](https://npmjs.com/package/abstract-chunk-store) instance
* `opts.size` - size of the chunks in the chunk store
* `opts.available` - next free chunk index to use, set if loading a previously
saved file with data from `'available'` events

## kdb.query(q, opts={}, cb)

Query for results with `q`, an array of `[min,max]` arrays for each dimension.
The results are given as an array of points in `cb(err, results)`. Each element
in `results` has a `point` and `value` property.

* `opts.depth` - add depth information to each matching point when true in a
`depth` property (default: `false`)
* `opts.index` - add `[chunkIndex,pointIndex]` pairs to each matching point when
true in an `index` property (default: `false`)

## var stream = kdb.queryStream(q, opts={})

Return a readable `stream` of query results from the query `q`.

## kdb.insert(pt, value, cb)

Insert `value` (a 32-bit integer) at a point `pt`.

## kdb.on('available', function (n) {})

Index `n` of the next available chunk to use.

Save `n` and pass as `opts.available` to future kdb instances that load from the
same file.

### data types

These data types are provided under string aliases:

* `float` (`float32`)
* `double` (`float64`)
* `uint8`
* `uint16`
{'fraction_non_alphanumeric': 0.15619596541786743, 'fraction_numerical': 0.026512968299711816, 'mean_word_length': 3.8974600188146753, 'pattern_counts': {'":': 0, '<': 5, '<?xml version=': 0, '>': 3, 'https://': 4, 'lorem ipsum': 0, 'www.': 2, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '6837947', 'n_tokens_mistral': 1909, 'n_tokens_neox': 1786, 'n_words': 645}