<commit_before><p align="center">
<img 
    src="logo.png" 
    width="307" height="150" border="0" alt="BuntDB">
<br>
<a href="https://travis-ci.org/tidwall/buntdb"><img src="https://travis-ci.org/tidwall/buntdb.svg?branch=master" alt="Build Status"></a>
<img src="https://img.shields.io/badge/coverage-96%25-green.svg?style=flat" alt="Code Coverage">
<a href="https://godoc.org/github.com/tidwall/buntdb"><img src="https://godoc.org/github.com/tidwall/buntdb?status.svg" alt="GoDoc"></a>
<img src="https://img.shields.io/badge/version-0.1.0-green.svg" alt="Version">
</p>

====

BuntDB is a low-level, in-memory, key/value store in pure Go. 
It persists to disk, is ACID compliant, and uses locking for multiple
readers and a single writer. It supports custom indexes and geospatial 
data. It's ideal for projects that need a dependable database and favor 
speed over data size.

The desire to create BuntDB stems from the need for a new embeddable
database for [Tile38](https://github.com/tidwall/tile38). One that can work 
both as a performant [Raft Store](https://github.com/tidwall/raft-boltdb), 
and a Geospatial database.

Much of the API is inspired by the fantastic [BoltDB](https://github.com/boltdb/bolt),
an amazing key/value store that can handle terabytes of data on disk.

Features
========

- In-memory database for [fast reads and writes](https://github.com/tidwall/raft-boltdb#benchmarks)
- Embeddable with a [simple API](https://godoc.org/github.com/tidwall/buntdb)
- [Spatial indexing](#spatial-indexes) for up to 20 dimensions; Useful for Geospatial data
- Create [custom indexes](#custom-indexes) for any data type
- [Built-in types](#built-in-types) that are easy to get up & running; String, Uint, Int, Float
- Flexible [iteration](#iterating) of data; ascending, descending, and ranges
- [Durable append-only file](#append-only-file) format for persistence. 
- Option to evict old items with an [expiration](#data-expiration) TTL
- Tight codebase, under 1K loc using the `cloc` command
- ACID semantics with locking [transactions](#transactions) that support rollbacks

Getting Started
===============

## Installing

To start using BuntDB, install Go and run `go get`:

```sh
$ go get github.com/tidwall/buntdb
```

This will retrieve the library.


## Opening a database

The primary object in BuntDB is a `DB`. To open or create your 
database, use the `buntdb.Open()` function:

```go
package main

import (
	"log"

	"github.com/tidwall/buntdb"
)

func main() {
	// Open the data.db file. It will be created if it doesn't exist.
	db, err := buntdb.Open("data.db")
	if err != nil {
		log.Fatal(err)
	}
	defer db.Close()
	
	... 
}
```

## Transactions
All reads and writes must be performed from inside a transaction. BuntDB can have one write transaction opened at a time, but can have many concurrent read transactions. Each transaction maintains a stable view of the database. In other words, once a transaction has begun, the data for that transaction cannot be changed by other transactions.

Transactions run in a function that exposes a `Tx` object, which represents the transaction state. While inside a transaction, all database operations should be performed using this object. You should never access the origin `DB` object while inside a transaction. Doing so may have side-effects, such as blocking your application.

When a transaction fails, it will roll back, and revert all changes that occurred to the database during that transaction. There's a single return value that you can use to close the transaction. For read/write transactions, returning an error this way will force the transaction to roll back. When a read/write transaction succeeds all changes are persisted to disk.

### Read-only Transactions
A read-only transaction should be used when you don't need to make changes to the data. The advantage of a read-only transaction is that there can be many running concurrently.

```go
err := db.View(func(tx *buntdb.Tx) error {
    ...
    return nil
})
```

### Read/write Transactions
A read/write transaction is used when you need to make changes to your data. There can only be one read/write transaction running at a time. So make sure you close it as soon as you are done with it. 

```go
err := db.Update(func(tx *buntdb.Tx) error {
    ...
    return nil
})
```

## Setting and getting key/values

To set a value you must open a read/write transaction:

```go
err := db.Update(func(tx *buntdb.Tx) error {
    err := tx.Set("mykey", "myvalue", nil)
    return err
})
```


To get the value:

```go
err := db.View(func(tx *buntdb.Tx) error {
    val, err := tx.Get("mykey") 
    if err != nil{
    	return err
    }
    fmt.Printf("value is %s\n", val) 
    return nil
})
```

Getting non-existent values will case an `ErrNotFound` error.

### Iterating
All keys/value pairs are ordered in the database by the key. To iterate over the keys:

```go
err := db.View(func(tx *buntdb.Tx) error {
err := tx.Ascend("", func(key, value string) bool{
    	fmt.Printf("key: %s, value: %s\n", key, value)
    })
    return err
})
```

There is also `AscendGreaterOrEqual`, `AscendLessThan`, `AscendRange`, `Descend`, `DescendLessOrEqual`, `DescendGreaterThan`, and `DescendRange`. Please see the [documentation](https://godoc.org/github.com/tidwall/buntdb) for more information on these functions.


## Custom Indexes
Initially all data is stored in a single [B-tree](https://en.wikipedia.org/wiki/B-tree) with each item having one key and one value. All of these items are ordered by the key. This is great for quickly getting a value from a key or [iterating](#iterating) over the keys.

You can also create custom indexes that allow for ordering and [iterating](#iterating) over values. A custom index also uses a B-tree, but it's more flexible because it allows for custom ordering.

For example, let's say you want to create an index for ordering names:

```go
db.CreateIndex("names", "*", buntdb.IndexString)
```

This will create an index named `names` which stores and sorts all values. The second parameter is a pattern that is used to filter on keys. A `*` wildcard argument means that we want to accept all keys. `IndexString` is a built-in function that performs case-insensitive ordering on the values

Now you can add various names:

```go<commit_msg>mention btree implementation<commit_after><p align="center">
<img 
    src="logo.png" 
    width="307" height="150" border="0" alt="BuntDB">
<br>
<a href="https://travis-ci.org/tidwall/buntdb"><img src="https://travis-ci.org/tidwall/buntdb.svg?branch=master" alt="Build Status"></a>
<img src="https://img.shields.io/badge/coverage-96%25-green.svg?style=flat" alt="Code Coverage">
<a href="https://godoc.org/github.com/tidwall/buntdb"><img src="https://godoc.org/github.com/tidwall/buntdb?status.svg" alt="GoDoc"></a>
<img src="https://img.shields.io/badge/version-0.1.0-green.svg" alt="Version">
</p>

====

BuntDB is a low-level, in-memory, key/value store in pure Go. 
It persists to disk, is ACID compliant, and uses locking for multiple
readers and a single writer. It supports custom indexes and geospatial 
data. It's ideal for projects that need a dependable database and favor 
speed over data size.

The desire to create BuntDB stems from the need for a new embeddable
database for [Tile38](https://github.com/tidwall/tile38). One that can work 
both as a performant [Raft Store](https://github.com/tidwall/raft-boltdb), 
and a Geospatial database.

Much of the API is inspired by the fantastic [BoltDB](https://github.com/boltdb/bolt),
an amazing key/value store that can handle terabytes of data on disk.

Features
========

- In-memory database for [fast reads and writes](https://github.com/tidwall/raft-boltdb#benchmarks)
- Embeddable with a [simple API](https://godoc.org/github.com/tidwall/buntdb)
- [Spatial indexing](#spatial-indexes) for up to 20 dimensions; Useful for Geospatial data
- Create [custom indexes](#custom-indexes) for any data type
- [Built-in types](#built-in-types) that are easy to get up & running; String, Uint, Int, Float
- Flexible [iteration](#iterating) of data; ascending, descending, and ranges
- [Durable append-only file](#append-only-file) format for persistence. 
- Option to evict old items with an [expiration](#data-expiration) TTL
- Tight codebase, under 1K loc using the `cloc` command
- ACID semantics with locking [transactions](#transactions) that support rollbacks

Getting Started
===============

## Installing

To start using BuntDB, install Go and run `go get`:

```sh
$ go get github.com/tidwall/buntdb
```

This will retrieve the library.


## Opening a database

The primary object in BuntDB is a `DB`. To open or create your 
database, use the `buntdb.Open()` function:

```go
package main

import (
	"log"

	"github.com/tidwall/buntdb"
)

func main() {
	// Open the data.db file. It will be created if it doesn't exist.
	db, err := buntdb.Open("data.db")
	if err != nil {
		log.Fatal(err)
	}
	defer db.Close()
	
	... 
}
```

## Transactions
All reads and writes must be performed from inside a transaction. BuntDB can have one write transaction opened at a time, but can have many concurrent read transactions. Each transaction maintains a stable view of the database. In other words, once a transaction has begun, the data for that transaction cannot be changed by other transactions.

Transactions run in a function that exposes a `Tx` object, which represents the transaction state. While inside a transaction, all database operations should be performed using this object. You should never access the origin `DB` object while inside a transaction. Doing so may have side-effects, such as blocking your application.

When a transaction fails, it will roll back, and revert all changes that occurred to the database during that transaction. There's a single return value that you can use to close the transaction. For read/write transactions, returning an error this way will force the transaction to roll back. When a read/write transaction succeeds all changes are persisted to disk.

### Read-only Transactions
A read-only transaction should be used when you don't need to make changes to the data. The advantage of a read-only transaction is that there can be many running concurrently.

```go
err := db.View(func(tx *buntdb.Tx) error {
    ...
    return nil
})
```

### Read/write Transactions
A read/write transaction is used when you need to make changes to your data. There can only be one read/write transaction running at a time. So make sure you close it as soon as you are done with it. 

```go
err := db.Update(func(tx *buntdb.Tx) error {
    ...
    return nil
})
```

## Setting and getting key/values

To set a value you must open a read/write transaction:

```go
err := db.Update(func(tx *buntdb.Tx) error {
    err := tx.Set("mykey", "myvalue", nil)
    return err
})
```


To get the value:

```go
err := db.View(func(tx *buntdb.Tx) error {
    val, err := tx.Get("mykey") 
    if err != nil{
    	return err
    }
    fmt.Printf("value is %s\n", val) 
    return nil
})
```

Getting non-existent values will case an `ErrNotFound` error.

### Iterating
All keys/value pairs are ordered in the database by the key. To iterate over the keys:

```go
err := db.View(func(tx *buntdb.Tx) error {
err := tx.Ascend("", func(key, value string) bool{
    	fmt.Printf("key: %s, value: %s\n", key, value)
    })
    return err
})
```

There is also `AscendGreaterOrEqual`, `AscendLessThan`, `AscendRange`, `Descend`, `DescendLessOrEqual`, `DescendGreaterThan`, and `DescendRange`. Please see the [documentation](https://godoc.org/github.com/tidwall/buntdb) for more information on these functions.


## Custom Indexes
Initially all data is stored in a single [B-tree](https://en.wikipedia.org/wiki/B-tree) with each item having one key and one value. All of these items are ordered by the key. This is great for quickly getting a value from a key or [iterating](#iterating) over the keys. Feel free to take a peruse at the [B-tree implementation](https://github.com/tidwall/btree) that BuntDB uses.

You can also create custom indexes that allow for ordering and [iterating](#iterating) over values. A custom index also uses a B-tree, but it's more flexible because it allows for custom ordering.

For example, let's say you want to create an index for ordering names:

```go
db.CreateIndex("names", "*", buntdb.IndexString)
```

This will create an index named `names` which stores and sorts all values. The second parameter is a pattern that is used to filter on keys. A `*` wildcard argument means that we want to accept all keys. `IndexString` is a built-in function that performs case-insensitive ordering on the values

Now you can add various names:

```go
{'fraction_non_alphanumeric': 0.11054488435907488, 'fraction_numerical': 0.003292826342610741, 'mean_word_length': 4.183258837870785, 'pattern_counts': {'":': 0, '<': 27, '<?xml version=': 0, '>': 27, 'https://': 27, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '1477270', 'n_tokens_mistral': 3984, 'n_tokens_neox': 3753, 'n_words': 1610}