<commit_before># trindex

[![GoDoc](https://godoc.org/github.com/flosch/trindex?status.png)](https://godoc.org/github.com/flosch/trindex)
[![Build Status](https://travis-ci.org/flosch/trindex.svg?branch=master)](https://travis-ci.org/flosch/trindex)
[![gratipay](http://img.shields.io/badge/gratipay-support%20trindex-brightgreen.svg)](https://gratipay.com/flosch/)

trindex is a trigram search library for terms written for and in Go (in alpha stage!). It provides a very simple API
and ships with its own database.

I put up a demo page online using trindex. I indexed all German wikidata lemmas (4064962 titles in total) and
made them available for search: https://www.florian-schlachter.de/trindex/

The Wikidata example (build & query) is in the repository as well: [`examples/wikidata`](https://github.com/flosch/trindex/tree/master/examples/wikidata).

```go
idx := trindex.NewIndex("trindex.db")
defer idx.Close()

dataset := []string{
    "Mallorca", "Ibiza", "Menorca", "Pityusen", "Formentera", 
    "Berlin", "New York", "Yorkshire",
}

for _, data := range dataset {
    id := idx.Insert(data)
    // Use ID to connect the term with the associated dataset;
    // for example save the ID in your SQL database about travel destinations
}

results := idx.Query("malorka", 3)

// Returns a sorted list of 3 results including the ID and
// a confidence number ("Similarity"; 1 = best match) 
```

trindex relies heavily on caching; it's API is safe for concurrent use. Please make sure that you'll call `idx.Close()`
**in any case** on application shutdown (to flush inserted data to disk).

## Related blog posts

 * [trindex: A trigram search library for Go](https://www.florian-schlachter.de/post/trindex/) [26th Oct 2014]<commit_msg>README fix<commit_after># trindex

[![GoDoc](https://godoc.org/github.com/flosch/trindex?status.png)](https://godoc.org/github.com/flosch/trindex)
[![Build Status](https://travis-ci.org/flosch/trindex.svg?branch=master)](https://travis-ci.org/flosch/trindex)
[![gratipay](http://img.shields.io/badge/gratipay-support%20trindex-brightgreen.svg)](https://gratipay.com/flosch/)

trindex is a trigram search library for terms written for and in Go (in alpha stage!). It provides a very simple API
and ships with its own database.

I put up a demo page online using trindex. I indexed all German wikidata lemmas (4064962 titles in total) and
made them available for search: https://www.florian-schlachter.de/trindex/

The Wikidata example (build & query) is in the repository as well: [`examples/wikidata`](https://github.com/flosch/trindex/tree/master/examples/wikidata).

```go
idx := trindex.NewIndex("trindex.db")
defer idx.Close()

dataset := []string{
    "Mallorca", "Ibiza", "Menorca", "Pityusen", "Formentera", 
    "Berlin", "New York", "Yorkshire",
}

for _, data := range dataset {
    id := idx.Insert(data)
    // Use ID to connect the term with the associated dataset;
    // for example save the ID in your SQL database about travel destinations
}

results := idx.Query("malorka", 3, 0.3)

// Returns a sorted list of 3 results including the ID and
// a confidence number ("Similarity"; 1 = best match) >= 0.3 
```

trindex relies heavily on caching; it's API is safe for concurrent use. Please make sure that you'll call `idx.Close()`
**in any case** on application shutdown (to flush inserted data to disk).

## Related blog posts

 * [trindex: A trigram search library for Go](https://www.florian-schlachter.de/post/trindex/) [26th Oct 2014]
{'fraction_non_alphanumeric': 0.1368180519851471, 'fraction_numerical': 0.011425307055127107, 'mean_word_length': 4.412673879443585, 'pattern_counts': {'":': 0, '<': 3, '<?xml version=': 0, '>': 4, 'https://': 16, 'lorem ipsum': 0, 'www.': 4, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '3152343', 'n_tokens_mistral': 1206, 'n_tokens_neox': 1119, 'n_words': 364}