<commit_before>QLBridge - a Go SQL Rumtine Engine
====================================================

This is a SQL script VM to process data with sql for embedded use, 
includes a native go lexer, parser.  Extend with native go functions.


### QLBridge Features and Goals
* execution of sql queries against your data, embedable, not coupled to storage layer
* extend VM with custom go functions, provide rich basic library of functions
* provide example backends (csv, elasticsearch, etc)

### Example VM Runtime for Reading a Csv via Stdio, and evaluating


See example in [qlcsv](https://github.com/araddon/qlbridge/tree/master/examples/qlcsv)
folder for a CSV reader, parser, evaluation engine.

```sh

./qlcsv -sql 'select 
		user_id, email, item_count * 2, yy(reg_date) > 10 
	FROM stdio' < users.csv

```
```go

func main() {

	// load all of our built-in functions
	builtins.LoadAllBuiltins()

	// Add a custom function to the VM to make available to SQL language
	expr.FuncAdd("email_is_valid", EmailIsValid)
<commit_msg>spelling fixes<commit_after>QLBridge - a Go SQL Runtime Engine
====================================================

This is a SQL execution engine to process data with sql for embedded use, 
includes a native go lexer, parser.  Extend with native go functions.


### QLBridge Features and Goals
* execution of sql queries against your data, embedable, not coupled to storage layer
* extend VM with custom go functions, provide rich basic library of functions
* provide example backends (csv, elasticsearch, etc)

### Example VM Runtime for Reading a Csv via Stdio, and evaluating


See example in [qlcsv](https://github.com/araddon/qlbridge/tree/master/examples/qlcsv)
folder for a CSV reader, parser, evaluation engine.

```sh

./qlcsv -sql 'select 
		user_id, email, item_count * 2, yy(reg_date) > 10 
	FROM stdio' < users.csv

```
```go

func main() {

	// load all of our built-in functions
	builtins.LoadAllBuiltins()

	// Add a custom function to the VM to make available to SQL language
	expr.FuncAdd("email_is_valid", EmailIsValid)

{'fraction_non_alphanumeric': 0.12921890067502412, 'fraction_numerical': 0.0028929604628736743, 'mean_word_length': 4.48941798941799, 'pattern_counts': {'":': 0, '<': 5, '<?xml version=': 0, '>': 5, 'https://': 2, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '2783320', 'n_tokens_mistral': 618, 'n_tokens_neox': 576, 'n_words': 255}