<commit_before>sqlite3-reutil
==============
Regular Expressions and Scalar/Vector Math for SQLite3

Usage
-----
### Regular Expressions
reutil adds to SQLite's builtin LIKE and GLOB expressions by adding full featured regular expression support.
It's based on another [module that used PCRE.](https://github.com/ralight/sqlite3-pcre).
Whereas its predecessor supported only matches, this supports searches, matches, and formatted replacements,
as implemented by the
[Boost project regex module](http://www.boost.org/doc/libs/1_55_0/libs/regex/doc/html/index.html).

 - [Regex syntax reference](http://www.boost.org/doc/libs/1_55_0/libs/regex/doc/html/boost_regex/syntax/perl_syntax.html) (Perl compatible)
 - [Replacement format reference](http://www.boost.org/doc/libs/1_55_0/libs/regex/doc/html/boost_regex/format/perl_format.html) (also Perl compatible)

Examples:
```sql
SELECT * FROM table WHERE column MATCH "<tag [^>]+>";
SELECT * FROM table WHERE column SEARCH "is the (thir|four)teenth of May";
SELECT sub("(\w+) lives by lake (\w+)", "$1 thinks $2 is cool.", column) FROM table;
```

### Vector Math
SQLite omits a lot of functions, to reduce API footprint. As such, the functions provided here probably not as bulletproof as those in SQLite. But in practice they're still pretty useful:

```sql
sqlite> .load sqlite3-reutil
sqlite> SELECT sin(4);
-0.756802495307928
sqlite> SELECT vshow(sin(vread('3 7 6 1.2')));
0.14112 0.656987 -0.279415 0.932039
```

The following functions are included:
- Regular Expressions
 - `match(regular expression, subject)`: Match the regular expression against the subject, only at the beginning.
 - `search(regular expression, subject)`: Search through the subject to see if there is a match for the regular expression anywhere within the text
 - `sub(regular expression, format string, subject)`: Replace any matches of the regular expression with the format string, with may contain backticks of the form:
 - `\0`: Replaced with the whole match
 - `\1`: Replaced with the first captured group
 - `\2`: Replaced with the second captured group
 - See the Boost Regex reference above for more information.
- Math
  - Create Vectors
    - `vzero(length)`: Create a 0-vector of a specific length
    - `vone(length)`: Create a 1-vector of a specific length
    - `vread(text)`: Read space-separated floating point values from text into a vector
    - `vshow(text)`: Format a vector as a human readable string compatible with vread()
 - Unary Operators on Vectors or Scalars
    - `sin(V)`
    - `asin(V)`
    - `cos(V)`
    - `acos(V)`
    - `tan(V)`
    - `atan(V)`
    - `log(V)`
    - `exp(V)`
    - `pow(V)`
    - `sqrt(V)`
 - Binary Operators on any combination of vector and scalars
    - `add(V, V)`
    - `subtract(V, V)`
    - `mult(V, V)`
    - `div(V, V)`
 - Vector operations (operate only on vectors)
    - `vsum(V)`: Compute the sum of a vector

Install
-------

It includes a QMake project file, but you can also compile it by hand. It's very small.
```sh
# Download
git pull https://github.com/SeanTater/sqlite3-reutil.git && cd sqlite3-reutil
make
```<commit_msg>Fix README formatting<commit_after>sqlite3-reutil
==============
Regular Expressions and Scalar/Vector Math for SQLite3

Usage
-----
### Regular Expressions
reutil adds to SQLite's builtin LIKE and GLOB expressions by adding full featured regular expression support.
It's based on another [module that used PCRE.](https://github.com/ralight/sqlite3-pcre).
Whereas its predecessor supported only matches, this supports searches, matches, and formatted replacements,
as implemented by the
[Boost project regex module](http://www.boost.org/doc/libs/1_55_0/libs/regex/doc/html/index.html).

 - [Regex syntax reference](http://www.boost.org/doc/libs/1_55_0/libs/regex/doc/html/boost_regex/syntax/perl_syntax.html) (Perl compatible)
 - [Replacement format reference](http://www.boost.org/doc/libs/1_55_0/libs/regex/doc/html/boost_regex/format/perl_format.html) (also Perl compatible)

Examples:
```sql
SELECT * FROM table WHERE column MATCH "<tag [^>]+>";
SELECT * FROM table WHERE column SEARCH "is the (thir|four)teenth of May";
SELECT sub("(\w+) lives by lake (\w+)", "$1 thinks $2 is cool.", column) FROM table;
```

### Vector Math
SQLite omits a lot of functions, to reduce API footprint. As such, the functions provided here probably not as bulletproof as those in SQLite. But in practice they're still pretty useful:

```sql
sqlite> .load sqlite3-reutil
sqlite> SELECT sin(4);
-0.756802495307928
sqlite> SELECT vshow(sin(vread('3 7 6 1.2')));
0.14112 0.656987 -0.279415 0.932039
```

The following functions are included:
- Regular Expressions
 - `match(regular expression, subject)`: Match the regular expression against the subject, only at the beginning.
 - `search(regular expression, subject)`: Search through the subject to see if there is a match for the regular expression anywhere within the text
 - `sub(regular expression, format string, subject)`: Replace any matches of the regular expression with the format string, with may contain backticks of the form:
 - `\0`: Replaced with the whole match
 - `\1`: Replaced with the first captured group
 - `\2`: Replaced with the second captured group
 - See the Boost Regex reference above for more information.
- Math
  - Create Vectors
    - `vzero(length)`: Create a 0-vector of a specific length
    - `vone(length)`: Create a 1-vector of a specific length
    - `vread(text)`: Read space-separated floating point values from text into a vector
    - `vshow(text)`: Format a vector as a human readable string compatible with vread()
 - Unary Operators on Vectors or Scalars
    - `sin(V)`
    - `asin(V)`
    - `cos(V)`
    - `acos(V)`
    - `tan(V)`
    - `atan(V)`
    - `log(V)`
    - `exp(V)`
    - `pow(V)`
    - `sqrt(V)`
 - Binary Operators on any combination of vector and scalars
    - `add(V, V)`
    - `subtract(V, V)`
    - `mult(V, V)`
    - `div(V, V)`
 - Vector operations (operate only on vectors)
    - `vsum(V)`: Compute the sum of a vector


Install
-------

It includes a QMake project file, but you can also compile it by hand. It's very small.
```sh
# Download
git pull https://github.com/SeanTater/sqlite3-reutil.git && cd sqlite3-reutil
make
```
{'fraction_non_alphanumeric': 0.12382018876979684, 'fraction_numerical': 0.02367621180611102, 'mean_word_length': 3.876755070202808, 'pattern_counts': {'":': 0, '<': 5, '<?xml version=': 0, '>': 13, 'https://': 4, 'lorem ipsum': 0, 'www.': 6, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 2, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '6374599', 'n_tokens_mistral': 2102, 'n_tokens_neox': 1923, 'n_words': 724}