<commit_before>### Nano

Nano provides numerical optimization and machine learning utilities. For example it can be used to train models such as multi-layer perceptrons (classical neural networks) and convolution networks.


#### Compilation [![Build Status](https://travis-ci.org/accosmin/nano.svg?branch=0.4)](https://travis-ci.org/accosmin/nano)

Use a C++14 compiler and install Eigen3.3+, LibArchive, Zlib and DevIL. Nano is tested on Linux ([gcc 4.9+ | clang 3.6+], CMake 3.1+, Ninja or Make) and OSX (AppleClang7+, homebrew, CMake 3.1+, Ninja or Make). The code is written to be cross-platform, so it may work (with minor fixes) on other platforms as well (e.g. Windows/MSVC).

The easiest way to compile (and install) is to run:
```
bash build.sh --build-type release --generator ninja
```
The test programs and utilities will be found in the `build-release` directory.

To build the debugging version with or without address, memory and thread sanitizers (if available) run:
```
bash build.sh --build-type debug [--asan|--msan|--tsan] --generator ninja
```

It is recommended to use libc++ with clang by issuing the following command:
```
bash build.sh --build-type release --compiler clang++-3.8 --libc++ --generator ninja
```

To display the list of available build options invoke:
```
bash build.sh --help
```
<commit_msg>update readme (use cmake directly to build library)<commit_after>### Nano

Nano provides numerical optimization and machine learning utilities. For example it can be used to train models such as multi-layer perceptrons (classical neural networks) and convolution networks.


#### Compilation [![Build Status](https://travis-ci.org/accosmin/nano.svg?branch=0.4)](https://travis-ci.org/accosmin/nano)

Use a C++14 compiler and install Eigen3.3+, LibArchive, Zlib and DevIL. Nano is tested on Linux ([gcc 4.9+ | clang 3.6+], CMake 3.1+, Ninja or Make) and OSX (AppleClang7+, homebrew, CMake 3.1+, Ninja or Make). The code is written to be cross-platform, so it may work (with minor fixes) on other platforms as well (e.g. Windows/MSVC).

The easiest way to compile (and install) is to run:
```
mkdir build-release && cd build-release
cmake -G "Ninja" -DCMAKE_BUILD_TYPE=Release
```
The test programs and utilities will be found in the `build-release` directory.

To build the debugging version with or without address, memory and thread sanitizers (if available) run:
```
mkdir build-debug && cd build-debug
cmake -G "Ninja" -DCMAKE_BUILD_TYPE=Release -DNANO_WITH_[ASAN|MSAN|TSAN]=ON
```

It is recommended to use libc++ with clang by issuing the following command:
```
cmake -G "Ninja" -DCMAKE_BUILD_TYPE=Release -DCMAKE_CXX_COMPILER="clang++-4.0" -DNANO_WITH_LIBCPP=ON
```


{'fraction_non_alphanumeric': 0.11636497968230514, 'fraction_numerical': 0.012560029553010712, 'mean_word_length': 4.560574948665297, 'pattern_counts': {'":': 0, '<': 3, '<?xml version=': 0, '>': 3, 'https://': 4, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '7400645', 'n_tokens_mistral': 900, 'n_tokens_neox': 814, 'n_words': 340}