<commit_before>ClConvolve
==========

OpenCL library to train deep convolutional networks
- C++
- OpenCL
- Deep convolutional

Target usage:
- 19 x 19 Go boards, eg something similar to [Clark and Storkey](http://arxiv.org/abs/1412.3409) or [Maddison, Huang, Sutskever and Silver](http://arxiv.org/abs/1412.6564)
- Also works on MNIST 28 x 28 boards (but you need to pad them to 29 x 29; the size of the board/image should be an odd number)

Neural Net API
==============

*Basic fully connected layers*

Example, with 1 fully connected layer:
```
#include "NeuralNet.h"

NeuralNet *net = NeuralNet::maker()->planes(2)->boardSize(1)->instance();
net->fullyConnectedMaker()->planes(2)->boardSize(1)->insert();
net->print();
for( int epoch = 0; epoch < 100; epoch++ ) {
    net->epochMaker()
       ->learningRate(3)->batchSize(4)->numExamples(4)
       ->inputData(mydata)->expectedOutputs(myExpectedResults)
       ->run();
    cout << "Loss L " << net->calcLoss(expectedOutputs) << endl;
    AccuracyHelper::printAccuracy( numImages, numClasses, trainingLabels, net->getResults() );
}
delete net;
```
Notes:
* fully connected layers run on cpu for now

*Convolutional layers*

For convolutional layer, you can do:
```
net->ConvolutionalMaker()->numFilters(2)->filterSize(5)->insert();
```
Or:
```
net->ConvolutionalMaker()->numFilters(2)->filterSize(5)->padZeros()->insert();
```
Notes:
* convolutional layers forward-prop runs on gpu, via OpenCL
* back-prop is on cpu for now

Options:
* add `->padZeros()` to pad the input board with zeros, so the output board and input board are the same size
* add `->biased()` to add per-filter biases

*General*

You can print the network like this:
```
net->print();
```
Data format
===========

Input data should be provided in a contiguous array.  "group by" order should be:

* training example id
* input plane<commit_msg>add relu to doc<commit_after>ClConvolve
==========

OpenCL library to train deep convolutional networks
- C++
- OpenCL
- Deep convolutional

Target usage:
- 19 x 19 Go boards, eg something similar to [Clark and Storkey](http://arxiv.org/abs/1412.3409) or [Maddison, Huang, Sutskever and Silver](http://arxiv.org/abs/1412.6564)
- Also works on MNIST 28 x 28 boards (but you need to pad them to 29 x 29; the size of the board/image should be an odd number)

Neural Net API
==============

*Basic fully connected layers*

Example, with 1 fully connected layer:
```
#include "NeuralNet.h"

NeuralNet *net = NeuralNet::maker()->planes(2)->boardSize(1)->instance();
net->fullyConnectedMaker()->planes(2)->boardSize(1)->insert();
net->print();
for( int epoch = 0; epoch < 100; epoch++ ) {
    net->epochMaker()
       ->learningRate(3)->batchSize(4)->numExamples(4)
       ->inputData(mydata)->expectedOutputs(myExpectedResults)
       ->run();
    cout << "Loss L " << net->calcLoss(expectedOutputs) << endl;
    AccuracyHelper::printAccuracy( numImages, numClasses, trainingLabels, net->getResults() );
}
delete net;
```

Notes:
* fully connected layers run on cpu for now

*Convolutional layers*

For convolutional layer, you can do:
```
net->ConvolutionalMaker()->numFilters(2)->filterSize(5)->insert();
```
Or:
```
net->ConvolutionalMaker()->numFilters(2)->filterSize(5)->padZeros()->insert();
```
Notes:
* convolutional layers forward-prop runs on gpu, via OpenCL
* back-prop is on cpu for now

Options:
* add `->padZeros()` to pad the input board with zeros, so the output board and input board are the same size

*General*

Options:
* add `->biased()` to add per-filter biases
* add `->relu()` to switch to Relu activation (default is Tanh)

You can print the network like this:
```
net->print();
```
Data format
===========

Input data should be provided in a contiguous array.  "group by" order should be:

* training example id
* input plane
{'fraction_non_alphanumeric': 0.15735641227380015, 'fraction_numerical': 0.023078940466824023, 'mean_word_length': 4.667161961367014, 'pattern_counts': {'":': 0, '<': 17, '<?xml version=': 0, '>': 62, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '5138143', 'n_tokens_mistral': 1297, 'n_tokens_neox': 1173, 'n_words': 403}