<commit_before># Online Regression

> Online regression via [Stochastic Gradient Descent][stochastic-gradient-descent].


<section class="usage">

## Usage

``` javascript
var onlineSGDRegression = require( '@stdlib/math/ml/online-sgd-regression' );
```

#### onlineSGDRegression( \[options\] )

Creates an online linear regression model fitted via [stochastic gradient descent][stochastic-gradient-descent]. The module performs [L2 regularization][l2-regularization] of the model coefficients, shrinking them towards zero by penalizing the squared [euclidean norm][euclidean-norm] of the coefficients.

``` javascript
var randu = require( '@stdlib/math/base/random/randu' );
var model = onlineSGDRegression();

var x1;
var x2;
var i;
var y;

// Update model as data comes in...
for ( i = 0; i < 100000; i++ ) {
    x1 = randu();
    x2 = randu();
    y = 3.0 * x1 + -3.0 * x2 + 2.0;
    model.update( [ x1, x2 ], y );
}
```

The function accepts the following `options`:

* __learningRate__: `string` denoting the learning rate to use. Can be `constant`, `pegasos` or `basic`. Default: `basic`.
* __loss__: `string` denoting the loss function to use. Can be `squaredError`, `epsilonInsensitive` or `huber`. Default: `squaredError`.
* __epsilon__: insensitivity parameter. Default: `0.1`.
* __lambda__: regularization parameter. Default: `1e-3`.
* __eta0__: constant learning rate. Default: `0.02`.
* __intercept__: `boolean` indicating whether to include an intercept. Default: `true`.

``` javascript
model = onlineSGDRegression({
    'loss': 'squaredError',
    'lambda': 1e-4
});
```

The `learningRate` decides how fast or slow the weights will be updated towards the optimal weights. Let `i` denote the current iteration of the algorithm (i.e. the number of data points having arrived). The possible learning rates are:

| Option          | Definition                |
|:---------------:|:-------------------------:|
| basic (default) |  1000.0 / ( i + 1000.0 )  |
| constant        |  eta0                     |
| pegasos         |  1.0 / ( lambda * i )     |

The used loss function is specified via the `loss` option. The available options are:

* __epsilonInsensitive__: Penalty is the absolute value of the error whenever the absolute error exceeds epsilon and zero otherwise.
* __huber__: Squared-error loss for observations with error smaller than epsilon in magnitude, linear loss otherwise. Should be used in order to decrease the influence of outliers on the model fit.
* __squaredError__: Squared error loss, i.e. the squared difference of the observed and fitted values.

The `lambda` parameter determines the amount of shrinkage inflicted on the model coefficients:

``` javascript
var createRandom = require( '@stdlib/math/base/random/randu' ).factory;

var opts = { 'seed': 23 };
var rand = createRandom( opts );

var model = onlineSGDRegression({
    'lambda': 1e-5
});

for ( i = 0; i < 100; i++ ) {
    x1 = randu();
    x2 = randu();
    y = 3.0 * x1 + -3.0 * x2 + 2.0;
    model.update( [ x1, x2 ], y );
}

var coefs = model.coefs;
// returns [ ~2.9997, ~-2.9994, ~1.9999 ]

rand = createRandom( opts );
model = onlineSGDRegression({
    'lambda': 1e-2
});

for ( i = 0; i < 100; i++ ) {
    x1 = randu();
    x2 = randu();
    y = 3.0 * x1 + -3.0 * x2 + 2.0;
    model.update( [ x1, x2 ], y );
}

coefs = model.coefs;
// returns [ ~2.7446, ~-2.5017, ~1.9323 ]
```<commit_msg>Add noise and fix rand calls<commit_after># Online Regression

> Online regression via [Stochastic Gradient Descent][stochastic-gradient-descent].


<section class="usage">

## Usage

``` javascript
var onlineSGDRegression = require( '@stdlib/math/ml/online-sgd-regression' );
```

#### onlineSGDRegression( \[options\] )

Creates an online linear regression model fitted via [stochastic gradient descent][stochastic-gradient-descent]. The module performs [L2 regularization][l2-regularization] of the model coefficients, shrinking them towards zero by penalizing the squared [euclidean norm][euclidean-norm] of the coefficients.

``` javascript
var randu = require( '@stdlib/math/base/random/randu' );
var normal = require( '@stdlib/math/base/random/normal' );
var model = onlineSGDRegression();

var x1;
var x2;
var i;
var y;

// Update model as data comes in...
for ( i = 0; i < 100000; i++ ) {
    x1 = randu();
    x2 = randu();
    y = 3.0 * x1 + -3.0 * x2 + 2.0 + normal( 0.0, 1.0 );
    model.update( [ x1, x2 ], y );
}
```

The function accepts the following `options`:

* __learningRate__: `string` denoting the learning rate to use. Can be `constant`, `pegasos` or `basic`. Default: `basic`.
* __loss__: `string` denoting the loss function to use. Can be `squaredError`, `epsilonInsensitive` or `huber`. Default: `squaredError`.
* __epsilon__: insensitivity parameter. Default: `0.1`.
* __lambda__: regularization parameter. Default: `1e-3`.
* __eta0__: constant learning rate. Default: `0.02`.
* __intercept__: `boolean` indicating whether to include an intercept. Default: `true`.

``` javascript
model = onlineSGDRegression({
    'loss': 'squaredError',
    'lambda': 1e-4
});
```

The `learningRate` decides how fast or slow the weights will be updated towards the optimal weights. Let `i` denote the current iteration of the algorithm (i.e. the number of data points having arrived). The possible learning rates are:

| Option          | Definition                |
|:---------------:|:-------------------------:|
| basic (default) |  1000.0 / ( i + 1000.0 )  |
| constant        |  eta0                     |
| pegasos         |  1.0 / ( lambda * i )     |

The used loss function is specified via the `loss` option. The available options are:

* __epsilonInsensitive__: Penalty is the absolute value of the error whenever the absolute error exceeds epsilon and zero otherwise.
* __huber__: Squared-error loss for observations with error smaller than epsilon in magnitude, linear loss otherwise. Should be used in order to decrease the influence of outliers on the model fit.
* __squaredError__: Squared error loss, i.e. the squared difference of the observed and fitted values.

The `lambda` parameter determines the amount of shrinkage inflicted on the model coefficients:

``` javascript
var createRandom = require( '@stdlib/math/base/random/randu' ).factory;

var opts = { 'seed': 23 };
var rand = createRandom( opts );

var model = onlineSGDRegression({
    'lambda': 1e-5
});

for ( i = 0; i < 100; i++ ) {
    x1 = rand();
    x2 = rand();
    y = 3.0 * x1 + -3.0 * x2 + 2.0;
    model.update( [ x1, x2 ], y );
}

var coefs = model.coefs;
// returns [ ~2.9997, ~-2.9994, ~1.9999 ]

rand = createRandom( opts );
model = onlineSGDRegression({
    'lambda': 1e-2
});

for ( i = 0; i < 100; i++ ) {
    x1 = rand();
    x2 = rand();
    y = 3.0 * x1 + -3.0 * x2 + 2.0;
    model.update( [ x1, x2 ], y );
}

coefs = model.coefs;
// returns [ ~2.7446, ~-2.5017, ~1.9323 ]
```
{'fraction_non_alphanumeric': 0.14343727338651197, 'fraction_numerical': 0.033647570703408264, 'mean_word_length': 3.280571073867163, 'pattern_counts': {'":': 0, '<': 11, '<?xml version=': 0, '>': 7, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '2560498', 'n_tokens_mistral': 2369, 'n_tokens_neox': 2139, 'n_words': 780}