<commit_before><a href="http://promisesaplus.com/">
    <img src="http://promisesaplus.com/assets/logo-small.png" alt="Promises/A+ logo" title="Promises/A+ 1.0 compliant" align="right" />
</a>

banditdb-egreedy
================

[![Build Status](https://travis-ci.org/banditdb/egreedy.svg)](https://travis-ci.org/banditdb/egreedy)

A Promises/A+, [multi-armed bandit](http://en.wikipedia.org/wiki/Multi-armed_bandit) implemented with an epsilon-greedy algorithm.

This implemention is based on [<em>Bandit Algorithms for Website Optimization</em>](http://shop.oreilly.com/product/0636920027393.do) and related empirical research in ["Algorithms for the multi-armed bandit problem"](https://d2w9gswcdc2jtf.cloudfront.net/research/Algorithms+for+the+multi-armed+bandit+problem.pdf).


## Quick start

1. Create a bandit with 3 arms and epsilon 0.25

    ```
    var Bandit = require('banditdb-egreedy');

    var bandit = new Bandit({
        arms: 3,
        epsilon: 0.25
    });
    ```

2. Select an arm (for exploration or exploitation, according to the algorithm)

    ```
    bandit.select().then(function (arm) {
        console.log('pulled arm=' + arm);
    });
    ```

3. Report the reward earned from a chosen arm

    ```
    bandit.reward(1, 1).then(function (rewards) {
        console.log('arm rewards are currently=' + rewards);
    });
    ```


## Configuration

#### Load the bandit algorithm

Install from npm

```
npm install banditdb-egreedy --save
```

Require in your project

```
var Bandit = require('banditdb-egreedy');
```

#### Instantiate a bandit

This algorithm defaults to 2 arms and eplsilon 0.5

```
var bandit = new Bandit();
```

The constructor accepts an options object that supports two parameters:

- `arms`: the number of arms over which the bandit can operate
- `epsilon`: the desired level of epsilon (from 0=never explore/always exploit to 1=always explore/never exploit)

```
var bandit = new Bandit({
    arms: 4,
    epsilon: 0.75
});
```


## API

All banditdb algorithms, including this implementation, provide the same Promises/A+ interface.

#### `bandit.select()`

Choose an arm to play, according to the specified bandit algorithm.

**Arguments**

_None_

**Returns**

A promise that resolves to a Number corresponding to the associated arm index.

**Example**

```
> var Bandit = require('banditdb-egreedy');
> var bandit = new Bandit();
> bandit.select().then(function (arm) { console.log(arm); });

0
```

#### `bandit.reward(arm, reward)`

Inform the algorithm about the payoff from a given arm.

**Arguments**

- `arm` (Integer): the arm index (provided from `bandit.select()`)
- `reward` (Number): the observed reward value (which can be 0, to indicate no reward)

**Returns**

A promise that resolves to an Array of the current reward state of each arm; each position in the array corresponds to the associated arm index.

**Example**

```
> var Bandit = require('banditdb-egreedy');
> var bandit = new Bandit();
> bandit.reward(0, 1).then(function (rewards) { console.log(rewards); });

[1, 0]

> bandit.reward(1, 1).then(function (rewards) { console.log(rewards); });

[1, 1]

> bandit.reward(1, 0).then(function (rewards) { console.log(rewards); });

[1, 0.66666]
```

#### `bandit.serialize()`

Obtain a persistable JSON object representing the internal state of the algorithm.

**Arguments**

_None_

**Returns**

A promise that resolves to an Object representing parameters required to reconstruct algorithm state.

**Example**

```
> var Bandit = require('banditdb-egreedy');
> var bandit = new Bandit();
> bandit.serialize().then(function (state) { console.log(state); });

{
    arms: 2,
    epsilon: 0.5,
    counts: [ 0, 0 ],
    values: [ 0, 0 ]
}
```

#### `bandit.load(state)`

Restore an instance of a bandit to a perviously serialized algorithm state. This method overrides any options parameters passed at instantiation.

**Arguments**

- `state` (Object): a serialized algorithm state (provided from `bandit.serialize()`)

**Returns**

A promise that resolves to an Array of the current reward state of each arm; each position in the array corresponds to the associated arm index.

**Example**

```
> var state = { arms: 2, epsilon: 0.5, counts: [ 1, 2 ], values: [ 1, 0.5 ] };
> var Bandit = require('banditdb-egreedy');
> var bandit = new Bandit();
> bandit.load(state).then(function (rewards) { console.log(rewards); });

[1, 0.5]
```

#### `bandit.n`

(Number) An instance property representing the total number of recorded reward samples, updated at each `bandit.reward()` call.

**Example**

```
> var Bandit = require('banditdb-egreedy');
> var bandit = new Bandit();
> bandit.reward(0, 1).then(function (rewards) { console.log(rewards); });
> bandit.n;

1
```


## Tests

To run the full unit test suite

```
npm test<commit_msg>readme clarification<commit_after><a href="http://promisesaplus.com/">
    <img src="http://promisesaplus.com/assets/logo-small.png" alt="Promises/A+ logo" title="Promises/A+ 1.0 compliant" align="right" />
</a>

banditdb-egreedy
================

[![Build Status](https://travis-ci.org/banditdb/egreedy.svg)](https://travis-ci.org/banditdb/egreedy)

A Promises/A+, [multi-armed bandit](http://en.wikipedia.org/wiki/Multi-armed_bandit) implemented with an epsilon-greedy algorithm.

This implemention is based on [<em>Bandit Algorithms for Website Optimization</em>](http://shop.oreilly.com/product/0636920027393.do) and related empirical research in ["Algorithms for the multi-armed bandit problem"](https://d2w9gswcdc2jtf.cloudfront.net/research/Algorithms+for+the+multi-armed+bandit+problem.pdf).


## Quick start

1. Create a bandit with 3 arms and epsilon 0.25

    ```
    var Bandit = require('banditdb-egreedy');

    var bandit = new Bandit({
        arms: 3,
        epsilon: 0.25
    });
    ```

2. Select an arm (for exploration or exploitation, according to the algorithm)

    ```
    bandit.select().then(function (arm) {
        console.log('pulled arm=' + arm);
    });
    ```

3. Report the reward earned from a chosen arm

    ```
    bandit.reward(1, 1).then(function (rewards) {
        console.log('arm rewards are currently=' + rewards);
    });
    ```


## Configuration

#### Load the bandit algorithm

Install from npm

```
npm install banditdb-egreedy --save
```

Require in your project

```
var Bandit = require('banditdb-egreedy');
```

#### Instantiate a bandit

This algorithm defaults to 2 arms and eplsilon 0.5

```
var bandit = new Bandit();
```

The constructor accepts an options object that supports two parameters:

- `arms`: the number of arms over which the bandit can operate
- `epsilon`: the desired level of epsilon (from 0=never explore/always exploit to 1=always explore/never exploit)

```
var bandit = new Bandit({
    arms: 4,
    epsilon: 0.75
});
```


## API

All banditdb algorithms, including this implementation, provide the same Promises/A+ interface.

#### `bandit.select()`

Choose an arm to play, according to the specified bandit algorithm.

**Arguments**

_None_

**Returns**

A promise that resolves to a Number corresponding to the associated arm index.

**Example**

```
> var Bandit = require('banditdb-egreedy');
> var bandit = new Bandit();
> bandit.select().then(function (arm) { console.log(arm); });

0
```

#### `bandit.reward(arm, reward)`

Inform the algorithm about the payoff from a given arm.

**Arguments**

- `arm` (Integer): the arm index (provided from `bandit.select()`)
- `reward` (Number): the observed reward value (which can be 0, to indicate no reward)

**Returns**

A promise that resolves to an Array of the current reward state of each arm; each position in the array corresponds to the associated arm index.

**Example**

```
> var Bandit = require('banditdb-egreedy');
> var bandit = new Bandit();
> bandit.reward(0, 1).then(function (rewards) { console.log(rewards); });

[1, 0]

> bandit.reward(1, 1).then(function (rewards) { console.log(rewards); });

[1, 1]

> bandit.reward(1, 0).then(function (rewards) { console.log(rewards); });

[1, 0.66666]
```

#### `bandit.serialize()`

Obtain a persistable JSON object representing the internal state of the algorithm.

**Arguments**

_None_

**Returns**

A promise that resolves to an Object representing parameters required to reconstruct algorithm state.

**Example**

```
> var Bandit = require('banditdb-egreedy');
> var bandit = new Bandit();
> bandit.serialize().then(function (state) { console.log(state); });

{
    arms: 2,
    epsilon: 0.5,
    counts: [ 0, 0 ],
    values: [ 0, 0 ]
}
```

#### `bandit.load(state)`

Restore an instance of a bandit to a perviously serialized algorithm state. This method overrides any options parameters passed at instantiation.

**Arguments**

- `state` (Object): a serialized algorithm state (provided from `bandit.serialize()`)

**Returns**

A promise that resolves to an Array of the current reward state of each arm; each position in the array corresponds to the associated arm index.

**Example**

```
> var state = { arms: 2, epsilon: 0.5, counts: [ 1, 2 ], values: [ 1, 0.5 ] };
> var Bandit = require('banditdb-egreedy');
> var bandit = new Bandit();
> bandit.load(state).then(function (rewards) { console.log(rewards); });

[1, 0.5]
```

#### `bandit.n`

(Number) An instance property representing the total number of recorded reward samples, updated at each `bandit.reward()` call.

**Example**

```
> var Bandit = require('banditdb-egreedy');
> var bandit = new Bandit();
> bandit.reward(0, 1).then(function () { console.log(bandit.n); });

1
```


## Tests

To run the full unit test suite

```
npm test
{'fraction_non_alphanumeric': 0.15095124233288285, 'fraction_numerical': 0.016633745711612434, 'mean_word_length': 4.155412647374062, 'pattern_counts': {'":': 0, '<': 13, '<?xml version=': 0, '>': 50, 'https://': 6, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '1385768', 'n_tokens_mistral': 3272, 'n_tokens_neox': 3068, 'n_words': 990}