<commit_before># Bandits in TF-Agents

The <b>Multi-Armed Bandit problem (MAB)</b> is a special case of Reinforcement
Learning (RL): an agent collects rewards in an environment by taking some
actions after observing some state of the environment. The main difference
between general RL and MAB is that in MAB, we assume that the action taken by
the agent does not influence the next state of the environment. Therefore,
agents do not model state transitions, credit rewards to past actions, or
"plan ahead" to get to reward-rich states. Due to this very fact, the notion of
episodes is not used in MAB, unlike in general RL.

In many bandits use cases, the state of the environment is observed. These are
known as <b>contextual bandits</b> problems, and can be thought of as a
generalization of multi-armed bandits where the agent has access to additional
context in each round.

To get started with Bandits in TF-Agents, we recommend checking the following
Colab tutorial:
[`tf_agents/g3doc/tutorials/bandits_tutorial.ipynb`](https://github.com/tensorflow/agents/tree/master/tf_agents/g3doc/tutorials/bandits_tutorial.ipynb).

## Agents

Currently the following algorithms are available:

* [`LinUCB`](https://github.com/tensorflow/agents/tree/master/tf_agents/bandits/agents/lin_ucb_agent.py):
[__A Contextual Bandit Approach to Personalized News Article Recommendation__ Li et al., 2010.](https://arxiv.org/abs/1003.0146)
* [`Linear Thompson Sampling`](https://github.com/tensorflow/agents/tree/master/tf_agents/bandits/agents/linear_thompson_sampling_agent.py):
[__Thompson Sampling for Contextual Bandits with Linear Payoffs__ Agrawal et al., 2013.](http://proceedings.mlr.press/v28/agrawal13.pdf)
* [`Neural Epsilon Greedy`](https://github.com/tensorflow/agents/tree/master/tf_agents/bandits/agents/neural_epsilon_greedy_agent.py):
[__Bandit Algorithms__ Lattimore et al., 2019](https://tor-lattimore.com/downloads/book/book.pdf)
* [`Neural LinUCB`](https://github.com/tensorflow/agents/tree/master/tf_agents/bandits/agents/neural_linucb_agent.py):
[__Deep Bayesian Bandits Showdown: An Empirical Comparison of Bayesian Deep Networks for Thompson Sampling__ Riquelme et al., 2018]
(https://arxiv.org/abs/1802.09127)
* [`Thompson Sampling with Dropout`](https://github.com/tensorflow/agents/tree/master/tf_agents/bandits/agents/dropout_thompson_sampling_agent.py):
[__Deep Bayesian Bandits Showdown: An Empirical Comparison of Bayesian Deep Networks for Thompson Sampling__ Riquelme et al., 2018]
(https://arxiv.org/abs/1802.09127)
* [`Multi-objective neural agent`](https://github.com/tensorflow/agents/tree/master/tf_agents/bandits/agents/greedy_multi_objective_neural_agent.py):
[__Designing multi-objective multi-armed bandits algorithms: a study__  Drugan et al., 2013.](https://ieeexplore.ieee.org/document/6707036)<commit_msg>Fix broken link in README.<commit_after># Bandits in TF-Agents

The <b>Multi-Armed Bandit problem (MAB)</b> is a special case of Reinforcement
Learning (RL): an agent collects rewards in an environment by taking some
actions after observing some state of the environment. The main difference
between general RL and MAB is that in MAB, we assume that the action taken by
the agent does not influence the next state of the environment. Therefore,
agents do not model state transitions, credit rewards to past actions, or
"plan ahead" to get to reward-rich states. Due to this very fact, the notion of
episodes is not used in MAB, unlike in general RL.

In many bandits use cases, the state of the environment is observed. These are
known as <b>contextual bandits</b> problems, and can be thought of as a
generalization of multi-armed bandits where the agent has access to additional
context in each round.

To get started with Bandits in TF-Agents, we recommend checking our [`bandits tutorial`]
(https://github.com/tensorflow/agents/blob/master/docs/tutorials/bandits_tutorial.ipynb).

## Agents

Currently the following algorithms are available:

* [`LinUCB`](https://github.com/tensorflow/agents/tree/master/tf_agents/bandits/agents/lin_ucb_agent.py):
[__A Contextual Bandit Approach to Personalized News Article Recommendation__ Li et al., 2010.](https://arxiv.org/abs/1003.0146)
* [`Linear Thompson Sampling`](https://github.com/tensorflow/agents/tree/master/tf_agents/bandits/agents/linear_thompson_sampling_agent.py):
[__Thompson Sampling for Contextual Bandits with Linear Payoffs__ Agrawal et al., 2013.](http://proceedings.mlr.press/v28/agrawal13.pdf)
* [`Neural Epsilon Greedy`](https://github.com/tensorflow/agents/tree/master/tf_agents/bandits/agents/neural_epsilon_greedy_agent.py):
[__Bandit Algorithms__ Lattimore et al., 2019](https://tor-lattimore.com/downloads/book/book.pdf)
* [`Neural LinUCB`](https://github.com/tensorflow/agents/tree/master/tf_agents/bandits/agents/neural_linucb_agent.py):
[__Deep Bayesian Bandits Showdown: An Empirical Comparison of Bayesian Deep Networks for Thompson Sampling__ Riquelme et al., 2018]
(https://arxiv.org/abs/1802.09127)
* [`Thompson Sampling with Dropout`](https://github.com/tensorflow/agents/tree/master/tf_agents/bandits/agents/dropout_thompson_sampling_agent.py):
[__Deep Bayesian Bandits Showdown: An Empirical Comparison of Bayesian Deep Networks for Thompson Sampling__ Riquelme et al., 2018]
(https://arxiv.org/abs/1802.09127)
* [`Multi-objective neural agent`](https://github.com/tensorflow/agents/tree/master/tf_agents/bandits/agents/greedy_multi_objective_neural_agent.py):
[__Designing multi-objective multi-armed bandits algorithms: a study__  Drugan et al., 2013.](https://ieeexplore.ieee.org/document/6707036)
{'fraction_non_alphanumeric': 0.10051592243373066, 'fraction_numerical': 0.022060131649172744, 'mean_word_length': 5.567757009345795, 'pattern_counts': {'":': 0, '<': 11, '<?xml version=': 0, '>': 11, 'https://': 24, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 2, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '2823489', 'n_tokens_mistral': 1902, 'n_tokens_neox': 1690, 'n_words': 526}