<commit_before>C++ Robust And Approximate Markov decision processes
==============================================

.. role:: cpp(code)
    :language: c++

A simple and easy to use library to solve Markov decision processes and *robust* Markov decision processes. 

The library supports standard finite or infinite horizon discounted MDPs [Puterman2005]_. The library assumes *maximization* over actions. The states and actions must be finite.

The robust model extends the regular MDPs [Iyengar2005]_. The library allows to model uncertainty in *both* the transition and rewards, unlike some published papers on this topic. This is modeled by adding an outcome to each action. The outcome is assumed to be minimized by nature, similar to [Filar1997]_.

In summary, the MDP problem being solved is:

.. math::

    v(s) = \max_{a \in \mathcal{A}} \min_{o \in \mathcal{O}} \sum_{s\in\mathcal{S}} ( r(s,a,o,s') + \gamma P(s,a,o,s') v(s') ) ~.

Here, :math:`\mathcal{S}` are the states, :math:`\mathcal{A}` are the actions, :math:`\mathcal{O}` are the outcomes. 
<commit_msg>README.rst edited online with Bitbucket<commit_after>Robust And Approximate Markov decision processes
===========================================

.. role:: cpp(code)
    :language: c++

A simple and easy to use C++ library to solve Markov decision processes and *robust* Markov decision processes. 

The library supports standard finite or infinite horizon discounted MDPs [Puterman2005]_. The library assumes *maximization* over actions. The states and actions must be finite.

The robust model extends the regular MDPs [Iyengar2005]_. The library allows to model uncertainty in *both* the transition and rewards, unlike some published papers on this topic. This is modeled by adding an outcome to each action. The outcome is assumed to be minimized by nature, similar to [Filar1997]_.

In summary, the MDP problem being solved is:

.. math::

    v(s) = \max_{a \in \mathcal{A}} \min_{o \in \mathcal{O}} \sum_{s\in\mathcal{S}} ( r(s,a,o,s') + \gamma P(s,a,o,s') v(s') ) ~.

Here, :math:`\mathcal{S}` are the states, :math:`\mathcal{A}` are the actions, :math:`\mathcal{O}` are the outcomes. 

{'fraction_non_alphanumeric': 0.14582371942778033, 'fraction_numerical': 0.011075219197046609, 'mean_word_length': 4.675392670157068, 'pattern_counts': {'":': 0, '<': 3, '<?xml version=': 0, '>': 3, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '5744705', 'n_tokens_mistral': 654, 'n_tokens_neox': 613, 'n_words': 274}