<commit_before>Creating Multiagent Environments
================================
In addition to single agent control, Flow supports the use of
multiple agents with individual controllers and distinct rewards.

To do this, we use RLlib's multiagent support. All that changes is
that: instead of receiving a list of actions and returning a single observation
and a single reward, we now receive a dictionary of actions and
return a dictionary of rewards and a dictionary of observations.

The keys of the dictionary are IDs of the agent policies.

**Note that you must also subclass MultiEnv.**

A brief example of a multiagent env:
::

    from flow.envs.multiagent_env import MultiEnv
    class MultiAgentAccelEnv(AccelEnv, MultiEnv):
    """Example MultiAgent environment"""
    def _apply_rl_actions(self, rl_actions):
        rl_ids = []
        rl_action_list = []
        for rl_id, action in rl_actions.items():
            rl_ids.append(rl_id)
            rl_action_list.append(action)
        self.k.vehicle.apply_acceleration(rl_ids, rl_action_list)

    def compute_reward(self, rl_actions, **kwargs):
        """In this example all agents receive a reward of 10"""
        reward_dict = {}
        for rl_id, action in rl_actions.items():
            reward_dict[rl_id] = 10
        return reward_dict

    def get_state(self, **kwargs):
        """Here every agent gets its speed"""
        # speed normalizer
        obs_dict = {}
        for rl_id in self.k.vehicle.get_rl_ids():
            obs_dict[rl_id] = self.k.vehicle.get_speed(rl_id)
        return obs_dict


For further details look at our
`Multiagent examples <https://github.com/flow-project/flow/tree/master/examples/rllib/multiagent_exps>`_<commit_msg>DOC fix indent and improve wording for multiagent<commit_after>Creating Multiagent Environments
================================
In addition to single agent control, Flow supports the use of
multiple agents with individual controllers and distinct rewards.

To do this, we use RLlib's multiagent support.
The changes for multiagent environments are as follows:
instead of receiving a list of actions and returning a single observation
and a single reward, we now receive a dictionary of actions and
return a dictionary of rewards and a dictionary of observations.

The keys of the dictionary are IDs of the agent policies.

**Note that you must also subclass MultiEnv.**

A brief example of a multiagent env:
::

    from flow.envs.multiagent_env import MultiEnv

    class MultiAgentAccelEnv(AccelEnv, MultiEnv):
        """Example MultiAgent environment"""

        def _apply_rl_actions(self, rl_actions):
            rl_ids = []
            rl_action_list = []
            for rl_id, action in rl_actions.items():
                rl_ids.append(rl_id)
                rl_action_list.append(action)
            self.k.vehicle.apply_acceleration(rl_ids, rl_action_list)

        def compute_reward(self, rl_actions, **kwargs):
            """In this example all agents receive a reward of 10"""
            reward_dict = {}
            for rl_id, action in rl_actions.items():
                reward_dict[rl_id] = 10
            return reward_dict

        def get_state(self, **kwargs):
            """Here every agent gets its speed"""
            # speed normalizer
            obs_dict = {}
            for rl_id in self.k.vehicle.get_rl_ids():
                obs_dict[rl_id] = self.k.vehicle.get_speed(rl_id)
            return obs_dict


For further details look at our
`Multiagent examples <https://github.com/flow-project/flow/tree/master/examples/rllib/multiagent_exps>`_.
{'fraction_non_alphanumeric': 0.08939939108773873, 'fraction_numerical': 0.0022142264046498754, 'mean_word_length': 3.033482142857143, 'pattern_counts': {'":': 0, '<': 5, '<?xml version=': 0, '>': 5, 'https://': 2, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '4594924', 'n_tokens_mistral': 1031, 'n_tokens_neox': 971, 'n_words': 351}