We study factors that affect specialisation such as agent number, task throughput (limiting the number of times one task can be performed before another task: in our case this means limiting the number of resources that can be in area two, we also call it bottleneck) and reward degradation over time. Agents are independent learners, trained through two state of the art reinforcement learning algorithms: Dueling Double Deep Q-Network (DDDQN) [1]}, [2]} with agent-independent prioritised experience replay buffers [3]} and Asynchronous Advantage Actor-Critic (A2C) [4]}, both using Convolutional Neural Networks as function approximators. The former is a value-based method, that learns a value function and then acts greedily with respect to it. The policy is intrinsically given by the value function itself and the actions are selected according to a time-decreasing probability called epsilon (\(\epsilon \) -greedy) that controls whenever they will be sampled according to the current estimated value function or from a uniform random distribution (\(\epsilon \)  can be seen to decrease linearly and equally for all agents in Figure REF ). On the other hand, the algorithm A2C, is a mix of value based and policy gradient approaches, in which we have two networks, an actor who maps states to actions and a critic (value based) who learns how well the actor is performing and contributes to improve the policy. This is performed through the actor policy gradient, who aims to change the policy to maximize the value estimations of the critic. In this case the exploration is controlled by the entropy of the policy (how uncertain and similar to a uniform distribution it is), computed in the actor network loss function, without any external parameters.
<FIGURE><FIGURE>