To improve training efficiency, we utilize parallel training. Specifically, we employ multiple learning actors, referred to as workers, as proposed in [1]}. A total of \(N_\text{worker}\)  workers are executed in multiple instances of the training environment in parallel. Each instance is simulated with a different random seed and initialized with a random resource assignment to the vehicles, with the first action being also randomized. Each worker asynchronously gathers epochs of experience (of length \(L_\text{epoch}\) ) in its own environment. After every epoch, the worker updates the parameters of a single global policy and a value function that are shared by all the workers. In the end, we obtain the global policy that is trained using multiple workers, to be deployed in the target environment. This approach is beneficial in terms of speeding up the training thanks to a more efficient exploration of the state-space of the environment under distinct policies of multiple actors, and the added diversity helps to learn a general policy applicable to similar environments [1]}.
