The key idea in SEED RL to overcome these drawbacks is that model inference, and trajectory collection is made in learner instead of actors. The whole design is somehow a single-machine structure with remote environments. So we can use batching inference and accelerators such as GPUs and TPUs [1]} - AI accelerator application-specific integrated circuit, have multiple environments on the actor, avoid data transfer bottleneck by keeping parameters and states locally in the learner, and there is just a single task in each actor (which is conceptually an environment handler). Additionally, growing the number of parameters does not increase the demand for more actors.
<FIGURE>