Nevertheless, a new problem could arise in this model. For each environment iteration, every actor sends its observation to the learner, making a decision, and sending the inferred action back to the actor. So this method can cause latency compared to previous methods in which the inference was not centralized. To decrease this latency, [1]} introduce an efficient library based on gRPC (a high-performance open-source RPC framework), including asynchronous streaming RPCs (each actor's connection to the learner is kept open). In this framework, a batching module is employed that batches multiple actor inference calls in a well-organized way. Furthermore, the framework utilizes Unix domain sockets in cases where actors can fit on the same machine, so latency and CPU overhead is minimized.
