Agent architectures. The speaker and the listener are designed as two independent RL agents. Both agents have the same architecture without sharing weights or gradient values. They both have a 2-layer Convolutional Neural Network (CNN) that generates an \(8-32\)  bit representation \(s\)  of the environment. In the case of the listener, this representation of the environment gets concatenated with the message received from the speaker. In both cases, the vector gets passed into a fully connected layer that generates the agent's action. Agents with memory, have an additional single-layer LSTM [1]} after their fully connected layer.
