A communication-based multi-agent fully cooperative task in the partially observable setting can be formulated as a Dec-POMDP [1]}, consisting of a tuple
\( G=\langle I,S,\ \Omega ,O,M,F,U,P,R,\ n,\gamma \rangle \) , where \( I\equiv \left\lbrace 1,\ldots ,n\right\rbrace  \)  describes the set of agents, \( S \)  is denotes true state of
the environment, \( \mathrm {\Omega } \)  denotes the set of joint observations. At each time step, due to partially observability, an agent obtains
its observation \( o\in \mathrm {\Omega } \)  based on the observation function \( O\left(s,\mathbf {u}\right):S\times \mathbf {U}\rightarrow \mathrm {\Omega } \) ,
where \( \mathbf {U}\equiv U^n \)  is the set of joint actions. Each agent has an action-observation history \( \tau \in T \equiv (\Omega \times U)^\ast \) .
The message \(m\in M\)  is produced by the message function \( F(\tau ): T \rightarrow M \)  and then propagated to others in the communication.
Each agent i chooses an action \(u_i\in U\)  by a stochastic policy \(\pi _i\left(u_i|\tau _i,m_i\right):\left(T\times M\right)^\ast \times U\rightarrow \left[0,1\right]\) ,
forming a joint action \( \mathbf {u}\in \mathbf {U} \) , which leads to a shared reward according to the reward function \( r(s,\mathbf {u}):S \times \mathbf {U} \rightarrow \mathbb {R}\) 
and a transition on the environment through the state transition function \( P\left(s^\prime | s,a\right):S\times \mathbf {U}\rightarrow S\) .
The goal of the task is to learn the decentralised action-value by maximising the total expected return \( R^t=\sum _{t=0}^{T}{\gamma ^tr^t} \) ,
where \(T\)  is the length of the episode and \( \gamma \in \left[0,1\right) \)  is a discount factor.
