In order to efficiently handle the continuous power action space, the DDPG unit is adopted to determine the power allocation. DDPG is an actor-critic off-policy RL algorithm [1]}, which utilizes the advantages of DPG theorem [2]} and the DQN algorithm. As shown in the right part of Fig. REF , the DDPG unit uses the actor network to generate the deterministic action and uses the critic network to evaluate the reward of the state-action pair. Similar to DQN, the target network is also used for the actor and critic network in DDPG in order to improve the stability. After obtaining the selected sub-band for the \(k\) -th agent with the observed state \(\mathbf {s}_k(t)\) , DDPG is applied to allocate power for this agent on the corresponding sub-band. Specifically, the \(k\) -th agent uses the actor network to deterministically generate power allocation \(a_k^p(t)\)  via the same state \(\mathbf {s}_k(t)\) , \(a_k^p(t) = \pi (\mathbf {s}_k(t),\mathbf {\phi })\) , where \(\pi (\mathbf {s}_k(t),\mathbf {\phi })\)  denotes the policy of the actor network with the network weights \(\mathbf {\phi }\) . In order to balance the exploration and the exploitation, a stochastic noise is introduced [1]}. Hence, the decision made by the actor network on power allocation for the agent \(k\)  is given by
\(a_k^p(t) = [\pi (\mathbf {s}_k(t),\mathbf {\phi })+ \eta ]_{0}^{P_{max}},\forall k\) 
