Deep Reinforcement Learning for Distributed and Uncoordinated Cognitive Radios Resource Allocation
This paper presents a novel deep reinforcement learning-based resource
allocation technique for the multi-agent environment presented by a cognitive
radio network where the interactions of the agents during learning may lead to
a non-stationary environment. The resource allocation technique presented in
this work is distributed, not requiring coordination with other agents. It is
shown by considering aspects specific to deep reinforcement learning that the
presented algorithm converges in an arbitrarily long time to equilibrium
policies in a non-stationary multi-agent environment that results from the
uncoordinated dynamic interaction between radios through the shared wireless
environment. Simulation results show that the presented technique achieves a
faster learning performance compared to an equivalent table-based Q-learning
algorithm and is able to find the optimal policy in 99% of cases for a
sufficiently long learning time. In addition, simulations show that our DQL
approach requires less than half the number of learning steps to achieve the
same performance as an equivalent table-based implementation. Moreover, it is
shown that the use of a standard single-agent deep reinforcement learning
approach may not achieve convergence when used in an uncoordinated interacting
multi-radio scenario