We consider the problem of distributed stochastic optimization in networks. Each node adjusts its action in order to optimize the global utility of the network, which is defined as the sum of local utilities of all nodes. Since the computation of the gradient may require much information exchange, we consider here that each node only has a noisy numerical observation of its local utility. This assumption is quite realistic, especially when the system is too complicated or constantly changing. At each time, nodes may exchange the observation of their numerical local utilities to estimate the global utility. Under the assumptions whether each node has collected the local utilities of all the other nodes or only part of these utilities, we propose two stochastic perturbation based distributed algorithms. We use tools from stochastic approximation to prove that both algorithms converge to the optimum. We then apply our algorithm to a power control problem in wireless networks and present numerical results that corroborate our claim.
* The first tutorial-style book that gives all the relevant theory at the right level of rigor, for the wireless communications engineer. * Bridges the gap between theory and practice by giving examples and case studies showing how game theory can solve real-word problems. * Contains algorithms and techniques to implement game theory in wireless terminals. Written by leading experts in the field, Game Theory and Learning for Wireless Networks Covers how theory can be used to solve prevalent problems in wireless networks such as power control, resource allocation or medium access control. With the emphasis now on promoting 'green' solutions in the wireless field where power consumption is minimized, there is an added focus on developing network solutions that maximizes the use of the spectrum available. With the growth of distributed wireless networks such as Wi-Fi and the Internet; the push to develop ad hoc and cognitive networks has led to a considerable interest in applying game theory to wireless communication systems. Game Theory and Learning for Wireless Networks is the first comprehensive resource of its kind, and is ideal for wireless communications R&D engineers and graduate students. Samson Lasaulce is a senior CNRS researcher at the Laboratory of Signals and Systems (LSS) at Suplec, Gif-sur-Yvette, France. He is also a part-time professor in the Department of Physics at cole Polytechnique, Palaiseau, France. Hamidou Tembine is a professor in the Department of Telecommunications at Suplec, Gif-sur-Yvette, France. Merouane Debbah is a professor at Suplec, Gif-sur-Yvette, France. He is the holder of the Alcatel-Lucent chair in flexible radio since 2007. The first tutorial style book that gives all the relevant theory, at the right level of rigour, for the wireless communications engineer Bridges the gap between theory and practice by giving examples and case studies showing how game theory can solve real world resource allocation problems Contains algorithms and techniques to implement game theory in wireless terminals In this paper, a decentralized and self-organizing mechanism for small cell networks (such as micro-, femto- and picocells) is proposed. In particular, an application to the case in which small cell networks aim to mitigate the interference caused to the macrocell network, while maximizing their own spectral efficiencies, is presented. The proposed mechanism is based on new notions of reinforcement learning (RL) through which small cells jointly estimate their time-average performance and optimize their probability distributions with which they judiciously choose their transmit configurations. Here, a minimum signal to interference plus noise ratio (SINR) is guaranteed at the macrocell user equipment (UE), while the small cells maximize their individual performances. The proposed RL procedure is fully distributed as every small cell base station requires only an observation of its instantaneous performance which can be obtained from its UE. Furthermore, it is shown that the proposed mechanism always converges to an epsilon Nash equilibrium when all small cells share the same interest. In addition, this mechanism is shown to possess better convergence properties and incur less overhead than existing techniques such as best response dynamics, fictitious play or classical RL. Finally, numerical results are given to validate the theoretical findings, highlighting the inherent tradeoffs facing small cells, namely exploration exploitation, myopic foresighted behavior and complete incomplete information.