Traditional radio systems are strictly co-designed on the lower levels of the OSI stack for compatibility and efficiency. Although this has enabled the success of radio communications, it has also introduced lengthy standardization processes and imposed static allocation of the radio spectrum. Various initiatives have been undertaken by the research community to tackle the problem of artificial spectrum scarcity by both making frequency allocation more dynamic and building flexible radios to replace the static ones. There is reason to believe that just as computer vision and control have been overhauled by the introduction of machine learning, wireless communication can also be improved by utilizing similar techniques to increase the flexibility of wireless networks. In this work, we pose the problem of discovering low-level wireless communication schemes ex-nihilo between two agents in a fully decentralized fashion as a reinforcement learning problem. Our proposed approach uses policy gradients to learn an optimal bi-directional communication scheme and shows surprisingly sophisticated and intelligent learning behavior. We present the results of extensive experiments and an analysis of the fidelity of our approach.
Since Widrow's early work, research into the use of neural architectures and machine learning techniques to address problems in communications has taken off in many directions. Ibnkahla's comprehensive review of the intersection of communications and machine learning @cite_9 is a testament to the impressive efforts made in this area of research. His survey lists various learning-based approaches to adaptive equalization, nonlinear channel modeling, coding, error correcting codes, spread spectrum applications, network planning, modulation detection and many more. Due to the vastness of this field, we refer the reader to the review. Research in making radio agents adaptive to achieve cooperative goals began in 1999, when Mitola introduced the concept of the cognitive radio @cite_13 . His proposal was for cognitive radios to use model-based reasoning to achieve competency in radio related tasks using both supervised and unsupervised learning. A review of cognitive radio work in years after that is in @cite_15 .
@cite_9 @cite_13 @cite_15
2168078104 2065715415
Contributors. Foreword 1. Foreword 2. Preface. Acknowledgements. Introduction. Chapter 1: Biologically Inspired Networking. Chapter 2: The Role of Autonomic Networking in Cognitive Networks. Chapter 3: Adaptive Networks. Chapter 4: Self-Managing Networks. Chapter 5: Machine Learning for Cognitive Networks: Technology Assessment and Research Challenges. Chapter 6: Cross-Layer Design and Optimization in Wireless Networks. Chapter 7: Cognitive Radio Architecture. Chapter 8: The Wisdom of Crowds: Cognitive Ad hoc Networks. Chapter 9: Distributed Learning and Reasoning in Cognitive Networks: Methods and Design Decisions. Chapter 10: The Semantic Side of Cognitive Radio. Chapter 11: Security Issues in Cognitive Radio Networks. Chapter 12: Intrusion Detection in Cognitive Networks. Chapter 13: Erasure Tolerant Coding for Cognitive Radios. Index. Cognitive radio is viewed as a novel approach for improving the utilization of a precious natural resource: the radio electromagnetic spectrum. The cognitive radio, built on a software-defined radio, is defined as an intelligent wireless communication system that is aware of its environment and uses the methodology of understanding-by-building to learn from the environment and adapt to statistical variations in the input stimuli, with two primary objectives in mind: spl middot highly reliable communication whenever and wherever needed; spl middot efficient utilization of the radio spectrum. Following the discussion of interference temperature as a new metric for the quantification and management of interference, the paper addresses three fundamental cognitive tasks. 1) Radio-scene analysis. 2) Channel-state estimation and predictive modeling. 3) Transmit-power control and dynamic spectrum management. This work also discusses the emergent behavior of cognitive radio. The spectrum sensing problem has gained new aspects with cognitive radio and opportunistic spectrum access concepts. It is one of the most challenging issues in cognitive radio systems. In this paper, a survey of spectrum sensing methodologies for cognitive radio is presented. Various aspects of spectrum sensing problem are studied from a cognitive radio perspective and multi-dimensional spectrum sensing concept is introduced. Challenges associated with spectrum sensing are given and enabling spectrum sensing methods are reviewed. The paper explains the cooperative sensing concept and its various forms. External sensing algorithms and other alternative sensing methods are discussed. Furthermore, statistical modeling of network traffic and utilization of these models for prediction of primary user behavior is studied. Finally, sensing features of some current wireless standards are given.