Traditional radio systems are strictly co-designed on the lower levels of the OSI stack for compatibility and efficiency. Although this has enabled the success of radio communications, it has also introduced lengthy standardization processes and imposed static allocation of the radio spectrum. Various initiatives have been undertaken by the research community to tackle the problem of artificial spectrum scarcity by both making frequency allocation more dynamic and building flexible radios to replace the static ones. There is reason to believe that just as computer vision and control have been overhauled by the introduction of machine learning, wireless communication can also be improved by utilizing similar techniques to increase the flexibility of wireless networks. In this work, we pose the problem of discovering low-level wireless communication schemes ex-nihilo between two agents in a fully decentralized fashion as a reinforcement learning problem. Our proposed approach uses policy gradients to learn an optimal bi-directional communication scheme and shows surprisingly sophisticated and intelligent learning behavior. We present the results of extensive experiments and an analysis of the fidelity of our approach.
Since Widrow's early work, research into the use of neural architectures and machine learning techniques to address problems in communications has taken off in many directions. Ibnkahla's comprehensive review of the intersection of communications and machine learning @cite_9 is a testament to the impressive efforts made in this area of research. His survey lists various learning-based approaches to adaptive equalization, nonlinear channel modeling, coding, error correcting codes, spread spectrum applications, network planning, modulation detection and many more. Due to the vastness of this field, we refer the reader to the review. Research in making radio agents adaptive to achieve cooperative goals began in 1999, when Mitola introduced the concept of the cognitive radio @cite_13 . His proposal was for cognitive radios to use model-based reasoning to achieve competency in radio related tasks using both supervised and unsupervised learning. A review of cognitive radio work in years after that is in @cite_15 .
@cite_9 @cite_13 @cite_15
2071707134 2102295378 2179880807 1983578087
Cognitive radio is viewed as a novel approach for improving the utilization of a precious natural resource: the radio electromagnetic spectrum. The cognitive radio, built on a software-defined radio, is defined as an intelligent wireless communication system that is aware of its environment and uses the methodology of understanding-by-building to learn from the environment and adapt to statistical variations in the input stimuli, with two primary objectives in mind: spl middot highly reliable communication whenever and wherever needed; spl middot efficient utilization of the radio spectrum. Following the discussion of interference temperature as a new metric for the quantification and management of interference, the paper addresses three fundamental cognitive tasks. 1) Radio-scene analysis. 2) Channel-state estimation and predictive modeling. 3) Transmit-power control and dynamic spectrum management. This work also discusses the emergent behavior of cognitive radio. We propose a probabilistic model based on stochastic geometry to analyze cognitive radio in a large wireless network with randomly located users sharing the medium with carrier sensing multiple access. Analytical results are derived on the impact of the interaction between primary and secondary users, on their medium access probability, coverage probability and throughput. These results can be seen as the continuation of the theory of priorities in queueing theory to spatial processes. They give insight into the guarantees that can be offered to primary users and more generally on the possibilities offered by cognitive radio to improve the effectiveness of spectrum utilization in such networks. In this paper, we consider a communication scenario in which the primary and the cognitive radios wish to communicate to different receivers, subject to mutual interference. In the model that we use, the cognitive radio has noncausal knowledge of the primary radio's codeword. We characterize the largest rate at which the cognitive radio can reliably communicate under the constraint that 1) no rate degradation is created for the primary user, and 2) the primary receiver uses a single-user decoder just as it would in the absence of the cognitive radio. The result holds in a ldquolow-interferencerdquo regime in which the cognitive radio is closer to its receiver than to the primary receiver. In this regime, our results are subsumed by the results derived in a concurrent and independent work (Wu , 2007). We also demonstrate that, in a ldquohigh-interferencerdquo regime, multiuser decoding at the primary receiver is optimal from the standpoint of maximal jointly achievable rates for the primary and cognitive users. This paper describes a new class of attacks specific to cognitive radio networks. Wireless devices that can learn from their environment can also be taught things by malicious elements of their environment. By putting artificial intelligence in charge of wireless network devices, we are allowing unanticipated, emergent behavior, fitting a perhaps distorted or manipulated level of optimality. The state space for a cognitive radio is made up of a variety of learned beliefs and current sensor inputs. By manipulating radio sensor inputs, an adversary can affect the beliefs of a radio, and consequently its behavior. In this paper we focus primarily on PHY-layer issues, describing several classes of attacks and giving specific examples for dynamic spectrum access and adaptive radio scenarios. These attacks demonstrate the capabilities of an attacker who can manipulate the spectral environment when a radio is learning. The most powerful of which is a self-propagating AI virus that could interactively teach radios to become malicious. We then describe some approaches for mitigating the effectiveness of these attacks by instilling some level of "common sense" into radio systems, and requiring learned beliefs to expire and be relearned. Lastly we provide a road-map for extending these ideas to higher layers in the network stack.