Traditional radio systems are strictly co-designed on the lower levels of the OSI stack for compatibility and efficiency. Although this has enabled the success of radio communications, it has also introduced lengthy standardization processes and imposed static allocation of the radio spectrum. Various initiatives have been undertaken by the research community to tackle the problem of artificial spectrum scarcity by both making frequency allocation more dynamic and building flexible radios to replace the static ones. There is reason to believe that just as computer vision and control have been overhauled by the introduction of machine learning, wireless communication can also be improved by utilizing similar techniques to increase the flexibility of wireless networks. In this work, we pose the problem of discovering low-level wireless communication schemes ex-nihilo between two agents in a fully decentralized fashion as a reinforcement learning problem. Our proposed approach uses policy gradients to learn an optimal bi-directional communication scheme and shows surprisingly sophisticated and intelligent learning behavior. We present the results of extensive experiments and an analysis of the fidelity of our approach.
Since Widrow's early work, research into the use of neural architectures and machine learning techniques to address problems in communications has taken off in many directions. Ibnkahla's comprehensive review of the intersection of communications and machine learning @cite_9 is a testament to the impressive efforts made in this area of research. His survey lists various learning-based approaches to adaptive equalization, nonlinear channel modeling, coding, error correcting codes, spread spectrum applications, network planning, modulation detection and many more. Due to the vastness of this field, we refer the reader to the review. Research in making radio agents adaptive to achieve cooperative goals began in 1999, when Mitola introduced the concept of the cognitive radio @cite_13 . His proposal was for cognitive radios to use model-based reasoning to achieve competency in radio related tasks using both supervised and unsupervised learning. A review of cognitive radio work in years after that is in @cite_15 .
@cite_9 @cite_13 @cite_15
2168078104 2065715415
Contributors. Foreword 1. Foreword 2. Preface. Acknowledgements. Introduction. Chapter 1: Biologically Inspired Networking. Chapter 2: The Role of Autonomic Networking in Cognitive Networks. Chapter 3: Adaptive Networks. Chapter 4: Self-Managing Networks. Chapter 5: Machine Learning for Cognitive Networks: Technology Assessment and Research Challenges. Chapter 6: Cross-Layer Design and Optimization in Wireless Networks. Chapter 7: Cognitive Radio Architecture. Chapter 8: The Wisdom of Crowds: Cognitive Ad hoc Networks. Chapter 9: Distributed Learning and Reasoning in Cognitive Networks: Methods and Design Decisions. Chapter 10: The Semantic Side of Cognitive Radio. Chapter 11: Security Issues in Cognitive Radio Networks. Chapter 12: Intrusion Detection in Cognitive Networks. Chapter 13: Erasure Tolerant Coding for Cognitive Radios. Index. Cognitive radio is viewed as a novel approach for improving the utilization of a precious natural resource: the radio electromagnetic spectrum. The cognitive radio, built on a software-defined radio, is defined as an intelligent wireless communication system that is aware of its environment and uses the methodology of understanding-by-building to learn from the environment and adapt to statistical variations in the input stimuli, with two primary objectives in mind: spl middot highly reliable communication whenever and wherever needed; spl middot efficient utilization of the radio spectrum. Following the discussion of interference temperature as a new metric for the quantification and management of interference, the paper addresses three fundamental cognitive tasks. 1) Radio-scene analysis. 2) Channel-state estimation and predictive modeling. 3) Transmit-power control and dynamic spectrum management. This work also discusses the emergent behavior of cognitive radio. The spectrum sensing problem has gained new aspects with cognitive radio and opportunistic spectrum access concepts. It is one of the most challenging issues in cognitive radio systems. In this paper, a survey of spectrum sensing methodologies for cognitive radio is presented. Various aspects of spectrum sensing problem are studied from a cognitive radio perspective and multi-dimensional spectrum sensing concept is introduced. Challenges associated with spectrum sensing are given and enabling spectrum sensing methods are reviewed. The paper explains the cooperative sensing concept and its various forms. External sensing algorithms and other alternative sensing methods are discussed. Furthermore, statistical modeling of network traffic and utilization of these models for prediction of primary user behavior is studied. Finally, sensing features of some current wireless standards are given. Cognitive radio is widely expected to be the next Big Bang in wireless communications. Spectrum sensing, that is, detecting the presence of the primary users in a licensed spectrum, is a fundamental problem for cognitive radio. As a result, spectrum sensing has reborn as a very active research area in recent years despite its long history. In this paper, spectrum sensing techniques from the optimal likelihood ratio test to energy detection, matched filtering detection, cyclostationary detection, eigenvalue-based sensing, joint space-time sensing, and robust sensing methods are reviewed. Cooperative spectrum sensing with multiple receivers is also discussed. Special attention is paid to sensing methods that need little prior information on the source signal and the propagation channel. Practical challenges such as noise power uncertainty are discussed and possible solutions are provided. Theoretical analysis on the test statistic distribution and threshold setting is also investigated. In the past decade, cognitive radio and cooperative communication techniques have been proposed in the literature for efficiently utilizing the radio resources. Cognitive radio is an emerging technology intended to enhance the utilization of the radio frequency spectrum. The cooperative communication system, with the same total power and bandwidth of legacy wireless communication systems, can increase the data rate of the future wireless communication system. A combination of cognitive radio with cooperative communication can further improve the future wireless network performance. Efficient resource allocation in cooperative cognitive radio network (CRN) is essential in order to meet the challenges of future wireless networks. In this article, a survey of resource allocation in cooperative CRN is presented. We discuss the taxonomy of objectives and protocols used in the literature for resource allocation in cooperative CRN. This paper also highlights the use of power control, cooperation types, network configurations and decision types used in cooperative CRN. Finally, directions for future research are outlined. Software radios are emerging as platforms for multiband multimode personal communications systems. Radio etiquette is the set of RF bands, air interfaces, protocols, and spatial and temporal patterns that moderate the use of the radio spectrum. Cognitive radio extends the software radio with radio-domain model-based reasoning about such etiquettes. Cognitive radio enhances the flexibility of personal services through a radio knowledge representation language. This language represents knowledge of radio etiquette, devices, software modules, propagation, networks, user needs, and application scenarios in a way that supports automated reasoning about the needs of the user. This empowers software radios to conduct expressive negotiations among peers about the use of radio spectrum across fluents of space, time, and user context. With RKRL, cognitive radio agents may actively manipulate the protocol stack to adapt known etiquettes to better satisfy the user's needs. This transforms radio nodes from blind executors of predefined protocols to radio-domain-aware intelligent agents that search out ways to deliver the services the user wants even if that user does not know how to obtain them. Software radio provides an ideal platform for the realization of cognitive radio. A novel approach of training data augmentation and domain adaptation is presented to support machine learning applications for cognitive radio. Machine learning provides effective tools to automate cognitive radio functionalities by reliably extracting and learning intrinsic spectrum dynamics. However, there are two important challenges to overcome, in order to fully utilize the machine learning benefits with cognitive radios. First, machine learning requires significant amount of truthed data to capture complex channel and emitter characteristics, and train the underlying algorithm (e.g., a classifier). Second, the training data that has been identified for one spectrum environment cannot be used for another one (e.g., after channel and emitter conditions change). To address these challenges, a generative adversarial network (GAN) with deep learning structures is used to 1) generate additional synthetic training data to improve classifier accuracy, and 2) adapt training data to spectrum dynamics. This approach is applied to spectrum sensing by assuming only limited training data without knowledge of spectrum statistics. Machine learning classifiers are trained to detect signals using either limited, augmented or adapted training data. Results show that training data augmentation increases the classifier accuracy significantly and this increase is sustained with domain adaptation while spectrum conditions change. Cognitive radios are promising solutions to the problem of overcrowded and inefficient licensed spectrum. In this work we explore the throughput potential of cognitive communication. We summarize different cognitive radio techniques that underlay, overlay and interweave the transmissions of the cognitive user with those of the licensed users. Recently proposed models for cognitive radios based on the overlay technique are described. For the interweave technique, we present a two switch' cognitive radio model and develop inner and outer bounds on the secondary radio capacity. Using the two switch model, we investigate the inherent tradeoff between the sensitivity of primary detection and the cognitive link capacity. With numerical results, we compare the throughputs achieved by the secondary user in the different models. Cognitive radio promises a low-cost, highly flexible alternative to the classic single-frequency band, single-protocol wireless device. By sensing and adapting to its environment, such a device is able to fill voids in the wireless spectrum and can dramatically increase spectral efficiency. In this paper, the cognitive radio channel is defined as a two-sender, two-receiver interference channel in which sender 2 obtains the encoded message sender 1 plans to transmit. We consider two cases: in the genie-aided cognitive radio channel, sender 2 is noncausally presented the data to be transmitted by sender 1 while in the causal cognitive radio channel, the data is obtained causally. The cognitive radio at sender 2 may then choose to transmit simultaneously over the same channel, as opposed to waiting for an idle channel as is traditional for a cognitive radio. Our main result is the development of an achievable region which combines Gel'fand-Pinkser coding with an achievable region construction for the interference channel. In the additive Gaussian noise case, this resembles dirty-paper coding, a technique used in the computation of the capacity of the Gaussian multiple-input multiple-output (MIMO) broadcast channel. Numerical evaluation of the region in the Gaussian noise case is performed, and compared to an inner bound, the interference channel, and an outer bound, a modified Gaussian MIMO broadcast channel. Results are also extended to the case in which the message is causally obtained. Wireless cognitive radio (CR) is a newly emerging paradigm that attempts to opportunistically transmit in licensed frequencies, without affecting the pre-assigned users of these bands. To enable this functionality, such a radio must predict its operational parameters, such as transmit power and spectrum. These tasks, collectively called spectrum management, is difficult to achieve in a dynamic distributed environment, in which CR users may only take local decisions, and react to the environmental changes. In this paper, we introduce a multi-agent reinforcement learning approach based spectrum management. Our approach uses value functions to evaluate the desirability of choosing different transmission parameters, and enables efficient assignment of spectrums and transmit powers by maximizing long-term reward. We then investigate various real-world scenarios, and compare the communication performance using different sets of learning parameters. We also apply Kanerva-based function approximation to improve our approach's ability to handle large cognitive radio networks and evaluate its effect on communication performance. We conclude that our reinforcement learning based spectrum management can significantly reduce the interference to the licensed users, while maintaining a high probability of successful transmissions in a cognitive radio ad hoc network. Radio frequency (RF) energy transfer and harvesting techniques have recently become alternative methods to power the next-generation wireless networks. As this emerging technology enables proactive energy replenishment of wireless devices, it is advantageous in supporting applications with quality-of-service requirements. In this paper, we present a comprehensive literature review on the research progresses in wireless networks with RF energy harvesting capability, which is referred to as RF energy harvesting networks (RF-EHNs). First, we present an overview of the RF-EHNs including system architecture, RF energy harvesting techniques, and existing applications. Then, we present the background in circuit design as well as the state-of-the-art circuitry implementations and review the communication protocols specially designed for RF-EHNs. We also explore various key design issues in the development of RF-EHNs according to the network types, i.e., single-hop networks, multiantenna networks, relay networks, and cognitive radio networks. Finally, we envision some open research directions. Cognitive networks enable efficient sharing of the radio spectrum. Multi-hop cognitive network is a cooperative network in which cognitive users take help of their neighbors to forward data to the destination. Control signals used to enable cooperation communicate through a common control channel (CCC). Such usage introduces conditions like channel saturation which degrades the overall performance of the network. Thus, exchanging control information is a major challenge in cognitive radio networks. This paper proposes an alternative MAC protocol for multi-hop cognitive radio networks in which the use of a CCC is avoided. The scheme is applicable in heterogeneous environments where channels have different bandwidths and frequencies of operation. It inherently provides a solution to issues like CCC saturation problem, Denial of Service attacks (DoS) and multi-channel hidden problem. The proposed protocol is shown to provide better connectivity and higher throughput than a CCC based protocol, especially when the network is congested. Dynamic spectrum access has been proposed as a means to share scarce radio resources, and requires devices to follow protocols that use resources in a proper, disciplined manner. For a cognitive radio network to achieve this goal, spectrum policies and the ability to enforce them are necessary. Detection of an unauthorized (anomalous) usage is one of the critical issues in spectrum etiquette enforcement. In this paper, we present a network structure for dynamic spectrum access and formulate the anomalous usage detection problem using statistical significance testing. The detection problem is classified into two subproblems. For the case where no authorized signal is present, we describe the existing cooperative sensing schemes and investigate the impact of signal path loss on their performance. For the case where an authorized signal is present, we propose three methods that detect anomalous transmissions by making use of the characteristics of radio propagation. Analytical models are formulated for two special cases and, due to the intractability of the general problem, we present an algorithm using machine learning techniques to solve the general case. Our simulation results show that our approaches can effectively detect unauthorized spectrum usage with high detection rate and low false positive rate. The estimation of the communication performance achievable with respect to environmental factors and configuration parameters plays a key role in the optimization process performed by a Cognitive Radio according to the original definition by Mitola [1]. In this paper we propose the use of Multilayered Feedforward Neural Networks as an effective technique for real-time characterization of the communication performance which is based on measurements carried out by the device and therefore offers some interesting learning capabilities. Cognitive radio is a novel concept that enables wireless systems to sense the environment, adapt, and learn from previous experience to improve the quality of the communication. However, CR requires a flexible and adaptive physical layer in order to perform the required tasks efficiently. In this article, CR systems and their requirement of a physical layer are discussed, and the orthogonal frequency division multiplexing technique is investigated as a candidate transmission technology for CR. The challenges that arise from employing OFDM in CR systems are identified. The cognitive properties of some OFDM-based wireless standards also are discussed to indicate the trend toward a more cognitive radio. With the development of cognitive radio technologies, dynamic spectrum access becomes a promising approach to increase the efficiency of spectrum utilization and solve spectrum scarcity problem. Under dynamic spectrum access, unlicensed wireless users (secondary users) can dynamically access the licensed bands from legacy spectrum holders (primary users) on an opportunistic basis. While most primary users in existing works assume secondary transmissions as negative interference and don't actively involve them into the primary transmission, in this paper, motivated by the idea of cooperative communication, we propose a cooperative cognitive radio framework, where primary users, aware of the existence of secondary users, may select some of them to be the cooperative relay, and in return lease portion of the channel access time to them for their own data transmission. Secondary users cooperating with primary transmissions have the right to decide their payment made for primary user in order to achieve a proportional access time to the wireless media. Both primary and secondary users target at maximizing their utilities in terms of their transmission rate and revenue payment. This model is formulated as a Stackelberg game and a unique Nash Equilibrium point is achieved in analytical format. Based on the analysis we discuss the condition under which cooperation will increase the performance of the whole system. Both analytical result and numerical result show that the cooperative cognitive radio framework is a promising framework under which the utility of both primary and secondary system are maximized. Spectrum sensing is required for many cognitive radio applications, including spectral awareness, interoperability, and dynamic spectrum access. Any approach is highly dependent on the signal environment in which it operates. Noise, multipath channel distortion, sample-rate offsets, and center-frequency mis-estimations can make feature-based signal detection difficult. Additionally, as these values change in a dynamic environment, statically-trained classifiers cannot track the evolving class statistics. Unsupervised learning allows a cognitive radio system to evolve its classifier as the radio environment evolves, without the need for expert-annotated signal samples. While this allows improved classifier performance, previous work has demonstrated that unsupervised techniques developed without security in mind suffer from classifier manipulation through transmission of carefully crafted signals that shift decision boundaries. This paper develops countermeasures to the class manipulation attacks that mitigates their efficacy, and shows the robustness of unsupervised learning for signal classification in an evolving RF environment. Spectrum sensing, which aims at detecting spectrum holes, is the precondition for the implementation of cognitive radio (CR). Collaborative spectrum sensing among the cognitive radio nodes is expected to improve the ability of checking complete spectrum usage. Due to hardware limitations, each cognitive radio node can only sense a relatively narrow band of radio spectrum. Consequently, the available channel sensing information is far from being sufficient for precisely recognizing the wide range of unoccupied channels. Aiming at breaking this bottleneck, we propose to apply matrix completion and joint sparsity recovery to reduce sensing and transmission requirements and improve sensing results. Specifically, equipped with a frequency selective filter, each cognitive radio node senses linear combinations of multiple channel information and reports them to the fusion center, where occupied channels are then decoded from the reports by using novel matrix completion and joint sparsity recovery algorithms. As a result, the number of reports sent from the CRs to the fusion center is significantly reduced. We propose two decoding approaches, one based on matrix completion and the other based on joint sparsity recovery, both of which allow exact recovery from incomplete reports. The numerical results validate the effectiveness and robustness of our approaches. In particular, in small-scale networks, the matrix completion approach achieves exact channel detection with a number of samples no more than 50 of the number of channels in the network, while joint sparsity recovery achieves similar performance in large-scale networks. We consider a decentralized cognitive radio network in which autonomous secondary users seek spectrum opportunities in licensed spectrum bands. We assume that the primary users' channel occupancy follows a Markovian evolution, and formulate the spectrum sensing problem as a Decentralized Partially Observable Markov Decision Process (DEC-POMDP). We develop a distributed Reinforcement Learning (RL) algorithm that allows each autonomous cognitive radio to distributively learn its own spectrum sensing policy. The resulting decentralized sensing policy enables secondary users to non-cooperatively reach an equilibrium that leads to high utilization of idle channels while minimizing the collisions among secondary cognitive radios. Moreover, we propose a decentralized channel access policy that permits controlling, with high accuracy, the collision probability with primary users. Our numerical results validate the robustness of this collision probability control as the sensing noise changes. They also show the efficiency of the proposed learning algorithm in improving the spectrum utilization. Cognitive Radio (CR) is a next-generation wireless communication system that enables unlicensed users to exploit underutilized licensed spectrum to optimize the utilization of the overall radio spectrum. A Distributed Cognitive Radio Network (DCRN) is a distributed wireless network established by a number of unlicensed users in the absence of fixed network infrastructure such as a base station. Context awareness and intelligence are the capabilities that enable each unlicensed user to observe and carry out its own action as part of the joint action on its operating environment for network-wide performance enhancement. These capabilities can be applied in various application schemes in CR networks such as Dynamic Channel Selection (DCS), congestion control, and scheduling. In this paper, we apply Reinforcement Learning (RL), including single-agent and multi-agent approaches, to achieve context awareness and intelligence. Firstly, we show that the RL approach achieves a joint action that provides better network-wide performance in respect to DCS in DCRNs. The multi-agent approach is shown to provide higher levels of stability compared to the single-agent approach. Secondly, we show that RL achieves high level of fairness. Thirdly, we show the effects of network density and various essential parameters in RL on the network-wide performance.