Traditional radio systems are strictly co-designed on the lower levels of the OSI stack for compatibility and efficiency. Although this has enabled the success of radio communications, it has also introduced lengthy standardization processes and imposed static allocation of the radio spectrum. Various initiatives have been undertaken by the research community to tackle the problem of artificial spectrum scarcity by both making frequency allocation more dynamic and building flexible radios to replace the static ones. There is reason to believe that just as computer vision and control have been overhauled by the introduction of machine learning, wireless communication can also be improved by utilizing similar techniques to increase the flexibility of wireless networks. In this work, we pose the problem of discovering low-level wireless communication schemes ex-nihilo between two agents in a fully decentralized fashion as a reinforcement learning problem. Our proposed approach uses policy gradients to learn an optimal bi-directional communication scheme and shows surprisingly sophisticated and intelligent learning behavior. We present the results of extensive experiments and an analysis of the fidelity of our approach.
Since Widrow's early work, research into the use of neural architectures and machine learning techniques to address problems in communications has taken off in many directions. Ibnkahla's comprehensive review of the intersection of communications and machine learning @cite_9 is a testament to the impressive efforts made in this area of research. His survey lists various learning-based approaches to adaptive equalization, nonlinear channel modeling, coding, error correcting codes, spread spectrum applications, network planning, modulation detection and many more. Due to the vastness of this field, we refer the reader to the review. Research in making radio agents adaptive to achieve cooperative goals began in 1999, when Mitola introduced the concept of the cognitive radio @cite_13 . His proposal was for cognitive radios to use model-based reasoning to achieve competency in radio related tasks using both supervised and unsupervised learning. A review of cognitive radio work in years after that is in @cite_15 .
@cite_9 @cite_13 @cite_15
2071707134 2102295378 2179880807 1983578087 2963079272 2962981131 2123927904 2118292831 2102617152 2157564789 2163905491 2141915025 2160081863 2064008241 2097536950 1976294375 2044301459 2540574196 2168078104 2111744010
Cognitive radio is viewed as a novel approach for improving the utilization of a precious natural resource: the radio electromagnetic spectrum. The cognitive radio, built on a software-defined radio, is defined as an intelligent wireless communication system that is aware of its environment and uses the methodology of understanding-by-building to learn from the environment and adapt to statistical variations in the input stimuli, with two primary objectives in mind: spl middot highly reliable communication whenever and wherever needed; spl middot efficient utilization of the radio spectrum. Following the discussion of interference temperature as a new metric for the quantification and management of interference, the paper addresses three fundamental cognitive tasks. 1) Radio-scene analysis. 2) Channel-state estimation and predictive modeling. 3) Transmit-power control and dynamic spectrum management. This work also discusses the emergent behavior of cognitive radio. We propose a probabilistic model based on stochastic geometry to analyze cognitive radio in a large wireless network with randomly located users sharing the medium with carrier sensing multiple access. Analytical results are derived on the impact of the interaction between primary and secondary users, on their medium access probability, coverage probability and throughput. These results can be seen as the continuation of the theory of priorities in queueing theory to spatial processes. They give insight into the guarantees that can be offered to primary users and more generally on the possibilities offered by cognitive radio to improve the effectiveness of spectrum utilization in such networks. In this paper, we consider a communication scenario in which the primary and the cognitive radios wish to communicate to different receivers, subject to mutual interference. In the model that we use, the cognitive radio has noncausal knowledge of the primary radio's codeword. We characterize the largest rate at which the cognitive radio can reliably communicate under the constraint that 1) no rate degradation is created for the primary user, and 2) the primary receiver uses a single-user decoder just as it would in the absence of the cognitive radio. The result holds in a ldquolow-interferencerdquo regime in which the cognitive radio is closer to its receiver than to the primary receiver. In this regime, our results are subsumed by the results derived in a concurrent and independent work (Wu , 2007). We also demonstrate that, in a ldquohigh-interferencerdquo regime, multiuser decoding at the primary receiver is optimal from the standpoint of maximal jointly achievable rates for the primary and cognitive users. This paper describes a new class of attacks specific to cognitive radio networks. Wireless devices that can learn from their environment can also be taught things by malicious elements of their environment. By putting artificial intelligence in charge of wireless network devices, we are allowing unanticipated, emergent behavior, fitting a perhaps distorted or manipulated level of optimality. The state space for a cognitive radio is made up of a variety of learned beliefs and current sensor inputs. By manipulating radio sensor inputs, an adversary can affect the beliefs of a radio, and consequently its behavior. In this paper we focus primarily on PHY-layer issues, describing several classes of attacks and giving specific examples for dynamic spectrum access and adaptive radio scenarios. These attacks demonstrate the capabilities of an attacker who can manipulate the spectral environment when a radio is learning. The most powerful of which is a self-propagating AI virus that could interactively teach radios to become malicious. We then describe some approaches for mitigating the effectiveness of these attacks by instilling some level of "common sense" into radio systems, and requiring learned beliefs to expire and be relearned. Lastly we provide a road-map for extending these ideas to higher layers in the network stack. A novel approach of training data augmentation and domain adaptation is presented to support machine learning applications for cognitive radio. Machine learning provides effective tools to automate cognitive radio functionalities by reliably extracting and learning intrinsic spectrum dynamics. However, there are two important challenges to overcome, in order to fully utilize the machine learning benefits with cognitive radios. First, machine learning requires significant amount of truthed data to capture complex channel and emitter characteristics, and train the underlying algorithm (e.g., a classifier). Second, the training data that has been identified for one spectrum environment cannot be used for another one (e.g., after channel and emitter conditions change). To address these challenges, a generative adversarial network (GAN) with deep learning structures is used to 1) generate additional synthetic training data to improve classifier accuracy, and 2) adapt training data to spectrum dynamics. This approach is applied to spectrum sensing by assuming only limited training data without knowledge of spectrum statistics. Machine learning classifiers are trained to detect signals using either limited, augmented or adapted training data. Results show that training data augmentation increases the classifier accuracy significantly and this increase is sustained with domain adaptation while spectrum conditions change. We introduce learned attention models into the radio machine learning domain for the task of modulation recognition by leveraging spatial transformer networks and introducing new radio domain appropriate transformations. This attention model allows the network to learn a localization network capable of synchronizing and normalizing a radio signal blindly with zero knowledge of the signal's structure based on optimization of the network for classification accuracy, sparse representation, and regularization. Using this architecture we are able to outperform our prior results in accuracy vs signal to noise ratio against an identical system without attention, however we believe such an attention model has implication far beyond the task of modulation recognition. We consider a decentralized cognitive radio network in which autonomous secondary users seek spectrum opportunities in licensed spectrum bands. We assume that the primary users' channel occupancy follows a Markovian evolution, and formulate the spectrum sensing problem as a Decentralized Partially Observable Markov Decision Process (DEC-POMDP). We develop a distributed Reinforcement Learning (RL) algorithm that allows each autonomous cognitive radio to distributively learn its own spectrum sensing policy. The resulting decentralized sensing policy enables secondary users to non-cooperatively reach an equilibrium that leads to high utilization of idle channels while minimizing the collisions among secondary cognitive radios. Moreover, we propose a decentralized channel access policy that permits controlling, with high accuracy, the collision probability with primary users. Our numerical results validate the robustness of this collision probability control as the sensing noise changes. They also show the efficiency of the proposed learning algorithm in improving the spectrum utilization. The estimation of the communication performance achievable with respect to environmental factors and configuration parameters plays a key role in the optimization process performed by a Cognitive Radio according to the original definition by Mitola [1]. In this paper we propose the use of Multilayered Feedforward Neural Networks as an effective technique for real-time characterization of the communication performance which is based on measurements carried out by the device and therefore offers some interesting learning capabilities. Cognitive radio promises a low-cost, highly flexible alternative to the classic single-frequency band, single-protocol wireless device. By sensing and adapting to its environment, such a device is able to fill voids in the wireless spectrum and can dramatically increase spectral efficiency. In this paper, the cognitive radio channel is defined as a two-sender, two-receiver interference channel in which sender 2 obtains the encoded message sender 1 plans to transmit. We consider two cases: in the genie-aided cognitive radio channel, sender 2 is noncausally presented the data to be transmitted by sender 1 while in the causal cognitive radio channel, the data is obtained causally. The cognitive radio at sender 2 may then choose to transmit simultaneously over the same channel, as opposed to waiting for an idle channel as is traditional for a cognitive radio. Our main result is the development of an achievable region which combines Gel'fand-Pinkser coding with an achievable region construction for the interference channel. In the additive Gaussian noise case, this resembles dirty-paper coding, a technique used in the computation of the capacity of the Gaussian multiple-input multiple-output (MIMO) broadcast channel. Numerical evaluation of the region in the Gaussian noise case is performed, and compared to an inner bound, the interference channel, and an outer bound, a modified Gaussian MIMO broadcast channel. Results are also extended to the case in which the message is causally obtained. Cognitive radio is an emerging technology that aims for efficient spectrum usage by allowing unlicensed (secondary) users to access licensed frequency bands under the condition of protecting the licensed (primary) users from harmful interference. The latter condition constraints the achievable throughput of a cognitive radio network, which should therefore access a wideband spectrum in order to provide reliable and efficient services to its users. In this paper, we study the problem of designing the optimal sensing time and power allocation strategy, in order to maximize the ergodic throughput of a cognitive radio that employs simultaneous multiband detection and operates under two different schemes, namely the wideband sensing-based spectrum sharing (WSSS) and the wideband opportunistic spectrum access (WOSA) scheme. We consider average transmit and interference power constraints for both schemes, in order to effectively protect the primary users from harmful interference, propose two algorithms that acquire the optimal sensing time and power allocation under imperfect spectrum sensing for the two schemes and discuss the effect of the average transmit and interference power constraint on the optimal sensing time. Finally, we provide simulation results to compare the two schemes and validate our theoretical analysis. Wireless cognitive radio (CR) is a newly emerging paradigm that attempts to opportunistically transmit in licensed frequencies, without affecting the pre-assigned users of these bands. To enable this functionality, such a radio must predict its operational parameters, such as transmit power and spectrum. These tasks, collectively called spectrum management, is difficult to achieve in a dynamic distributed environment, in which CR users may only take local decisions, and react to the environmental changes. In this paper, we introduce a multi-agent reinforcement learning approach based spectrum management. Our approach uses value functions to evaluate the desirability of choosing different transmission parameters, and enables efficient assignment of spectrums and transmit powers by maximizing long-term reward. We then investigate various real-world scenarios, and compare the communication performance using different sets of learning parameters. We also apply Kanerva-based function approximation to improve our approach's ability to handle large cognitive radio networks and evaluate its effect on communication performance. We conclude that our reinforcement learning based spectrum management can significantly reduce the interference to the licensed users, while maintaining a high probability of successful transmissions in a cognitive radio ad hoc network. In this paper, optimal multi-channel cooperative sensing strategies in cognitive radio networks are investigated. A cognitive radio network with multiple potential channels is considered. Secondary users cooperatively sense the channels and send the sensing results to a coordinator, in which energy detection with a soft decision rule is employed to estimate whether there are primary activities in the channels. An optimization problem is formulated, which maximizes the throughput of secondary users while keeping detection probability for each channel above a pre-defined threshold. In particular, two sensing modes are investigated: slotted-time sensing mode and continuous-time sensing mode. With a slotted-time sensing mode, the sensing time of each secondary user consists of a number of mini-slots, each of which can be used to sense one channel. The initial optimization problem is shown to be a nonconvex mixed-integer problem. A polynomial-complexity algorithm is proposed to solve the problem optimally. With a continuous-time sensing mode, the sensing time of each secondary user for a channel can be any arbitrary continuous value. The initial nonconvex problem is converted into a convex bilevel problem, which can be successfully solved by existing methods. Numerical results are presented to demonstrate the effectiveness of our proposed algorithms. "Cognitive radio" is an emerging technique to improve the utilization of radio frequency spectrum in wireless networks. In this paper, we consider the problem of spectrum sharing among a primary user and multiple secondary users. We formulate this problem as an oligopoly market competition and use a noncooperative game to obtain the spectrum allocation for secondary users. Nash equilibrium is considered as the solution of this game. We first present the formulation of a static game for the case where all secondary users have the current information of the adopted strategies and the payoff of each other. However, this assumption may not be realistic in some cognitive radio systems. Therefore, we consider the case of bounded rationality in which the secondary users gradually and iteratively adjust their strategies based on the observations on their previous strategies. The speed of adjustment of the strategies is controlled by the learning rate. The stability condition of the dynamic behavior for this spectrum sharing scheme is investigated. The numerical results reveal the dynamics of distributed dynamic adaptation of spectrum sharing strategies. In this paper, we consider a sensing-based spectrum sharing scenario and present an efficient decentralized algorithm to maximize the total throughput of the cognitive radio users by optimizing jointly both the detection operation and the power allocation, taking into account the influence of the sensing accuracy. This optimization problem can be formulated as a distributed non-cooperative power allocation game, which can be solved by using an alternating direction optimization method. The transmit power budget of the cognitive radio users and the constraint related to the rate-loss of the primary user due to the interference are considered in the scheme. Finally, we use variational inequality theory in order to find the existence and uniqueness of the Nash equilibrium for our proposed distributed non-cooperative game. Cognitive radio and Dynamic Spectrum Access (DSA) enable wireless users to share a wide range of available spectrums. In this paper, we study joint spectrum allocation and scheduling problems in cognitive radio wireless networks with the objectives of achieving fair spectrum sharing. A novel Multi-Channel Contention Graph (MCCG) is proposed to characterize the impact of interference under the protocol model in such networks. Based on the MCCG, we present an optimal algorithm to compute maximum throughput solutions. As simply maximizing throughput may result in a severe bias on resource allocation, we take fairness into consideration by presenting optimal algorithms as well as fast heuristics to compute fair solutions based on a simplified max-min fairness model and the well-known proportional fairness model. Numerical results show that the performance given by our heuristic algorithms is very close to that of the optimal solution, and our proportional fair algorithms achieve a good tradeoff between throughput and fairness. In addition, we extend our research to the physical interference model, and propose effective heuristics for solving the corresponding problems. An Aloha-like spectrum access scheme without negotiation is considered for multiuser and multichannel cognitive radio systems. To avoid collisions incurred by the lack of coordination, each secondary user learns how to select channels according to its experience. Multiagent reinforcement leaning (MARL) is applied for the secondary users to learn good strategies of channel selection. Specifically, the framework of Q-learning is extended from single user case to multiagent case by considering other secondary users as a part of the environment. The dynamics of the Q-learning are illustrated using a Metrick-Polak plot, which shows the traces of Q-values in the two-user case. For both complete and partial observation cases, rigorous proofs of the convergence of multiagent Q-learning without communications, under certain conditions, are provided using the Robins-Monro algorithm and contraction mapping, respectively. The learning performance (speed and gain in utility) is evaluated by numerical simulations. In this paper we study the resource allocation problem for the multiuser orthogonal frequency division multiplexing (OFDM)-based cognitive radio (CR) systems with proportional rate constraints. The mutual interference introduced by primary user (PU) and cognitive radio user (also referred to secondary user, SU) makes the optimization problem of CR systems more complex. Moreover, the interference introduced to PUs must be kept under a given threshold. In this paper, the highest achievable rate of each OFDM subchannel is calculated by jointly considering the channel gain and interference level. First, a subchannel is assigned to the SU with the highest achievable rate. The remaining subchannels are always allocated to the SU that suffers the severest unjustness. Second, an efficient bit allocation algorithm is developed to maximize the sum capacity, which is again based on the highest achievable rate of each subchannel. Finally, an adjustment procedure is designed to maintain proportional fairness. Simulation results show that the proposed algorithm maximizes the sum capacity while keeping the proportional rate constraints satisfied. The algorithm exhibits a good tradeoff between sum capacity maximization and proportional fairness. Furthermore, the proposed algorithm has lower complexity compared with other algorithms, rendering it promising for practical applications. Copyright © 2011 John Wiley & Sons, Ltd. (By jointly considering the channel gain and the interference level of the OFDM-based cognitive radio systems, the highest achievable rate of each subchannel can be calculated, which indicates the ability of carrying bits of the subchannel. Using the normalized cost index list, an efficient algorithm is proposed to maximize the sum capacity of the cognitive OFDM systems while keeping the proportional rate constraints satisfied.) Motivated by cognitive radio applications, we consider mitigating the effect of interference by exploiting known properties about its signal structure. Specifically, we analyze communication between a source and destination with an interferer that induces random variations in the source-destination channel. The interferer transmits a sequence chosen uniformly from a randomly generated codebook, which has an i.i.d. structure, or a superposition structure. It is assumed that both the encoder and decoder know the interferer's codebook. We first provide a definition of capacity for these settings. When the encoder knows the interferer's message noncausally, it can use Gel'fand-Pinsker (GP) encoding to precode against interference. Alternatively, it can encode by taking into account that the interference is a codeword, to enable the decoder to decode both messages. It is demonstrated by an example that the latter can outperform GP encoding. Two upper bounds to the performance of this channel are then presented. Next, a more realistic scenario is considered in which the interference is learned at the cognitive encoder causally through a noisy channel. It is shown that for the case of i.i.d. generated interference, this information has no value. In contrast, when the interference is a codeword from an i.i.d. generated codebook, this fact can be exploited to obtain higher rates between the cognitive pair. Software radios are emerging as platforms for multiband multimode personal communications systems. Radio etiquette is the set of RF bands, air interfaces, protocols, and spatial and temporal patterns that moderate the use of the radio spectrum. Cognitive radio extends the software radio with radio-domain model-based reasoning about such etiquettes. Cognitive radio enhances the flexibility of personal services through a radio knowledge representation language. This language represents knowledge of radio etiquette, devices, software modules, propagation, networks, user needs, and application scenarios in a way that supports automated reasoning about the needs of the user. This empowers software radios to conduct expressive negotiations among peers about the use of radio spectrum across fluents of space, time, and user context. With RKRL, cognitive radio agents may actively manipulate the protocol stack to adapt known etiquettes to better satisfy the user's needs. This transforms radio nodes from blind executors of predefined protocols to radio-domain-aware intelligent agents that search out ways to deliver the services the user wants even if that user does not know how to obtain them. Software radio provides an ideal platform for the realization of cognitive radio. Cognitive radios (CRs) have a great potential to improve spectrum utilization by enabling users to access the spectrum dynamically without disturbing licensed primary radios (PRs). A key challenge in operating these radios as a network is how to implement an efficient medium access control (MAC) mechanism that can adaptively and efficiently allocate transmission powers and spectrum among CRs according to the surrounding environment. Most existing works address this issue via suboptimal heuristic approaches or centralized solutions. In this paper, we propose a novel joint power channel allocation scheme that improves the performance through a distributed pricing approach. In this scheme, the spectrum allocation problem is modeled as a noncooperative game, with each CR pair acting as a player. A price-based iterative water-filling (PIWF) algorithm is proposed, which enables CR users to reach a good Nash equilibrium (NE). This PIWF algorithm can be implemented distributively with CRs repeatedly negotiating their best transmission powers and spectrum. Simulation results show that the social optimality of the NE solution is dramatically improved through pricing. Depending on the different orders according to which CRs take actions, we study sequential and parallel versions of the PIWF algorithm. We show that the parallel version converges faster than the sequential version. We then propose a corresponding MAC protocol to implement our resource management schemes. The proposed MAC allows multiple CR pairs to be first involved in an admission phase, then iteratively negotiate their transmission powers and spectrum via control-packet exchanges. Following the negotiation phase, CRs proceed concurrently with their data transmissions. Simulations are used to study the performance of our protocol and demonstrate its effectiveness in terms of improving the overall network throughput and reducing the average power consumption.