Deep neural network (DNN) based approaches hold significant potential for reinforcement learning (RL) and have already shown remarkable gains over state-of-art methods in a number of applications. The effectiveness of DNN methods can be attributed to leveraging the abundance of supervised data to learn value functions, Q-functions, and policy function approximations without the need for feature engineering. Nevertheless, the deployment of DNN-based predictors with very deep architectures can pose an issue due to computational and other resource constraints at test-time in a number of applications. We propose a novel approach for reducing the average latency by learning a computationally efficient gating function that is capable of recognizing states in a sequential decision process for which policy prescriptions of a shallow network suffices and deeper layers of the DNN have little marginal utility. The overall system is adaptive in that it dynamically switches control actions based on state-estimates in order to reduce average latency without sacrificing terminal performance. We experiment with a number of alternative loss-functions to train gating functions and shallow policies and show that in a number of applications a speed-up of up to almost 5X can be obtained with little loss in performance.
Finally, our approach draws inspiration from cognitive science and neuroscience. @cite_4 suggests that there are two systems in the brain, being coordinated by the uncertainty of the decision under a competition rule. @cite_19 implies that the switching of two systems that are with different levels of accuracy, reflect time, and metabolic cost, saves human energy budgets while maintaining a moderately good decision-making ability. This is reminiscent of our combination of gating, SNN and DNN to meet budget constraints.
@cite_19 @cite_4
2167362547 2540490069 2859887037 2024516306
A broad range of neural and behavioral data suggests that the brain contains multiple systems for behavioral choice, including one associated with prefrontal cortex and another with dorsolateral striatum. However, such a surfeit of control raises an additional choice problem: how to arbitrate between the systems when they disagree. Here, we consider dual-action choice systems from a normative perspective, using the computational theory of reinforcement learning. We identify a key trade-off pitting computational simplicity against the flexible and statistically efficient use of experience. The trade-off is realized in a competition between the dorsolateral striatal and prefrontal systems. We suggest a Bayesian principle of arbitration between them according to uncertainty, so each controller is deployed when it should be most accurate. This provides a unifying account of a wealth of experimental evidence about the factors favoring dominance by either system. In this paper, we study resource allocation for a multicarrier-based cognitive radio (CR) network. More specifically, we investigate the problem of secondary users’ energy-efficiency (EE) maximization problem under secondary total power and primary interference constraints. First, assuming cooperation among the secondary base stations (BSs), a centralized approach is considered to solve the EE optimization problem for the CR network where the primary and secondary users are using either orthogonal frequency-division multiplexing (OFDM) or filter bank based multicarrier (FBMC) modulations. We propose an alternating-based approach to solve the joint power-subcarrier allocation problem. More importantly, in the first place, subcarriers are allocated using a heuristic method for a given feasible power allocation. Then, we conservatively approximate the nonconvex power control problem and propose a joint successive convex approximation-Dinkelbach algorithm (SCADA) to efficiently obtain a solution to the nonconvex power control problem. The proposed algorithm is shown to converge to a solution that coincides with the stationary point of the original nonconvex power control subproblem. Moreover, we propose a dual decomposition-based decentralized version of the SCADA. Second, under the assumption of no cooperation among the secondary BSs, we propose a fully distributed power control algorithm from the perspective of game theory. The proposed algorithm is shown to converge to a Nash-equilibrium (NE) point. Moreover, we propose a sufficient condition that guarantees the uniqueness of the achieved NE. Extensive simulation analyses are further provided to highlight the advantages and demonstrate the efficiency of our proposed schemes. In this paper, we propose to use Deep Neural Networks (DNNs) to solve so-called Team Decision (TD) problems, in which decentralized Decision Makers (DMs) aim at maximizing a common utility on the basis of locally available Channel State Information (CSI) without any additional communication or iteration. In the proposed configuration -coined Team DNNs (T-DNNs)-, the decision at each DM is approximated using a DNN and the weights of all DNNs are jointly trained, even though the implementation remains fundamentally decentralized. Turning to a practical application, the problem of decentralized link scheduling in Interference Channels (IC) is reformulated as a TD problem so that the T-DNNs approach can be applied. After adequate training, the scheduling obtained using the T-DNNs flexibly adapts to the decentralized CSI configuration to outperform other scheduling algorithms, thus proposing a novel efficient solution to a problem that has remained elusive for years. This paper investigates the use of full duplex relaying (FDR) in cognitive radio systems. Cognitive full duplex relay networks (CogFRNs) offer the advantage not only of increasing spectral efficiency by spectrum sharing but also of extending coverage through the use of relays. Concurrent transmissions at the source and relay in CogFRNs can overcome a loss of resource efficiency in a way that conventional half duplex relay (HDR) systems cannot. However, in CogFRNs, the primary user experiences interference from the secondary source and relay simultaneously due to the effects of full duplexing. Satisfying the interference constraint by simply reducing transmission power results in performance degradation for the secondary user. What is therefore needed is a way to optimize the transmission powers at the secondary source and relay. To address this need, we propose an optimal power allocation scheme based on minimizing the outage probability in CogFRNs. We then analyze the outage probability of the secondary user in the noise-limited and interference-limited environments. In addition, we also propose an outage-constrained power allocation scheme which does not require the instantaneous channel state information (CSI) for the link between the primary and secondary users. Simulation results show that the proposed schemes achieve performance improvement in terms of outage probability.