Cellular network configuration plays a critical role in network performance. In current practice, network configuration depends heavily on field experience of engineers and often remains static for a long period of time. This practice is far from optimal. To address this limitation, online-learning-based approaches have great potentials to automate and optimize network configuration. Learning-based approaches face the challenges of learning a highly complex function for each base station and balancing the fundamental exploration-exploitation tradeoff while minimizing the exploration cost. Fortunately, in cellular networks, base stations (BSs) often have similarities even though they are not identical. To leverage such similarities, we propose kernel-based multi-BS contextual bandit algorithm based on multi-task learning. In the algorithm, we leverage the similarity among different BSs defined by conditional kernel embedding. We present theoretical analysis of the proposed algorithm in terms of regret and multi-task-learning efficiency. We evaluate the effectiveness of our algorithm based on a simulator built by real traces.
Recently, learning-based methods are proposed @cite_0 @cite_19 @cite_10 @cite_4 . In @cite_0 , the authors propose a tailored form of reinforcement learning to adaptively select the optimal antenna configuration in a time-varying environment. In @cite_4 , the authors use Q-learning with compact state representation for traffic offloading. In @cite_10 , the authors design a generalized global bandit algorithm to control the transmit power in the cellular coverage optimization problem. In all these papers, BS similarities are not considered, and thus require more exploration. In @cite_19 , the authors study the pilot power configuration problem and design a Gibbs-sampling-based online learning algorithm so as to maximize the throughput of users. In comparison, they make the assumption that all BSs are equal while we allow different BSs to learn different mappings.
@cite_0 @cite_19 @cite_10 @cite_4
2125890412 2125725268 2163190245 2791827017
This paper first provides a brief survey on existing traffic offloading techniques in wireless networks. Particularly as a case study, we put forward an online reinforcement learning framework for the problem of traffic offloading in a stochastic heterogeneous cellular network (HCN), where the time-varying traffic in the network can be offloaded to nearby small cells. Our aim is to minimize the total discounted energy consumption of the HCN while maintaining the quality-of-service (QoS) experienced by mobile users. For each cell (i.e., a macro cell or a small cell), the energy consumption is determined by its system load, which is coupled with system loads in other cells due to the sharing over a common frequency band. We model the energy-aware traffic offloading problem in such HCNs as a discrete-time Markov decision process (DTMDP). Based on the traffic observations and the traffic offloading operations, the network controller gradually optimizes the traffic offloading strategy with no prior knowledge of the DTMDP statistics. Such a model-free learning framework is important, particularly when the state space is huge. In order to solve the curse of dimensionality, we design a centralized @math -learning with compact state representation algorithm, which is named @math -learning. Moreover, a decentralized version of the @math -learning is developed based on the fact the macro base stations (BSs) can independently manage the operations of local small-cell BSs through making use of the global network state information obtained from the network controller. Simulations are conducted to show the effectiveness of the derived centralized and decentralized @math -learning algorithms in balancing the tradeoff between energy saving and QoS satisfaction. Universal mobile telecommunication system (UMTS) networks should be deployed according to cost-effective strategies that optimize a cost objective and satisfy target quality-of-service (QoS) requirements. In this paper, we propose novel algorithms for joint uplink downlink UMTS radio planning with the objective of minimizing total power consumption in the network. Specifically, we define two components of the radio planning problem: 1) continuous-based site placement and 2) integer-based site selection. In the site-placement problem, our goal is to find the optimal locations of UMTS base stations (BSs) in a certain geographic area with a given user distribution to minimize the total power expenditure such that a satisfactory level of downlink and uplink signal-to-interference ratio (SIR) is maintained with bounded outage constraints. We model the problem as a constrained optimization problem with SIR-based uplink and downlink power control scheme. An algorithm is proposed and implemented using pattern search techniques for derivative-free optimization with augmented Lagrange multiplier estimates to support general constraints. In the site-selection problem, we aim to select the minimum set of BSs from a fixed set of candidate sites that satisfies quality and outage constraints. We develop an efficient elimination algorithm by proposing a method for classifying BSs that are critical for network coverage and QoS. Finally, the problem is reformulated to take care of location constraints whereby the placement of BSs in a subset of the deployment area is not permitted due to, e.g., private property limitations or electromagnetic radiation constraints. Experimental results and optimal tradeoff curves are presented and analyzed for various scenarios. In this paper, a reinforcement learning-based throughput on demand (ToD) provisioning dynamic power management method (RLTDPM) is proposed for sustaining perpetual operation and satisfying the ToD requirements for today's energy harvesting wireless sensor node (EHWSN). The RLTDPM monitors the environmental state of the EHWS and adjusts their operational duty cycle under criteria of energy neutrality to meet the demanded throughput. Outcomes of these observation-adjustment interactions are then evaluated by feedback reward that represents how well the ToD requests are met; subsequently, the observation-adjustment-evaluation process, so-called reinforcement learning, continues. After the learning process, the RLTDPM is able to autonomously adjust the duty cycle for satisfying the ToD requirement, and in doing so, sustain the perpetual operation of the EHWSN. Simulations of the proposed RLTDPM on a wireless sensor node powered by a battery and solar cell for image sensing tasks were performed. Experimental results demonstrate that the achieved demanded throughput is improved 10.7 for the most stringent ToD requirement, while the residual battery energy of the RLTDPM is improved 7.4 compared with an existing DPM algorithm for EHWSN with image sensing purpose. Motivated by the engineering problem of cellular coverage optimization, we propose a novel multiarmed bandit model called generalized global bandit. We develop a series of greedy algorithms that have the capability to handle nonmonotonic but decomposable reward functions, multidimensional global parameters, and switching costs. The proposed algorithms are rigorously analyzed under the multiarmed bandit framework, where we show that they achieve bounded regret, and hence, they are guaranteed to converge to the optimal arm in finite time. The algorithms are then applied to the cellular coverage optimization problem to achieve the optimal tradeoff between sufficient small cell coverage and limited macroleakage without prior knowledge of the deployment environment. The performance advantage of the new algorithms over existing bandits solutions is revealed analytically and further confirmed via numerical simulations. The key element behind the performance improvement is a more efficient “trial and error” mechanism, in which any trial will help improve the knowledge of all candidate power levels.