Energy harvesting (EH) is a promising technique to fulfill the long-term and self-sustainable operations for Internet of things (IoT) systems. In this paper, we study the joint access control and battery prediction problems in a small-cell IoT system including multiple EH user equipments (UEs) and one base station (BS) with limited uplink access channels. Each UE has a rechargeable battery with finite capacity. The system control is modeled as a Markov decision process without complete prior knowledge assumed at the BS, which also deals with large sizes in both state and action spaces. First, to handle the access control problem assuming causal battery and channel state information, we propose a scheduling algorithm that maximizes the uplink transmission sum rate based on reinforcement learning (RL) with deep Q-network (DQN) enhancement. Second, for the battery prediction problem, with a fixed round-robin access control policy adopted, we develop a RL based algorithm to minimize the prediction loss (error) without any model knowledge about the energy source and energy arrival process. Finally, the joint access control and battery prediction problem is investigated, where we propose a two-layer RL network to simultaneously deal with maximizing the sum rate and minimizing the prediction loss: the first layer is for battery prediction, the second layer generates the access policy based on the output from the first layer. Experiment results show that the three proposed RL algorithms can achieve better performances compared with existing benchmarks.
The increasing popularity of micro-scale power-scavenging techniques for wireless sensor networks (WSNs) is paving the way to energy-autonomous sensing systems. To sustain perpetual operations, however, environmentally powered devices must adapt their workload to the stochastic nature of ambient sources. Energy prediction models, which estimate the future expected energy intake, are effective tools to support the development of proactive power management strategies. In this paper, we present profile energy prediction model (Pro-Energy), an energy prediction model for multi-source energy-harvesting WSNs that leverages past energy observations to forecast future energy availability. We then propose Pro-Energy with variable-length timeslots (Pro-Energy-VLT), an extension of Pro-Energy that combines our energy predictor with timeslots of variable lengths to adapt to the dynamics of the power source. To assess the performance of our proposed solutions, we use real-life solar and wind traces, as well as publicly available traces of solar irradiance and wind speed. A comparative performance evaluation shows that Pro-Energy significantly outperforms the state-of-the-art energy predictors, by improving the prediction accuracy of up to 67 . Moreover, by adapting the granularity of the prediction timeslots to the dynamics of the energy source, Pro-Energy-VLT further improves the prediction accuracy, while reducing the memory footprint and the energy overhead of energy forecasting. Traditionally, we rely highly upon fossil fuels for our energy provisioning, but there are drawbacks to using these fossil fuels: the risk for depletion in the future; the increased cost as already observed during the last few years, and the high impact on climate change. One virtually carbon-neutral alternative to fossil fuels are renewable energy sources, like solar energy. In this paper, we investigate the energy and network performance of a wireless access network powered by the traditional electricity grid and a photovoltaic panel system. An energy-aware management system for the future wireless access networks is proposed. This system consists of the management of the energy provisioning and storage system and the application of the proposed energy-saving strategies which aim to reduce the energy footprint through the traditional grid in case a renewable energy shortage occurs. To evaluate the network’s performance, this paper proposes a deployment tool with the above described energy-aware management system. The results show that it is promising to further investigate a more evolved and complex energy-aware management system. Traditional wireless sensor networks (WSNs) face the problem of a limited-energy source, typically batteries, resulting in the need for careful and effective utilization of the energy source. However, inevitable energy depletion will eventually disturb the operation of a WSN. Energy harvesting (EH) technology is acquiring particular interest, because it has the potential to provide a continuous energy supply in battery-powered WSNs. Solar energy is the most effective environmental energy for EH-WSNs because of its high energy intensity, which comes from a non-controllable source. Therefore, the prediction of future energy availability is a critical issue, as the amount of the harvestable energy may vary over time. In this paper, a novel solar energy prediction algorithm with Q-learning, called Q-learning-based solar energy prediction (QL-SEP), is proposed. Q-learning is an effective way of predicting future actions based on past observations. The distinctive feature of QL-SEP is that not only past days’ observations but also the current weather conditions are considered for prediction. The performance of QL-SEP is simulated in this paper using real-world measurements obtained from a solar panel in comparison with the state-of-art approaches.