This paper studies online transmission policies for an energy harvesting transmitter. Unlike the existing online policies which more or less require the knowledge on the future behavior of the energy and data arrival processes, we consider a practical but significantly more challenging scenario where the energy and data arrival processes are assumed to be totally unknown. Our design is formulated as a robust-optimal control problem which aims to optimize the worst-case performance. The transmission policy is designed only based on the current battery energy level and the data queue length directly monitored by the transmitter itself. Specifically, we apply an event-trigger approach in which the transmitter continuously monitors the battery energy and data queue length, and triggers an event when a significant change occurs in either of them. Once an event is triggered, the transmission policy is updated by the solution to the robust-optimal control problem. We consider both the transmission time and throughput as the performance metrics and formulated two optimization problems. The solutions are given in either a simple analytical form or an easy-to-implement algorithm.
The second framework uses parameter-independent methodologies. @cite_6 , the energy and data arrival processes were formulated as time-homogeneous Markov chains without knowing the transition matrix, and Q-learning was applied to perform online optimization on the transmission policy. By Lyapunov optimization technique combined with the idea of weight perturbation, @cite_20 proposed a generic utility-maximization policy, under the assumption that the amount of harvested energy in each time slot is independent and identically distributed (i.i.d.) but its statistical parameters are totally unknown. Although these parameter-independent policies require less knowledge on the energy and data arrival processes, the stochastic models of the energy and data arrival processes still need to be known exactly.
@cite_20 @cite_6
2006689319 2065154528
Communication over a broadband fading channel powered by an energy harvesting transmitter is studied. Assuming non-causal knowledge of energy data arrivals and channel gains, optimal transmission schemes are identified by taking into account the energy cost of the processing circuitry as well as the transmission energy. A constant processing cost for each active sub-channel is assumed. Three different system objectives are considered: i) throughput maximization, in which the total amount of transmitted data by a deadline is maximized for a backlogged transmitter with a finite capacity battery; ii) energy maximization, in which the remaining energy in an infinite capacity battery by a deadline is maximized such that all the arriving data packets are delivered; iii) transmission completion time minimization, in which the delivery time of all the arriving data packets is minimized assuming infinite size battery. For each objective, a convex optimization problem is formulated, the properties of the optimal transmission policies are identified, and an algorithm which computes an optimal transmission policy is proposed. Finally, based on the insights gained from the offline optimizations, low-complexity online algorithms performing close to the optimal dynamic programming solution for the throughput and energy maximization problems are developed under the assumption that the energy data arrivals and channel states are known causally at the transmitter. This paper studies online transmission policies for an energy harvesting transmitter. Unlike the existing online policies which more or less require the knowledge on the future behavior of the energy and data arrival processes, we consider a practical but significantly more challenging scenario where the energy and data arrival processes are assumed to be totally unknown. Our design is formulated as a robust-optimal control problem which aims to optimize the worst-case performance. The transmission policy is designed only based on the current battery energy level and the data queue length directly monitored by the transmitter itself. Specifically, we apply an event-trigger approach in which the transmitter continuously monitors the battery energy and data queue length, and triggers an event when a significant change occurs in either of them. Once an event is triggered, the transmission policy is updated by the solution to the robust-optimal control problem. We consider both the transmission time and throughput as the performance metrics and formulated two optimization problems. The solutions are given in either a simple analytical form or an easy-to-implement algorithm. A time-slotted multiple access wireless system with N transmitting nodes, each equipped with an energy harvesting (EH) device and a rechargeable battery of finite capacity, is studied. The energy arrival process at each node is modeled as an independent two-state Markov process, such that a node either harvests one unit of energy, or none, at each time slot (TS). The access point (AP) schedules a subset of K nodes to transmit over K orthogonal channels at each TS. The maximum total throughput is studied for a backlogged system without the knowledge of the EH processes and nodes' battery states at the AP. The problem is identified as a partially observable Markov decision process, and the optimal policy for the general model is studied numerically. Under certain assumptions regarding the EH processes and the battery sizes, the optimal scheduling policy is characterized explicitly, and is shown to be myopic. A multi-access wireless network with N transmitting nodes, each equipped with an energy harvesting (EH) device and a rechargeable battery of finite capacity, is studied. At each time slot (TS) a node is operative with a certain probability, which may depend on the availability of data, or the state of its channel. The energy arrival process at each node is modelled as an independent two-state Markov process, such that, at each TS, a node either harvests one unit of energy, or none. At each TS a subset of the nodes is scheduled by the access point (AP) . The scheduling policy that maximises the total throughput is studied assuming that the AP does not know the states of either the EH processes or the batteries. The problem is identified as a restless multi-armed bandit (RMAB) problem, and an upper bound on the optimal scheduling policy is found. Under certain assumptions regarding the EH processes and the battery sizes, the optimality of the myopic policy (MP) is proven. For the general case, the performance of MP is compared numerically to the upper bound. Wireless systems comprised of rechargeable nodes have a significantly prolonged lifetime and are sustainable. A distinct characteristic of these systems is the fact that the nodes can harvest energy throughout the duration in which communication takes place. As such, transmission policies of the nodes need to adapt to these harvested energy arrivals. In this paper, we consider optimization of point-to-point data transmission with an energy harvesting transmitter which has a limited battery capacity, communicating in a wireless fading channel. We consider two objectives: maximizing the throughput by a deadline, and minimizing the transmission completion time of the communication session. We optimize these objectives by controlling the time sequence of transmit powers subject to energy storage capacity and causality constraints. We, first, study optimal offline policies. We introduce a directional water-filling algorithm which provides a simple and concise interpretation of the necessary optimality conditions. We show the optimality of an adaptive directional water-filling algorithm for the throughput maximization problem. We solve the transmission completion time minimization problem by utilizing its equivalence to its throughput maximization counterpart. Next, we consider online policies. We use stochastic dynamic programming to solve for the optimal online policy that maximizes the average number of bits delivered by a deadline under stochastic fading and energy arrival processes with causal channel state feedback. We also propose near-optimal policies with reduced complexity, and numerically study their performances along with the performances of the offline and online optimal policies under various different configurations. We consider the optimal online packet scheduling problem in a single-user energy harvesting wireless communication system, where energy is harvested from natural renewable sources, making future energy arrivals instants and amounts random in nature. The most general case of arbitrary energy arrivals is considered where neither the future energy arrival instants or amount, nor their distribution is known. The problem considered is to adaptively change the transmission rate according to the causal energy arrival information, such that the time by which all packets are delivered is minimized. We assume that all bits have arrived and are ready at the source before the transmission begins. For a minimization problem, the utility of an online algorithm is tested by finding its competitive ratio or competitiveness that is defined to be the maximum of the ratio of the gain of the online algorithm with the optimal offline algorithm over all input sequences. We derive a lower and upper bound on the competitive ratio of any online algorithm to minimize the total transmission time in an energy harvesting system. The upper bound is obtained using a lazy' transmission policy that chooses its transmission power to minimize the transmission time assuming that no further energy arrivals are going to occur in future. The lazy transmission policy is shown to be strictly two-competitive. We also derive an adversarial lower bound that shows that competitive ratio of any online algorithm is at least 1.325. We consider the optimal packet scheduling problem in a single-user energy harvesting wireless communication system. In this system, both the data packets and the harvested energy are modeled to arrive at the source node randomly. Our goal is to adaptively change the transmission rate according to the traffic load and available energy, such that the time by which all packets are delivered is minimized. Under a deterministic system setting, we assume that the energy harvesting times and harvested energy amounts are known before the transmission starts. For the data traffic arrivals, we consider two different scenarios. In the first scenario, we assume that all bits have arrived and are ready at the transmitter before the transmission starts. In the second scenario, we consider the case where packets arrive during the transmissions, with known arrival times and sizes. We develop optimal off-line scheduling policies which minimize the time by which all packets are delivered to the destination, under causality constraints on both data and energy arrivals. We consider the transmission completion time minimization problem in a single-user energy harvesting wireless communication system. In this system, both the data packets and the harvested energy are modelled to arrive at the source node randomly. Our goal is to adaptively change the transmission rate according to the traffic load and available energy, such that the transmission completion time is minimized. Under a deterministic system setting, we assume that the energy harvesting times and harvested energy amounts are known before the transmission starts. For the data traffic arrivals, we consider two different scenarios. In the first scenario, we assume that all bits have arrived and are ready at the transmitter before the transmission starts. In the second scenario we consider, packets arrive during the transmissions with known arriving times and sizes. We develop optimal off-line scheduling policies which minimize the overall transmission completion time under causality constraints on both data and energy arrivals. Energy harvesting (EH) two-hop communications are considered. The transmitter and the relay harvest energy from the environment and use it exclusively for transmitting data. A data arrival process is assumed at the transmitter. At the relay, a finite data buffer is used to store the received data. We consider a realistic scenario in which the EH nodes have only local causal knowledge, i.e., at any time instant, each EH node only knows the current value of its EH process, channel state, and data arrival process. Our goal is to find a power allocation policy to maximize the throughput at the receiver. We show that because the EH nodes have local causal knowledge, the two-hop communication problem can be separated into two point-to-point problems. Consequently, independent power allocation problems are solved at each EH node. To find the power allocation policy, reinforcement learning with linear function approximation is applied. Moreover, to perform function approximation two feature functions which consider the data arrival process are introduced. Numerical results show that the proposed approach has only a small degradation as compared to the offline optimum case. Furthermore, we show that with the use of the proposed feature functions a better performance is achieved compared to standard approximation techniques. This paper considers a two-user Gaussian interference channel with energy harvesting transmitters. Different than conventional battery powered wireless nodes, energy harvesting transmitters have to adapt transmission to availability of energy at a particular instant. In this setting, the optimal power allocation problem to maximize the sum throughput with a given deadline is formulated. The convergence of the proposed iterative coordinate descent method for the problem is proved and the short-term throughput maximizing offline power allocation policy is found. Examples for interference regions with known sum capacities are given with directional waterfilling interpretations. Next, stochastic data arrivals are addressed. Finally, online and or distributed near-optimal policies are proposed. Performance of the proposed algorithms are demonstrated through simulations. A sensor node that is sending measurement updates regarding some physical phenomenon to a destination is considered. The sensor relies on energy harvested from nature to transmit its updates, and is equipped with a finite @math -sized battery to save its harvested energy. Energy recharges the battery incrementally in units, according to a Poisson process, and one update consumes one energy unit to reach the destination. The setting is online, where the energy arrival times are revealed causally after the energy is harvested. The goal is to update the destination in a timely manner, namely, such that the long term average age of information is minimized, subject to energy causality constraints. The age of information at a given time is defined as the time spent since the latest update has reached the destination. It is shown that the optimal update policy follows a renewal structure, where the inter-update times are independent, and the time durations between any two consecutive events of submitting an update and having @math units of energy remaining in the battery are independent and identically distributed for a given @math . The optimal renewal policy for the case of @math energy units is explicitly characterized, and it is shown that it has an energy-dependent threshold structure, where the sensor updates only if the age grows above a certain threshold that is a function of the amount of energy in its battery. Wireless networks with energy harvesting battery equipped nodes are quickly emerging as a viable option for future wireless networks with extended lifetime. Equally important to their counterpart in the design of energy harvesting radios are the design principles that this new networking paradigm calls for. In particular, unlike wireless networks considered to date, the energy replenishment process and the storage constraints of the rechargeable batteries need to be taken into account in designing efficient transmission strategies. In this work, such transmission policies for rechargeable nodes are considered, and optimum solutions for two related problems are identified. Specifically, the transmission policy that maximizes the short term throughput, i.e., the amount of data transmitted in a finite time horizon is found. In addition, the relation of this optimization problem to another, namely, the minimization of the transmission completion time for a given amount of data is demonstrated, which leads to the solution of the latter as well. The optimum transmission policies are identified under the constraints on energy causality, i.e., energy replenishment process, as well as the energy storage, i.e., battery capacity. For battery replenishment, a model with discrete packets of energy arrivals is considered. The necessary conditions that the throughput-optimal allocation satisfies are derived, and then the algorithm that finds the optimal transmission policy with respect to the short-term throughput and the minimum transmission completion time is given. Numerical results are presented to confirm the analytical findings. A point-to-point wireless communication system in which the transmitter is equipped with an energy harvesting device and a rechargeable battery, is studied. Both the energy and the data arrivals at the transmitter are modeled as Markov processes. Delay-limited communication is considered assuming that the underlying channel is block fading with memory, and the instantaneous channel state information is available at both the transmitter and the receiver. The expected total transmitted data during the transmitter's activation time is maximized under three different sets of assumptions regarding the information available at the transmitter about the underlying stochastic processes. A learning theoretic approach is introduced, which does not assume any a priori information on the Markov processes governing the communication system. In addition, online and offline optimization problems are studied for the same setting. Full statistical knowledge and causal information on the realizations of the underlying stochastic processes are assumed in the online optimization problem, while the offline optimization problem assumes non-causal knowledge of the realizations in advance. Comparing the optimal solutions in all three frameworks, the performance loss due to the lack of the transmitter's information regarding the behaviors of the underlying Markov processes is quantified. In this paper, we consider delay-optimal charging scheduling of the electric vehicles (EVs) at a charging station with multiple charge points. The charging station is equipped with renewable energy generation devices and can also buy energy from power grids. The uncertainty of the arrival of the EV, the intermittence of the renewable energy, and the variation of the grid power price are taken into account and described as independent Markov processes. Meanwhile, the required charging energy for each EV is random. The goal is to minimize the mean waiting time for EVs under the long-term constraint on the cost. We propose queue mapping to convert the EV queue to the charging demand queue, and we prove the equivalence between the minimization of the two queues' average length. Then, we focus on the minimization for the average length of the charging demand queue under the long-term cost constraint. We propose a framework of Markov decision process (MDP) to investigate this constrained stochastic optimization problem. The system state includes the charging demand queue length, the charging demand arrival, the energy level in the storage battery of the renewable energy, the renewable energy arrival, and the grid power price. Additionally, the number of charging demands and the allocated energy from the storage battery compose the 2-D policy. We derive two necessary conditions of the optimal policy. Moreover, we discuss the reduction of the 2-D policy to be the number of charging demands only. We give the sets of system states for which charging no demand and charging as many demands as possible are optimal, respectively. Finally, we investigate the proposed policies numerically. We consider the minimization of the transmission completion time with a battery limited energy harvesting transmitter in an M-user AWGN broadcast channel where the transmitter is able to harvest energy from the nature, using a finite storage capacity rechargeable battery. The harvested energy is modeled to arrive (be harvested) at the transmitter during the course of transmissions at arbitrary time instants. The transmitter has fixed number of packets for each receiver. Due to the finite battery capacity, energy may overflow without being utilized for data transmission. We derive the optimal offline transmission policy that minimizes the time by which all of the data packets are delivered to their respective destinations. We analyze the structural properties of the optimal transmission policy using a dual problem. We find the optimal total transmit power sequence by a directional water-filling algorithm. We prove that there exist M-1 cut-off power levels such that user i is allocated the power between the i-1st and the ith cut-off power levels subject to the availability of the allocated total power level. Based on these properties, we propose an algorithm that gives the globally optimal offline policy. The proposed algorithm uses directional water-filling repetitively. Finally, we illustrate the optimal policy and compare its performance with several suboptimal policies under different settings. Transmission rate adaptation in wireless devices provides a unique opportunity to trade off data service rate with energy consumption. In this paper, we study optimal rate control to minimize transmission energy expenditure subject to strict deadline or other quality-of-service (QoS) constraints. Specifically, the system consists of a wireless transmitter with controllable transmission rate and with strict QoS constraints on data transmission. The goal is to obtain a rate-control policy that minimizes the total transmission energy expenditure while ensuring that the QoS constraints are met. Using a novel formulation based on cumulative curves methodology, we obtain the optimal transmission policy and show that it has a simple and appealing graphical visualization. Utilizing the optimal ldquoofflinerdquo results, we then develop an online transmission policy for an arbitrary stream of packet arrivals and deadline constraints, and show, via simulations, that it is significantly more energy-efficient than a simple head-of-line drain policy. Finally, we generalize the optimal policy results to the case of time-varying power-rate functions. An energy-harvesting sensor node that is sending status updates to a destination is considered. The sensor is equipped with a battery of finite size to save its incoming energy, and consumes one unit of energy per status update transmission, which is delivered to the destination instantly over an error-free channel. The setting is online in which the harvested energy is revealed to the sensor causally over time, and the goal is to design status update transmission policy such that the long term average age of information (AoI) is minimized. AoI is defined as the time elapsed since the latest update has reached at the destination. Two energy arrival models are considered: a random battery recharge (RBR) model, and an incremental battery recharge (IBR) model. In both models, energy arrives according to a Poisson process with unit rate, with values that completely fill up the battery in the RBR model, and with values that fill up the battery incrementally, unit-by-unit, in the IBR model. The key approach to characterizing the optimal status update policy for both models is showing the optimality of renewal policies, in which the inter-update times follow a specific renewal process that depends on the energy arrival model and the battery size. It is then shown that the optimal renewal policy has an energy-dependent threshold structure, in which the sensor sends a status update only if the AoI grows above a certain threshold that depends on the energy available. For both the RBR and the IBR models, the optimal energy-dependent thresholds are characterized explicitly, i.e., in closed-form, in terms of the optimal long term average AoI. It is also shown that the optimal thresholds are monotonically decreasing in the energy available in the battery, and that the smallest threshold, which comes in effect when the battery is full, is equal to the optimal long term average AoI. A competitive analysis for the online and offline optimization problems for a slotted energy harvesting (EH) wireless communication system is studied. The objective is to design online strategies that minimize the competitive rate gap that is defined as the maximum gap between the optimal rates that can be achieved by the offline and online policies over all possible energy arrival profiles. It is shown that the competitive rate gap is upper-bounded by the logarithm of the number of slots, and a myopic online transmission policy is proposed that achieves a lower rate gap. We investigate performance limits of a multiple access communication system with energy harvesting nodes where the utility function is taken to be the long-term average sum-throughput. We assume a causal structure for energy arrivals and study the problem in the continuous time regime. For this setting, we first characterize a storage dam model that captures the dynamics of a battery with energy harvesting and variable transmission power. Using this model, we next establish an upper bound on the throughput problem as a function of battery capacity. We also formulate a nonlinear optimization problem to determine optimal achievable power policies for transmitters. Applying a calculus of variations technique, we then derive Euler-Lagrange equations as necessary conditions for optimum power policies in terms of a system of coupled partial integro-differential equations. Based on a Gauss-Seidel algorithm, we devise an iterative algorithm to solve these equations. We also propose a fixed-point algorithm for the symmetric multiple access setting in which the statistical descriptions of energy harvesters are identical. To further support our iterative algorithms, along with the analysis, comprehensive numerical results are also obtained. We consider managing the freshness of status updates sent from a source (such as a sensor) to a monitoring node. The time-varying availability of energy at the sender limits the rate of update packet transmissions. At any time, the age of information is defined as the amount of time since the most recent update was successfully received. An offline solution that minimizes not only the time average age, but also the peak age for an arbitrary energy replenishment profile is derived. The related decision problem under stochastic energy arrivals at the sender is studied through a discrete time dynamic programming formulation, and the structure of the optimal policy that minimizes the expected age is shown. It is found that tracking the expected value of the current age (which is a linear operation), together with the knowledge of the current energy level at the sender side is sufficient for generating an optimal threshold policy. An effective online heuristic, Balance Updating (BU), that achieves performance close to an omniscient (offline) policy is proposed. Simulations of the policies indicate that they can significantly improve the age over greedy approaches. An extension of the formulation to stochastically formed updates is considered.