In this paper, we study the problem of dynamic channel allocation for URLLC traffic in a multiuser multi-channel wireless network where urgent packets have to be successfully received in a timely manner. We formulate the problem as a finite-horizon Markov Decision Process with a stochastic constraint related to the QoS requirement, defined as the packet loss rate for each user. We propose a novel weighted formulation that takes into account both the total expected reward (number of successfully decoded packets) and the risk which we define as the QoS requirement violation. First, we use the value iteration algorithm to find the optimal policy, which assumes a perfect knowledge of the controller of all the parameters, namely the channel statistics. We then propose a Q-learning algorithm where the controller learns the optimal policy without having knowledge of neither the CSI nor the channel statistics. We illustrate the performance of our algorithms with numerical studies.
The issue of deadline-constrained traffic scheduling has been investigated by several works including @cite_20 @cite_18 @cite_14 @cite_16 . For example, in @cite_16 , the authors study the problem of dynamic channel allocation in a single user multi-channel system with service costs and deadline-constrained traffic. They propose online algorithms to enable the controller to learn the optimal policy based on Thompson sampling for multi-armed bandit problems. The MDP framework and reinforcement learning approaches for downlink packet scheduling are considered in @cite_20 @cite_14 @cite_13 @cite_4 @cite_8 @cite_12 . In @cite_20 , the authors propose an MDP for deadline-constrained packet scheduling problem and use dynamic programming to find the optimal scheduling policies. The authors do not consider QoS constraints in the scheduling problem.
@cite_18 @cite_14 @cite_4 @cite_8 @cite_16 @cite_13 @cite_12 @cite_20
2136340918 2056459334 2027034953 2043605850
We consider the problem of resource allocation in downlink OFDMA systems for multi service and unknown environment. Due to users' mobility and intercell interference, the base station cannot predict neither the Signal to Noise Ratio (SNR) of each user in future time slots nor their probability distribution functions. In addition, the traffic is bursty in general with unknown arrival. The probability distribution functions of the SNR, channel state and traffic arrival density are then unknown. Achieving a multi service Quality of Service (QoS) while optimizing the performance of the system (e.g. total throughput) is a hard and interesting task since it depends on the unknown future traffic and SNR values. In this paper we solve this problem by modeling the multiuser queuing system as a discrete time linear dynamic system. We develop a robust H∞ controller to regulate the queues of different users. The queues and Packet Drop Rates (PDR) are controlled by proposing a minimum data rate according to the demanded service type of each user. The data rate vector proposed by the controller is then fed as a constraint to an instantaneous resource allocation framework. This instantaneous problem is formulated as a convex optimization problem for instantaneous subcarrier and power allocation decisions. Simulation results show small delays and better fairness among users. This paper studies the problem of congestion control and scheduling in ad hoc wireless networks that have to support a mixture of best-effort and real-time traffic. Optimization and stochastic network theory have been successful in designing architectures for fair resource allocation to meet long-term throughput demands. However, to the best of our knowledge, strict packet delay deadlines were not considered in this framework previously. In this paper, we propose a model for incorporating the quality-of-service (QoS) requirements of packets with deadlines in the optimization framework. The solution to the problem results in a joint congestion control and scheduling algorithm that fairly allocates resources to meet the fairness objectives of both elastic and inelastic flows and per-packet delay requirements of inelastic flows. We consider the question of obtaining tight delay guarantees for throughout-optimal link scheduling in arbitrary topology wireless ad-hoc networks. Two classes of scheduling policies are considered: 1) a maximum queue-length weighted independent set scheduling policy and 2) a randomized independent set scheduling policy where the set scheduling probabilities are selected optimally. Both policies stabilize all queues for any set of feasible packet arrival rates, and are therefore throughput-optimal. For these policies, we show that the average packet delay is bounded by a constant that depends on the chromatic number of the interference graph, and the arrival slack in the system-a metric representing the overall load on the network. We prove that this upper bound is asymptotically tight in the sense that there exist classes of topologies where the expected delay attained by any scheduling policy is lower bounded by the same constant. We extend our upper bounds to the case of multi-hop sessions. Through simulations, we study how our analysis compares with actual delays computed for i.i.d., Markovian, and trace-driven packet arrival processes. Delay-sensitive traffic services, such as live streaming video, voice over IP and multimedia teleconferencing, requires low end-to-end delay in order to maintain its interactive and streaming nature. In recent years, the popularity of delay-sensitive applications has been rapidly growing. In this study, the authors consider the delay-sensitive user scheduling and power control design for heterogeneous networks (HetNets). They model the problem as an infinite-horizon average cost Markov decision process MDP. Based on the dynamics of channel state information (CSI) and queue state information (QSI), they focus on finding an adaptive user scheduling and power control policy to minimise the average delay of network subjected to the power constraints of heterogeneous base stations. In order to reduce the computational complexity and facilitate the distributive implementation, the authors derive a distributive stochastic learning algorithm which only requires local CSI and local QSI to determine the optimal user scheduling and power control policy. The simulation results confirm that significant delay performance gain can be achieved compared with various baseline schemes.