Recent research on Software-Defined Networking (SDN) strongly promotes the adoption of distributed controller architectures. To achieve high network performance, designing a scheduling function (SF) to properly dispatch requests from each switch to suitable controllers becomes critical. However, existing literature tends to design the SF targeted at specific network settings. In this paper, a reinforcement-learning-based (RL) approach is proposed with the aim to automatically learn a general, effective, and efficient SF. In particular, a new dispatching system is introduced in which the SF is represented as a neural network that determines the priority of each controller. Based on the priorities, a controller is selected using our proposed probability selection scheme to balance the trade-off between exploration and exploitation during learning. In order to train a general SF, we first formulate the scheduling function design problem as an RL problem. Then a new training approach is developed based on a state-of-the-art deep RL algorithm. Our simulation results show that our RL approach can rapidly design (or learn) SFs with optimal performance. Apart from that, the trained SF can generalize well and outperforms commonly used scheduling heuristics under various network settings.
Recently, a completely different design approach based on RL has been studied in the literature @cite_4 @cite_13 @cite_1 . @cite_13 proposed an RL-based approach to automatically allocate the server resources in data centers. DeepRM @cite_4 tackled the multi-resource cluster scheduling problem using policy search to optimize various objectives, e.g., average job completion time and resource utilization. @cite_1 leveraged the delay-tolerant feature of IoT traffic and developed an RL-based scheduler to handle traffic variation so that the network utilization can be constantly optimized.
@cite_13 @cite_1 @cite_4
2096989632 2020196954 2100787464 2770542169
The proliferation of high-volume time-critical Web services such as online trading calls for a scalable server design that allows meeting individual response-time guarantees of real time transactions. A main challenge is to honor these guarantees despite unpredictability in incoming server load. The extremely high volume of real-time service requests mandates constant-time scheduling and schedulability analysis algorithms (as opposed to polynomial or logarithmic ones in the number of current requests). The paper makes two major contributions towards developing an architecture and theoretical foundations for scalable real-time servers operating in dynamic environments. First, we derive a tight utilization bound for schedulability of aperiodic tasks (requests) that allows implementing a constant time schedulability test on the server. We demonstrate that Liu and Layland's schedulable utilization bound of ln 2 does not apply to aperiodic tasks, and prove that an optimal arrival-time independent scheduling policy will meet all aperiodic task deadlines if utilization is maintained below 1 1+ spl radic (1 2). Second, we show that aperiodic deadline-monotonic scheduling is the optimal arrival-time-independent scheduling policy for aperiodic tasks. This result is used to optimally prioritize server requests. Evaluation of a utilization control loop that maintains server utilization below the bound shows that the approach is effective in meeting all individual deadlines in a high performance real-time server. It has recently been proposed that Internet energy costs, both monetary and environmental, can be reduced by exploiting temporal variations and shifting processing to data centers located in regions where energy currently has low cost. Lightly loaded data centers can then turn off surplus servers. This paper studies online algorithms for determining the number of servers to leave on in each data center, and then uses these algorithms to study the environmental potential of geographical load balancing (GLB). A commonly suggested algorithm for this setting is “receding horizon control” (RHC), which computes the provisioning for the current time by optimizing over a window of predicted future loads. We show that RHC performs well in a homogeneous setting, in which all servers can serve all jobs equally well; however, we also prove that differences in propagation delays, servers, and electricity prices can cause RHC perform badly, So, we introduce variants of RHC that are guaranteed to perform as well in the face of such heterogeneity. These algorithms are then used to study the feasibility of powering a continent-wide set of data centers mostly by renewable sources, and to understand what portfolio of renewable energy is most effective. Efficiently utilizing off-chip DRAM bandwidth is a critical issuein designing cost-effective, high-performance chip multiprocessors(CMPs). Conventional memory controllers deliver relativelylow performance in part because they often employ fixed,rigid access scheduling policies designed for average-case applicationbehavior. As a result, they cannot learn and optimizethe long-term performance impact of their scheduling decisions,and cannot adapt their scheduling policies to dynamic workloadbehavior.We propose a new, self-optimizing memory controller designthat operates using the principles of reinforcement learning (RL)to overcome these limitations. Our RL-based memory controllerobserves the system state and estimates the long-term performanceimpact of each action it can take. In this way, the controllerlearns to optimize its scheduling policy on the fly to maximizelong-term performance. Our results show that an RL-basedmemory controller improves the performance of a set of parallelapplications run on a 4-core CMP by 19 on average (upto 33 ), and it improves DRAM bandwidth utilization by 22 compared to a state-of-the-art controller. Minimizing job scheduling time is a fundamental issue in data center networks that has been extensively studied in recent years. The incoming jobs require different CPU and memory units, and span different number of time slots. The traditional solution is to design efficient heuristic algorithms with performance guarantee under certain assumptions. In this paper, we improve a recently proposed job scheduling algorithm using deep reinforcement learning and extend it to multiple server clusters. Our study reveals that deep reinforcement learning method has the potential to outperform traditional resource allocation algorithms in a variety of complicated environments.