FIELD

One or more aspects of example embodiments relate to workload traffic distribution, and more particularly to systems and methods for spike detection and load balancing.

BACKGROUND

Generally, the amount of traffic (or load) targeted towards cloud data centers fluctuate based on user requests. This traffic may be bursty (e.g., sudden peak in requests) and may require a high degree of resource reallocation. Often, however, the nature of uncertain workload traffic my require many requests to be reallocated on-the-fly, which may result in performance degradation. For example, traffic arriving at a cloud data center may be proportional to scheduling delays, where the higher the variability the longer the scheduling delays. Thus, if resources are committed, changing schedules may result in unsatisfied Service-Level Agreements (SLA).

These issues may be exacerbated for users who rent or own multiple server nodes (e.g., Docker containers, virtual machines, and the like) to process (or execute) requests. For example, these server nodes are generally hosted by a cloud computing vendor, and may be located on physical servers that often host other users' service nodes. Moreover, these physical servers are oftentimes oversold to multiple users, such that multiple users share the overpromised resource. As a result, a user having a workload spike (e.g., sending a large amount of jobs or requests) may not only cause issues for the user on the shared resource, but may also slow down the physical host server to affect other users.

Accordingly, systems and methods for spike detection and intelligent load balancing may be desired.

The above information disclosed in this Background section is for enhancement of understanding of the background of the present disclosure, and therefore, it may contain information that does not constitute prior art.

SUMMARY

According to one or more example embodiments, a load balancing system, includes: a centralized queue; a pool of resource nodes connected to the centralized queue; one or more processors; and memory connected to the one or more processors and storing instructions that, when executed by the one or more processors, cause the one or more processors to: monitor a queue status of the centralized queue to identify a bursty traffic period; calculate an index value for a load associated with the bursty traffic period; select a load balancing strategy based on the index value; distribute the load to the pool of resource nodes based on the load balancing strategy; observe a state of the pool of resource nodes in response to the load balancing strategy; calculate a reward based on the observed state; and adjust the load balancing strategy based on the reward.

In one or more example embodiments, the index value may correspond to a Gittins Index (GI) value.

In one or more example embodiments, to calculate the index value, the instructions may further cause the one or more processors to: calculate a plurality of GI values for the load associated with the bursty traffic period; and output a greatest one of the plurality of GI values as a new spike value.

In one or more example embodiments, to distribute the load to the pool of resource nodes, the instructions may further cause the one or more processors to: adjust resource requirements for the pool of resource nodes based on the new spike value to generate a new compute environment; and generate a schedule of job request distributions associated with the load for the new compute environment.

In one or more example embodiments, to select the load balancing strategy, the instructions may further cause the one or more processors to: calculate an incremental load managing value for each resource in the pool of resource nodes; and calculate a base resource parameter for each resource in the pool of resource nodes.

In one or more example embodiments, the load balancing strategy may be selected based on the resource with the greatest base resource parameter.

In one or more example embodiments, to calculate the reward, the instructions may further cause the one or more processors to: calculate a Q-function based on the index value, an input state, the selected load balancing strategy, and a reward function.

In one or more example embodiments, the Q-function may be calculated according to: {circumflex over (Q)}(s,a):=r(s,a)+γ maxaiQ(s′,ai), wherein: s and s′ may correspond to past and present state GI index values, respectively, a may correspond to different load values; and r may correspond to an immediate reward parameter.

In one or more example embodiments, the instructions may further cause the one or more processors to: apply an exponentially weighted moving average to the Q-function calculation.

In one or more example embodiments, the load may correspond to a Poisson traffic distribution.

According to one or more example embodiments, a method for load balancing, includes: monitoring, by one or more processors, a queue status of a centralized queue to identify a bursty traffic period; calculating, by the one or more processors, an index value for a load associated with the bursty traffic period; selecting, by the one or more processors, a load balancing strategy based on the index value; distributing, by the one or more processors, the load to a pool of resource nodes based on the load balancing strategy; observing, by the one or more processors, a state of the pool of resource nodes in response to the load balancing strategy; calculating, by the one or more processors, a reward based on the observed state; and adjusting, by the one or more processors, the load balancing strategy based on the reward.

In one or more example embodiments, the index value may correspond to a Gittins Index (GI) value.

In one or more example embodiments, the calculating of the index value may include: calculating, by the one or more processors, a plurality of GI values for the load associated with the bursty traffic period; and outputting, by the one or more processors, a greatest one of the plurality of GI values as a new spike value.

In one or more example embodiments, the distributing of the load to the pool of resource nodes may include: adjusting, by the one or more processors, resource requirements for the pool of resource nodes based on the new spike value to generate a new compute environment; and generating, by the one or more processors, a schedule of job request distributions associated with the load for the new compute environment.

In one or more example embodiments, the selecting of the load balancing strategy may include: calculating, by the one or more processors, an incremental load managing value for each resource in the pool of resource nodes; and calculating, by the one or more processors, a base resource parameter for each resource in the pool of resource nodes.

In one or more example embodiments, the load balancing strategy may be selected based on the resource with the greatest base resource parameter.

In one or more example embodiments, the calculating of the reward may include: calculating, by the one or more processors, a Q-function based on the index value, an input state, the selected load balancing strategy, and a reward function.

In one or more example embodiments, the Q-function may be calculated according to: {circumflex over (Q)}(s,a):=r(s,a)+γ maxaiQ(s′,ai), wherein: s and s′ may correspond to past and present state GI index values, respectively, a may correspond to different load values; and r may correspond to an immediate reward parameter.

In one or more example embodiments, the method may further include: applying, by the one or more processors, an exponentially weighted moving average to the Q-function calculation.

In one or more example embodiments, the load may correspond to a Poisson traffic distribution.

