Modern web services rely on Content Delivery Networks (CDNs) to efficiently deliver contents to end users. In order to minimize the experienced communication cost, it is necessary to send the end user's requests to the nearest servers. However, it is shown that this naive method causes some servers to get overloaded. Similarly, when distributing the requests to avoid overloading, the communication cost increases. This is a well-known trade-off between communication cost and load balancing in CDNs. In this work, by introducing a new meta-heuristic algorithm, we try to optimize this trade-off, that is, to have less-loaded servers at lower experienced communication cost. This trade-off is even better managed when we optimize the way servers update their information of each others' load. The proposed scheme, which is based on Honeybee algorithm, is an implementation of bees algorithm which is known for solving continuous optimization problems. Our proposed version for CDNs is a combination of a request redirecting method and a server information update algorithm. To evaluate the suggested method in a large-scale network, we leveraged our newly developed CDN simulator which takes into account all the important network parameters in the scope of our problem. The simulation results show that our proposed scheme achieves a better trade-off between the communication cost and load balancing in CDNs, compared to previously proposed schemes.
The power of multiple choices @cite_23 is a classic paradigm for load balancing which is used in many practical implementations of request routing algorithms. Simplicity and achieving much better results in comparison to a simple randomized method are the main advantages of this paradigm. In @cite_10 the authors have implemented an extension to this paradigm, called . In this method, each server can receive the clients' requests and forward them to the most appropriate server. The best server is chosen between a random server and it's next neighbor. The next-neighbor load sharing method leads to better response time as the authors claimed. In addition, Gardner and Balter have used the power of two choices concept to propose a new theoretical dispatching model @cite_24 . In this algorithm, each request is copied in multiple servers' queues. This way, the requests would wait in multiple queues at the same time, resulting in the minimum waiting time across queues. The authors showed that copying each request in only two servers will reduce average response time noticeably.
@cite_24 @cite_10 @cite_23
2015774390 2963728009 2101438247 2109055312
In a Fork-Join (FJ) queueing system an upstream fork station splits incoming jobs into N tasks to be further processed by N parallel servers, each with its own queue; the response time of one job is determined, at a downstream join station, by the maximum of the corresponding tasks' response times. This queueing system is useful to the modelling of multi-service systems subject to synchronization constraints, such as MapReduce clusters or multipath routing. Despite their apparent simplicity, FJ systems are hard to analyze. This paper provides the first computable stochastic bounds on the waiting and response time distributions in FJ systems. We consider four practical scenarios by combining 1a) renewal and 1b) non-renewal arrivals, and 2a) non-blocking and 2b) blocking servers. In the case of non blocking servers we prove that delays scale as O(logN), a law which is known for first moments under renewal input only. In the case of blocking servers, we prove that the same factor of log N dictates the stability region of the system. Simulation results indicate that our bounds are tight, especially at high utilizations, in all four scenarios. A remarkable insight gained from our results is that, at moderate to high utilizations, multipath routing 'makes sense' from a queueing perspective for two paths only, i.e., response times drop the most when N = 2; the technical explanation is that the resequencing (delay) price starts to quickly dominate the tempting gain due to multipath transmissions. We consider load balancing in a network of caching servers delivering contents to end users. Randomized load balancing via the so-called power of two choices is a well-known approach in parallel and distributed systems. In this framework, we investigate the tension between storage resources, communication cost, and load balancing performance. To this end, we propose a randomized load balancing scheme which simultaneously considers cache size limitation and proximity in the server redirection process. In contrast to the classical power of two choices setup, since the memory limitation and the proximity constraint cause correlation in the server selection process, we may not benefit from the power of two choices. However, we prove that in certain regimes of problem parameters, our scheme results in the maximum load of order @math (here @math is the network size). This is an exponential improvement compared to the scheme which assigns each request to the nearest available replica. Interestingly, the extra communication cost incurred by our proposed scheme, compared to the nearest replica strategy, is small. Furthermore, our extensive simulations show that the trade-off trend does not depend on the network topology and library popularity profile details. Since many Internet applications employ a multitier architecture, in this article, we focus on the problem of analytically modeling the behavior of such applications. We present a model based on a network of queues where the queues represent different tiers of the application. Our model is sufficiently general to capture (i) the behavior of tiers with significantly different performance characteristics and (ii) application idiosyncrasies such as session-based workloads, tier replication, load imbalances across replicas, and caching at intermediate tiers. We validate our model using real multitier applications running on a Linux server cluster. Our experiments indicate that our model faithfully captures the performance of these applications for a number of workloads and configurations. Furthermore, our model successfully handles a comprehensive range of resource utilization---from 0 to near saturation for the CPU---for two separate tiers. For a variety of scenarios, including those with caching at one of the application tiers, the average response times predicted by our model were within the 95p confidence intervals of the observed average response times. Our experiments also demonstrate the utility of the model for dynamic capacity provisioning, performance prediction, bottleneck identification, and session policing. In one scenario, where the request arrival rate increased from less than 1500 to nearly 4200 requests minute, a dynamic provisioning technique employing our model was able to maintain response time targets by increasing the capacity of two of the tiers by factors of 2 and 3.5, respectively. In this paper we investigate the potential benefits of coordinated congestion control for multipath data transfers, and contrast with uncoordinated control. For static random path selections, we show the worst-case throughput performance of uncoordinated control behaves as if each user had but a single path (scaling like log(log(N)) log(N) where N is the system size, measured in number of resources). Whereas coordinated control gives a throughput allocation bounded away from zero, improving on both uncoordinated control and on the greedy-least loaded path selection of e.g. Mitzenmacher. We then allow users to change their set of routes and introduce the notion of a Nash equilibrium. We show that with RTT bias (as in TCP Reno), uncoordinated control can lead to inefficient equilibria. With no RTT bias, both uncoordinated or coordinated Nash equilibria correspond to desirable welfare maximising states. Moreover, simple path reselection polices that shift to paths with higher net benefit can find these states.