Access to fiber-optic connectivity in the Internet is traditionally offered either via lit circuits or dark fiber. Economic (capex vs. opex) and operational considerations (latency, capacity) dictate the choice between these two offerings, but neither may effectively address the specific needs of modern-day enterprises or service providers over a range of use scenarios. In this paper, we describe a new approach for fiber-optic connectivity in the Internet that we call GreyFiber. The core idea of GreyFiber is to offer flexible access to fiber-optic paths between end points (e.g., datacenters or colocation facilities) over a range of timescales. We identify and discuss operational issues and systems challenges that need to be addressed to make GreyFiber a viable and realistic option for offering flexible access to infrastructure (similar to cloud computing). We investigate the efficacy of GreyFiber with a prototype implementation deployed in the GENI and CloudLab testbeds. Our scaling experiments show that 50 circuits can be provisioned within a minute. We also show that backup paths can be provisioned 28 times faster than an OSPF-based solution during failure maintenance events. Our experiments also examine GreyFiber overhead demands and show that the time spent in circuit creation is dependent on the network infrastructure, indicating avenues for future improvements.
Infrastructure provisioning. In the context of datacenter and WAN settings, infrastructure provisioning has been of interest to both industrial and academic communities @cite_44 @cite_8 @cite_42 @cite_67 . SDN-based provisioning approaches include B4 @cite_27 , SWAN @cite_51 , Owan @cite_44 , and others @cite_56 @cite_73 @cite_28 @cite_10 @cite_26 , each of which aims at improving the utilization of inter-datacenter and wide area networks. A survey of related efforts are available here @cite_25 @cite_64 . We posit that deployment of such efforts along with acquiring access to physical paths (via IRU or ) between DCs has the potential to produce better performance results than considering either of these solutions in isolation. In particular, we argue that such an environment, which considers provisioning and access to physical paths in tandem, can facilitate improvements at the physical layer @cite_95 , network layer optimizations @cite_48 , and cross-layer enhancements, e.g. , @cite_96 @cite_19 .
@cite_67 @cite_26 @cite_64 @cite_8 @cite_28 @cite_10 @cite_48 @cite_42 @cite_96 @cite_56 @cite_44 @cite_95 @cite_27 @cite_19 @cite_73 @cite_51 @cite_25
2052220570 2619707440 2167025919 2106675331
Current capacity planning practices based on heavy over-provisioning of power infrastructure hurt (i) the operational costs of data centers as well as (ii) the computational work they can support. We explore a combination of statistical multiplexing techniques to improve the utilization of the power hierarchy within a data center. At the highest level of the power hierarchy, we employ controlled underprovisioning and over-booking of power needs of hosted workloads. At the lower levels, we introduce the novel notion of soft fuses to flexibly distribute provisioned power among hosted workloads based on their needs. Our techniques are built upon a measurement-driven profiling and prediction framework to characterize key statistical properties of the power needs of hosted workloads and their aggregates. We characterize the gains in terms of the amount of computational work (CPU cycles) per provisioned unit of power Computation per Provisioned Watt (CPW). Our technique is able to double the CPWoffered by a Power Distribution Unit (PDU) running the e-commerce benchmark TPC-W compared to conventional provisioning practices. Over-booking the PDU by 10 based on tails of power profiles yields a further improvement of 20 . Reactive techniques implemented on our Xen VMM-based servers dynamically modulate CPU DVFS states to ensure power draw below the limits imposed by soft fuses. Finally, information captured in our profiles also provide ways of controlling application performance degradation despite overbooking. The 95th percentile of TPC-W session response time only grew from 1.59 sec to 1.78 sec--a degradation of 12 . Increasingly, proposals for new datacenter networking fabrics employ some form of traffic scheduling---often to avoid congestion, mitigate queuing delays, or avoid timeouts. Fundamentally, practical implementations require estimating upcoming traffic demand. Unfortunately, as our results show, it is difficult to accurately predict demand in typical datacenter applications more than a few milliseconds ahead of time. We explore the impact of errors in demand estimation on traffic scheduling in circuit-switched networks. We show that even relatively small estimation errors such as shifting the arrival time of at most 30 of traffic by a few milliseconds can lead to suboptimal schedules that dramatically reduce network efficiency. Existing systems cope by provisioning extra capacity---either on each circuit, or through the addition of a separate packet-switched fabric. We show through simulation that indirect traffic routing is a powerful technique for recovering from the inefficiencies of suboptimal scheduling under common datacenter workloads, performing as well as networks with 16 extra circuit bandwidth or a packet switch with 6 of the circuit bandwidth. Datacenter networks provide high path diversity for traffic between machines. Load balancing traffic across these paths is important for both, latency- and throughput-sensitive applications. The standard load balancing techniques used today obliviously hash a flow to a random path. When long flows collide on the same path, this might lead to long lasting congestion while other paths could be underutilized, degrading performance of other flows as well. Recent proposals to address this shortcoming incur significant implementation complexity at the host that would actually slow down short flows (MPTCP), depend on relatively slow centralized controllers for rerouting large congesting flows (Hedera), or require custom switch hardware, hindering near-term deployment (DeTail). We propose FlowBender, a novel technique that: (1) Load balances distributively at the granularity of flows instead of packets, avoiding excessive packet reordering. (2) Uses end-host-driven rehashing to trigger dynamic flow-to-path assignment. (3) Recovers from link failures within a Retransmit Timeout (RTO). (4) Amounts to less than 50 lines of critical kernel code and is readily deployable in commodity data centers today. (5) Is very robust and simple to tune. We evaluate FlowBender using both simulations and a real testbed implementation, and show that it improves average and tail latencies significantly compared to state of the art techniques without incurring the significant overhead and complexity of other load balancing schemes. Data center power infrastructure incurs massive capital costs, which typically exceed energy costs over the life of the facility. To squeeze maximum value from the infrastructure, researchers have proposed over-subscribing power circuits, relying on the observation that peak loads are rare. To ensure availability, these proposals employ power capping, which throttles server performance during utilization spikes to enforce safe power budgets. However, because budgets must be enforced locally -- at each power distribution unit (PDU) -- local utilization spikes may force throttling even when power delivery capacity is available elsewhere. Moreover, the need to maintain reserve capacity for fault tolerance on power delivery paths magnifies the impact of utilization spikes. In this paper, we develop mechanisms to better utilize installed power infrastructure, reducing reserve capacity margins and avoiding performance throttling. Unlike conventional high-availability data centers, where collocated servers share identical primary and secondary power feeds, we reorganize power feeds to create shuffled power distribution topologies. Shuffled topologies spread secondary power feeds over numerous PDUs, reducing reserve capacity requirements to tolerate a single PDU failure. Second, we propose Power Routing, which schedules IT load dynamically across redundant power feeds to: (1) shift slack to servers with growing power demands, and (2) balance power draw across AC phases to reduce heating and improve electrical stability. We describe efficient heuristics for scheduling servers to PDUs (an NP-complete problem). Using data collected from nearly 1000 servers in three production facilities, we demonstrate that these mechanisms can reduce the required power infrastructure capacity relative to conventional high-availability data centers by 32 without performance degradation.