A Survey on Efforts to Evolve the Control Plane of Inter-Domain Routing <s> Inter-Domain Communication <s> The networking equipment market is being transformed by the need for greater openness and flexibility, not only for research purposes but also for in-house innovation by the equipment owners. In contrast to networking gear following the model of computer mainframes, where closed software runs on proprietary hardware, the software-defined networking approach effectively decouples the data from the control plane via an open API (i.e., OpenFlow protocol) that allows the (remote) control of packet forwarding engines. Motivated by this scenario, we propose RouteFlow, a commodity routing architecture that combines the line-rate performance of commercial hardware with the flexibility of open-source routing stacks (remotely) running on general purpose computers. The outcome is a novel point in the design space of commodity routing solutions with far-reaching implications towards virtual routers and IP networks as a service. This paper documents the progress achieved in the design and prototype implementation of our work and outlines our research agenda that calls for a community-driven approach. <s> BIB001 </s> A Survey on Efforts to Evolve the Control Plane of Inter-Domain Routing <s> Inter-Domain Communication <s> Software-defined networking empowers network operators with more flexibility to program their networks. With SDN, network management moves from codifying functionality in terms of low-level device configurations to building software that facilitates network management and debugging. By separating the complexity of state distribution from network specification, SDN provides new ways to solve long-standing problems in networking --- routing, for instance --- while simultaneously allowing the use of security and dependability techniques, such as access control or multi-path. However, the security and dependability of the SDN itself is still an open issue. In this position paper we argue for the need to build secure and dependable SDNs by design. As a first step in this direction we describe several threat vectors that may enable the exploit of SDN vulnerabilities. We then sketch the design of a secure and dependable SDN control platform as a materialization of the concept here advocated. We hope that this paper will trigger discussions in the SDN community around these issues and serve as a catalyser to join efforts from the networking and security & dependability communities in the ultimate goal of building resilient control planes. <s> BIB002 </s> A Survey on Efforts to Evolve the Control Plane of Inter-Domain Routing <s> Inter-Domain Communication <s> Software defined networking (SDN) [1] works as a centralized control model. However, the Internet is managed by owners of different domains, which makes the centralized control does not work for inter-domain. Controlling the flow of data packets in a global network requires each NOS (Network Operating System) to have a relative global network view to determine the next NOS hop. Hence, NOSes are required to exchange reachability and topology information between the inter-domain networks. How such information is efficiently exchanged has not been well addressed so far. Thus, this demo proposes a West-East Bridge (WE-Bridge) mechanism for different SDN administrative domains to peer and cooperate with each other: (1) we design a peer-to-peer based high performance network information exchange mechanism. We define what network information should be exchanged and how such information is efficiently exchanged among interdomain SDN peers. (2) To achieve a resilient peer-to-peer control plane of heterogeneous NOSes, we propose a ‘maximum connection degree’ based connection algorithm [4]. (3) Considering the privacy, we propose to virtualize the SDN network view, and only exchange the virtualized network view to construct the relative global network view. Applications above WE-Bridge carry out the route translation, calculation, and installation. We deployed WE-Bridge together with two use cases to four SDN networks CERNET (China Education and Research Network), Internet2 in USA, CSTNET (China Science and Technology Network), and SURFNET (the national research and education network of Netherlands). <s> BIB003 </s> A Survey on Efforts to Evolve the Control Plane of Inter-Domain Routing <s> Inter-Domain Communication <s> The Internet's interdomain routing architecture has undergone only minor changes since its inception. It presents issues difficult to solve such as the difficulty in deploying new features, understanding its behavior and dynamics, identification and faults correction. Efforts have been made to address these issues but implementation of new applications onto the Internet architecture is a hard task due to the need to deploy directly to routers, a task that is compounded by the requirement of global acceptance of new protocols and modifications. This work presents the Inter-SDN component, a novel mechanism to provide interdomain routing using the Software-Defined Networking (SDN) approach. We discuss the main issues of current interdomain routing, describe the Inter-SDN component, its behavior and experimental evaluation. We also demonstrate that building interdomain mechanisms through the SDN approach is not complex, and discuss how our solution takes advantage of the SDN features to address issues on interdomain routing. <s> BIB004 </s> A Survey on Efforts to Evolve the Control Plane of Inter-Domain Routing <s> Inter-Domain Communication <s> Applications using the Internet already have access to some topology ::: information of Internet Service Provider (ISP) networks. For example, ::: views to Internet routing tables at Looking Glass servers are ::: available and can be practically downloaded to many network ::: application clients. What is missing is knowledge of the underlying ::: network topologies from the point of view of ISPs. In other words, ::: what an ISP prefers in terms of traffic optimization -- and a way to ::: distribute it. The Application-Layer Traffic Optimization (ALTO) ::: services defined in this document provide network information (e.g., ::: basic network location structure and preferences of network paths) ::: with the goal of modifying network resource consumption patterns while ::: maintaining or improving application performance. The basic ::: information of ALTO is based on abstract maps of a network. These maps ::: provide a simplified view, yet enough information about a network for ::: applications to effectively utilize them. Additional services are ::: built on top of the maps. This document describes a protocol ::: implementing the ALTO services. Although the ALTO services would ::: primarily be provided by ISPs, other entities, such as content service ::: providers, could also provide ALTO services. Applications that could ::: use the ALTO services are those that have a choice to which end points ::: to connect. Examples of such applications are peer-to-peer (P2P) and ::: content delivery networks. <s> BIB005 </s> A Survey on Efforts to Evolve the Control Plane of Inter-Domain Routing <s> Inter-Domain Communication <s> The Border Gateway Protocol (BGP) was designed almost three decades ago and has many limitations relating to its fully distributed nature, policy enforcement capabilities, scalability, security and complexity. For example, the control plane can take several minutes to converge after a routing change; this may be unacceptable for real-time network services. Despite many research proposals for incremental improvements and clean-slate redesigns of how inter-domain routing should work, BGP is likely one of the most ossified protocols of the Internet architecture and it has not retrofitted the proposed ideas. In this work, we propose a radical, incrementally deployable Internet routing paradigm in which the control plane of multiple networks is logically centralized. This follows the Software Defined Networking (SDN) paradigm, although at the inter-domain level involving multiple Autonomous Systems (AS). Multi-domain SDN centralization can be realized by outsourcing routing functions to an external contractor, which provides inter-domain routing services facilitated through a multi-AS network controller. The proposed model promises to become a vehicle for evolving BGP and uses the bird's eye view over several networks to benefit aspects of inter-domain routing, such as convergence properties, policy conflict resolution, inter-domain troubleshooting, and collaborative security. In addition to the proposed paradigm, we introduce a publicly available emulation platform built on top of Mininet and the Quagga routing software, for experimenting on hybrid BGP-SDN AS-level networks. As a proof of concept, we focus specifically on exploiting multi-domain centralization to improve BGP's slow convergence. We build and make publicly available a first multi-AS controller tailored to this use case and demonstrate experimentally that SDN centralization helps to linearly reduce BGP convergence times and churn rates with expanding SDN deployments. <s> BIB006 </s> A Survey on Efforts to Evolve the Control Plane of Inter-Domain Routing <s> Inter-Domain Communication <s> In a number of environments, a component external to a network is ::: called upon to perform computations based on the network topology and ::: current state of the connections within the network, including Traffic ::: Engineering (TE) information. This is information typically ::: distributed by IGP routing protocols within the network. This ::: document describes a mechanism by which link-state and TE information ::: can be collected from networks and shared with external components ::: using the BGP routing protocol. This is achieved using a new BGP ::: Network Layer Reachability Information (NLRI) encoding format. The ::: mechanism is applicable to physical and virtual IGP links. The ::: mechanism described is subject to policy control. Applications of ::: this technique include Application-Layer Traffic Optimization (ALTO) ::: servers and Path Computation Elements (PCEs). <s> BIB007 </s> A Survey on Efforts to Evolve the Control Plane of Inter-Domain Routing <s> Inter-Domain Communication <s> The Internet reserves numerous potential path diversity among densely connected autonomous systems (ASes). However, the Internet routing is controlled by Border Gateway Protocol (BGP), which has limitations in the path diversity expression and cooperation between ASes. The emergence of software defined networking (SDN) scheme provides flexible control over networks. In this paper, we leverage the programmability of SDN, and propose a new routing control plane, named RCS. It can support flexible inter-domain forwarding control, and enable network functions chaining along inter-domain paths. In RCS, network functions are abstracted and disseminated between SDN ASes. Customer networks can set up their desired routing paths for particular applications, such as multipath routing. RCS can be deployed incrementally to apply SDN to current inter-domain settings. In our primary analysis based on BGP data, the provision of customizable forwarding control at only a few ASes can lead to a large of potential path diversity. <s> BIB008 </s> A Survey on Efforts to Evolve the Control Plane of Inter-Domain Routing <s> Inter-Domain Communication <s> Software-defined networking (SDN) scheme decouples network control plane and data plane, which can improve the flexibility of traffic management in networks. OpenFlow is a promising implementation instance of SDN scheme and has been applied to enterprise networks and data center networks in practice. However, it has less effort to spread SDN control scheme over the Internet to conquer the ossification of inter-domain routing. In this paper, we further innovate to the SDN inter-domain routing inspired by the OpenFlow protocol. We apply SDN flow-based routing control to inter-domain routing and propose a fine-granularity inter-domain routing mechanism, named SDI (Software Defined Inter-domain routing). It enables inter-domain routing to support the flexible routing policy by matching multiple fields of IP packet header. We also propose a method to reduce redundant flow entries for inter-domain settings. And, we implement a prototype and deploy it on a multi-domain testbed. <s> BIB009 </s> A Survey on Efforts to Evolve the Control Plane of Inter-Domain Routing <s> Inter-Domain Communication <s> The border gateway protocol (BGP) is the Internet’s default inter-domain routing protocol that manages connectivity among autonomous systems (ASes). Over the past two decades many anomalies of BGP have been identified that threaten its stability and reliability. This survey discusses and classifies these anomalies and discusses the 20 most significant techniques used to identify them. Our classification is based on the broad category of approach, BGP features used to identify the anomaly, effectiveness in identifying the anomaly and effectiveness in identifying which AS was the location of the event that caused the anomaly. We also discuss a number of key requirements for the next generation of BGP anomaly detection techniques. <s> BIB010 </s> A Survey on Efforts to Evolve the Control Plane of Inter-Domain Routing <s> Inter-Domain Communication <s> This document enhances the BGP Cease NOTIFICATION message ::: "Administrative Shutdown" and "Administrative Reset" subcodes for ::: operators to transmit a short freeform message to describe why a BGP ::: session was shutdown or reset. This document updates RFC 4486. <s> BIB011 </s> A Survey on Efforts to Evolve the Control Plane of Inter-Domain Routing <s> Inter-Domain Communication <s> In distributed SDN architectures, the network is controlled by a cluster of multiple controllers. This distributed approach permits to meet the scalability and reliability requirements of large operational networks. Despite that, a logical centralized view of the network state should be guaranteed, enabling the simple development of network applications. Achieving a consistent network state requires a consensus protocol, which generates control traffic among the controllers whose timely delivery is crucial for network performance. We focus on the state-of-art ONOS controller, designed to scale to large networks, based on a cluster of self-coordinating controllers. In particular, we study the inter-controller control traffic due to the adopted consistency protocols. Based on real traffic measurements and the analysis of the adopted consistency protocols, we develop some empirical models to quantify the traffic exchanged among the controllers, depending on the considered shared data structures, the current network state (e.g., topology) and the occurring network events (e.g., flow or host addition). Our models provide a formal tool to be integrated into the design and dimension the control network interconnecting the controllers. Our results are of paramount importance for the proper design of large SDN networks, in which the control plane is implemented in-band and cannot exploit dedicated network resources. <s> BIB012
The control communication between different domains is also an important criterion to be analyzed in related works. Table 5 depicts the related works that evolve the inter-domain control plane routing using some inter-domain communication mechanism. For ASs that exchange control information with the BGP protocol, they can use the BGP attributes for that purpose. For example, an AS may set MED or Communities' values to alter how the control plane of other ASs selects routes [83, 84] . However, the BGP attributes have limited scope and effectiveness BIB010 , and consequently, new signaling mechanisms between different control planes emerged to overcome those restrictions. To tackle the ability of the BGP advertising just one best route per prefix, ADD-PATH is an extension to the BGP that uses its capability to advertise, identify and add multiple paths to a destination  . The standard behavior of the BGP produces and advertises only one best-path to a given address prefix. If two NLRIs are advertised with the same value of the address prefix, the latest advertisement will override the previous one in the RIB. ADD-PATH allows the advertisement of multiple paths for the same address prefix without the new paths implicitly replacing any previous ones. The main idea of this BGP extension is instead of using the address prefix as the primary identifier of a path, it uses a Path Identifier, in which each path receives a unique identifier, and that allows multiple values of address prefixes to exist in the RIB. By exchanging, the topology and information about the current state of the network among routing components are typically used as the interior routing protocol (for example, OSPF) for such tasks. Generally, the ASs are not willing to reveal their internal infrastructure or the business relationship information among other ASs because of security or business concerns. However, in some scenarios, the link-state or Traffic Engineering (TE) information can be shared with external components of a domain, for example, applications that require end-to-end TE (this is the case of the SIREN BIB006 proposal where the external control plane was responsible for computing the routing of the customer domain or Multi-Protocol Label Switching Traffic Engineering (MPLS-TE) with Path Computation Element (PCE). The North-Bound Distribution of Link-State and Traffic Engineering (TE) Information BIB007 describes a mechanism to collect and share link-state and TE information with external components by a new BGP NLRI encoding format. Another extension for BGP to improve the communication among BGP neighbors is the BGP Administrative Shutdown Communication BIB011 , defined in RFC 8203. It tackles the lack of information when a BGP session was reset or shutdown. Thus, the solution adds a short text message as part of the notification message of the BGP  . The purpose of that message is to notify operators about the event that caused the BGP closure. RouteFlow BIB001 controls and configures flows of OpenFlow switches using Quagga  as the main engine. RouteFlow becomes a proxy for the OpenFlow controller, where all the network logic follows traditional network protocols with inter-domain routing executing the BGP protocol. Furthermore, RouteFlow provides virtualized IP routing services over OpenFlow-enabled hardware, with the main idea of the proposed architecture adopting a west/eastbound interface to make a deeper integration of the routing engines of an ISP and its customers. As recent surveys indicate, there is not yet a standard west/eastbound SDN interface to exchange network control information between different SDN controllers  BIB003 BIB002 . Thereby, to allow the communication between SDN controllers in the inter-domain ecosystem, the authors in BIB003 proposed an interface for SDN to exchange reachability and topology information, the WE-Bridge BIB003 . The goals were to present a peer-to-peer mechanism that has to be resilient, secure and exchange network control information for ASs. WE-Bridge was used by RCS BIB008 and SDI BIB009 to exchange control information between SDN controllers. To exchange routing information between SDN domains (a domain that deploys SDN technologies), the Inter-SDN Routing Component BIB004 was proposed to tackle the problem of integration between the different control planes. Using TCP connections, the new exchange mechanism is inspired by the BGP and incorporates messages similar to the BGP (OPEN, UPDATE, KEEPALIVE and NOTIFICATION), uses RIBs (RIB-In, Local RIB, RIB-Out) and simplifies the route selection decision process. The Inter-SDN claims that the way to solve inter-domain issues is through architectural abstraction (SDN applications) and extensibility properties (the network programmability). Application-Layer Traffic Optimization (ALTO) BIB005 (defined in RFC 7285) is the most detailed specification for a standardization of the east/westbound interface for SDN controllers. It provides a network information service with the goal of exporting resources and parameters to network applications. ALTO is based on abstract maps of a network that simplify the state of a network and the applications that can effectively use them. Summary: Traditional network approaches for inter-domain communications focus on adding new capabilities or modifications to the BGP's messages. However, those approaches require a long cycle (months or years) of development until they reach the production environment. The SDN and traditional network have one point in common: those initiatives tackle the challenge of managing a huge number of network devices that requires coordination among multiple control planes. In fact, to overcome BGP control plane limitations for inter-domain communication, SDN approaches seek an integration with BGP and coordination between SDN controllers. Additionally, SDN network applications require complex network information that is hidden through inter-domain routing to allow the propriety execution of optimization algorithms in the network BIB012 .