The Smart Grid is not just about the digitalization of the Power Grid. In its more visionary acceptation, it is a model of energy management in which the users are engaged in producing energy as well as consuming it, while having information systems fully aware of the energy demand-response of the network and of dynamically varying prices. A natural question is then: to make the Smart Grid a reality will the Distribution Grid have to be updated? We assume a positive answer to the question and we consider the lower layers of Medium and Low Voltage to be the most affected by the change. In our previous work, we have analyzed samples of the Dutch Distribution Grid in our previous work and we have considered possible evolutions of these using synthetic topologies modeled after studies of complex systems in other technological domains in another previous work. In this paper, we take an extra important further step by defining a methodology for evolving any existing physical Power Grid to a good Smart Grid model thus laying the foundations for a decision support system for utilities and governmental organizations. In doing so, we consider several possible evolution strategies and apply then to the Dutch Distribution Grid. We show how more connectivity is beneficial in realizing more efficient and reliable networks. Our proposal is topological in nature, and enhanced with economic considerations of the costs of such evolutions in terms of cabling expenses and economic benefits of evolving the Grid.
The concept of network evolution of a technological infrastructure is considered in @cite_31 where a model for the evolution of the Internet Autonomous System (AS) is provided. As most works (e.g., @cite_16 @cite_37 ), this one focuses on creating a model of the existing evolution of the Internet rather than proposing new ways of evolving networks. The interesting aspect on the modeling of the Internet is the better results of the proposed model Parallel Addition and Rewiring Growth (PARG) compared to other models in the literature to capture specific features of the AS topology. In particular, the model is able to capture the dissortativeness of hubs (i.e., the most connected nodes in the network) and linear local assortativeness of nodes. The model proposed proves successful, however one must recognize that the AS connection are mainly logical and are due more to business and contractual agreements than physical interconnections, therefore missing part of the problem that is essential in the case of the topology.
@cite_31 @cite_16 @cite_37
2124637492 2164066487 2168896238 2120652359 2144089718 2151200985 2082986408 2139708889 2072251439 1990271487 2055654888 1846523235 2106335950 2101888944 137863291 1571874655 2962949867 2164511281 2151078464 2140083824
The emergence of order in natural systems is a constant source of inspiration for both physical and biological sciences. While the spatial order characterizing for example the crystals has been the basis of many advances in contemporary physics, most complex systems in nature do not offer such high degree of order. Many of these systems form complex networks whose nodes are the elements of the system and edges represent the interactions between them. Traditionally complex networks have been described by the random graph theory founded in 1959 by Paul Erdohs and Alfred Renyi. One of the defining features of random graphs is that they are statistically homogeneous, and their degree distribution (characterizing the spread in the number of edges starting from a node) is a Poisson distribution. In contrast, recent empirical studies, including the work of our group, indicate that the topology of real networks is much richer than that of random graphs. In particular, the degree distribution of real networks is a power-law, indicating a heterogeneous topology in which the majority of the nodes have a small degree, but there is a significant fraction of highly connected nodes that play an important role in the connectivity of the network. The scale-free topology of real networks has very important consequences on their functioning. For example, we have discovered that scale-free networks are extremely resilient to the random disruption of their nodes. On the other hand, the selective removal of the nodes with highest degree induces a rapid breakdown of the network to isolated subparts that cannot communicate with each other. The non-trivial scaling of the degree distribution of real networks is also an indication of their assembly and evolution. Indeed, our modeling studies have shown us that there are general principles governing the evolution of networks. Most networks start from a small seed and grow by the addition of new nodes which attach to the nodes already in the system. This process obeys preferential attachment: the new nodes are more likely to connect to nodes with already high degree. We have proposed a simple model based on these two principles wich was able to reproduce the power-law degree distribution of real networks. Perhaps even more importantly, this model paved the way to a new paradigm of network modeling, trying to capture the evolution of networks, not just their static topology. Characterizing the evolution of Internet topology is important to our understanding of the Internet architecture and its interplay with technical, economic and social forces. A major challenge in obtaining empirical data on topology evolution is to identify real topology changes from the observed topology changes, since the latter can be due to either topology changes or transient routing dynamics. In this paper, we formulate the topology liveness problem and propose a solution based on the analysis of BGP data. We find that the impact of transient routing dynamics on topology observation decreases exponentially over time, and that the real topology dynamics consist of a constant-rate birth process and a constant-rate death process. Our model enables us to infer real topology changes from observation data with a given confidence level. We demonstrate the usefulness of the model by applying it to three applications: providing more accurate views of the topology, evaluating theoretical evolution models, and empirically characterizing the trends of topology evolution. We find that customer networks and provider networks have distinct evolution trends, which can provide an important input to the design of future Internet routing architecture. Internet connectivity at the AS level, defined in terms of pairwise logical peering relationships, is constantly evolving. This evolution is largely a response to economic, political, and technological changes that impact the way ASs conduct their business. We present a new framework for modeling this evolutionary process by identifying a set of criteria that ASs consider either in establishing a new peering relationship or in reassessing an existing relationship. The proposed framework is intended to capture key elements in the decision processes underlying the formation of these relationships. We present two decision processes that are executed by an AS, depending on its role in a given peering decision, as a customer or a peer of another AS. When acting as a peer, a key feature of the AS’s corresponding decision model is its reliance on realistic inter-AS traffic demands. To reflect the enormous heterogeneity among customer or peer ASs, our decision models are flexible enough to accommodate a wide range of AS-specific objectives. We demonstrate the potential of this new framework by considering different decision models in various realistic “what if” experiment scenarios. We implement these decision models to generate and study the evolution of the resulting AS graphs over time, and compare them against observed historical evolutionary features of the Internet at the AS level. An understanding of the topological structure of the Internet is needed for quite a number of networking tasks, e. g., making decisions about peering relationships, choice of upstream providers, inter-domain traffic engineering. One essential component of these tasks is the ability to predict routes in the Internet. However, the Internet is composed of a large number of independent autonomous systems (ASes) resulting in complex interactions, and until now no model of the Internet has succeeded in producing predictions of acceptable accuracy.We demonstrate that there are two limitations of prior models: (i) they have all assumed that an Autonomous System (AS) is an atomic structure - it is not, and (ii) models have tended to oversimplify the relationships between ASes. Our approach uses multiple quasi-routers to capture route diversity within the ASes, and is deliberately agnostic regarding the types of relationships between ASes. The resulting model ensures that its routing is consistent with the observed routes. Exploiting a large number of observation points, we show that our model provides accurate predictions for unobserved routes, a first step towards developing structural mod-els of the Internet that enable real applications. Our goal is to understand the evolution of the Autonomous System (AS) ecosystem over the last decade. Instead of focusing on abstract topological properties, we classify ASes into a number of "species" depending on their function and business type. Further, we consider the semantics of inter-AS links, in terms of customer-provider versus peering relations. We find that the available historic datasets from RouteViews and RIPE are not sufficient to infer the evolution of peering links, and so we restrict our focus to customer-provider links. Our findings highlight some important trends in the evolution of the Internet over the last decade, and hint at what the Internet is heading towards. After an exponential increase phase until 2001, the Internet now grows linearly in terms of both ASes and inter-AS links. The growth is mostly due to enterprise networks and content access providers at the periphery of the Internet. The average path length remains almost constant mostly due to the increasing multihoming degree of transit and content access providers. In recent years, enterprise networks prefer to connect to small transit providers, while content access providers connect equally to both large and small transit providers. The AS species differ significantly from each other with respect to their rewiring activity; content access providers are the most active. A few large transit providers act as "attractors" or "repellers" of customers. For many providers, strong attractiveness precedes strong repulsiveness by 3-9 months. Finally, in terms of regional growth, we find that the AS ecosystem is now larger and more dynamic in Europe than in North America. We study the large-scale topological and dynamical properties of real Internet maps at the autonomous system level, collected in a 3-yr time interval. We find that the connectivity structure of the Internet presents statistical distributions settled in a well-defined stationary state. The large-scale properties are characterized by a scale-free topology consistent with previous observations. Correlation functions and clustering coefficients exhibit a remarkable structure due to the underlying hierarchical organization of the Internet. The study of the Internet time evolution shows a growth dynamics with aging features typical of recently proposed growing network models. We compare the properties of growing network models with the present real Internet data analysis. Based on the empirical analysis of the dependency network in 18 Java projects, we develop a novel model of network growth which considersboth: an attachment mechanism and the addition of new nodes with a heterogeneous distribution of their initial degree, k0. Empirically we find that the cumulative degree distributions of initial degrees and of the final network, follow power-law behaviors: P(k0) ∝ k01−α, and P(k) ∝ k1−γ, respectively. For the total number of links as a function of the network size, we find empirically K(N) ∝ Nβ, where β is (at the beginning of thenetwork evolution) between 1.25 and 2, while converging to ∼ 1 for large N. This indicates a transition from a growth regime with increasing network density towards a sustainable regime, which revents a collapse because of ever increasing dependencies. Our theoretical framework is able to predict relations between the exponents α, β, γ, which also link issues of software engineering and developer activity. These relations are verified by means of computer simulations and empirical investigations. They indicate that the growth of real Open Source Software networks occurs on the edge between two regimes, which are either dominated by the initial degree distribution of added nodes, or by the preferential attachment mechanism. Hence, the heterogeneous degree distribution of newly added nodes, found empirically, is essential to describe the laws of sustainable growth in networks. Network generators that capture the Internet's large-scale topology are crucial for the development of efficient routing protocols and modeling Internet traffic. Our ability to design realistic generators is limited by the incomplete understanding of the fundamental driving forces that affect the Internet's evolution. By combining several independent databases capturing the time evolution, topology, and physical layout of the Internet, we identify the universal mechanisms that shape the Internet's router and autonomous system level topology. We find that the physical layout of nodes form a fractal set, determined by population density patterns around the globe. The placement of links is driven by competition between preferential attachment and linear distance dependence, a marked departure from the currently used exponential laws. The universal parameters that we extract significantly restrict the class of potentially correct Internet models and indicate that the networks created by all available topology generators are fundamentally different from the current Internet. We propose a model to create synthetic networks that may also serve as a narrative of a certain kind of infrastructure network evolution. It consists of an initialization phase with the network extending tree-like for minimum cost and a growth phase with an attachment rule giving a trade-off between cost-optimization and redundancy. Furthermore, we implement the feature of some lines being split during the grid's evolution. We show that the resulting degree distribution has an exponential tail and may show a maximum at degree two, suitable to observations of real-world power grid networks. In particular, the mean degree and the slope of the exponential decay can be controlled in partial independence. To verify to which extent the degree distribution is described by our analytic form, we conduct statistical tests, showing that the hypothesis of an exponential tail is well-accepted for our model data. Motivated by data on the evolution of the Internet and World Wide Web we consider scenarios of self-organization of the nonlinearly growing networks into free-scale structures. We find that the accelerating growth of the networks establishes their structure. For the growing networks with preferential linking and increasing density of links, two scenarios are possible. In one of them, the value of the exponent @math of the connectivity distribution is between 3 2 and 2. In the other, @math and the distribution is necessarily non-stationary. We report a study of the correlations among topological, weighted and spatial properties of large infrastructure networks. We review the empirical results obtained for the air-transportation infrastructure that motivates a network modeling approach which integrates the various attributes of this network. In particular, we describe a class of models which include a weight-topology coupling and the introduction of geographical attributes during the network evolution. The inclusion of spatial features is able to capture the appearance of non-trivial correlations between the traffic flows, the connectivity pattern and the actual distances of vertices. The anomalous fluctuations in the betweenness-degree correlation function observed in empirical studies are also recovered in the model. The presented results suggest that the interplay between topology, weights and geographical constraints is a key ingredient in order to understand the structure and evolution of many real-world networks. The study of the Internet topology has recently received much attention from the research community. In particular, the observation that the network graph has interesting properties, such as power laws, that might be explored in a myriad of ways. Most of the work in characterizing the Internet graph is based on the physical network graph, i.e., the connectivity graph. In this paper we investigate how logical relationships between nodes of the AS graph can be used to gain insight to its structure. We characterize the logical graph using various metrics and identify the presence of power laws in the number of customers that a provider has. Using these logical relationships we define a structural model of the AS graph. The model highlights the hierarchical nature of logical relationships and the preferential connection to larger providers. We also investigate the consistency of this model over time and observe interesting properties of the hierarchical structure. The discovery of networks is a fundamental problem arising in numerous fields of science and technology, including communication systems, biology, sociology, and neuroscience. Unfortunately, it is often difficult, or impossible, to obtain data that directly reveal network structure, and so one must infer a network from incomplete data. This paper considers inferring network structure from "co-occurrence" data: observations that identify which network components (e.g., switches, routers, genes) carry each transmission but do not indicate the order in which they handle the transmission. Without order information, the number of networks that are consistent with the data grows exponentially with the size of the network (i.e., the number of nodes). Yet, the basic engineering evolutionary principles underlying most networks strongly suggest that not all data-consistent networks are equally likely. In particular, nodes that co-occur in many observations are probably closely connected. With this in mind, we model the co-occurrence observations as independent realizations of a random walk on the network, subjected to a random permutation to account for the lack of order information. Treating permutations as missing data, we derive an expectation-maximization (EM) algorithm for estimating the random walk parameters. The model and EM algorithm significantly simplify the problem, but the computational complexity of the reconstruction process does grow exponentially in the length of each transmission path. For networks with long paths, the exact e-step may be computationally intractable. We propose a polynomial-time Monte Carlo EM algorithm based on importance sampling and derive conditions that ensure convergence of the algorithm with high probability. Simulations and experiments with Internet measurements demonstrate the promise of this approach. In this paper, we empirically study the evolution of large scale Internet topology at the autonomous system (AS) level. The network size grows in an exponential form, obeying the famous Moore's law. We theoretically predict that the size of the AS-level Internet will double every 5.32 years. We apply the k-core decomposition method on the real Internet, and find that the size of a k-core with larger k is nearly stable over time. In addition, the maximal coreness is very stable after 2003. In contrast to the predictions of most previous models, the maximal degree of the Internet is also relatively stable versus time. We use the edge-exchange operation to obtain the randomized networks with the same degree sequence. A systematical comparison is drawn, indicating that the real Internet is more loosely connected, and both the full Internet and the nucleus are more disassortative than their randomized versions. Many real-world domains can be represented as large node-link graphs: backbone Internet routers connect with 70,000 other hosts, mid-sized Web servers handle between 20,000 and 200,000 hyperlinked documents, and dictionaries contain millions of words defined in terms of each other. Computational manipulation of such large graphs is common, but previous tools for graph visualization have been limited to datasets of a few thousand nodes. Visual depictions of graphs and networks are external representations that exploit human visual processing to reduce the cognitive load of many tasks that require understanding of global or local structure. We assert that the two key advantages of computer-based systems for information visualization over traditional paper-based visual exposition are interactivity and scalability. We also argue that designing visualization software by taking the characteristics of a target user's task domain into account leads to systems that are more effective and scale to larger datasets than previous work. This thesis contains a detailed analysis of three specialized systems for the interactive exploration of large graphs, relating the intended tasks to the spatial layout and visual encoding choices. We present two novel algorithms for specialized layout and drawing that use quite different visual metaphors. The H3 system for visualizing the hyperlink structures of web sites scales to datasets of over 100,000 nodes by using a carefully chosen spanning tree as the layout backbone, 3D hyperbolic geometry for a Focus+Context view, and provides a fluid interactive experience through guaranteed frame rate drawing. The Constellation system features a highly specialized 2D layout intended to spatially encode domain-specific information for computational linguists checking the plausibility of a large semantic network created from dictionaries. The Planet Multicast system for displaying the tunnel topology of the Internet's multicast backbone provides a literal 3D geographic layout of arcs on a globe to help MBone maintainers find misconfigured long-distance tunnels. Each of these three systems provides a very different view of the graph structure, and we evaluate their efficacy for the intended task. We generalize these findings in our analysis of the importance of interactivity and specialization for graph visualization systems that are effective and scalable. In this paper we apply data mining analysis to study the topology of the Internet thus creating a new processing framework To the best of our knowledge this is one of the rst studies that focus on the Internet topology at the router level i e each node is a router The size K nodes and the nature of the graph are such that new analysis methods have to be employed First we suggest computationally expensive metrics to characterize topological properties Then we present an e cient approximation algorithm that makes the calculation of these metrics possible Finally we demonstrate the initial results of our framework For example we show that we can identify central routers and poorly connected or even isolated nodes We also nd that the Internet is surprisingly resilient to random link and router failures having only small changes in the connectivity for fewer than failures Our framework seems a promising step towards understanding and characterizing the Internet topology and possible other real communication graphs such as web graphs It is widely believed that the success of deep convolutional networks is based on progressively discarding uninformative variability about the input with respect to the problem at hand. This is supported empirically by the difficulty of recovering images from their hidden representations, in most commonly used network architectures. In this paper we show that this loss of information is not a necessary condition to learn representations that generalize well on complicated problems, such as ImageNet. Via a cascade of homeomorphic layers, we build the i-RevNet, a network that can be fully inverted up to the final projection onto the classes, i.e. no information is discarded. Building an invertible architecture is difficult, for example, because the local inversion is ill-conditioned, we overcome this by providing an explicit inverse. An analysis of i-RevNet’s learned representations suggests an explanation of the good accuracy by a progressive contraction and linear separation with depth. To shed light on the nature of the model learned by the i-RevNet we reconstruct linear interpolations between natural images representations. An accurate Internet topology graph is important in many areas of networking, from deciding ISP business relationships to diagnosing network anomalies. Most Internet mapping efforts have derived the network structure, at the level of interconnected autonomous systems (ASes), from a limited number of either BGP- or traceroute- based data sources. While techniques for charting the topology continue to improve, the growth of the number of vantage points is significantly outpaced by the rapid growth of the Internet. In this paper, we argue that a promising approach to revealing the hidden areas of the Internet topology is through active measurement from an observation platform that scales with the growing Internet. By leveraging measurements performed by an extension to a popular P2P system, we show that this approach indeed exposes significant new topological information. Based on traceroute measurements from more than 992,000 IPs in over 3,700 ASes distributed across the Internet hierarchy, our proposed heuristics identify 23,914 new AS links not visible in the publicly-available BGP data - 12.86 more customer-provider links and 40.99 more peering links, than previously reported. We validate our heuristics using data from a tier-1 ISP and show that they correctly filter out all false links introduced by public IP-to-AS mapping. We have made the identified set of links and their inferred relationships publically available We present a detailed study of network evolution by analyzing four large online social networks with full temporal information about node and edge arrivals. For the first time at such a large scale, we study individual node arrival and edge creation processes that collectively lead to macroscopic properties of networks. Using a methodology based on the maximum-likelihood principle, we investigate a wide variety of network formation strategies, and show that edge locality plays a critical role in evolution of networks. Our findings supplement earlier network models based on the inherently non-local preferential attachment. Based on our observations, we develop a complete model of network evolution, where nodes arrive at a prespecified rate and select their lifetimes. Each node then independently initiates edges according to a "gap" process, selecting a destination for each edge according to a simple triangle-closing model free of any parameters. We show analytically that the combination of the gap distribution with the node lifetime leads to a power law out-degree distribution that accurately reflects the true network in all four cases. Finally, we give model parameter settings that allow automatic evolution and generation of realistic synthetic networks of arbitrary scale. Modeling Internet growth is important both for understanding the current network and to predict and improve its future. To date, Internet models have typically attempted to explain a subset of the following characteristics: network structure, traffic flow, geography, and economy. In this paper we present a discrete, agent-based model, that integrates all of them. We show that the model generates networks with topologies, dynamics, and more speculatively spatial distributions that are similar to the Internet.