A key challenge within the social network literature is the problem of network generation - that is, how can we create synthetic networks that match characteristics traditionally found in most real world networks? Important characteristics that are present in social networks include a power law degree distribution, small diameter and large amounts of clustering; however, most current network generators, such as the Chung Lu and Kronecker models, largely ignore the clustering present in a graph and choose to focus on preserving other network statistics, such as the power law distribution. Models such as the exponential random graph model have a transitivity parameter, but are computationally difficult to learn, making scaling to large real world networks intractable. In this work, we propose an extension to the Chung Lu ran- dom graph model, the Transitive Chung Lu (TCL) model, which incorporates the notion of a random transitive edge. That is, with some probability it will choose to connect to a node exactly two hops away, having been introduced to a 'friend of a friend'. In all other cases it will follow the standard Chung Lu model, selecting a 'random surfer' from anywhere in the graph according to the given invariant distribution. We prove TCL's expected degree distribution is equal to the degree distribution of the original graph, while being able to capture the clustering present in the network. The single parameter required by our model can be learned in seconds on graphs with millions of edges, while networks can be generated in time that is linear in the number of edges. We demonstrate the performance TCL on four real- world social networks, including an email dataset with hundreds of thousands of nodes and millions of edges, showing TCL generates graphs that match the degree distribution, clustering coefficients and hop plots of the original networks.
One method that can model clustering and can learn the associated parameter is the Exponential Random Graph Model (ERGM) @cite_0 . ERGMs define a probability distribution over the set of possible graphs with a log-linear model that uses feature counts of local graph properties. However, these models are typically hard to train as each update of the Fisher scoring function takes @math . With real-world networks numbering in the hundreds of thousands if not millions of nodes, this makes ERGMs impossible to fit.
@cite_0
2024818197 2145402497 2143270803 2084983846
Exponential-family random graph models (ERGMs) provide a principled and flexible way to model and simulate features common in social networks, such as propensities for homophily, mutuality, and friend-of-a-friend triad closure, through choice of model terms (sufficient statistics). However, those ERGMs modeling the more complex features have, to date, been limited to binary data: presence or absence of ties. Thus, analysis of valued networks, such as those where counts, measurements, or ranks are observed, has necessitated dichotomizing them, losing information and introducing biases. In this work, we generalize ERGMs to valued networks. Focusing on modeling counts, we formulate an ERGM for networks whose ties are counts and discuss issues that arise when moving beyond the binary case. We introduce model terms that generalize and model common social network features for such data and apply these methods to a network dataset whose values are counts of interactions. We describe some of the capabilities of the ergm package and the statistical theory underlying it. This package contains tools for accomplishing three important, and inter-related, tasks involving exponential-family random graph models (ERGMs): estimation, simulation, and goodness of fit. More precisely, ergm has the capability of approximating a maximum likelihood estimator for an ERGM given a network data set; simulating new network data sets from a fitted ERGM using Markov chain Monte Carlo; and assessing how well a fitted ERGM does at capturing characteristics of a particular network data set. The growing availability of network data and of scientific interest in distributed systems has led to the rapid development of statistical models of network structure. Typically, however, these are models for the entire network, while the data consists only of a sampled sub-network. Parameters for the whole network, which is what is of interest, are estimated by applying the model to the sub-network. This assumes that the model is consistent under sampling, or, in terms of the theory of stochastic processes, that it defines a projective family. Focusing on the popular class of exponential random graph models (ERGMs), we show that this apparently trivial condition is in fact violated by many popular and scientifically appealing models, and that satisfying it drastically limits ERGM’s expressive power. These results are actually special cases of more general results about exponential families of dependent random variables, which we also prove. Using such results, we offer easily checked conditions for the consistency of maximum likelihood estimation in ERGMs, and discuss some possible constructive responses. This paper studies networks where all nodes are distributed on a unit square A=Δ [- [1 2], [1 2]]2 following a Poisson distribution with known density ρ and a pair of nodes separated by an Euclidean distance x are directly connected with probability grρ(x)=Δg(x rρ), independent of the event that any other pair of nodes are directly connected. Here, g:[0,∞)→ [0,1] satisfies the conditions of rotational invariance, nonincreasing monotonicity, integral boundedness, and g(x)=o(1 (x2log2x)) ; further, rρ=√ (logρ+b) (Cρ) where C=∫ℜ2g(||x||)dx and b is a constant. Denote the aforementioned network by G(Xρ,grρ,A). We show that as ρ→ ∞, 1) the distribution of the number of isolated nodes in G(Xρ,grρ,A) converges to a Poisson distribution with mean e-b ; 2) asymptotically almost surely (a.a.s.) there is no component in G(Xρ,grρ,A) of fixed and finite order k >; 1; c) a.a.s. the number of components with an unbounded order is one. Therefore, as ρ→ ∞, the network a.a.s. contains a unique unbounded component and isolated nodes only; a sufficient and necessary condition for G(Xρ,grρ,A) to be a.a.s. connected is that there is no isolated node in the network, which occurs when b→ ∞ as ρ→ ∞. These results expand recent results obtained for connectivity of random geometric graphs from the unit disk model and the fewer results from the log-normal model to the more general and more practical random connection model.