<jupyter_start><jupyter_text>This tutorial is Part 2 of an introduction to social network analysis in Python. It covers how to sample and visualize network data. The primary example used for replication is <NAME>'s (2005) paper on relationships between political blogs ahead of the 2004 election. The paper is available here and is the source of all figures included below. We use the GML file available [here](http://www.thomaspadilla.org/data/network/polblogs/polblogs.gml) or [here](http://www-personal.umich.edu/~mejn/netdata/) ([documentation](http://www.thomaspadilla.org/data/network/polblogs/polblogs.txt)). Table of Contents1&nbsp;&nbsp;Importing external data files2&nbsp;&nbsp;Downsampling graphs2.1&nbsp;&nbsp;Snowball sample2.1.1&nbsp;&nbsp;Snowball sampling with single-source-shortest-paths2.1.2&nbsp;&nbsp;Snowball sampling with breadth-first search2.2&nbsp;&nbsp;Node and edge samples3&nbsp;&nbsp;Plotting3.1&nbsp;&nbsp;Graph Layouts3.1.1&nbsp;&nbsp;Default (spring) layout3.1.2&nbsp;&nbsp;Random layout3.1.3&nbsp;&nbsp;Kamada-Kawai (force-directed) layout3.1.4&nbsp;&nbsp;Layouts overview3.1.5&nbsp;&nbsp;Tuning graph layouts3.2&nbsp;&nbsp;Styling the plot aesthetic3.2.1&nbsp;&nbsp;Plotting the default3.2.2&nbsp;&nbsp;Change sizes of nodes, edges, or arrows3.2.3&nbsp;&nbsp;Change colors of nodes or edges3.2.4&nbsp;&nbsp;Change styling of nodes or arrows3.2.5&nbsp;&nbsp;Turn plot features on and off3.2.6&nbsp;&nbsp;Adding text3.3&nbsp;&nbsp;Drawing attention to selected features3.3.1&nbsp;&nbsp;Drawing selected nodes3.3.2&nbsp;&nbsp;Drawing selected node labels3.3.3&nbsp;&nbsp;Drawing selected edges3.3.4&nbsp;&nbsp;Drawing selected edge labels3.4&nbsp;&nbsp;Illustrating discrete attributes3.4.1&nbsp;&nbsp;Coloring nodes by label3.4.2&nbsp;&nbsp;Coloring edges by node label3.5&nbsp;&nbsp;Illustrating continuous attributes3.5.1&nbsp;&nbsp;Sizing nodes by value3.5.1.1&nbsp;&nbsp;Degree3.5.1.2&nbsp;&nbsp;Pagerank3.5.2&nbsp;&nbsp;Coloring nodes by value3.5.3&nbsp;&nbsp;Sizing edges by value3.5.4&nbsp;&nbsp;Degree histogram Let's get started! First, we need to import all of the packages we'll use to do our analysis.<jupyter_code>import pandas as pd               # For analyzing tabular data
import numpy as np                # For working with arrays and numerical operations
import networkx as nx             # For network data specifically
from networkx.drawing.nx_agraph import graphviz_layout  # ...and graphing network data
import matplotlib.pyplot as plt   # For making plots
from pyvis import network as pv   # For making interactive plots
import random                     # For generating random numbers

%matplotlib inline<jupyter_output><empty_output><jupyter_text>Importing external data files We will be using a dataset from <NAME>'s (2005) paper: _The political blogosphere and the 2004 US election: Divided they blog._The paper built a dataset of political blogs as follows:- They compiled a list of important blogs from blog curation sites (e.g. BlogCatalog).- They crawled the front pages of these blogs and got all other linked blogs on these pages.- If the other blog had at least 17 citations in the dataset, they kept it and got its links.- They labeled the political orientation of the blogs using the blog curation sites and their own inspections.        The final dataset had:- 759 liberal blogs- 735 conservative blogsKey findings from this analysis:> "91% of  links originating within either the conservative or liberal communities stay within that community...Conservative blogs show a greater tendency to link." Reference: https://networkx.github.io/documentation/stable/reference/readwrite/index.html First, let's practice importing files from external data sources. There are many different formats for disseminating network data, but some helpful ones supported by NetworkX include:```pythonnx.read_gml(filepath)                         GMLnx.read_edgelist(filepath, delimiter=',')     A delimited edgelistnx.from_pandas_edgelist(df)                   Pandas dataframe in edge list formatnx.from_pandas_adjacency(df)                  Pandas dataframe in adjacency matrix format``` We'll get started with the dataset of blog links.Note: Because some edges are duplicated, we had to modify the original file by adding a line to indicate that it is a multigraph. So, the dataset on this server is not perfectly identical to the original.<jupyter_code># Let's use NetworkX's read_gml to bring in the raw data file
G = nx.read_gml("data/polblogs.gml")
G
# As discussed, note that the graph above is a multigraph
# Let's now convert it from a multigraph to a directed graph
P = nx.DiGraph(G)
# And, we inspect
print(nx.info(P))<jupyter_output><empty_output><jupyter_text>The dataset does not seem to be a perfect match for the paper. For example, we have 1940 links while they had 1494. But, it's close. We can also confirm that our degree counts roughly match those reported for Daily Kos (338), Eschelon (264), and Instapundit (277).<jupyter_code># Code to get all blog names containing a given keyword: [i for i in list(P.nodes) if 'instapundit' in i]

# Let's check if individual blog degrees match:
P.in_degree(['dailykos.com', 'atrios.blogspot.com', 'instapundit.com'])<jupyter_output><empty_output><jupyter_text>Downsampling graphs We can see that the graph is (relatively) large and pretty well-connected: 1490 nodes and 19,025 edges. This is not actually "large" by network standards (Facebook has 1.7 billion users) but on my computer the plots are still running slow.Since we will be making _many_ plots, and since it's generally helpful for large graphs, let's learn to downsample. I will demonstrate two different approaches.  Snowball sample One common method for sampling graphs is _snowball sampling_. The basic idea is as follows:1. Start with set of seed nodes.2. Add any friends of the seed nodes.2. Add any friends of the friends of the seed nodes.2. Add any friends of the friends of the (friends of the...) seed nodes.<jupyter_code>source = "matthewyglesias.com"
p = nx.single_source_shortest_path(P,source,cutoff=2)

sample = list(p.keys())
Q = P.subgraph(sample)

T1 = nx.bfs_tree(Q, source = 'thewashingtonnote.com', depth_limit=1 )
T2 = nx.bfs_tree(Q, source = 'thewashingtonnote.com', depth_limit=2 )
T3 = nx.bfs_tree(Q, source = 'thewashingtonnote.com', depth_limit=3 )

pos = nx.kamada_kawai_layout(Q)

nx.draw(Q, pos=pos, arrows=False, edge_color='grey', width=0.5, node_color='grey')

nx.draw_networkx_nodes(Q, pos=pos, nodelist = T3.nodes, node_color='yellow')
nx.draw_networkx_edges(Q, pos=pos, edgelist = T3.edges, edge_color='orange', width=2)
nx.draw_networkx_nodes(Q, pos=pos, nodelist = T2.nodes, node_color='orange')
nx.draw_networkx_edges(Q, pos=pos, edgelist = T2.edges, edge_color='red', width=2.2)
nx.draw_networkx_nodes(Q, pos=pos, nodelist = T1.nodes, node_color='red')
nx.draw_networkx_edges(Q, pos=pos, edgelist = T1.edges, edge_color='darkred', width=2)
nx.draw_networkx_nodes(Q, pos=pos, nodelist = ['thewashingtonnote.com'], node_color='darkred')

plt.show()<jupyter_output><empty_output><jupyter_text>We will use a few helper functions, including some from Python's `random` package:    ```pythonnx.single_source_shortest_path(G, source, cutoff=n)   Get all shortest paths from source node within distance nnx.bfs_tree(G, source, depth_limit=n )                Conduct a breadth-first search of depth nS = G.subgraph(...)        Select a subgraphrandom.seed(n)             Set a "seed" for the random number generator. This ensures that your code is replicable.random.sample([...], n)    Choose n items from a list of items```  Snowball sampling with single-source-shortest-paths Actually, last week we learned an algorithmic approach to implementing this with NetworkX. Remember that we had a function, `single_source_shortest_path`, that would give a list of all shortest paths from a source node. Well, that function takes an optional argument, `cutoff`, that limits the maximum length of the shortest paths returned.This can help us find our sample: we start from our source node, and then walk away in all directions until we hit the cutoff. Let's try this for the blogger <NAME>.<jupyter_code># Let's make him the path
source = "matthewyglesias.com"

# Let's find what other nodes can be reached in two steps or less
paths = nx.single_source_shortest_path(P,source,cutoff=2)

# Our sample will be all of these nodes
sample = list(paths.keys())
len(sample)
# Let's take the subgraph of P containing this sample
S = P.subgraph(sample)
nx.draw(S)<jupyter_output><empty_output><jupyter_text>Snowball sampling with breadth-first search Note that this is equivalent to a breadth-first tree search on the graph.<jupyter_code># Let's do a BFS from the source with a depth of 2
sample = nx.bfs_tree(Q, source, depth_limit=2 )

# Let's collect the subgraph
S = P.subgraph(sample)

# Let's confirm that it's the same size
len(sample)<jupyter_output><empty_output><jupyter_text>Node and edge samples Let's try another strategy, just for fun.1. We will first randomly sample 500 nodes from the graph (reduce the number of nodes).2. Then, we will randomly sample 500 edges from these nodes (reduce the number of edges).3. Finally, we will keep only the largest connected component (reduce the number of small isolated components).<jupyter_code># Set the seed for a random number
random.seed(1)
# Step 1 --------------------------------------------------

# Choose a random sample of p's nodes 
random_nodes = random.sample(P.nodes, 500)

# Get the subgraph containing of P these nodes
R = P.subgraph(random_nodes)

print(nx.info(R))
# Step 2 --------------------------------------------------

# Choose a random sample of p's edges 
random_edges = random.sample(R.edges, 500)

# Get the subgraph containing of P these edges
R = P.edge_subgraph(random_edges)
# Step 3 --------------------------------------------------

# Get the biggest connected component
R = max(nx.strongly_connected_component_subgraphs(R), key=len)

nx.draw(R)<jupyter_output><empty_output><jupyter_text>Plotting  Exercise: Now, it's time to see what we've got! Before we start, let's practice drawing a graph of our own. Try plotting the following adjacency matrix:<jupyter_code># Import an example graph that I made
df = pd.read_csv('data/sample_graph.csv', index_col="Unnamed: 0").astype(int)
df<jupyter_output><empty_output><jupyter_text>Graph Layouts Reference: https://networkx.github.io/documentation/stable/reference/drawing.htmlmodule-networkx.drawing.layout A basic challenge for plotting networks is how to lay out the nodes and edges.- One option is to use a **fixed layout** design, such as circular layouts (which put all nodes in a circle) or geographic layouts (which put nodes at their geographic location on a map).- An alternative is to use an **algorithm** to determine the layout of nodes. Common classes of algorithms are:    - **Force-directed layouts:** These algorithms generally balance two forces: a baseline repulsion between all nodes, and a countervailing attraction between connected nodes. - **Spectral layouts:** These algorithms perform dimensionality reduction to divide the graph into different clusters.You might have to experiment a bit to get the layout you want. Let's try it!  Default (spring) layout<jupyter_code># We start with the default graph. 
nx.draw(S)<jupyter_output><empty_output><jupyter_text>Random layout<jupyter_code># Actually, it's not bad! If we drew it with a random layout, it would look worse...
nx.draw_random(S)<jupyter_output><empty_output><jupyter_text>Kamada-Kawai (force-directed) layout<jupyter_code># Maybe there are some other alternatives?
nx.draw_kamada_kawai(S)<jupyter_output><empty_output><jupyter_text>Layouts overview<jupyter_code># Here I'll just plot a short summary
fig, ax = plt.subplots(2,3, figsize=[15,10])

ax1,ax2,ax3,ax4,ax5,ax6 = ax[0][0],ax[0][1],ax[0][2],ax[1][0],ax[1][1],ax[1][2]
nx.draw(             S, ax = ax[0][0])
nx.draw_spring(      S, ax = ax[0][1])
nx.draw_kamada_kawai(S, ax = ax[0][2])
nx.draw_random(      S, ax = ax[1][0])
nx.draw_circular(    S, ax = ax[1][1])
nx.draw_spectral(    S, ax = ax[1][2])

ax[0][0].set_title("draw")
ax[0][1].set_title("draw_spring")
ax[0][2].set_title("draw_kamada_kawai")
ax[1][0].set_title("draw_random")
ax[1][1].set_title("draw_circular")
ax[1][2].set_title("draw_spectral")

plt.show()<jupyter_output><empty_output><jupyter_text>Tuning graph layouts We can also tweak individual algorithms. Before we start with that, though, we need to understand a little bit about NetworkX.- Above, we have seen NetworkX's `draw` commands that directly incorporate the desired layout, e.g. `draw_spring`.- However, we can also plot graphs with a given layout in two steps:  1. Request the layout positions.  2. Pass the layout positions to the `draw` command.  To tweak the positions, you'll need to read the documentation to see what parameters are available to you. For example, let's look into the [spring_layout](https://networkx.github.io/documentation/stable/reference/generated/networkx.drawing.layout.spring_layout.html).<jupyter_code># To tweak the layout, we'll need to work in two steps
pos = nx.spring_layout(S)        # First, we run the algorithm to get the positions
nx.draw(S, pos=pos)              # Next, we plot the graph using those positions
# For example, we can scale it down in size ...
pos = nx.spring_layout(S, scale=0.01)
nx.draw(S, pos=pos)    
# ... we can push nodes apart ...
pos = nx.spring_layout(S, k=5)
nx.draw(S, pos=pos)    
# ...or pull them together
pos = nx.spring_layout(S, k=0.01)
nx.draw(S, pos=pos)<jupyter_output><empty_output><jupyter_text>Styling the plot aesthetic For now, let's stick with the default Kamada-Kawai layout. We will set this one time and then use it subsequently below.<jupyter_code># Set the positions for the rest of this section
pos = nx.kamada_kawai_layout(S)<jupyter_output><empty_output><jupyter_text>Plotting the default Below, we can see the default plot that we've grown used to generating. But now, it takes the `pos` argument which determines the node positions.We can pass other arguments to the command to change other aspects of the graph styling:<jupyter_code># Draw the default
nx.draw(S, pos=pos)<jupyter_output><empty_output><jupyter_text>Change sizes of nodes, edges, or arrows<jupyter_code>nx.draw(S, pos=pos,
        node_size=200,       # Change size of node
        width=0.5,           # Change width of edge
        arrowsize=20         # Change size of arrow
       )<jupyter_output><empty_output><jupyter_text>Change colors of nodes or edges<jupyter_code>nx.draw(S,  pos=pos,
        node_size=200, 
        width=0.5,
        arrowsize=20,
# -----------------------
        node_color="darkseagreen",
        edge_color='seagreen'
       )<jupyter_output><empty_output><jupyter_text>Change styling of nodes or arrows<jupyter_code>nx.draw(S, pos=pos,
        node_size=200, 
        width=0.5,
        arrowsize=20,
        node_color="darkseagreen",
        edge_color='seagreen',
# -----------------------
        node_shape='p', 
        arrowstyle='->'
       )<jupyter_output><empty_output><jupyter_text>Turn plot features on and off<jupyter_code>nx.draw(S, pos=pos,
        node_size=200, 
        width=0.5,
        node_color="darkseagreen",
        edge_color='seagreen',
        node_shape='s', 
# -----------------------
        arrows=False,          # Control whether arrow is shown
        with_labels=True       # Control whether plot is labeled
       )<jupyter_output><empty_output><jupyter_text>Adding text<jupyter_code>nx.draw(S, pos=pos,
        node_size=200, 
        width=0.5,
        node_color="darkseagreen",
        edge_color='seagreen',
        node_shape='s', 
        arrows=False,
        with_labels=True,
# -----------------------
        font_size=10,
        font_color='darkslategrey',
        font_weight='bold',
        font_family='serif'
       )<jupyter_output><empty_output><jupyter_text>Drawing attention to selected featuresNow, we'll show how you can select and emphasize certain features. For example, you might want to tell a story about a given node.Essentially, the easiest way to do this is:1. Get the positions of the graph layout.2. Plot the underlying graph and nodes.3. Layer your styling on top.<jupyter_code># Let's start with a simple baseline graph
nx.draw(S, pos=pos,
        node_size=300, 
        node_color="seagreen",
        edge_color='darkseagreen',
        arrowstyle='->',
        arrowsize=15
       )<jupyter_output><empty_output><jupyter_text>Drawing selected nodes<jupyter_code># Let's plot the underlying graph
nx.draw(S, pos=pos,
        node_size=300, 
        node_color="seagreen",
        edge_color='darkseagreen',
        arrowstyle='->',
        arrowsize=15
       )

# ------------------------------------------------------
# We can choose a list of nodes we want to emphasize (in this case, one of <NAME>' pages)
selected_nodes = ['yglesias.typepad.com/matthew']

# Let's resize and recolor the node(s) of interest
nx.draw_networkx_nodes(S, pos=pos,
                      nodelist=selected_nodes,         # List of nodes to alter with new styling
                      node_color = 'palevioletred',
                      node_size=1000)<jupyter_output><empty_output><jupyter_text>Drawing selected node labels<jupyter_code># Let's plot the underlying graph
nx.draw(S, pos=pos,
        node_size=300, 
        node_color="seagreen",
        edge_color='darkseagreen',
        arrowstyle='->',
        arrowsize=15
       )

# We can choose a list of nodes we want to emphasize (in this case, one of <NAME>' pages)
selected_nodes = ['yglesias.typepad.com/matthew']

# Let's resize and recolor the node(s) of interest
nx.draw_networkx_nodes(S, pos=pos,
                      nodelist=selected_nodes,         # List of nodes to alter with new styling
                      node_color = 'palevioletred',
                      node_size=1000)
# ------------------------------------------------------

# We can create a dictionary of nodes to label, and the labels we want
selected_labels = {'yglesias.typepad.com/matthew': "matt"}

# Let's add these labels
nx.draw_networkx_labels(S, pos=pos,
                        labels=selected_labels,       # Dictionary of nodes to label
                        font_size=12)

plt.show()<jupyter_output><empty_output><jupyter_text>Drawing selected edges<jupyter_code># Let's plot the underlying graph
nx.draw(S, pos=pos,
        node_size=300, 
        node_color="seagreen",
        edge_color='darkseagreen',
        arrowstyle='->',
        arrowsize=15
       )

# We can choose a list of nodes we want to emphasize (in this case, one of <NAME>' pages)
selected_nodes = ['yglesias.typepad.com/matthew']

# Let's resize and recolor the node(s) of interest
nx.draw_networkx_nodes(S, pos=pos,
                      nodelist=selected_nodes,         # List of nodes to alter with new styling
                      node_color = 'palevioletred',
                      node_size=1000)

# We can create a dictionary of nodes to label, and the labels we want
selected_labels = {'yglesias.typepad.com/matthew': "matt"}

# Let's add these labels
nx.draw_networkx_labels(S, pos=pos,
                        labels=selected_labels,       # Dictionary of nodes to label
                        font_size=12)


# ------------------------------------------------------
# We can create a list of edges we want to plot <- in this case, all of Matt's
selected_edges = list(S.edges(['yglesias.typepad.com/matthew']))

# We can plot only these edges
nx.draw_networkx_edges(S, pos=pos,
                      edgelist=selected_edges,        # List of edges to restyle
                      edge_color = 'palevioletred',
                      width=2)

plt.show()<jupyter_output><empty_output><jupyter_text>Drawing selected edge labels<jupyter_code># Let's plot the underlying graph
nx.draw(S, pos=pos,
        node_size=300, 
        node_color="seagreen",
        edge_color='darkseagreen',
        arrowstyle='->',
        arrowsize=15
       )

# We can choose a list of nodes we want to emphasize (in this case, one of <NAME>' pages)
selected_nodes = ['yglesias.typepad.com/matthew']

# Let's resize and recolor the node(s) of interest
nx.draw_networkx_nodes(S, pos=pos,
                      nodelist=selected_nodes,         # List of nodes to alter with new styling
                      node_color = 'palevioletred',
                      node_size=1000)

# We can create a dictionary of nodes to label, and the labels we want
selected_labels = {'yglesias.typepad.com/matthew': "matt"}

# Let's add these labels
nx.draw_networkx_labels(S, pos=pos,
                        labels=selected_labels,       # Dictionary of nodes to label
                        font_size=12)


# We can create a list of edges we want to plot <- in this case, all of Matt's
selected_edges = list(S.edges(['yglesias.typepad.com/matthew']))

# We can plot only these edges
nx.draw_networkx_edges(S, pos=pos,
                      edgelist=selected_edges,        # List of edges to restyle
                      edge_color = 'palevioletred',
                      width=2)

# ------------------------------------------------------
# We can create a dictionary of edges we want to label, and their labels
selected_edge_labels = {("pandagon.net", "prospect.org/weblog"): "mutual"}

nx.draw_networkx_edge_labels(S, pos=pos,
                              edge_labels = selected_edge_labels) # Dictionary of edges to label

plt.show()<jupyter_output><empty_output><jupyter_text>Illustrating discrete attributes Of course, we might want to systematically render attributes of our graph. For example, in our dataset, there is an attribute called "value", which is set to 0 if the blog is liberal and 1 if it is conservative.Let's drop Matt's blog network and switch back to the sample we previously made, R.<jupyter_code># We'll plot R with the default Kamada Kawai layout
pos = nx.kamada_kawai_layout(R)
nx.draw(R, pos=pos)<jupyter_output><empty_output><jupyter_text>Exercise:     Let's tidy up the graph a bit. How would you...       - make the size of the nodes to be 100- make the color of the edges to be grey- make the style of arrows to be a "v" instead of a triangle? **Answer**:pos = nx.kamada_kawai_layout(R)nx.draw(R, pos=pos,         node_size=100,        edge_color='gray',        arrowstyle="->")  Coloring nodes by labelLet's highlight groups of nodes by changing the label color.<jupyter_code># We can see that each node has an attribute caleld "value" with political affiliation
# It also has information about the source of the blog
R.nodes.data()
# Let's extract political affiliation as a dictionary
political_affil = nx.get_node_attributes(R, 'value')
political_affil
# Let's make a list of node_ids and an empty list of node colors
node_ids = list(R.nodes)
node_colors = []

# Now, for each node, get its affiliation
for n in node_ids:                    
    if political_affil[n] == 0:       # 0 = liberal, let's color it blue
        node_colors.append("blue")
    elif political_affil[n] == 1:     # 1 = conservative, let's color it red
        node_colors.append("red")

# Let's inspect
print(node_ids[:10])
print(node_colors[:10])
# There is a one-line alternative: 
# node_colors = ["blue" if political_affil[n]==0 else "red" for n in node_ids]
nx.draw(R, pos=pos, 
        node_size=100,
        edge_color='gray',
        arrowstyle="->", fontsize=7, 
        # -------------------------
        nodelist = node_ids,      # Now we just supply the ids
        node_color = node_colors  # And the list of colors corresponding to them
       )<jupyter_output><empty_output><jupyter_text>Coloring edges by node label Now, let's try to replicate Adamic and Glance's classic figure on our graph subset. We want to use the same intuition: make a list of edges, and then color the edges.<jupyter_code># Let's make a list of edge_ids and an empty list of edge colors
edge_ids = list(R.edges)
edge_colors = []

for e in edge_ids:
    
    n1,n2 = e

    # Liberal links to liberal
    if political_affil[n1] == 0 and political_affil[n2] == 0:
        edge_colors.append("steelblue")
    
    # Conservative links to conservative
    elif political_affil[n1] == 1 and political_affil[n2] == 1:
        edge_colors.append("lightcoral")
   
    # Links across parties
    elif political_affil[n1] != political_affil[n2]:
        edge_colors.append("orange")
nx.draw(R, pos=pos, 
        node_size=100,
        arrowstyle="->", fontsize=7, 
        nodelist = node_ids,
        node_color = node_colors,
        # -------------------------
        edgelist = edge_ids,          # Now we just supply the edge ids
        edge_color = edge_colors      # and the list of colors corresponding to them
       )<jupyter_output><empty_output><jupyter_text>Exercise:    We almost replicated the graph, but not exactly. We use orange links to represent any cross-party ties, but in Adamic and Glance, the orange links only go from liberal to conservative. They use purple links to indicate the opposite direction. How would we modify the code to match their coloring scheme? **Answer**: edge_ids = list(R.edges)  Let's make a list of edge_ids and an empty list of edge colorsedge_colors = []for e in edge_ids:    n1,n2 = e    if political_affil[n1] == 0 and political_affil[n2] == 0:        edge_colors.append("steelblue")      Liberal links to liberal    elif political_affil[n1] == 1 and political_affil[n2] == 1:        edge_colors.append("lightcoral")     Conservative links to conservative    elif political_affil[n1] == 0 and political_affil[n2] == 1:        edge_colors.append("orange")         Liberal to conservative    elif political_affil[n1] == 1 and political_affil[n2] == 0:        edge_colors.append("purple")         Conservative to liberalnx.draw(R, pos=pos,         node_size=100,        arrowstyle="->", fontsize=7,         nodelist = node_ids,        node_color = node_colors,        edgelist = edge_ids,           Now we just supply the edge ids        edge_color = edge_colors       and the list of colors corresponding to them       )  Illustrating continuous attributes We can also use size to inform us about relevant quantities in the graph. For example, we might want to make important nodes larger, or plot the weights on edges.  Sizing nodes by value  Note: When sizing nodes by value, it sometimes helps to scale the value in order to draw a more dramatic contrast. We can do this by multiplying the raw value or exponentiating. You can experiment with the scaling constants until you get the appearance you want.  Degree<jupyter_code># Let's make a list of node_ids and an empty list of node sizes
node_ids = list(R.nodes)
node_sizes = []

# Add the scaled degrees for each node
for n in node_ids:
    node_sizes.append((2*R.degree(n))**1.75)   # Experiment with scaling here
# Plot
nx.draw(R, pos=pos, 
        node_size=node_sizes,                  # Supply a list of node sizes
        arrowstyle="->", 
        fontsize=7, 
        node_color = node_colors,
        edgelist = edge_ids,
        edge_color = edge_colors,
        nodelist = node_ids
       )<jupyter_output><empty_output><jupyter_text>Pagerank We can use the same general formula for any calculated value. For example, we can size nodes by PageRank:<jupyter_code># Get a dictionary of page ranks
pr = nx.pagerank(R)

# Let's make a list of node_ids and an empty list of node sizes
node_ids = list(R.nodes)
node_sizes = []

# Calculate the node sizes
for n in node_ids:
    node_sizes.append(4000*pr[n])              # Pagerank values are small (they sum to 1), so we scale aggressively
# Plot
nx.draw(R, pos=pos, 
        node_size=node_sizes,                  # Supply a list of node sizes
        arrowstyle="->", 
        fontsize=7, 
        node_color = node_colors,
        edgelist = edge_ids,
        edge_color = edge_colors,
        nodelist = node_ids
       )<jupyter_output><empty_output><jupyter_text>Coloring nodes by value  We might want to show multiple features. In that case, we can also use color as an informative dimension. Let's keep the node ids and node sizes from above, but add a color dimension that captures in-degree:<jupyter_code># Initialize a list to store the variable associated with the color
node_color_values = []

# Add the (numeric) values to the list
for n in node_ids:
    node_color_values.append(R.in_degree(n))
nx.draw(R, pos=pos, 
        node_size=node_sizes,
        arrowstyle="->", 
        fontsize=7, 
        node_color = node_color_values,   # Supply the (numeric) data for the color variable
        cmap = plt.get_cmap('viridis'),   # Map the numeric data to a color scheme
        edgelist = edge_ids,
        edge_color = 'lightgrey',
        nodelist = node_ids,
        arrows=False
       )<jupyter_output><empty_output><jupyter_text>Note:  We can see that in-degree does not perfectly correspond with PageRank. The nodes with the highest in degree (the lightest) are not always the ones with the highest PageRank (the biggest). PageRank depends on the weight of inbound links (and not just the number of them).  Sizing edges by value Finally, we might want to use weight values to change the edges sizing. Here, let's weight a tie by how many neighbors its endpoints have in common. By now, you can probably guess how to do this:<jupyter_code># Initialize a list of edge weights
edge_weights = []

# For each edge, append a weight
for n1,n2 in edge_ids:
    neighbors = nx.common_neighbors(nx.Graph(R),n1,n2)   # Get common neighbors of the end points
    n_neighbors = len(list(neighbors))                   # Count how many of these neighbors they have in common
    edge_weights.append(n_neighbors)                     # Add to the list (+1, in case no common neighbors)

# Plot
nx.draw(R, pos=pos, 
        node_size=node_sizes,
        arrowstyle="->", 
        fontsize=7, 
        node_color = node_colors,
        edgelist = edge_ids,
        edge_color = edge_colors,
        nodelist = node_ids,
        width=edge_weights,                              # Supply the edge weights
        edge_labels = [str(i) for i in edge_weights]
       )<jupyter_output><empty_output><jupyter_text>Exercise: - Why are some edges missing?- How would you fix the code so that even edges with no common neighbors are drawn? **Answer**: edge_weights = []for n1,n2 in edge_ids:    neighbors = nx.common_neighbors(nx.Graph(R),n1,n2)    Get common neighbors of the end points    n_neighbors = len(list(neighbors))                    Count how many of these neighbors they have in common    edge_weights.append(1+n_neighbors)                      Add to the list (+1, in case no common neighbors)nx.draw(R, pos=pos,         node_size=node_sizes,        arrowstyle="->",         fontsize=7,         node_color = node_colors,        edgelist = edge_ids,        edge_color = edge_colors,        nodelist = node_ids,        width=edge_weights,                               Supply the edge weights        edge_labels = [str(i) for i in edge_weights]       )  Exercise: How would you change the code so that random weights are drawn?_Note_ You can draw random numbers between zero and one using `random.random()`. **Answer**: edge_weights = []for e in edge_ids:    edge_weights.append(2*random.random())                  Add a random weightnx.draw(R, pos=pos,         node_size=node_sizes,        arrowstyle="->",         fontsize=7,         node_color = node_colors,        edgelist = edge_ids,        edge_color = edge_colors,        nodelist = node_ids,        width=edge_weights,                               Supply the edge weights        edge_labels = [str(i) for i in edge_weights]       )  Degree histogram<jupyter_code># Get a list of all degrees
degrees = list(dict(P.degree()).values())

# Plot a histogram of the degree distribution
plt.hist(degrees, bins=50, range = [0,200])
plt.show()<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.10518925328423877, 'fraction_numerical': 0.013386588448216682, 'mean_word_length': 3.366807466957351, 'pattern_counts': {'":': 0, '<': 142, '<?xml version=': 0, '>': 157, 'https://': 3, 'lorem ipsum': 0, 'www.': 3, 'xml': 0}, 'pii_count': 2, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 1, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '33', 'dir': 'jupyter-structured-clean-dedup', 'id': '651466', 'license': 'BSD-3-Clause', 'n_tokens_mistral': 9896, 'n_tokens_neox': 9407, 'n_words': 3323, 'path': 'tutorial_part_2_visualize.ipynb', 'repo_name': 'khof312/networks_tutorial', 'size': '32104'}