<jupyter_start><jupyter_text>Creating a feature matrix from a networkx graphIn this notebook we will look at a few ways to quickly create a feature matrix from a networkx graph.[Networkx basic tutorial](http://pynetwork.readthedocs.io/en/latest/networkx_basics.html)<jupyter_code># %load ../_data/standard_import.txt

%matplotlib inline

import numpy as np
import pandas as pd
import matplotlib as mpl
import matplotlib.pyplot as plt
import seaborn as sns
import networkx as nx
import pickle

plt.style.use('seaborn-white')<jupyter_output><empty_output><jupyter_text>Data  Import edges and nodes into networkx 2.X Networkx 1.X files cannot be read by networkx 2.X```pythonfile = './major_us_cities.dms'G = nx.read_gpickle(file)print(nx.info(G))list(G.edges(data=True))[:10]list(G.nodes(data=True))[:10]edges = pd.DataFrame(list(G.edges(data=True)))nodes = pd.DataFrame(list(G.nodes(data=True)))nodes.sample()edges.sample() Use a lambda to pull out the attributes from the attributes dictionary in column 1nodes['location'] = nodes.loc[:, 1].map(lambda x: x['location'])nodes['population'] = nodes.loc[:, 1].map(lambda x: x['population'])del nodes[1]nodes.sample() Use a lambda to pull out the attributes from the attributes dictionary in column 1edges['weight'] = edges.loc[:, 2].map(lambda x: x['weight'])del edges[2]edges.sample()edges.to_csv('major_us_cities_edges.csv')nodes.to_csv('major_us_cities_email_nodes.csv')```<jupyter_code>!find .. | grep -i major_us_cities
nodes = pd.read_csv('../_data/major_us_cities_nodes.csv', index_col=0, 
                    names=['node', 'location', 'population'])
edges = pd.read_csv('../_data/major_us_cities_edges.csv', index_col=0,
                   names=['n1', 'n2', 'weight'])
nodes.sample(3)
edges.sample(3)
# First create graph from edges, then add nodes
G = nx.from_pandas_dataframe(edges, 'n1', 'n2', edge_attr='weight')
print(nx.info(G))
list(G.edges(data=True))[:5]
list(G.nodes(data=True))[:5]

# remove header
try:
    G.remove_node('0')
    G.remove_node('1')
    G.remove_edge('0', '1')
except:
    None
_ = [G.add_node(nodes.loc[n, 'node'], 
                location=nodes.loc[n, 'location'], 
                population=nodes.loc[n, 'population']) for n in nodes.index]
list(G.nodes(data=True))[:10]<jupyter_output><empty_output><jupyter_text>Node based features<jupyter_code># Initialize the dataframe, using the nodes as the index
df = pd.DataFrame(index=G.nodes())<jupyter_output><empty_output><jupyter_text>Extracting attributesUsing `nx.get_node_attributes` it's easy to extract the node attributes in the graph into DataFrame columns.<jupyter_code>df['location'] = pd.Series(nx.get_node_attributes(G, 'location'))
df['population'] = pd.Series(nx.get_node_attributes(G, 'population'))

df.head()<jupyter_output><empty_output><jupyter_text>Creating node based featuresMost of the networkx functions related to nodes return a dictionary, which can also easily be added to our dataframe.<jupyter_code>list(nx.degree(G))[:10]
pd.Series([d[1] for d in G.degree()]).head()
df['clustering'] = pd.Series(nx.clustering(G))
df['degree'] = pd.Series([d[1] for d in G.degree()]).values
df.head()<jupyter_output><empty_output><jupyter_text>Edge based features<jupyter_code>list(G.edges(data=True))[:5]
# Initialize the dataframe, using the edges as the index
df = pd.DataFrame(index=G.edges(data=True))<jupyter_output><empty_output><jupyter_text>Extracting attributesUsing `nx.get_edge_attributes`, it's easy to extract the edge attributes in the graph into DataFrame columns.<jupyter_code>weight = list(nx.get_edge_attributes(G, 'weight').values())
weight[:5]
# Remove weight from index
df.index = [(edge[0], edge[1]) for edge in df.index]
df['weight'] = weight
df.sample(3)<jupyter_output><empty_output><jupyter_text>Creating edge based featuresMany of the networkx functions related to edges return a nested data structures. We can extract the relevant data using list comprehension.<jupyter_code>df['preferential attachment'] = [i[2] for i in nx.preferential_attachment(G, df.index)]
df.sample(3)<jupyter_output><empty_output><jupyter_text>In the case where the function expects two nodes to be passed in, we can map the index to a lamda function.<jupyter_code>df['Common Neighbors'] = df.index.map(lambda city: len(list(nx.common_neighbors(G, city[0], city[1]))))
df.sample(3)<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.1409257561869844, 'fraction_numerical': 0.01008249312557287, 'mean_word_length': 6.360876897133221, 'pattern_counts': {'":': 0, '<': 37, '<?xml version=': 0, '>': 37, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '9', 'dir': 'jupyter-structured-clean-dedup', 'id': '290420', 'license': 'MIT', 'n_tokens_mistral': 1509, 'n_tokens_neox': 1462, 'n_words': 353, 'path': 'networkx_graph_features.ipynb', 'repo_name': 'aixpact/networks', 'size': '4364'}