<jupyter_start><jupyter_text>652 Week 2 Tips and Tricks  <NAME> | <NAME>   Before starting your assignments...  ... make a clean copy of your notebook in case you have to start over.In Coursera:`File > Make a copy...`  Don't overload the autograder  Some cells take quite a while to compute. Work on one problem at a time, comment out other cells until you're read to submit.  Make sure your notebook runs, however.```Kernel -> Restart & Run All```  Load some libraries to get started...<jupyter_code>import networkx as nx
import matplotlib.pyplot as plt
import json
%matplotlib inline<jupyter_output><empty_output><jupyter_text>Make a graph to use for examples (from 652 W1 tutorial)<jupyter_code>G = nx.Graph()
G.add_node(1) # add a single node
G.add_nodes_from([2, 3, 4, 5, 6, 7, 8, 9, 0]) # add a list of nodes
G.add_edge(1, 2) # add a single edge
G.add_edges_from([(1, 2), (1, 3), (2, 4), (2, 5), (3,5), 
                  (3, 7), (4,8), (5,9),(5,0),(6,0),(7,8),(8,9)]) 
# G.remove_edge(1, 2)<jupyter_output><empty_output><jupyter_text>___ Save computationally intensive results **Reminder from W1 Tips & Tricks** Some computations, such as PageRank, can take quite a while to run. Once you run them once, you can save the files to make your notebook run more quickly. `json` files are a good choice for saving dictionaries.Note that this is not a requirement, but just a suggestion to make your notebook run faster.<jupyter_code># clear out file from previous demonstration
!rm my_pagerank.json
saved_pr = None
pr_file = 'my_pagerank.json'

# look for a pagerank file
try:
    with open(pr_file, 'r') as f:
        load_pr  = json.load(f)
        # for some reason keys come back as strings
        # make sure to convert to integers
        saved_pr = {int(n):pr for n,pr in load_pr.items()}
        
# if there was no file, recalulate the pagerank        
except:
    
    print("can't find file: ",pr_file)
    print()
    print("recalculating pagerank")
    print()
    my_pr = nx.pagerank(G)
    
    with open(pr_file, 'w') as f:
        json.dump(my_pr, f)

# saved_pr = my_pr # don't set this yet, demonstration below.
my_pr
# is there a saved pagerank file?
print(saved_pr)
my_pr = None # reset from above

# try to open an existing pagerank file
try:
    with open(pr_file, 'r') as f:
        load_pr  = json.load(f)
        # for some reason keys come back as strings
        # make sure to convert to integers
        saved_pr = {int(n):pr for n,pr in load_pr.items()}

# if it is not there, do something (print an error)
except:
    print("can't find file: ", pr_file)
    
saved_pr
type(saved_pr)
# is my_pr still there
print(my_pr)
my_pr = saved_pr
my_pr
type(my_pr)<jupyter_output><empty_output><jupyter_text>___ Use NetworKit to calculate PageRank [NetworKit](https://networkit.github.io) is a python wrapper around algorithms written in C++, so it is **a lot** faster than NetworkX.```! conda install -y -c conda-forge networkit  pip does not install correctly``` **Disclaimer:** there are small differences in the PageRank values due to differences in float precision. I have tested networkit PageRanks with Questions 6, 9 and 12 of Assignment 2, and as far as I can tell, they should be acceptable to the autograder.    NetworKit FAQ **Q:** Why do we use NetworkX instead of NetworKit if the latter is faster?**A1:** NetworkX is a more established library with more functions/algorithms available, but it's really good to be aware of more efficient options such as NetworKit for certain tasks.**A2:** The documentation for NetworkX is easier to follow and more extensive. There are a lot of answered StackOverflow questions, and a lot of outside resources that use NetworkX. ___<jupyter_code>try:
    import networkit as nk
except:
    ! conda install -y -c conda-forge networkit # pip does not install correctly<jupyter_output><empty_output><jupyter_text>NetworKit Nodes NetworKit  differs from NetworkX in how it assigns node labels.See: https://stackoverflow.com/questions/60285134/retrieving-original-node-names-in-networkit<jupyter_code># mapping node labels from networkX graph to networkit graph
nxG = G                     # networkX Graph
nkG = nk.nxadapter.nx2nk(nxG) # converted to networkit Graph

# Get mapping from node ids in nxG to node ids in nkG
idmap_x_to_k = dict((id, u) for (id, u) in zip(G.nodes(), range(G.number_of_nodes())))
# mapping is networkX node: networkit node
idmap_x_to_k
# mapping node ids from networkit graph back to networkX 
idmap_k_to_x = {v:k for k,v in idmap_x_to_k.items()}
idmap_k_to_x<jupyter_output><empty_output><jupyter_text>A possibly useful function to compute PageRanks with NetworKit  This function 1. takes a NetworkX graph `G` and an `alpha` factor (default is `alpha=0.85`)2. converts the NetworkX graph `G` to a NetworKit graph `nkG`, 3. maps the NetworkX node names to NetworKit node numbers,4. runs `nk.centrality.PageRank` on the converted graph (so fast!) with the provided `alpha`5. uses the `.ranking()` method to return a list of node, pagerank tuples, ranked by PageRank6. converts the NetworKit PageRank rank list to a NetworkX-style PageRank dictionary7. maps the NetworkKit node labels back to NetworkX node labels.I hope you find the function useful!<jupyter_code># import networkit as nk # uncomment this line if you haven't `imported networkit as nk`

def nk_pagerank(G,alpha=0.85):
    
    # convert networkX graph to networkit
    nkG = nk.nxadapter.nx2nk(G)
    
    # Get mapping from node ids in nxG to node ids in nkG
    idmap_x_to_k = dict((id, u) for (id, u) in zip(G.nodes(), range(G.number_of_nodes())))
    idmap_k_to_x = {v:k for k,v in idmap_x_to_k.items()}
    
    # initialize networkit pagerank
    nk_pr = nk.centrality.PageRank(nkG, damp=alpha)
    
    # run networkit pagerank
    nk_pr.run()
    
    # nk_pr.ranking() returns a list of tuples instead of a dictionary
    # convert list of tuples and node labels into a dictionary for networkX
    nx_pr = {idmap_k_to_x[n]:pr for n,pr in nk_pr.ranking()}
    
    return nx_pr
# check node mappings from networkit to network X
pr_nk = nk_pagerank(G, alpha = 0.85)
{i:pr_nk[i] for i in range(len(pr_nk))}
pr_nx = nx.pagerank(G)
{i:pr_nx[i] for i in range(len(pr_nx))}
sorted([round(v,4) for k,v in pr_nx.items()], reverse=True)
sorted([round(v,4) for k,v in pr_nk.items()], reverse=True)<jupyter_output><empty_output><jupyter_text>Save a lot of time by using networkit AND saving pagerank files<jupyter_code>pr_file = 'my_pagerank.json'

# look for a pagerank file
try:
    with open(pr_file, 'r') as f:
        nx_pr_s = json.load(f)
        # for some reason keys come back as strings
        # make sure to convert to integers
        nk_pr   = {int(n):pr for n,pr in nx_pr_s.items()}
        
# if there was no file, recalulate the pagerank        
except:
    
    print("can't find file: ", pr_file)
    print()
    print("recalculating pagerank")
    print()

    nk_pr = nk_pagerank(G)
    
    with open(pr_file, 'w') as f:
        json.dump(nx_pr, f)

nk_pr
# checking Graph conversion (node labels)
# do the pageranks go with the correct node labels?
{k:nk_pr[k] for k in range(len(nk_pr))}
# checking Graph conversion (node labels)
# do the pageranks go with the correct node labels?
nx_pr = nx.pagerank(G)
{k:nx_pr[k] for k in range(len(nx_pr))}
# are the pagerank sort orders the same?
value_sort_nx = sorted([(v,k) for k,v in nx_pr.items()])
{k:v for v,k in value_sort_nx}
# are the pagerank sort orders the same?
value_sort_nk = sorted([(v,k) for k,v in nk_pr.items()])
{k:v for v,k in value_sort_nk}
# are the nodes in the same order?
# zip()
nx_ascending = [node for value,node in value_sort_nx]
nk_ascending = [node for value,node in value_sort_nk]
list(zip(nx_ascending, nk_ascending))<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.0968609865470852, 'fraction_numerical': 0.010506085842408712, 'mean_word_length': 3.981493299298022, 'pattern_counts': {'":': 0, '<': 31, '<?xml version=': 0, '>': 33, 'https://': 2, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 1, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '7', 'dir': 'jupyter-structured-clean-dedup', 'id': '206391', 'license': 'MIT', 'n_tokens_mistral': 2735, 'n_tokens_neox': 2628, 'n_words': 984, 'path': 'notebooks/week_two.ipynb', 'repo_name': 'benmerrilll/Network-Analysis-Tips-and-Tricks', 'size': '7815'}