<jupyter_start><jupyter_text>This notebook convert a mailing list (or a set of mailing lists) into a network of interactionWhat it does:-it creates a network of interaction between senders and receivers of emails, on one or more mailing lists-it generates a .gexf file that can be imported in Gephi for visualization and analysisParameters to set options:-it can look in one or more mailing lists, according to how many urls are set in the ‘urls’ variable; networks are aggregated across mailing lists-it can filter the network by date; set the variable 'date_from' and 'date_to' with a date frame consistent with the data<jupyter_code>%matplotlib inline
from bigbang.archive import Archive
from bigbang.archive import load as load_archive
import bigbang.parse as parse
import bigbang.analysis.graph as graph
import bigbang.ingress.mailman as mailman
import bigbang.analysis.process as process
import networkx as nx
import matplotlib.pyplot as plt
import pandas as pd
from pprint import pprint as pp
import pytz
import os 
#Insert a list of archive names
archives_names = ["6lo"]

cwd = os.getcwd()  

archives_paths = list()
for archive_name in archives_names:
    archives_paths.append('../../archives/'+archive_name+'.csv')
    

archives_list = [load_archive(archive_path).data for archive_path in archives_paths]
    
archives = Archive(pd.concat(archives_list))

archives_data = archives.data<jupyter_output><empty_output><jupyter_text>Set a valid date frame for building the network.<jupyter_code>#The oldest date and more recent date for the whole mailing lists are displayed, so you WON't set an invalid time frame 
print(archives_data['Date'].min())
print(archives_data['Date'].max())
#set the date frame
date_from = pd.datetime(2000,11,1,tzinfo=pytz.utc)
date_to = pd.datetime(2111,12,1,tzinfo=pytz.utc)<jupyter_output>/home/sb/anaconda3/envs/bigbang/lib/python3.7/site-packages/ipykernel_launcher.py:2: FutureWarning: The pandas.datetime class is deprecated and will be removed from pandas in a future version. Import from datetime module instead.
  
/home/sb/anaconda3/envs/bigbang/lib/python3.7/site-packages/ipykernel_launcher.py:3: FutureWarning: The pandas.datetime class is deprecated and will be removed from pandas in a future version. Import from datetime module instead.
  This is separate from the ipykernel package so we can avoid doing imports until<jupyter_text>Filter data according to date frame and export to .gexf file<jupyter_code>def filter_by_date(df,d_from,d_to):
    return df[(df['Date'] > d_from) & (df['Date'] < d_to)]
#create filtered network
archives_data_filtered = filter_by_date(archives_data, date_from, date_to)
network = graph.messages_to_interaction_graph(archives_data_filtered)
#export the network in a format that you can open in Gephi. 

#insert a file name
file_name = 'architecture_discuss_for_gephi.gexf'

network = nx.write_gexf(network, cwd+file_name)<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.0755873340143003, 'fraction_numerical': 0.007831120190670752, 'mean_word_length': 5.7540229885057474, 'pattern_counts': {'":': 0, '<': 13, '<?xml version=': 0, '>': 13, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '3', 'dir': 'jupyter-structured-clean-dedup', 'id': '207847', 'license': 'MIT', 'n_tokens_mistral': 928, 'n_tokens_neox': 892, 'n_words': 316, 'path': 'examples/network_analysis/InteractionNetwork.ipynb', 'repo_name': 'nllz/bigbang', 'size': '2937'}