<jupyter_start><jupyter_text>Exploring node2vecObjectives: - Research and understand node2vec concept and module's API - Explore node2vec capabilities, applications and limitations - Implement a link prediction model to predict social links of Twitch streamers  Node2VecReferences:- [node2vec: Scalable Feature Learning for Networks](https://arxiv.org/pdf/1607.00653.pdf)  What is node2vec ?Is an algorithmic framework for learning continuous feature representations for nodes in networks.![](https://cdn-images-1.medium.com/max/932/1*dIHSCxk8ZAR7vI7Fm3iw9A.png) How does it work (high-level) ?ndoe2vec simulates random walks to learn the representations for all the nodes in the network, while sampling nodes at every walk stop based on 2nd order Markov chain transition probabilities, using Alias Sampling for efficient sampling and using SGD (Stochastic Gradient Descent) as objective function for optimization.  DataSource: [Twitch Social Network](https://github.com/benedekrozemberczki/datasetstwitch-social-networks)**Description**These datasets used for node classification and transfer learning are Twitch user-user networks of gamers who stream in a certain language. Nodes are the users themselves and the links are mutual friendships between them. Vertex features are extracted based on the games played and liked, location and streaming habits. Datasets share the same set of node features, this makes transfer learning across networks possible. These social networks were collected in May 2018. The supervised task related to these networks is binary node classification - one has to predict whether a streamer uses explicit language.**Properties**- **Directed:** No.- **Node features:** Yes.- **Edge features:** No.- **Node labels:** Yes. Binary-labeled.- **Temporal:** No.**ENGB Data**- **Nodes:** 7,126- **Edges:** 35,324 - **Density:** 0.002- **Transitvity:** 0.042 **Imports**<jupyter_code>import pandas as pd
import numpy as np
import requests,json
import matplotlib.pyplot as plt
import seaborn as sns
import networkx as nx
import plotly.graph_objects as go
from tqdm.notebook import tqdm
tqdm.pandas()
import itertools
plt.style.use('seaborn')<jupyter_output><empty_output><jupyter_text>**Load Data**<jupyter_code>%%time
edges = pd.read_csv('https://raw.githubusercontent.com/benedekrozemberczki/datasets/master/twitch/ENGB/ENGB_edges.csv')
features = requests.get('https://raw.githubusercontent.com/benedekrozemberczki/datasets/master/twitch/ENGB/ENGB_features.json').json()
target = pd.read_csv('https://raw.githubusercontent.com/benedekrozemberczki/datasets/master/twitch/ENGB/ENGB_target.csv')<jupyter_output>CPU times: user 71.2 ms, sys: 19.8 ms, total: 91.1 ms
Wall time: 888 ms<jupyter_text>**Edges**<jupyter_code>print(f'Number of edges: {len(edges)}')
edges[:3]<jupyter_output>Number of edges: 35324<jupyter_text>**Target**<jupyter_code>print(f'Number of targets: {len(target)}')
target[:3]<jupyter_output>Number of targets: 7126<jupyter_text>**Features**<jupyter_code>for i,(k,v) in enumerate(features.items()):
    print(f'k:{k}\nv:{v}\n')
    if i == 3:
        break<jupyter_output>k:3032
v:[2605, 1191, 357, 2120, 861, 231, 3164, 920, 1907, 1612, 2327, 2549, 2524, 2912, 139, 83, 276, 2645, 2424, 2501, 2737, 1525, 751]

k:4032
v:[1521, 1191, 2334, 846, 3103, 3045, 920, 224, 810, 1369, 569, 2384, 2362, 1762, 3106, 751]

k:3945
v:[1550, 3152, 146, 2203, 846, 673, 2063, 920, 224, 3097, 329, 569, 2384, 2645, 608, 2936]

k:4677
v:[308, 3152, 357, 692, 653, 181, 861, 920, 1907, 2185, 1043, 139, 569]<jupyter_text>**Since we work with large size networks is recommended to use Plotly for plotting network graphs**<jupyter_code>def plot_network(G,pos_func=nx.kamada_kawai_layout):
    nodes_data = G.nodes(data=True)
    pos=pos_func(G)
    edge_trace = [go.Scatter(
    #   x0                   y0                   None
    x=[ pos.get(edge[0])[0], pos.get(edge[1])[0], None],
    #   x1                   y1                   None
    y=[ pos.get(edge[0])[1], pos.get(edge[1])[1], None],
    line=dict(width=0.2, color='gray'),
    hoverinfo='none',
    mode='lines') for edge in G.edges()]

    # create the nodes in plotly way
    node_trace = [go.Scatter(
        x=[pos.get(node)[0]], 
        y=[pos.get(node)[1]],
        mode='markers',
        text=f'{node} | Connections: {len(adjacencies)}',
        hoverinfo='text',
        
        textposition="middle center",
        textfont=dict(family='Arial',
                    size=10,
                    color='#000'
                    )) for node, adjacencies in G.adjacency()]
    fig = go.Figure(data=edge_trace + node_trace,
             layout=go.Layout(
                showlegend=False,
                hovermode='closest',
                margin=dict(b=0,l=5,r=5,t=0),
                xaxis=dict(showgrid=False, zeroline=False, showticklabels=False),
                yaxis=dict(showgrid=False, zeroline=False, showticklabels=False))
                )
    return fig<jupyter_output><empty_output><jupyter_text>**Edges Data**<jupyter_code>G = nx.from_pandas_edgelist(edges[:300],source='from',target='to')
plot_network(G)<jupyter_output><empty_output><jupyter_text>Applications & Capabilities<jupyter_code>try:
    from node2vec import Node2Vec
except:
    !pip install node2vec
    from node2vec import Node2Vec<jupyter_output><empty_output><jupyter_text>Similarity<jupyter_code>G = nx.from_pandas_edgelist(edges,source='from',target='to')
sim = Node2Vec(G, dimensions=10, walk_length=5, num_walks=10, p=1, q=1,workers=3)
sim_model = sim.fit(window=3, min_count=1)<jupyter_output><empty_output><jupyter_text>**Find Similar users by vector**<jupyter_code>similar_users = sim_model.wv.similar_by_vector(sim_model.wv.get_vector('2299'))
similar_users
def get_edge_list_by(id,graph):
    return [x for x in list(graph.edges()) if x[0] == int(id)]

def get_sub_graph_of_similarity(id,model,graph):
    sim_users = model.wv.similar_by_vector(model.wv.get_vector(id))
    sim_users = [x[0] for x  in sim_users]
    all_edges = [get_edge_list_by(x,graph) for x in sim_users]
    all_edges = list(itertools.chain(*all_edges))
    return graph.edge_subgraph(all_edges)
sim_sub_G = get_sub_graph_of_similarity('2299',sim_model,G);
plt.figure(figsize=(7,7))
nx.draw_networkx(sim_sub_G,pos=nx.spring_layout(sim_sub_G),font_size=11)<jupyter_output>/usr/local/lib/python3.6/dist-packages/gensim/matutils.py:737: FutureWarning:

Conversion of the second argument of issubdtype from `int` to `np.signedinteger` is deprecated. In future, it will be treated as `np.int64 == np.dtype(int).type`.<jupyter_text>Link Prediction **Let's generate few fake connections**<jupyter_code>min_node = edges.values.ravel().min()
max_node = edges.values.ravel().max()
fake_nodes = list()
fake_nodes_limit = 5000
counter = 0
pbar = tqdm(total=fake_nodes_limit)
while counter <= fake_nodes_limit:
    frm = np.random.randint(min_node,max_node)
    to = np.random.randint(min_node,max_node)
    if len(edges[(edges['from'] == frm) & (edges['to'] == to)]) == 0:
        pbar.update(counter)
        fake_nodes.append({
            'from': frm,
            'to': to
        })
        counter+=1
fake_nodes = pd.DataFrame(fake_nodes)
fake_nodes['link'] = 0
data = edges.copy()
data['link'] = 1
data = pd.concat([data,fake_nodes],ignore_index=False)
data.isnull().sum()
G = nx.from_pandas_edgelist(data,'from','to')
node2vec = Node2Vec(G, dimensions=128, walk_length=16, num_walks=50, workers=3)
# train node2vec model
link_model = node2vec.fit(window=7, min_count=1)
from sklearn.model_selection import train_test_split
from sklearn import metrics<jupyter_output><empty_output><jupyter_text>**Extract the vectors nodes**<jupyter_code>data_vec = data.copy()
data_vec['from_vec'] = data_vec['from'].apply(lambda x: link_model.wv.get_vector(str(x)))
data_vec['to_vec'] = data_vec['to'].apply(lambda x: link_model.wv.get_vector(str(x)))
data_vec['sum_vec'] = data_vec[['from_vec','to_vec']].sum(axis=1)
data_vec<jupyter_output><empty_output><jupyter_text>**Clustering**<jupyter_code>from sklearn.cluster import KMeans
clustering_data = pd.DataFrame({
    'node': list(G.nodes)
})
clustering_data['vec'] = clustering_data['node'].apply(lambda x: link_model.wv.get_vector(str(x)))
clustering_data[:3]<jupyter_output><empty_output><jupyter_text>**PCA**<jupyter_code>from sklearn.decomposition import PCA
pca = PCA(n_components=2)
pca_data = pca.fit_transform(clustering_data['vec'].values.tolist())
pca_data = pd.DataFrame({
    'comp_1':pca_data[:,0],
    'comp_2': pca_data[:,1]
})
clustering_data = pd.concat([clustering_data,pca_data],axis=1)
pca_data[:3]<jupyter_output><empty_output><jupyter_text>**Inertia**<jupyter_code>inertia = list()
for x in tqdm(range(1,31)):
    kmeans = KMeans(n_clusters=x).fit(clustering_data['vec'].values.tolist())
    inertia.append({
        'k': x,
        'i': kmeans.inertia_
    })
inertia = pd.DataFrame(inertia).set_index('k')
inertia.plot(figsize=(20,10));
k = 5
kmeans = KMeans(n_clusters=k).fit(clustering_data['vec'].values.tolist())
clustering_data['labels'] = kmeans.labels_
fig,ax = plt.subplots(1,1,figsize=(20,10))
color=iter(plt.cm.rainbow(np.linspace(0,1,k)))
for l in list(range(k)):
    c=next(color)
    d = clustering_data[clustering_data.labels == l]
    ax.scatter(d['comp_1'],d['comp_2'],label=l,color=c,alpha=.7)
ax.legend();<jupyter_output><empty_output><jupyter_text>**Split the data into Train / Test**<jupyter_code>train, test = train_test_split(data_vec,test_size=0.33,random_state=42,stratify=data_vec['link'])<jupyter_output><empty_output><jupyter_text>**Now let's train few models** **Logistic Regression**<jupyter_code>from sklearn.linear_model import LogisticRegression
lr = LogisticRegression(class_weight="balanced")
lr.fit(train['sum_vec'].values.tolist(), train['link'])
results = test.copy()
results['proba'] = lr.predict_proba(test['sum_vec'].values.tolist())[:,1]
metrics.roc_auc_score(results['link'],results['proba'])<jupyter_output><empty_output><jupyter_text>**XGBoost on Summed vectors**<jupyter_code>import xgboost as xgb<jupyter_output><empty_output><jupyter_text>**Balancing the weights**<jupyter_code>scale_pos_weight = np.sqrt(len(train[train.link == 0]) / len(train[train.link == 1]))<jupyter_output><empty_output><jupyter_text>**Train model**<jupyter_code>xgc=xgb.XGBClassifier(scale_pos_weight=scale_pos_weight)
xgc.fit(pd.DataFrame(train['sum_vec'].values.tolist()),train['link'])<jupyter_output><empty_output><jupyter_text>**Results**<jupyter_code>results = test.copy()
results['link_pred'] = xgc.predict(pd.DataFrame(test.sum_vec.values.tolist()))<jupyter_output><empty_output><jupyter_text>**Classification Report**<jupyter_code>print(metrics.classification_report(results.link,results.link_pred))
metrics.roc_auc_score(results['link'],results['link_pred'])<jupyter_output><empty_output><jupyter_text>**Confusion Matrix**<jupyter_code>sns.heatmap(metrics.confusion_matrix(results.link,results.link_pred),annot=True,fmt='d');<jupyter_output><empty_output><jupyter_text>**PyTorch**<jupyter_code># Torch
import torch
import torch.nn as nn
import torch.nn.functional as F
torch.backends.cudnn.deterministic = True<jupyter_output><empty_output><jupyter_text>**Convert values from pandas -> numpy -> torch tensors**<jupyter_code>x_from_train = torch.from_numpy(np.array(train['from_vec'].values.tolist())).type(torch.FloatTensor)
x_to_train = torch.from_numpy(np.array(train['to_vec'].values.tolist())).type(torch.FloatTensor)
y_train = torch.from_numpy(train['link'].values).type(torch.LongTensor)

x_from_test = torch.from_numpy(np.array(test['from_vec'].values.tolist())).type(torch.FloatTensor)
x_to_test = torch.from_numpy(np.array(test['to_vec'].values.tolist())).type(torch.FloatTensor)
y_test = torch.from_numpy(test['link'].values).type(torch.LongTensor)<jupyter_output><empty_output><jupyter_text>**Add the train and test tensors into DataLoaders**<jupyter_code>b_size = 256
train_dl = torch.utils.data.DataLoader(torch.utils.data.TensorDataset(x_from_train,x_to_train,y_train), batch_size=b_size, shuffle=True)
test_dl = torch.utils.data.DataLoader(torch.utils.data.TensorDataset(x_from_test,x_to_test,y_test), batch_size=b_size, shuffle=True)<jupyter_output><empty_output><jupyter_text>**Create and Train the Model**<jupyter_code>class Net(nn.Module):

    def __init__(self):
        super(Net,self).__init__()

        self.from_lin = nn.Linear(in_features=128,out_features=256)
        self.to_lin = nn.Linear(in_features=128,out_features=256)
        self.lin1 = nn.Linear(in_features=512,out_features=256)
        self.lin2 = nn.Linear(in_features=256,out_features=128)
        self.lin3 = nn.Linear(in_features=128,out_features=2)

    # here we take the input data and pass it through the chain of layers
    def forward(self,frm,to):
        frm = self.from_lin(frm)
        to = self.to_lin(to)
        x = torch.cat((frm,to),dim=1)
        # print(x.size())
        x = self.lin1(x)
        x = self.lin2(x)
        x = self.lin3(x)
        return x

# instance our model
model = Net()
# set the number of epochs
epochs = 100
# criterion aka loss function
criterion = nn.CrossEntropyLoss()

# optimizer
optimizer = torch.optim.Adam(model.parameters(), lr=0.003)

# create 3 lists to store the losses and accuracy at each epoch
train_losses, test_losses, accuracy = [0]*epochs, [0]*epochs,[0]*epochs


for e in tqdm(range(epochs)):

    for frm,to,label in train_dl:

        optimizer.zero_grad()
        frm = frm.float()
        to = to.float()
        # Comput train loss
        y_pred = model(frm,to)
        loss = criterion(y_pred, label)
        
        loss.backward()

        optimizer.step()

        # store train loss
        train_losses[e] = loss.item()
    
    for frm,to,label in test_dl:    
        # Compute the test stats
        with torch.no_grad():
            # Turn on all the nodes
            model.eval()
            frm = frm.float()
            to = to.float()
            # Comput test loss
            ps = model(frm,to)
            loss = criterion(ps, label)

            # store test loss
            test_losses[e] = loss.item()
            
            # Compute accuracy
            top_p, top_class = ps.topk(1, dim=1)
        
            equals = (top_class == label.view(*top_class.shape))
            
            # store accuracy
            accuracy[e] = torch.mean(equals.type(torch.FloatTensor))

# Print the final information
print(f'Accuracy  : {100*accuracy[-1].item():0.2f}%')
print(f'Train loss: {train_losses[-1]}')
print(f'Test loss : {test_losses[-1]}')
    
# Plot the results
fig,ax = plt.subplots(1,2,figsize=(20,5))
ax[0].set_ylabel('Accuracy')
ax[0].set_xlabel('Epochs')
ax[0].set_title('Model Accuracy')
ax[0].plot(accuracy)

ax[1].set_ylabel('Loss')
ax[1].set_xlabel('Epochs')
ax[1].set_title('Train/Test Losses')
ax[1].plot(train_losses, label='train')
ax[1].plot(test_losses, label='test')
ax[1].legend()   

plt.tight_layout()<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.14345738295318128, 'fraction_numerical': 0.035080698946245166, 'mean_word_length': 4.976484655241132, 'pattern_counts': {'":': 0, '<': 105, '<?xml version=': 0, '>': 106, 'https://': 6, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 4, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '27', 'dir': 'jupyter-structured-clean-dedup', 'id': '51769', 'license': 'RSA-MD', 'n_tokens_mistral': 5752, 'n_tokens_neox': 5343, 'n_words': 975, 'path': 'notebooks/2020_11_06_Twitch_Link_Prediction.ipynb', 'repo_name': 'cristianexer/trainings-and-tutorials', 'size': '14994'}