<jupyter_start><jupyter_text>Dataset : Protein-Protein Interaction Data Each graph : an unique human tissue Each node (a protein) : 50 features assigned from positional gene sets,  motif gene sets and immunological signatures data.  Label at each node : Each protein is labeled by a sequence of 121 numbers (0 or 1)  called gene ontology sets.  Ref : https://arxiv.org/pdf/1706.02216.pdf   section 4.2 The labels are assigned using the algorithm : https://www.ncbi.nlm.nih.gov/pmc/articles/PMC1941744/  What is the goal of the exercise ?  To build a graph neural network which will learn the gene ontology  (GO) ID's of each of the protein in a human cell and will predict the average ID's .<jupyter_code>import numpy as np
import torch<jupyter_output><empty_output><jupyter_text>Installing the DGL library<jupyter_code>pip install dgl
import networkx as nx
import dgl<jupyter_output><empty_output><jupyter_text>Download the PPID Dataset : https://github.com/williamleif/GraphSAGE/tree/master/example_data<jupyter_code>from dgl.data import PPIDataset
data_train, data_valid, data_test = PPIDataset('train'), PPIDataset('valid'), PPIDataset('test')<jupyter_output>Downloading /root/.dgl/ppi.zip from https://s3.us-east-2.amazonaws.com/dgl.ai/dataset/ppi.zip...
Extracting file to /root/.dgl/ppi
Loading G...
Loading G...
Loading G...<jupyter_text>How the induvidual dataset looks like ?<jupyter_code>len(data_train), len(data_valid), len(data_test)<jupyter_output><empty_output><jupyter_text>Structure of the dataset<jupyter_code>data_train[0]<jupyter_output><empty_output><jupyter_text>Each element of the dataset is a tuple : a graph, the features on each nodes, the GO ID on each node  The number of nodes change in each graph (a human tissue), but number of features assigned to each node (50) remains the same.  Similarly each protein comes with an unique GO ID. ( a sequence of 121 numbers consists of 0,1 )<jupyter_code>data_train[0][1].shape, data_train[0][2].shape,  data_train[3][1].shape, data_train[3][2].shape<jupyter_output><empty_output><jupyter_text>A look into graph structure<jupyter_code>data_train[0][0]
import matplotlib.pyplot as plt
plt.rcParams["figure.figsize"]=10,10
nx.draw(data_train[0][0].to_networkx(), with_labels=True)
plt.show()<jupyter_output><empty_output><jupyter_text>Use a function to assign features to the nodes and make tuples of (graph, node-labels)<jupyter_code>def ProcessGraph(data) : 
  
  dataset = []
  n_graph = len(data)
  
  for i_graph in range(n_graph) : 
    
    graph = data[i_graph][0]
    
    graph.ndata['x'] = torch.from_numpy(data[i_graph][1])
    
    label = torch.from_numpy(data[i_graph][2])
    
    dataset.append( (graph, label)  )
    
  return (dataset)
train_dataset = ProcessGraph(data_train)
valid_dataset = ProcessGraph(data_valid)
test_dataset = ProcessGraph(data_test)
#train_dataset<jupyter_output><empty_output><jupyter_text>Forming a graph minibatch In general different elements of a batch can have different shapes (the graphs are sparse)<jupyter_code>def collate(samples):
    # The input `samples` is a list of pairs
    #  (graph, label).
    graphs, labels = map(list, zip(*samples))
    batched_graph = dgl.batch(graphs) # --- the batch function creates the minibatch out of sparse graphs --- #
    return batched_graph, labels
import dgl.function as fn
import torch
import torch.nn as nn


# Sends a message of node feature h.
msg = fn.copy_src(src='h', out='m')

def reduce(nodes):
    """Take an average over all neighbor node features hu and use it to
    overwrite the original node feature."""
    accum = torch.mean(nodes.mailbox['m'], 1)
    return {'h': accum}

class NodeApplyModule(nn.Module):
    """Update the node feature hv with ReLU(Whv+b)."""
    def __init__(self, in_feats, out_feats, activation):
        super(NodeApplyModule, self).__init__()
        self.linear = nn.Linear(in_feats, out_feats)
        self.activation = activation

    def forward(self, node):
        h = self.linear(node.data['h'])
        h = self.activation(h)
        return {'h' : h}

class GCN(nn.Module):
    def __init__(self, in_feats, out_feats, activation):
        super(GCN, self).__init__()
        self.apply_mod = NodeApplyModule(in_feats, out_feats, activation)

    def forward(self, g, feature):
        # Initialize the node features with h.
        g.ndata['h'] = feature
        g.update_all(msg, reduce)
        g.apply_nodes(func=self.apply_mod)
        return g.ndata.pop('h')
import torch.nn.functional as F


class Classifier(nn.Module):
    def __init__(self, in_dim, hidden_dim1, hidden_dim2, hidden_dim3, n_classes):
        super(Classifier, self).__init__()

        self.layers = nn.ModuleList([
            GCN(in_dim, hidden_dim1, F.relu),
            GCN(hidden_dim1, hidden_dim2, F.relu),
            GCN(hidden_dim2, hidden_dim3, F.relu)])
        self.classify = nn.Linear(hidden_dim3, n_classes)

    def forward(self, g):
        # For undirected graphs, in_degree is the same as
        # out_degree.
        h = g.in_degrees().view(-1, 1).float()
        for conv in self.layers:
            h = conv(g, h)
        g.ndata['h'] = h
        hg = dgl.mean_nodes(g, 'h')
        return self.classify(hg)<jupyter_output><empty_output><jupyter_text>What's the network prdicticing ?  A : The average GO ID (the average is made over all the nodes)<jupyter_code>import torch.optim as optim
from torch.utils.data import DataLoader


# Use PyTorch's DataLoader and the collate function
# defined before.
data_loader = DataLoader(train_dataset, batch_size=32, shuffle=True,
                         collate_fn=collate)

val_data_loader = DataLoader(valid_dataset, batch_size=42, shuffle=True,
                         collate_fn=collate)

test_data_loader = DataLoader(test_dataset, batch_size=42, shuffle=True,
                         collate_fn=collate)

#data_loader.dataset
# Create model
model = Classifier(1, 256, 512, 128, 121)
len(data_loader.dataset)
model.train()
bg, label = data_loader.dataset[0]
label
prediction = model(bg)
prediction.shape, label.shape
def Loss(pred, label) : 
  
    loss = nn.L1Loss() 
    label_mod = torch.mean(label, 0)
    #print (pred.shape, label_mod.shape)
    #return torch.tensor(   torch.norm(pred- label_mod), requires_grad=True  )
    return loss(pred, label_mod)
loss = Loss(prediction, label.float() )
loss.item()

optimizer = optim.Adam(model.parameters(), lr=0.001)
# model.train()

## --------- Run the training loop ------------------- ##

n_epoch = 30

epoch_losses, epoch_val_losses = [], []
for epoch in range(1, n_epoch+1):
    epoch_loss = 0
    iter_batch = 0
    model.train()
    for i_node in range( len(data_loader.dataset)  ):
            
        bg, label = data_loader.dataset[i_node]
        prediction = model(bg)
        loss = Loss(prediction, label.float())
        optimizer.zero_grad()
        loss.backward()
        optimizer.step()
        epoch_loss += loss.detach().item()
        iter_batch += 1
    epoch_loss /= (iter_batch + 1)
    #print('Epoch {}, loss {:.4f}'.format(epoch, epoch_loss))
    epoch_losses.append(epoch_loss)
    
    
    epoch_val_loss = 0
    iter_batch_val = 0
    model.eval()
    for i_node in range( len(val_data_loader.dataset)  ):
         
        bg, label = data_loader.dataset[i_node]
        prediction = model(bg)
        loss = Loss(prediction, label.float())
        epoch_val_loss += loss.detach().item()
        iter_batch_val += 1
    epoch_val_loss /= (iter_batch_val + 1)
    #print('Epoch {}, loss {:.4f}'.format(epoch, epoch_loss))
    epoch_val_losses.append(epoch_val_loss)
    
    print('Epoch {}, loss {:.4f}, val_loss {:.4f}'.format(epoch, epoch_loss, epoch_val_loss))<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.11708451796329801, 'fraction_numerical': 0.01395709485655208, 'mean_word_length': 3.6536380036079374, 'pattern_counts': {'":': 0, '<': 40, '<?xml version=': 0, '>': 40, 'https://': 4, 'lorem ipsum': 0, 'www.': 1, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 2, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '10', 'dir': 'jupyter-structured-clean-dedup', 'id': '632388', 'license': 'Apache-2.0', 'n_tokens_mistral': 2658, 'n_tokens_neox': 2564, 'n_words': 656, 'path': 'GraphNNHomework.ipynb', 'repo_name': 'sanmayphy/course-v3', 'size': '7738'}