<jupyter_start><jupyter_text>Graph isomorphism networks> " /ᐠ｡▿｡ᐟ\\*ᵖᵘʳʳ* Redes neuronales de grafos, la moda que se viene"- toc: true- branch: master- badges: true- comments: true- author: <NAME>- categories: [pytorch,grafos,paper] Estas notas corresponden a la sección de código de una charla que di en el seminario [MachinLenin](https://twitter.com/machinlenin) en 2019. Esta es la [presentación](https://docs.google.com/presentation/d/1y9-4LMrm0UPqwYYBWwTMn-ixBazD8-nRnr6EIn53Nig/edit?usp=sharing) que usé y este el [repositorio](https://github.com/matiasbattocchia/gin).Esta es una implementación simple, con fines didácticos y para nada eficiente de la publicación [How powerful are graph neural networks?](https://arxiv.org/pdf/1810.00826.pdf). Para una implementación eficiente ver [GINConv](https://docs.dgl.ai/api/python/nn.pytorch.htmlginconv) de Deep Graph Library (DGL).La publicación mencionada fue la que elegí para comenzar a aprender sobre redes neuronales de grafos (*graph neural networks* o GNN). Unas redes interesantes porque no todo es texto, ni todo son imágenes, ni tablas... una vasta cantidad de información se representa en forma de grafo, y estas redes se especializan en esta estructura de datos.  PublicacionesHoy en día si tuviese que recomendar lecturas introductorias, serían los *reviews* de esta lista.**Reviews*** [Benchmarking Graph Neural Networks](https://arxiv.org/pdf/2003.00982.pdf)* [Representation Learning on Graphs: Methods and Applications](https://arxiv.org/abs/1709.05584) | http://snap.stanford.edu/proj/embeddings-www/**Arquitecturas*** [GAT](https://arxiv.org/pdf/1710.10903)* [GIN](https://arxiv.org/pdf/1810.00826)* [GCN](https://arxiv.org/abs/1609.02907)* [GraphSAGE](https://arxiv.org/pdf/1706.02216.pdf)<jupyter_code>import torch
import networkx as nx

%matplotlib inline<jupyter_output><empty_output><jupyter_text>Adjacency matrix<jupyter_code>G = nx.binomial_graph(5,0.5)

# TODO: numerar los nodos en el gráfico
nx.draw(G)
A = torch.tensor( nx.adjacency_matrix(G).todense(), dtype=torch.float32 )

A
X = torch.randint(low=0, high=2, size=(5,2), dtype=torch.float32)

X
A @ X<jupyter_output><empty_output><jupyter_text>Dataset<jupyter_code>import torch.utils.data

import importlib  
gnns = importlib.import_module('powerful-gnns.util')

class GraphDataset(torch.utils.data.Dataset):
    """ Levanta los datasets de Powerful-GNNS. """
    
    def __init__(self, dataset, degree_as_tag=False):
        self.data, self.classes = gnns.load_data(dataset, degree_as_tag)
        
        self.features = self.data[0].node_features.shape[1]
        
    def __len__(self):
        return len(self.data)
    
    def __getitem__(self, idx):
        graph = self.data[idx]
        adjacency_matrix = nx.adjacency_matrix( graph.g ).todense()
        
        item = {}

        item['adjacency_matrix'] = torch.tensor(adjacency_matrix, dtype=torch.float32)
        item['node_features'] = graph.node_features
        item['label'] = graph.label
        
        return item
DS = GraphDataset('PROTEINS')
DL = torch.utils.data.DataLoader(DS)<jupyter_output><empty_output><jupyter_text>Net modules<jupyter_code>class GINConv(torch.nn.Module):
    def __init__(self, hidden_dim):
        super().__init__()
        
        self.linear = torch.nn.Linear(hidden_dim, hidden_dim)

    def forward(self, A, X):
        """
        Params
        ------
        A [batch x nodes x nodes]: adjacency matrix
        X [batch x nodes x features]: node features matrix
        
        Returns
        -------
        X' [batch x nodes x features]: updated node features matrix
        """
        X = self.linear(X + A @ X)
        X = torch.nn.functional.relu(X)
        
        return X
class GNN(torch.nn.Module):
    
    def __init__(self, input_dim, hidden_dim, output_dim, n_layers):
        super().__init__()
        
        self.in_proj = torch.nn.Linear(input_dim, hidden_dim)
        
        self.convs = torch.nn.ModuleList()
        
        for _ in range(n_layers):
            self.convs.append(GINConv(hidden_dim))
        
        # In order to perform graph classification, each hidden state
        # [batch x nodes x hidden_dim] is concatenated, resulting in
        # [batch x nodes x hiddem_dim*(1+n_layers)], then aggregated
        # along nodes dimension, without keeping that dimension:
        # [batch x hiddem_dim*(1+n_layers)].
        self.out_proj = torch.nn.Linear(hidden_dim*(1+n_layers), output_dim)

    def forward(self, A, X):
        X = self.in_proj(X)

        hidden_states = [X]
        
        for layer in self.convs:
            X = layer(A, X)
            hidden_states.append(X)

        X = torch.cat(hidden_states, dim=2).sum(dim=1)

        X = self.out_proj(X)

        return X<jupyter_output><empty_output><jupyter_text>Train loop<jupyter_code>model = GNN(input_dim=DS.features, hidden_dim=3, output_dim=DS.classes, n_layers=3)

criterion = torch.nn.CrossEntropyLoss()
optimizer = torch.optim.Adam(model.parameters(), lr=0.01)

EPOCHS = 5

for epoch in range(EPOCHS):
    running_loss = 0.0
    
    for i, batch in enumerate(DL):
        A = batch['adjacency_matrix']
        X = batch['node_features']
        labels = batch['label']
                
        optimizer.zero_grad()

        outputs = model(A, X)
        loss = criterion(outputs, labels)
        loss.backward()
        optimizer.step()

        running_loss += loss.item()
    
    print(f'{epoch} - loss: {running_loss/(i+1)}')<jupyter_output>0 - loss: 1.2691171061103663
1 - loss: 0.32851389047806046
2 - loss: 0.944981049655904
3 - loss: 0.2894633889519622
4 - loss: 0.3056941165221455
{'fraction_non_alphanumeric': 0.11980915356069977, 'fraction_numerical': 0.03428167520763386, 'mean_word_length': 3.195700518902891, 'pattern_counts': {'":': 0, '<': 21, '<?xml version=': 0, '>': 22, 'https://': 11, 'lorem ipsum': 0, 'www.': 1, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '5', 'dir': 'jupyter-structured-clean-dedup', 'id': '486840', 'license': 'Apache-2.0', 'n_tokens_mistral': 2076, 'n_tokens_neox': 1918, 'n_words': 436, 'path': '_notebooks/2019-10-24-Graph-isomorphism-network.ipynb', 'repo_name': 'matiasbattocchia/datito', 'size': '5670'}