<jupyter_start><jupyter_text>Training a GNN for Graph Classification=======================================By the end of this tutorial, you will be able to-  Load a DGL-provided graph classification dataset.-  Understand what *readout* function does.-  Understand how to create and use a minibatch of graphs.-  Build a GNN-based graph classification model.-  Train and evaluate the model on a DGL-provided dataset.(Time estimate: 18 minutes)<jupyter_code>import dgl
import torch
import torch.nn as nn
import torch.nn.functional as F<jupyter_output><empty_output><jupyter_text>Overview of Graph Classification with GNN-----------------------------------------Graph classification or regression requires a model to predict certaingraph-level properties of a single graph given its node and edgefeatures.  Molecular property prediction is one particular application.This tutorial shows how to train a graph classification model for asmall dataset from the paper [How Powerful Are Graph NeuralNetworks](https://arxiv.org/abs/1810.00826).Loading Data------------<jupyter_code>import dgl.data

# Generate a synthetic dataset with 10000 graphs, ranging from 10 to 500 nodes.
dataset = dgl.data.GINDataset('PROTEINS', self_loop=True)<jupyter_output><empty_output><jupyter_text>The dataset is a set of graphs, each with node features and a singlelabel. One can see the node feature dimensionality and the number ofpossible graph categories of ``GINDataset`` objects in ``dim_nfeats``and ``gclasses`` attributes.<jupyter_code>print('Node feature dimensionality:', dataset.dim_nfeats)
print('Number of graph categories:', dataset.gclasses)<jupyter_output><empty_output><jupyter_text>Defining Data Loader--------------------A graph classification dataset usually contains two types of elements: aset of graphs, and their graph-level labels. Similar to an imageclassification task, when the dataset is large enough, we need to trainwith mini-batches. When you train a model for image classification orlanguage modeling, you will use a ``DataLoader`` to iterate over thedataset. In DGL, you can use the ``GraphDataLoader``.You can also use various dataset samplers provided in[`torch.utils.data.sampler`](https://pytorch.org/docs/stable/data.htmldata-loading-order-and-sampler).For example, this tutorial creates a training ``GraphDataLoader`` andtest ``GraphDataLoader``, using ``SubsetRandomSampler`` to tell PyTorchto sample from only a subset of the dataset.<jupyter_code>from dgl.dataloading import GraphDataLoader
from torch.utils.data.sampler import SubsetRandomSampler

num_examples = len(dataset)
num_train = int(num_examples * 0.8)

train_sampler = SubsetRandomSampler(torch.arange(num_train))
test_sampler = SubsetRandomSampler(torch.arange(num_train, num_examples))

train_dataloader = GraphDataLoader(
    dataset, sampler=train_sampler, batch_size=5, drop_last=False)
test_dataloader = GraphDataLoader(
    dataset, sampler=test_sampler, batch_size=5, drop_last=False)<jupyter_output><empty_output><jupyter_text>You can try to iterate over the created ``GraphDataLoader`` and see what itgives:<jupyter_code>it = iter(train_dataloader)
batch = next(it)
print(batch)<jupyter_output><empty_output><jupyter_text>As each element in ``dataset`` has a graph and a label, the``GraphDataLoader`` will return two objects for each iteration. Thefirst element is the batched graph, and the second element is simply alabel vector representing the category of each graph in the mini-batch.Next, we’ll talked about the batched graph.A Batched Graph in DGL----------------------In each mini-batch, the sampled graphs are combined into a single biggerbatched graph via ``dgl.batch``. The single bigger batched graph mergesall original graphs as separately connected components, with the nodeand edge features concatenated. This bigger graph is also a ``DGLGraph``instance (so you canstill treat it as a normal ``DGLGraph`` object as in[here](2_dglgraph.ipynb)). It however contains the informationnecessary for recovering the original graphs, such as the number ofnodes and edges of each graph element.<jupyter_code>batched_graph, labels = batch
print('Number of nodes for each graph element in the batch:', batched_graph.batch_num_nodes())
print('Number of edges for each graph element in the batch:', batched_graph.batch_num_edges())

# Recover the original graph elements from the minibatch
graphs = dgl.unbatch(batched_graph)
print('The original graphs in the minibatch:')
print(graphs)<jupyter_output><empty_output><jupyter_text>Define Model------------This tutorial will build a two-layer [Graph Convolutional Network(GCN)](http://tkipf.github.io/graph-convolutional-networks/). Each ofits layer computes new node representations by aggregating neighborinformation. If you have gone through the[introduction](1_introduction.ipynb), you will notice twodifferences:-  Since the task is to predict a single category for the *entire graph*   instead of for every node, you will need to aggregate the   representations of all the nodes and potentially the edges to form a   graph-level representation. Such process is more commonly referred as   a *readout*. A simple choice is to average the node features of a   graph with ``dgl.mean_nodes()``.-  The input graph to the model will be a batched graph yielded by the   ``GraphDataLoader``. The readout functions provided by DGL can handle   batched graphs so that they will return one representation for each   minibatch element.<jupyter_code>from dgl.nn import GraphConv

class GCN(nn.Module):
    def __init__(self, in_feats, h_feats, num_classes):
        super(GCN, self).__init__()
        self.conv1 = GraphConv(in_feats, h_feats)
        self.conv2 = GraphConv(h_feats, num_classes)
    
    def forward(self, g, in_feat):
        h = self.conv1(g, in_feat)
        h = F.relu(h)
        h = self.conv2(g, h)
        g.ndata['h'] = h
        return dgl.mean_nodes(g, 'h')<jupyter_output><empty_output><jupyter_text>Training Loop-------------The training loop iterates over the training set with the``GraphDataLoader`` object and computes the gradients, just likeimage classification or language modeling.<jupyter_code># Create the model with given dimensions
model = GCN(dataset.dim_nfeats, 16, dataset.gclasses)
optimizer = torch.optim.Adam(model.parameters(), lr=0.01)

for epoch in range(20):
    for batched_graph, labels in train_dataloader:
        pred = model(batched_graph, batched_graph.ndata['attr'].float())
        loss = F.cross_entropy(pred, labels)
        optimizer.zero_grad()
        loss.backward()
        optimizer.step()

num_correct = 0
num_tests = 0
for batched_graph, labels in test_dataloader:
    pred = model(batched_graph, batched_graph.ndata['attr'].float())
    num_correct += (pred.argmax(1) == labels).sum().item()
    num_tests += len(labels)

print('Test accuracy:', num_correct / num_tests)<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.10118875036242389, 'fraction_numerical': 0.005943751812119455, 'mean_word_length': 4.94229112833764, 'pattern_counts': {'":': 0, '<': 33, '<?xml version=': 0, '>': 33, 'https://': 2, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '8', 'dir': 'jupyter-structured-clean-dedup', 'id': '109139', 'license': 'Apache-2.0', 'n_tokens_mistral': 1976, 'n_tokens_neox': 1911, 'n_words': 709, 'path': '5_graph_classification.ipynb', 'repo_name': 'Geniussh/WSDM21-Hands-on-Tutorial', 'size': '6898'}