<jupyter_start><jupyter_text>In this tutorial we will explain how to integrate deep learning models into your Brancher pipline.   Building neural network modelsAll ```pytorch``` functions can be used in Brancher. These functions need to be imported from the Brancher `function` module which contains all ```pytorch``` functions acting on ```torch.Tensor``` in the form of Brancher functions acting on ```brancher.Variable```.We will define a stochastic convolutional network on the MNIST dataset. The first step is to import the dataset:<jupyter_code>import torchvision

# Create the data
image_size  = 28
num_classes = 10

train = torchvision.datasets.MNIST(root='./data',   train=True,  download=True, transform=None)
test  = torchvision.datasets.MNIST(root='./dataSo', train=False, download=True, transform=None)

dataset_size   = len(train)
input_variable = np.reshape(train.train_data.numpy(), newshape=(dataset_size, 1, image_size, image_size))
output_labels  = train.train_labels.numpy()<jupyter_output><empty_output><jupyter_text>In a Brancher model, datasets are stored in empirical variables. These are random variables that sample minibatches from a dataset.In a supervised problem we need two empirical variables, one for the input images and the other for the labels. However, these two variables need to be sampled jointly as each image should be associated with its correct label. In Brancher, we can implement this by generating a random variable ```RandomIndices```, and use it as ```indices``` for images and labels:<jupyter_code>from brancher.standard_variables import EmpiricalVariable as Empirical
from brancher.standard_variables import RandomIndices

# Data sampling model
minibatch_size = 7

minibatch_indices = RandomIndices( dataset_size=dataset_size, batch_size=minibatch_size, 
                                   name="indices", is_observed=True )

x = Empirical( input_variable, indices=minibatch_indices, 
               name="x", is_observed=True )

labels = Empirical( output_labels, indices=minibatch_indices, 
                    name="labels", is_observed=True )<jupyter_output><empty_output><jupyter_text>As a next step we import the `pytorch` function `conv2d` from `brancher.functions` and use it to build a stochastic 2D-convolutional layer with Gaussian weights. In order to do this we define the weights as a `NormalVariable`. We then use the Brancher `conv2d` and `relu` functions on the weights `Wk` and images `x`, and wrap the result of the layer `z` inside a `DeterministicVariable`.<jupyter_code>from brancher import functions as BF

from brancher.standard_variables import DeterministicVariable as Deterministic
from brancher.standard_variables import NormalVariable as Normal
from brancher.standard_variables import CategoricalVariable as Categorical

in_channels  = 1
out_channels = 5
image_size   = 28

# Define Gaussian convolutional kernels:
Wk = Normal( loc=np.zeros((out_channels, in_channels, 3, 3)),
             scale=np.ones((out_channels, in_channels, 3, 3)),
             name="Wk")

# Define output: 
z = Deterministic( BF.relu(BF.conv2d(x, Wk, padding=1)), name="z" )<jupyter_output><empty_output><jupyter_text>The randomized input image `x` is convolved with the random convolutional filter weights `Wk`. We can now run the forward pass by sampling from the model:<jupyter_code>num_samples = 6
z.get_sample(num_samples)["z"]<jupyter_output><empty_output><jupyter_text>Note that in each of these samples both the input and the weigths are sampled independently.We can now add a linear layer to this result to get a shallow convolutional classifier. We can do this by again defining random parameter variables as follows:<jupyter_code>num_classes = 10
Wl = Normal( loc   = np.zeros((num_classes, image_size*image_size*out_channels)),
             scale = np.ones((num_classes, image_size*image_size*out_channels)),
             name  = "Wl")

b  = Normal( loc   = np.zeros((num_classes, 1)),
             scale = np.ones((num_classes, 1)),
             name  = "b")

reshaped_z = BF.reshape(z, shape=(image_size*image_size*out_channels, 1))

k  = Categorical( logits = BF.linear(reshaped_z, Wl, b), 
                  name="k" )<jupyter_output><empty_output><jupyter_text>We had to reshape (flatten) the variable `z` to use it as input to the linear layer. Note that in Brancher you never need to explicitly consider the batch dimension. Batch properties are part of the data, not of the model and Brancher will handle them automatically! Observing the model and training the weights  Now that the model is defined we need to specifiy which variables are observed. The input image variable `x` was set up to be observed during model definition. The other variable to observe is the output `k` which needs to be observed using the real labels. To this aim, we simply need to call the `.observe` method on `k` with the label `EmpiricalVariable` as input.<jupyter_code>k.observe(labels)<jupyter_output><empty_output><jupyter_text>Done! We are now ready to learn the weights. If we are not concerned with quantifying uncertainty, we can train using maximum-a-posteriori (MAP).<jupyter_code>from brancher.inference import MAP
from brancher.inference import perform_inference
from brancher.variables import ProbabilisticModel

convolutional_model = ProbabilisticModel([k])

perform_inference( convolutional_model,
                   inference_method=MAP(),
                   number_iterations=600, #500
                   optimizer="Adam",
                   lr=0.005 )
loss_list = convolutional_model.diagnostics["loss curve"]
plt.plot(loss_list)<jupyter_output><empty_output><jupyter_text>We can now run the model on the test set by sampling the newly trained posterior distribution. To this aim we need to call the `.get_posterior_samples` method since `.get_sample` will result on a sample from the (untrained) prior. We also need to provide the test images as input to the this method:<jupyter_code>test_size = test.test_data.numpy().shape[0]
test_images = np.reshape(test.test_data.numpy(), newshape=(test_size, 1, image_size, image_size))

posterior_samples = convolutional_model.get_posterior_sample(num_samples = 1, 
                                                             input_values = {x: test_images[0:4,:]})<jupyter_output><empty_output><jupyter_text>The output is one-hot encoded. Let's get the predicted labels:<jupyter_code>np.argmax(posterior_samples["k"][0], axis=1)<jupyter_output><empty_output><jupyter_text>Using existing PyTorch models in a Brancher model We have seen how we can construct Brancher models with neural network building blocks. However, there are situations where we want to specify a `pytorch` model and use it within Brancher. In this case we can wrap the `pytorch` model using the `brancher.functions.BrancherFunction` class. Let's start by defining a `pytorch` neural network:<jupyter_code>import torch

class PytorchNetwork(torch.nn.Module):
    def __init__(self):
        super(PytorchNetwork, self).__init__()
        out_channels = 5
        image_size = 28
        self.l1 = torch.nn.Conv2d(in_channels=1, out_channels=out_channels, kernel_size=3, padding=1)
        self.f1 = torch.nn.ReLU()
        self.l2 = torch.nn.Linear(in_features=image_size ** 2 * out_channels, out_features=10)

    def __call__(self, x):
        h = self.f1(self.l1(x))
        h_shape = h.shape
        h = h.view((h_shape[0], np.prod(h_shape[1:])))
        logits = self.l2(h)
        return logits
    
network = PytorchNetwork()<jupyter_output><empty_output><jupyter_text>We can convert it into a brancher function via `BrancherFunction`:<jupyter_code>## Equivalent Brancher model ##
brancher_network = BF.BrancherFunction(network)<jupyter_output><empty_output><jupyter_text>However note that here we **do not** create latent variables. Instead we are treating the network as a black-box **function** with learnable parameters (learned via standard `pytorch`). We can not use Bayesian inference methods on the weights. You can learn this black-box-function via maximal likelihood. We can construct a full Brancher model as follows:<jupyter_code># Data sampling model #
minibatch_size = 4
minibatch_indices = RandomIndices( dataset_size=dataset_size, batch_size=minibatch_size,
                                   name="indices", is_observed=True )
x = Empirical( input_variable, indices=minibatch_indices,
               name="x", is_observed=True )
labels = Empirical( output_labels, indices=minibatch_indices,
                    name="labels", is_observed=True )

# Forward model #
k = Categorical( logits=brancher_network(x),
                 name="k" )
k.observe(labels)<jupyter_output><empty_output><jupyter_text>We can now train using `MaximumLikelihood`:<jupyter_code>from brancher.inference import MaximumLikelihood
from brancher.inference import perform_inference
from brancher.variables import ProbabilisticModel

convolutional_model = ProbabilisticModel([k])

perform_inference( convolutional_model,
                   inference_method=MaximumLikelihood(),
                   number_iterations=500,
                   optimizer="Adam",
                   lr=0.001 )
loss_list = convolutional_model.diagnostics["loss curve"]
plt.plot(loss_list)<jupyter_output><empty_output><jupyter_text>Note that `MaximumLikelihood` trains the probabilistic model itself (the prior in Bayesian terms) and not the posterior model. Therefore, we can test the model by calling the `.get_sample` method insteadòf `.get_posterior_sample` (the posterior is not even defined in this model as we do not have latent variables):<jupyter_code>test_size = test.test_data.numpy().shape[0]
test_images = np.reshape(test.test_data.numpy(), newshape=(test_size, 1, image_size, image_size))

np.argmax(convolutional_model.get_sample(1, input_values= {x: test_images[0:4,:]})["k"][0], axis=1)<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.08522899605702154, 'fraction_numerical': 0.0075826508947528055, 'mean_word_length': 4.595022624434389, 'pattern_counts': {'":': 0, '<': 57, '<?xml version=': 0, '>': 57, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 6, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '14', 'dir': 'jupyter-structured-clean-dedup', 'id': '106002', 'license': 'MIT', 'n_tokens_mistral': 2986, 'n_tokens_neox': 2880, 'n_words': 948, 'path': 'examples/notebooks/Deep learning .ipynb', 'repo_name': 'LucaAmbrogioni/Brancher-1', 'size': '9891'}