<jupyter_start><jupyter_text>Quick-startThis document helps you get up-and-running with `alr` immediately.It should give you a general idea of how to get started withthis package.<jupyter_code>import numpy as np
import torch
import torch.utils.data as torchdata

from torch.nn import functional as F
from torch import nn

from alr import MCDropout
from alr.acquisition import RandomAcquisition, BALD
from alr.utils import stratified_partition, eval_fwd_exp
from alr.data import DataManager, UnlabelledDataset
from alr.data.datasets import Dataset

np.random.seed(42)
torch.manual_seed(42)
data_loader_params = dict(pin_memory=True, num_workers=2, batch_size=32)
device = torch.device('cuda:0' if torch.cuda.is_available() else 'cpu')<jupyter_output><empty_output><jupyter_text>Firstly, we load and prepare our data.Note that we partitioned the training set into labelled and unlabelled setsusing `stratified partition` which balances the number of classes in the training pool:<jupyter_code># load training data
train, test = Dataset.MNIST.get()
train, pool = stratified_partition(train, Dataset.MNIST.about.n_class, size=20)
pool = UnlabelledDataset(pool)
len(train), len(test), len(pool)<jupyter_output><empty_output><jupyter_text>`MCDropout` lets us define a Bayesian NN. It provides an implementationfor `stochastic_forward` which we will use in the next section for theacquisition function.> Notice the dropout layers have been changed to their `Persistent` versions.<jupyter_code># instantiate a regular model and an optimiser.
class Net(nn.Module):
    def __init__(self):
        super().__init__()
        self.conv1 = nn.Conv2d(1, 32, 5)
        # 32 24 24
        self.dropout1 = nn.Dropout2d()
        # maxpool --
        # 32 12 12
        self.conv2 = nn.Conv2d(32, 64, 5)
        # 64 8 8
        self.dropout2 = nn.Dropout2d()
        # maxpool --
        # 64 4 4
        self.fc1 = nn.Linear(64 * 4 * 4, 128)
        self.dropout3 = nn.Dropout()
        self.fc2 = nn.Linear(128, 10)

    def forward(self, x):
        x = F.max_pool2d(self.dropout1(F.relu(self.conv1(x))), 2)
        x = F.max_pool2d(self.dropout2(F.relu(self.conv2(x))), 2)
        x = x.view(-1, 64 * 4 * 4)
        x = self.fc2(self.dropout3(F.relu(self.fc1(x))))
        return F.log_softmax(x, dim=1)

model = MCDropout(Net(), forward=20).to(device)
model.compile(criterion=torch.nn.NLLLoss(),
              optimiser=torch.optim.Adam(model.parameters()))
model<jupyter_output><empty_output><jupyter_text>Now, we can instantiate an acquisition functionand an associated `DataManager` instance:<jupyter_code>bald = BALD(
    eval_fwd_exp(model), device=device,
    batch_size=1024, pin_memory=True, num_workers=2
)
dm = DataManager(train, pool, bald)<jupyter_output><empty_output><jupyter_text>Finally, the vanilla acquisition loop looks like:<jupyter_code>ITERS = 24
EPOCHS = 50
accs = {}

# In each iteration, acquire `b` points
for i in range(ITERS):
    print(f"==== Iteration {i + 1} ({(i + 1) / ITERS:.2%}), "
          f"training size: {dm.n_labelled} ====")
    # reset weights to original values when the model was first created
    model.reset_weights()
    # fit = train
    result = model.fit(
        train_loader=torchdata.DataLoader(
            dm.labelled, shuffle=True, **data_loader_params
        ),
        train_acc=True, epochs=EPOCHS, device=device, quiet=True
    )
    # evaluate the model to obtain its test accuracy
    test_acc, test_loss = model.evaluate(
        data=torchdata.DataLoader(test, **data_loader_params),
        device=device, quiet=True
    )
    # display results
    result.reduce('last', inplace=True)
    accs[dm.n_labelled] = test_acc
    print(f"train_acc = {result.train_acc:.2f}, "
          f"train_loss = {result.train_loss:.2f}, "
          f"test_acc = {test_acc:.2f}, "
          f"test_loss = {test_loss:.2f}")
    # acquire `b` points from unlabelled pool
    dm.acquire(b=10)
accs<jupyter_output>==== Iteration 1 (4.17%), training size: 20 ====
train_acc = 1.00, train_loss = 0.24, test_acc = 0.63, test_loss = 1.88
==== Iteration 2 (8.33%), training size: 30 ====
{'fraction_non_alphanumeric': 0.11853291231479232, 'fraction_numerical': 0.029876123390818556, 'mean_word_length': 3.7828106852497094, 'pattern_counts': {'":': 0, '<': 20, '<?xml version=': 0, '>': 21, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '5', 'dir': 'jupyter-structured-clean-dedup', 'id': '470309', 'license': 'MIT', 'n_tokens_mistral': 1501, 'n_tokens_neox': 1390, 'n_words': 358, 'path': 'experiments/thesis/old/quickstart.ipynb', 'repo_name': 'jiahfong/alr', 'size': '4117'}