<jupyter_start><jupyter_text>Single Convolution Blog通过使用单个卷积 block 对单个图片的处理过程了，解卷积网络中 Conv 的使用过程。需要注意卷积层的 weight 和 bias 是初始值是通过均匀分布获取——均匀分布模式为：$$\mu(\sqrt{k}, \sqrt{k}) \\k = \frac{1}{C_{in} \times \displaystyle{\prod_{i=0}^m kernel\_size}}$$m 值与使用的卷积维度相关，一维卷积的值为 0，二维（[Conv2d](https://pytorch.org/docs/stable/nn.htmlconv2d)）为 1 以此类推。如果需要对 weight 和 bias 的数据进行调整，可以使用 [apply](https://pytorch.org/docs/master/nn.htmltorch.nn.Module.apply) 方法对参数权重进行调整。卷积层的计算过程是矩阵乘法计算的过程，在 [Image Kernels explained visually](http://setosa.io/ev/image-kernels/) 中进行了可视化表述卷积计算过程<jupyter_code>conv2d_1 = nn.Conv2d(1, 10, kernel_size=5)
conv2d_2 = nn.Conv2d(10, 20, kernel_size=5)
conv2d_1.weight
# 需要调整数据维度——图片数量 * 图像通道数 * 图像宽度 * 图像高度
test_data = sample_data[0][1]
test_data.resize_(1, *test_data.shape)
conv_sample = conv2d_1(test_data)
conv_sample<jupyter_output><empty_output><jupyter_text>池化层通常使用 MaxPooling 和 AveragePooling，但可选方法具有多种。池化计算中并没有权重的计算，因此在反馈计算中它并不需要进行权重更新<jupyter_code>pool_sample = F.max_pool2d(conv_sample, 2)
print(
    f"Origin data dimentions: {test_data.shape}"
    f"\nFirst convolution layers dimentions: {conv_sample.shape} with weight dimentions: {conv2d_1.weight.shape}"
    f"\nPooling layers dimentions: {pool_sample.shape} with filter dimentions: (2, 2)"
    f"\nSecond convolution layers dimentions: {conv2d_2(pool_sample).shape} with weight dimentions: {conv2d_2.weight.shape}"
    f"\nLast layer dimentions: {F.max_pool2d(conv2d_2(pool_sample), 2).shape}"
)
F.max_pool2d(conv2d_2(pool_sample), 2).view(1, -1).shape<jupyter_output><empty_output><jupyter_text>通过卷积层之后，需要处理进行 Flatten 之后才能作为全连接层的输入。<jupyter_code>fc1 = F.linear(
    F.max_pool2d(
        conv2d_2(pool_sample),
        2).view(-1, 320), # flatten 处理卷积层结果
    torch.randn(50, 320)
    )
fc2 = F.linear(F.relu(fc1), torch.randn(10, 50))
out = F.log_softmax(fc2, dim=1)
out
# 已经使用 log_softmax 处理数据，需要通过 nll_loss 计算损失值
F.nll_loss(out, sample_data[1][1].reshape((1, )))
class Net(nn.Module):
    def __init__(self):
        super().__init__()
        self.conv1 = nn.Conv2d(in_channels=1, out_channels=10, kernel_size=5)
        self.conv2 = nn.Conv2d(in_channels=10, out_channels=20, kernel_size=5)
        self.conv2_drop = nn.Dropout2d()
        self.fc1 = nn.Linear(320, 50)
        self.fc2 = nn.Linear(50, 10)

    def forward(self, x):
        x = F.relu(F.max_pool2d(self.conv1(x), 2))
        x = F.relu(F.max_pool2d(self.conv2_drop(self.conv2(x)), 2))
        x = x.view(-1, 320) # Maybe better common description x = x.view(1, -1)
        x = F.relu(self.fc1(x))
        #x = F.dropout(x,p=0.1, training=self.training)
        x = self.fc2(x)
        return F.log_softmax(x,dim=1)
model = Net()

# if is_cuda:
#     model.cuda()
# former code is worst
model = model.to(device)
optimizer = optim.SGD(model.parameters(),lr=0.01)
data , target = next(iter(train_loader))
# output = model(Variable(data.cuda())) 
output = model(Variable(data.to(device))) # better way
output.size()
target.size()
def fit(epoch,model,data_loader,phase='training',volatile=False):
    if phase == 'training':
        model.train()
    if phase == 'validation':
        model.eval()
        volatile=True
    running_loss = 0.0
    running_correct = 0
    for batch_idx , (data,target) in enumerate(data_loader):
        # if is_cuda:
        #    data,target = data.cuda(),target.cuda()
        data, target = data.to(device), target.to(device)
        # Varibale is deprecated
        # data , target = Variable(data,volatile),Variable(target) 
        if phase == 'training':
            optimizer.zero_grad()
        output = model(data)
        loss = F.nll_loss(output,target)
        
        running_loss += F.nll_loss(output,target,size_average=False).item()
        preds = output.data.max(dim=1,keepdim=True)[1]
        running_correct += preds.eq(target.data.view_as(preds)).cpu().sum()
        if phase == 'training':
            loss.backward()
            optimizer.step()
    
    loss = running_loss/len(data_loader.dataset)
    accuracy = 100. * running_correct/len(data_loader.dataset)
    
    print(f'{phase} loss is {loss:{5}.{2}} and {phase} accuracy is {running_correct}/{len(data_loader.dataset)}{accuracy:{10}.{4}}')
    return loss,accuracy
train_losses , train_accuracy = [],[]
val_losses , val_accuracy = [],[]
for epoch in range(1,20):
    epoch_loss, epoch_accuracy = fit(epoch,model,train_loader,phase='training')
    val_epoch_loss , val_epoch_accuracy = fit(epoch,model,test_loader,phase='validation')
    train_losses.append(epoch_loss)
    train_accuracy.append(epoch_accuracy)
    val_losses.append(val_epoch_loss)
    val_accuracy.append(val_epoch_accuracy)
plt.plot(range(1,len(train_losses)+1),train_losses,'bo',label = 'training loss')
plt.plot(range(1,len(val_losses)+1),val_losses,'r',label = 'validation loss')
plt.legend();
plt.plot(range(1,len(train_accuracy)+1),train_accuracy,'bo',label = 'train accuracy')
plt.plot(range(1,len(val_accuracy)+1),val_accuracy,'r',label = 'val accuracy')
plt.legend();<jupyter_output><empty_output><jupyter_text>Understanding 1d convolutions<jupyter_code>conv = nn.Conv1d(1,1,3,bias=False)
conv.weight
sample = torch.randn(1,1,7)
sample
conv(Variable(sample))<jupyter_output><empty_output><jupyter_text>Understand how padding works in Conv1d<jupyter_code>conv = nn.Conv1d(1,1,3,padding=2,bias=False)
conv.weight
sample = torch.randn(1,1,7)
sample
conv(Variable(sample))<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.14961354435038646, 'fraction_numerical': 0.029628266470371735, 'mean_word_length': 4.679205851619645, 'pattern_counts': {'":': 0, '<': 21, '<?xml version=': 0, '>': 21, 'https://': 2, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '5', 'dir': 'jupyter-structured-clean-dedup', 'id': '110857', 'license': 'MIT', 'n_tokens_mistral': 2290, 'n_tokens_neox': 2293, 'n_words': 315, 'path': 'Chapter05/Understanding Convolutions and building an MNIST image classifier.ipynb', 'repo_name': 'ZenRay/Deep-Learning-with-PyTorch', 'size': '5434'}