<jupyter_start><jupyter_text>Transposed Convolution:label:`sec_transposed_conv`The layers we introduced so far for convolutional neural networks, includingconvolutional layers (:numref:`sec_conv_layer`) and pooling layers (:numref:`sec_pooling`), often reduce the input width and height, or keep them unchanged. Applications such as semantic segmentation (:numref:`sec_semantic_segmentation`) and generative adversarial networks (:numref:`sec_dcgan`), however, require to predict values for each pixel and therefore needs to increase input width and height. Transposed convolution, also named fractionally-strided convolution :cite:`Dumoulin.Visin.2016` or deconvolution :cite:`Long.Shelhamer.Darrell.2015`, serves this purpose.<jupyter_code>import torch
from torch import nn
from d2l import torch as d2l<jupyter_output><empty_output><jupyter_text>Basic 2D Transposed ConvolutionLet us consider a basic case that both input and output channels are 1, with 0 padding and 1 stride. :numref:`fig_trans_conv` illustrates how transposed convolution with a $2\times 2$ kernel is computed on the $2\times 2$ input matrix.![Transposed convolution layer with a $2\times 2$ kernel.](../img/trans-conv.svg):label:`fig_trans_conv`We can implement this operation by giving matrix kernel $K$ and matrix input $X$.<jupyter_code>def trans_conv(X, K):
    h, w = K.shape
    Y = torch.zeros((X.shape[0] + h - 1, X.shape[1] + w - 1))
    for i in range(X.shape[0]):
        for j in range(X.shape[1]):
            Y[i: i + h, j: j + w] += X[i, j] * K
    return Y<jupyter_output><empty_output><jupyter_text>Remember the convolution computes results by `Y[i, j] = (X[i: i + h, j: j + w] * K).sum()` (refer to `corr2d` in :numref:`sec_conv_layer`), which summarizes input values through the kernel. While the transposed convolution broadcasts input values through the kernel, which results in a larger output shape.Verify the results in :numref:`fig_trans_conv`.<jupyter_code>X = torch.tensor([[0., 1], [2, 3]])
K = torch.tensor([[0., 1], [2, 3]])
trans_conv(X, K)<jupyter_output><empty_output><jupyter_text>Or we can use `nn.ConvTranspose2d` to obtain the same results. As `nn.Conv2d`, both input and kernel should be 4-D tensors.<jupyter_code>X, K = X.reshape(1, 1, 2, 2), K.reshape(1, 1, 2, 2)
tconv = nn.ConvTranspose2d(1, 1, kernel_size=2, bias=False)
tconv.weight.data = K
tconv(X)<jupyter_output><empty_output><jupyter_text>Padding, Strides, and ChannelsWe apply padding elements to the input in convolution, while they are applied to the output in transposed convolution. A $1\times 1$ padding means we first compute the output as normal, then remove the first/last rows and columns.<jupyter_code>tconv = nn.ConvTranspose2d(1, 1, kernel_size=2, padding=1, bias=False)
tconv.weight.data = K
tconv(X)<jupyter_output><empty_output><jupyter_text>Similarly, strides are applied to outputs as well.<jupyter_code>tconv = nn.ConvTranspose2d(1, 1, kernel_size=2, stride=2, bias=False)
tconv.weight.data = K
tconv(X)<jupyter_output><empty_output><jupyter_text>The multi-channel extension of the transposed convolution is the same as the convolution. When the input has multiple channels, denoted by $c_i$, the transposed convolution assigns a $k_h\times k_w$ kernel matrix to each input channel. If the output has a channel size $c_o$, then we have a $c_i\times k_h\times k_w$ kernel for each output channel.As a result, if we feed $X$ into a convolutional layer $f$ to compute $Y=f(X)$ and create a transposed convolution layer $g$ with the same hyperparameters as $f$ except for the output channel set to be the channel size of $X$, then $g(Y)$ should has the same shape as $X$. Let us verify this statement.<jupyter_code>X = torch.rand(size=(1, 10, 16, 16))
conv = nn.Conv2d(10, 20, kernel_size=5, padding=2, stride=3)
tconv = nn.ConvTranspose2d(20, 10, kernel_size=5, padding=2, stride=3)
tconv(conv(X)).shape == X.shape<jupyter_output><empty_output><jupyter_text>Analogy to Matrix TranspositionThe transposed convolution takes its name from the matrix transposition. In fact, convolution operations can also be achieved by matrix multiplication. In the example below, we define a $3\times 3$ input $X$ with a $2\times 2$ kernel $K$, and then use `corr2d` to compute the convolution output.<jupyter_code>X = torch.arange(9.0).reshape(3, 3)
K = torch.tensor([[0, 1], [2, 3]])
Y = d2l.corr2d(X, K)
Y<jupyter_output><empty_output><jupyter_text>Next, we rewrite convolution kernel $K$ as a matrix $W$. Its shape will be $(4, 9)$, where the $i^\mathrm{th}$ row present applying the kernel to the input to generate the $i^\mathrm{th}$ output element.<jupyter_code>def kernel2matrix(K):
    k, W = torch.zeros(5), torch.zeros((4, 9))
    k[:2], k[3:5] = K[0, :], K[1, :]
    W[0, :5], W[1, 1:6], W[2, 3:8], W[3, 4:] = k, k, k, k
    return W

W = kernel2matrix(K)
W<jupyter_output><empty_output><jupyter_text>Then the convolution operator can be implemented by matrix multiplication with proper reshaping.<jupyter_code>Y == torch.mv(W, X.reshape(-1)).reshape(2, 2)<jupyter_output><empty_output><jupyter_text>We can implement transposed convolution as a matrix multiplication as well by reusing `kernel2matrix`. To reuse the generated $W$, we construct a $2\times 2$ input, so the corresponding weight matrix will have a shape $(9, 4)$, which is $W^\top$. Let us verify the results.<jupyter_code>X = torch.tensor([[0.0, 1], [2, 3]])
Y = trans_conv(X, K)
Y == torch.mv(W.T, X.reshape(-1)).reshape(3, 3)<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.11970935513169846, 'fraction_numerical': 0.025068119891008173, 'mean_word_length': 5.539192399049881, 'pattern_counts': {'":': 0, '<': 45, '<?xml version=': 0, '>': 45, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '11', 'dir': 'jupyter-structured-clean-dedup', 'id': '44680', 'license': 'MIT', 'n_tokens_mistral': 1922, 'n_tokens_neox': 1808, 'n_words': 643, 'path': 'd2l-en/pytorch/chapter_computer-vision/transposed-conv.ipynb', 'repo_name': 'gr8khan/d2lai', 'size': '5505'}