<jupyter_start><jupyter_text>1x1 Convolution** References **1. [One by One Convolution](http://iamaaditya.github.io/2016/03/one-by-one-convolution/)2. [Lin, Min, <NAME>, and <NAME>. "Network in network." arXiv preprint arXiv:1312.4400 (2013).](https://arxiv.org/pdf/1312.4400v3.pdf)3. [VL_NNCONV - CNN convolution](http://www.vlfeat.org/matconvnet/mfiles/vl_nnconv/)4. [VGG Convolutional Neural Networks Practical](https://www.tumblr.com/blog/dimensionmismatch)5. [Convnet: Implementing Convolution Layer with Numpy](http://wiseodd.github.io/techblog/2016/07/16/convnet-conv-layer/)6. [Implement MATLAB's im2col 'sliding' in Python](https://stackoverflow.com/questions/30109068/implement-matlabs-im2col-sliding-in-python)7. [Rearrange image blocks into columns](https://www.mathworks.com/help/images/ref/im2col.html)  This 1x1 convolution is actually a matrix multiplication + addition~~~conv8_feat1 = single(reshape(1:36, [1 1 12 3]));squeeze(vl_nnconv(conv8_feat1, ...    single(reshape(1:48, [1 1 12 4])), ...    single(reshape(1:4, [1 4])), ...    'pad', [0 0 0 0], 'stride', 1))~~~~~~ans =  4×3 single matrix         651        1587        2523        1588        4252        6916        2525        6917       11309        3462        9582       15702~~~<jupyter_code>import tensorflow as tf
import numpy as np<jupyter_output><empty_output><jupyter_text>Ignoring singleton dimensions, the same can be achieved using np.dot(), which carries out matrix multiplication on 2-d arrays.n.b. _The transposition is applied because in MATLAB matrices are column-oriented, where as in Numpy they are row-oriented._<jupyter_code>A = np.arange(36).reshape((3, 12)).T + 1
F = np.arange(48).reshape((4, 12)).T + 1
B = np.arange(4) + 1
np.dot(F.T, A) + B[..., np.newaxis]<jupyter_output><empty_output><jupyter_text>Now replicate this operation with the definition of convolution in ref. 4. See also ref. 5, 6, 7.<jupyter_code>def conv(x, w):
    from skimage.util import view_as_windows
    im2col = lambda x, sz: view_as_windows(x, (sz[0],sz[1])).reshape((-1,sz[0]*sz[1])).T
    n_filters = w.shape[-1]
    # Put patches in x into columns
    X = im2col(x.copy(), (1, 1))
    print(x.shape, X.shape, w.shape)
    return X#np.dot(w.T.flatten(), X)

    # Suppose we have 20 of 3x3 filter: 20x1x3x3. W_col will be 20x9 matrix
#     W = W.reshape(n_filters, -1)

#     # 20x9 x 9x500 = 20x500
#     out = W_col @ X_col + b

#     # Reshape back from 20x500 to 5x20x10x10
#     # i.e. for each of our 5 images, we have 20 results with size of 10x10
#     out = out.reshape(n_filters, h_out, w_out, n_x)
#     out = out.transpose(3, 0, 1, 2)
conv(A.reshape((1, 1, 12, 3)), F.reshape((1, 1, 12, 4)))<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.14953959484346224, 'fraction_numerical': 0.08250460405156537, 'mean_word_length': 3.690846286701209, 'pattern_counts': {'":': 0, '<': 15, '<?xml version=': 0, '>': 15, 'https://': 4, 'lorem ipsum': 0, 'www.': 3, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '3', 'dir': 'jupyter-structured-clean-dedup', 'id': '120543', 'license': 'MIT', 'n_tokens_mistral': 1154, 'n_tokens_neox': 1003, 'n_words': 254, 'path': 'conv1x1.ipynb', 'repo_name': 'aleozlx/numpy-ipynb', 'size': '2726'}