<jupyter_start><jupyter_text>3 аттрибута тензора в pytorch:dtype - тип данных хранимых в тензоре (float32, etc.)device - на чём обрабатывается данный тензор (тензоры которые взаимодействую должны быть на одном проце/видяхе)layout - как данные расположены в памяти (не трогай это, дефолтная настройка норм) Создание тензоров (прямо как в NumPy):<jupyter_code># единичный тензор
print(torch.eye(3))
# нулевой
torch.zeros(2,2)
# единичный
torch.ones(2,2)<jupyter_output><empty_output><jupyter_text>Создание тензоров из данных:Factory function has more tweaking parameters than class constructor, so we will use them more often. They also infere the dtype (they get data in i.e. int32, they save it as int32).Beware that torch.Tensor and torch.tensor CREATE a copy of input data in memory, when torch.as_tensor and torch.from_numpy just mirror the existing data (so the last is more memory efficient, but if the original data is changed, tensor changed too. Data MUST be in NumPy array).For casual use - torch.tensor()For speed boost - torch.as_tensor()<jupyter_code>data = np.array([1,2,3])

t1 = torch.Tensor(data) # class constructor
t2 = torch.tensor(data) # factory function
t3 = torch.as_tensor(data) # factory function
t4 = torch.from_numpy(data) # factory function<jupyter_output><empty_output><jupyter_text>Reshaping the the tensors, squeezing.reshape() or .view()(одно и то же, просто разные названия) - прямое изменение размеров тензора.squeeze() - удаление всех осей с длинной 1..unsqueeze() - добавляет ось с длиной 1, это позволяет изменять ранг тензора.<jupyter_code>t = torch.tensor([
    [1,1,1,1],
    [2,2,2,2],
    [3,3,3,3]
], dtype=torch.float32)
t.shape
# number of elements check
print(torch.tensor(t.shape).prod())
print(t.numel())
print(t.reshape(2, 1, 2, 3)) # reshape to rank-4 tensor (from rank-2)
print(t.reshape(2, 6))
print(t.reshape(1,12)) # tensor rank 2
print(t.reshape(1,12).squeeze()) # tensor rank 1

# маленький трюк - если не знаем сколько компонентов будет в тензоре который мы хотим
# "вытянуть" сохранив второй ранг мы пишем "1, -1" во втором атрибуте .reshape
print(t.reshape(1,-1))

# если хотим перевести его в тензор первого ранга пишем "-1"
print(t.reshape(-1))<jupyter_output>tensor([[1., 1., 1., 1., 2., 2., 2., 2., 3., 3., 3., 3.]])
tensor([1., 1., 1., 1., 2., 2., 2., 2., 3., 3., 3., 3.])
tensor([[1., 1., 1., 1., 2., 2., 2., 2., 3., 3., 3., 3.]])
tensor([1., 1., 1., 1., 2., 2., 2., 2., 3., 3., 3., 3.])<jupyter_text>Flattening, Tensor size for neural network.flatten() - позволяет зарешейпить тензор любого ранга в тензор первого рангаОднако для нейросети абсолютно "плоские" данные нам не нужны, т.к. она должна различать batch (какие данные к какой введённой переменой относятся).Нейронка работает с тензорами ранга 4, вот что каждая из осей представляет(на примере тензора ниже):Batch - contains 3 imagesImage - contains 1 color channel (cause grayscale)Color channel - contains 4 arrays(rows/hight)4 arrays(rows/hight) - contains 4 pixel values (columns/width)<jupyter_code># create 3 "images" 4x4 pixels, grayscale

t1 = torch.tensor([
    [1,1,1,1],
    [1,1,1,1],
    [1,1,1,1],
    [1,1,1,1]
])
t2 = torch.tensor([
    [2,2,2,2],
    [2,2,2,2],
    [2,2,2,2],
    [2,2,2,2]
])
t3 = torch.tensor([
    [3,3,3,3],
    [3,3,3,3],
    [3,3,3,3],
    [3,3,3,3]
])

# stack them to make a tensor with rank 3 (new axis represents batch), now we have a 3
# "grayscale" pictures in the resulting tensor
t = torch.stack((t1, t2, t3))
t.shape
# and a new axis to this tensor to create a "batch" axis for discening the "pictures"
t = t.reshape(3, 1, 4, 4)
t<jupyter_output><empty_output><jupyter_text>how to use flattenWe can't flatten the whole tensor, cause it will become a simple vector and we won't be able to know where are the "start" and the "end" of each "picture". So we flatten this tensor in such a way that wa preserve the "batch" axis (flattening the color channel with hight and width axes), which tells us which "picture" is which.<jupyter_code># bad idea - got vector with 48 pixel, no way to know from which picture each pixel comes from
a = t.flatten()
print('bad  - ', a.shape)

# good idea - got 3 images with 16 pixels
t = t.flatten(start_dim=1) # start flattening from axis=1 (it`s an index)
print('good - ', t.shape)<jupyter_output>bad  -  torch.Size([48])
good -  torch.Size([3, 16])<jupyter_text>Broadcasting for element-wise operationsBroadcasting is transforming a lower rank tensor to match the shape of the tensor with which we want it to perform an element-wise operation.All element-wise operations works with tensors due to the broadcasting.<jupyter_code>print(t.eq(1)) # equal 1
print(t.abs()) # взятие модуля
print(t + 3)
print(t.neg())<jupyter_output>tensor([[ True,  True,  True,  True,  True,  True,  True,  True,  True,  True,
          True,  True,  True,  True,  True,  True],
        [False, False, False, False, False, False, False, False, False, False,
         False, False, False, False, False, False],
        [False, False, False, False, False, False, False, False, False, False,
         False, False, False, False, False, False]])
tensor([[1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1],
        [2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2],
        [3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3]])
tensor([[4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4],
        [5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5],
        [6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6]])
tensor([[-1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1],
        [-2, -2, -2, -2, -2, -2, -2, -2, -2, -2, -2, -2, -2, -2, -2, -2],
        [-3, -3, -3, -3, -3, -3, -3, -3, -3, -3, -3, -3, -3, -3, -3, -3]])<jupyter_text>Reduction operation - .ArgMax()Reduction operation on tensor is an operation that reduced the number of elements contained within the tensor.We can use .mean(),.argmax() returns the index of the max value inside the tensor<jupyter_code>t4 = torch.tensor([
    [0,1,0],
    [2,0,2],
    [0,3,0]
], dtype=torch.float32)
print(t4.shape)

print(t4.sum()) # sum all elements = reduce this tensor to one axis
print(t4.sum(dim=0))
print(t4.sum(dim=0).shape) # sum along axis 1 (sum columns) = reduce this tensor to one axis
print('value   = ', t4.max())     # MAX value 
print('flatten = ', t4.flatten()) # flatten tensor (count indices from '0')
print('index   = ', t4.argmax())  # index of the MAX value (yep, '3' has index=7)

print(t4.max(dim=0), '\n') # max values on axis=0
print(t4.max(dim=1)) # first tensor - max values, second tensor - their indices
# if we wanna get the results not as tensors, but as ints/floats/np.arrays:

print(t4.mean())             # got tensor
print(t4.mean().item(), '\n') # got float32

print(t4.mean(dim=0))                # got tensor
print(t4.mean(dim=0).tolist(), '\n') # got np.array of float32<jupyter_output>tensor(0.8889)
0.8888888955116272 

tensor([0.6667, 1.3333, 0.6667])
[0.6666666865348816, 1.3333333730697632, 0.6666666865348816]
{'fraction_non_alphanumeric': 0.15528839272935452, 'fraction_numerical': 0.0659796765421497, 'mean_word_length': 3.76346284935242, 'pattern_counts': {'":': 0, '<': 25, '<?xml version=': 0, '>': 25, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 4, 'word_list_counts': {'cursed_substrings.json': 24, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '7', 'dir': 'jupyter-structured-clean-dedup', 'id': '626589', 'license': 'MIT', 'n_tokens_mistral': 2996, 'n_tokens_neox': 2850, 'n_words': 918, 'path': 'Books and Courses/PyTorch/.ipynb_checkpoints/1 - tensor creation, reshaping, squeezing, flattening-checkpoint.ipynb', 'repo_name': 'FairlyTales/Machine_Learning_Courses', 'size': '6987'}