<jupyter_start><jupyter_text>Backpropagation (derivative) To calculate a derivative of a tensor/function of tensors it is necessary to add requires_grad=True. This way pytorch is tracking the tensor for gradient calculation.<jupyter_code>x = torch.tensor(2.0, requires_grad=True)
y = 2*x**4 + x**3 + 3*x**2 + 5*x + 1<jupyter_output><empty_output><jupyter_text>As shown below, it now automatically makes a gradient function for y, as y is a function of x.<jupyter_code>print(y)
type(y)
y.backward()
x.grad<jupyter_output><empty_output><jupyter_text>x.grad is an attribute of x, so without ()x.grad is a result of derivative y'(x): y'(x) = 93  Backpropagation on multiple steps<jupyter_code>x = torch.tensor([[1., 2., 3.], [3., 2., 1.]], requires_grad=True)
x
y = 3*x + 2
print(y)
z = 2*y**2<jupyter_output><empty_output><jupyter_text>function z have multi backwards automatically add for calculating derivatives<jupyter_code>print(z)
out = z.mean()
print(out)<jupyter_output>tensor(140., grad_fn=<MeanBackward0>)<jupyter_text>Backpropagation:<jupyter_code>out.backward()
print(x.grad)<jupyter_output>tensor([[10., 16., 22.],
        [22., 16., 10.]])
{'fraction_non_alphanumeric': 0.1409921671018277, 'fraction_numerical': 0.03220191470844212, 'mean_word_length': 6.32484076433121, 'pattern_counts': {'":': 0, '<': 20, '<?xml version=': 0, '>': 20, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '5', 'dir': 'jupyter-structured-clean-dedup', 'id': '529016', 'license': 'MIT', 'n_tokens_mistral': 453, 'n_tokens_neox': 420, 'n_words': 109, 'path': 'Courses/PyTorch for Deep Learning with Python/07 ANN/039_pytorch_gradients.ipynb', 'repo_name': 'industArk/AI_learn', 'size': '1149'}