<jupyter_start><jupyter_text>Gradients, Partial Derivatives, and the Chain Rule<jupyter_code>import platform
import numpy as np
import matplotlib
import matplotlib.pyplot as plt

print(platform.python_version())
print(np.__version__)
print(matplotlib. __version__)

#python version 3.9.7
#numpy version 1.21.2
#matplotlib version 3.5.0<jupyter_output>3.9.7
1.21.2
3.5.0<jupyter_text>The Partial DeravativeMeasures how much impact a single input has on a function's output.\The method for calculating a partial derivative is the same as for derivatives, simple have to repeat this process for each of the independent inputs.\`f(x, y, z)` -> `df(x, y, z)/dx`, `df(x, y, z)/dy`, `df(x, y, z)/dz`  The Partial Derivative of a Sum`f(x, y)` = `x + y` -> `df(x, y)/dx`, `df(x, y)/dy` -> `1+0`, `0+1`\Example:\`f(x, y)` = `2x + 3y**2` -> `df(x, y)/dx = 2`, `df(x, y)/dy = 6y`  The Partial Derivative of Multiplication`f(x, y)` = `x * y` -> `df(x,y)/dx = y`, `df(x, y)/dy = x`\Example:\`f(x, y, z)` = `3x**3z - y**2 + 5z + 2yz`\`df(x, y, z)/dx` -> `9x**2z`\`df(x, y, z)/dy` -> `-2 + 2z`\`df(x, y, z)/dz` -> `3x**3 + 2y + 5`  The Partial Derivative of Max`f(x, y) = max(x, y)` -> `df(x, y)/dx` = `dmax(x, y)/dx` = `(x>y)`  The GradientThe gradient is a vector composed of all of the partial derivatives of a function, calculated woth respect to each input variable\We'll be using **derivatives** of a single-parameter functions and **gradients** of multivariate functions to perform **gradient desenct** using the **chain rule**.  The Chain Rule<jupyter_code># code for a forward pass of an example neural network model
"""
loss = -np.log(
    np.sum(
        y* np.exp(
            np.dot(
                np.maximum(
                    0,
                    np.dot(
                        np.maximum(
                            0,
                            np.dot(
                                X, 
                                w1.T
                            ) + b1
                        ),
                        w2.T
                    ) + b2
                ),
                w3.T
            ) + b3
        ) /
        np.sum(
            np.exp(
                np.dot(
                    np.maximum(
                        0,
                        np.dot(
                            np.maximum(
                                0, 
                                np.dot(
                                    X, 
                                w1.T
                            ) + b1
                        ),
                        w2.T
                    ) + b2
                ),
                w3.T
            ) + b3
        ),
            axis=1
            keepdims=True
        )
    )
)
"""<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.13330911732655285, 'fraction_numerical': 0.02143116600072648, 'mean_word_length': 1.2854771784232366, 'pattern_counts': {'":': 0, '<': 8, '<?xml version=': 0, '>': 18, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 1, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '2', 'dir': 'jupyter-structured-clean-dedup', 'id': '412516', 'license': 'MIT', 'n_tokens_mistral': 928, 'n_tokens_neox': 876, 'n_words': 244, 'path': 'NNFS/08_nnfs_gradients_partial_derivatives_and_the_chain_rule.ipynb', 'repo_name': 'theaveas/DeepLearning', 'size': '2753'}