<jupyter_start><jupyter_text>练习--使用链式法则对复合函数球导数 $$f(x, y)=\frac{x+\sigma(y)}{\sigma(x)+(x+y)^{2}}$$<jupyter_code>x = 3 # example values
y = -4
import numpy as np
def sigmoid(X):
    return 1/(1+np.exp(-X))

## forward
a = x + y
b = sigmoid(x)
c = sigmoid(y)
d = x + c
e = a * a
f = b + e
fx =  d / f
            
## compute local gradients
              
DaDx = 1
DaDy = 1
DbDx = b * (1-b)
DcDy = c * (1-c)
DdDx = 1
DdDc = 1
DeDa = 2 * a
DfDb = 1
DfDe = 1
DfxDd = 1 / f
DfxDf = - d / f**2

            
## back propagation layer by layer
## 需要构建一个计算图，还要做裁剪
              
DfxDd = 1 / f
DfxDf = - d / f**2
              
                  
DdDy = DdDc * DcDy
DbDx = b * (1-b)
DeDx = DeDa * DaDx
DfDx = DfDb * DbDx + DfDe * DeDx

DfxDx = DfxDd * DdDx + DfxDf * DfDx 
DeDy = DeDa * DaDy
DfDy = DfDe * DeDy
DfxDy = DfxDd * DdDy + DfxDf * DfDy
DfxDy
import math
w = [2,-3,-3] # assume some random weights and data
x = [-1, -2]

# forward pass
dot = w[0]*x[0] + w[1]*x[1] + w[2]
f = 1.0 / (1 + math.exp(-dot)) # sigmoid function

# backward pass through the neuron (backpropagation)
ddot = (1 - f) * f # gradient on dot variable, using the sigmoid gradient derivation
dx = [w[0] * ddot, w[1] * ddot] # backprop into x
dw = [x[0] * ddot, x[1] * ddot, 1.0 * ddot] # backprop into w
# we're done! we have the gradients on the inputs to the circuit
dw, ddot<jupyter_output><empty_output><jupyter_text>使用向后传播算法计算f对于w的梯度：$$f(w, x)=\frac{1}{1+e^{-\left(w_{0} x_{0}+w_{1} x_{1}+w_{2}\right)}}$$<jupyter_code>y = x[0] * w[0] +  x[1] * w[1] + w[2]
z = sigmoid(y) 

Dy = z * (1 - z) 
DyDw = [x[0], x[1], 1]
DzDw = [x[0] * z * (1 - z) , x[1]* z * (1 - z), z * (1 - z) ]
DzDw
z * (1 - z)<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.1872791519434629, 'fraction_numerical': 0.035924617196702, 'mean_word_length': 2.5919661733615222, 'pattern_counts': {'":': 0, '<': 9, '<?xml version=': 0, '>': 9, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '2', 'dir': 'jupyter-structured-clean-dedup', 'id': '276667', 'license': 'MIT', 'n_tokens_mistral': 890, 'n_tokens_neox': 873, 'n_words': 219, 'path': 'notebook/backpropagation.ipynb', 'repo_name': 'chiechie/BasicAlgo', 'size': '1698'}