<jupyter_start><jupyter_text>linear regression 解析式直接求解<jupyter_code>df['x4'] = 1
X = df.iloc[:,(0,1,2,4)].values
y = df.y.values<jupyter_output><empty_output><jupyter_text>$y = Xw$  $ w = (X^T*X)^[-1]*X^T*y$<jupyter_code>inv_XX_T = inv(X.T.dot(X))
w = inv_XX_T.dot(X.T).dot(df.y.values)
w<jupyter_output><empty_output><jupyter_text>Resultsw1 = 2.97396653  w2 = -0.54139002  w3 = 0.97132913  b = 2.03076198<jupyter_code>qr(inv_XX_T)
X.shape
#solve(X,y)##只能解方阵<jupyter_output><empty_output><jupyter_text>梯度下降法求解- 目标函数选取要合适一些, 前边乘以适当的系数.- 注意检验梯度的计算是否正确...<jupyter_code>def f(w,X,y):    
    return ((X.dot(w)-y)**2/(2*1000)).sum()
def grad_f(w,X,y):
    return  (X.dot(w) - y).dot(X)/1000
w0 = np.array([100.0,100.0,100.0,100.0])
epsilon = 1e-10
alpha = 0.1
check_condition = 1
while  check_condition > epsilon:
    w0 += -alpha*grad_f(w0,X,y)
    check_condition = abs(grad_f(w0,X,y)).sum()
print w0<jupyter_output>[ 2.97396671 -0.5414066   0.97132728  2.03076759]<jupyter_text>随机梯度下降法求解- Stochastic gradient descent- 使用了固定步长    - 一开始用的0.1, 始终达不到给定的精度- 于是添加了判定条件用来更新步长.<jupyter_code>def cost_function(w,X,y):    
    return (X.dot(w)-y)**2/2
def grad_cost_f(w,X,y):
    return  (np.dot(X, w) - y)*X
w0 = np.array([1.0, 1.0, 1.0, 1.0])
epsilon = 1e-3
alpha = 0.01
# 生成随机index,用来随机索引数据.
random_index = np.arange(1000)
np.random.shuffle(random_index)
cost_value = np.inf #初始化目标函数值
while abs(grad_f(w0,X,y)).sum() > epsilon:
    for i in range(1000):
        w0 += -alpha*grad_cost_f(w0,X[random_index[i]],y[random_index[i]])
    
    #检查目标函数变化趋势, 如果趋势变化达到临界值, 更新更小的步长继续计算
    difference = cost_value - f(w0, X, y) 
    if difference < 1e-10:
        alpha *= 0.9
    cost_value = f(w0, X, y)
print w0<jupyter_output>[ 2.97376767 -0.54075842  0.97217986  2.03067711]
{'fraction_non_alphanumeric': 0.18238636363636362, 'fraction_numerical': 0.10852272727272727, 'mean_word_length': 4.452012383900929, 'pattern_counts': {'":': 0, '<': 20, '<?xml version=': 0, '>': 21, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '5', 'dir': 'jupyter-structured-clean-dedup', 'id': '896', 'license': 'MIT', 'n_tokens_mistral': 1050, 'n_tokens_neox': 988, 'n_words': 120, 'path': 'Linear_regression/Linear_regression.ipynb', 'repo_name': 'xpgeng/exercises_of_machine_learning', 'size': '1760'}