<jupyter_start><jupyter_text>线性回归进阶  2.1 自动计算梯度    |方法         |精确度         |是否支持任意代码      |备注               |   |----------  |:----------:  |:----------:       |-----:             |    |数值微分      |低            |是                 |实现琐碎            |  |符号微分      |高            |否                 |会构建一个完全不同的图 ||前向自动微分   |高            |是                 |基于二元树          ||反向自动微分   |高            |是                 |由Tensorflow实现    |表2-1.自动计算梯度的主要方法&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;TensorFlow是利用反向自动微分来实现自动计算梯度。<jupyter_code># 导入相关包
import tensorflow as tf
import numpy as np
from sklearn.datasets import fetch_california_housing
from sklearn.preprocessing import StandardScaler
import datetime
# 下载并整理数据
housing = fetch_california_housing()
m,n = housing.data.shape # 获取数据的行列数
housing_data_plus_bias = np.c_[np.ones((m,1)),housing.data] # 为数据添加偏差项，即添加y=ax+b中的b
# 数据预处理
scaler = StandardScaler().fit(housing_data_plus_bias)
scaled_housing_data_plus_bias = scaler.transform(housing_data_plus_bias)
# 创建计算图（一）
# 数据转换为常量
# 设置各种参数
n_epochs = 1000
global_learning_rate = 0.01
X = tf.constant(scaled_housing_data_plus_bias, dtype=tf.float32, name="X")
y = tf.constant(housing.target.reshape(-1,1), dtype=tf.float32, name="y") # 数据标签
XT = tf.transpose(X)
theta = tf.Variable(tf.random_uniform([n+1,1],-1.0,1.0),name="theta")     # 参数
y_pred =  tf.matmul(X, theta, name="prediction")                          # 预测值
error = y_pred-y                                                          # 误差
mse = tf.reduce_mean(tf.square(error), name="mse")                        # 均方误差(成本函数)
gradient = tf.gradients(mse, [theta])[0]                                  # 使用反向自动微分计算梯度
training_op = tf.assign(theta, theta-global_learning_rate*gradient)       # 训练<jupyter_output><empty_output><jupyter_text>2.2 使用优化器 &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;在TensorFLow中不仅可以自动求得梯度，还可以调用各种优化器对参数$\theta$进行优化。调用优化器时，只需在构建计算图步骤时更改对`training_op=...`赋值的语句即可，详情如下：<jupyter_code># 创建计算图（二）
n_epochs = 1000
global_learning_rate = 0.01
X = tf.constant(scaled_housing_data_plus_bias, dtype=tf.float32, name="X")
y = tf.constant(housing.target.reshape(-1,1), dtype=tf.float32, name="y") # 数据标签
XT = tf.transpose(X)
theta = tf.Variable(tf.random_uniform([n+1,1],-1.0,1.0),name="theta")     # 参数
y_pred =  tf.matmul(X, theta, name="prediction")                          # 预测值
error = y_pred-y                                                          # 误差
mse = tf.reduce_mean(tf.square(error), name="mse")                        # 均方误差(成本函数)
gradient = tf.gradients(mse, [theta])[0]                                  # 使用反向自动微分计算梯度
# 调用特定的优化器对参数进行优化
## 定义优化器(梯度下降)
# optimizer = tf.train.GradientDescentOptimizer(learning_rate = global_learning_rate)
## 定义优化器（动量）
optimizer = tf.train.MomentumOptimizer(learning_rate = global_learning_rate, momentum = 0.9)
training_op = optimizer.minimize(mse)<jupyter_output><empty_output><jupyter_text>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;创建会话，运行计算图，获得并观察结果<jupyter_code>init = tf.global_variables_initializer()                                  # 添加初始化节点

starttime = datetime.datetime.now()
with tf.Session() as sess:
    sess.run(init)
    
    for epoch in range(n_epochs):                                       # 逐步训练
        if epoch%100==0:
            print("Epoch:", epoch, "MSE=", mse.eval())                    # 每一步均方误差
        sess.run(training_op)                                             # 执行每一步训练，更新梯度
        
    best_theta = theta.eval()                                             # 训练完毕，返回最佳参数
    print("The best theta is", best_theta)
endtime = datetime.datetime.now()
print("The running time:", (endtime - starttime))<jupyter_output>Epoch: 0 MSE= 6.2275195
Epoch: 100 MSE= 4.809787
Epoch: 200 MSE= 4.803549
Epoch: 300 MSE= 4.803283
Epoch: 400 MSE= 4.8032584
Epoch: 500 MSE= 4.8032546
Epoch: 600 MSE= 4.8032546
Epoch: 700 MSE= 4.803254
Epoch: 800 MSE= 4.803253
Epoch: 900 MSE= 4.8032537
The best theta is [[-0.56934285]
 [ 0.82961434]
 [ 0.11875076]
 [-0.26551732]
 [ 0.3056884 ]
 [-0.00450322]
 [-0.03932609]
 [-0.8998975 ]
 [-0.87055224]]
The running time: 0:00:00.342895
{'fraction_non_alphanumeric': 0.13823314675103432, 'fraction_numerical': 0.05986858116330007, 'mean_word_length': 2.4193011647254576, 'pattern_counts': {'":': 0, '<': 12, '<?xml version=': 0, '>': 12, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 13, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '3', 'dir': 'jupyter-structured-clean-dedup', 'id': '228013', 'license': 'Apache-2.0', 'n_tokens_mistral': 1858, 'n_tokens_neox': 1810, 'n_words': 242, 'path': 'Practice/TensorFlow/Anaconda/LinearRegression/.ipynb_checkpoints/LinearRegression_3-checkpoint.ipynb', 'repo_name': 'Roggu123/Algorithm', 'size': '4109'}