<jupyter_start><jupyter_text>Softmax Regression Example  生成数据集， 看明白即可无需填写代码 '+' 从高斯分布采样 (X, Y) ~ N(3, 6, 1, 1, 0). 'o' 从高斯分布采样  (X, Y) ~ N(6, 3, 1, 1, 0) '*' 从高斯分布采样  (X, Y) ~ N(7, 7, 1, 1, 0)<jupyter_code>import tensorflow as tf
import matplotlib.pyplot as plt

from matplotlib import animation, rc
from IPython.display import HTML
import matplotlib.cm as cm
import numpy as np
%matplotlib inline

dot_num = 100
x_p = np.random.normal(3., 1, dot_num)
y_p = np.random.normal(6., 1, dot_num)
y = np.ones(dot_num)
C1 = np.array([x_p, y_p, y]).T

x_n = np.random.normal(6., 1, dot_num)
y_n = np.random.normal(3., 1, dot_num)
y = np.zeros(dot_num)
C2 = np.array([x_n, y_n, y]).T

x_b = np.random.normal(7., 1, dot_num)
y_b = np.random.normal(7., 1, dot_num)
y = np.ones(dot_num)*2
C3 = np.array([x_b, y_b, y]).T

plt.scatter(C1[:, 0], C1[:, 1], c='b', marker='+')
plt.scatter(C2[:, 0], C2[:, 1], c='g', marker='o')
plt.scatter(C3[:, 0], C3[:, 1], c='r', marker='*')

data_set = np.concatenate((C1, C2, C3), axis=0)
np.random.shuffle(data_set)<jupyter_output><empty_output><jupyter_text>建立模型建立模型类，定义loss函数，定义一步梯度下降过程函数填空一：在`__init__`构造函数中建立模型所需的参数填空二：实现softmax的交叉熵损失函数(不使用tf内置的loss 函数)<jupyter_code>epsilon = 1e-12
class SoftmaxRegression():
    def __init__(self):
        '''============================='''
        #todo 填空一，构建模型所需的参数 self.W, self.b 可以参考logistic-regression-exercise
        '''============================='''
        
        self.trainable_variables = [self.W, self.b]
    @tf.function
    def __call__(self, inp):
        logits = tf.matmul(inp, self.W) + self.b # shape(N, 3)
        pred = tf.nn.softmax(logits)
        return pred    
    
@tf.function
def compute_loss(pred, label):
    label = tf.one_hot(tf.cast(label, dtype=tf.int32), dtype=tf.float32, depth=3)
    '''============================='''
    #输入label shape(N, 3), pred shape(N, 3)
    #输出 losses shape(N,) 每一个样本一个loss
    #todo 填空二，实现softmax的交叉熵损失函数(不使用tf内置的loss 函数)
    '''============================='''
    loss = tf.reduce_mean(losses)
    
    accuracy = tf.reduce_mean(tf.cast(tf.equal(tf.argmax(label,axis=1), tf.argmax(pred, axis=1)), dtype=tf.float32))
    return loss, accuracy

@tf.function
def train_one_step(model, optimizer, x, y):
    with tf.GradientTape() as tape:
        pred = model(x)
        loss, accuracy = compute_loss(pred, y)
        
    grads = tape.gradient(loss, model.trainable_variables)
    optimizer.apply_gradients(zip(grads, model.trainable_variables))
    return loss, accuracy<jupyter_output><empty_output><jupyter_text>实例化一个模型，进行训练<jupyter_code>model = SoftmaxRegression()
opt = tf.keras.optimizers.SGD(learning_rate=0.01)
x1, x2, y = list(zip(*data_set))
x = list(zip(x1, x2))
for i in range(1000):
    loss, accuracy = train_one_step(model, opt, x, y)
    if i%50==49:
        print(f'loss: {loss.numpy():.4}\t accuracy: {accuracy.numpy():.4}')<jupyter_output>loss: 0.7527	 accuracy: 0.8067
loss: 0.6212	 accuracy: 0.8667
loss: 0.5481	 accuracy: 0.8933
loss: 0.5002	 accuracy: 0.8933
loss: 0.4658	 accuracy: 0.8933
loss: 0.4396	 accuracy: 0.89
loss: 0.4188	 accuracy: 0.8967
loss: 0.4018	 accuracy: 0.8933
loss: 0.3876	 accuracy: 0.8933
loss: 0.3755	 accuracy: 0.8967
loss: 0.3649	 accuracy: 0.8967
loss: 0.3557	 accuracy: 0.9
loss: 0.3476	 accuracy: 0.9
loss: 0.3403	 accuracy: 0.9
loss: 0.3337	 accuracy: 0.9
loss: 0.3277	 accuracy: 0.9
loss: 0.3223	 accuracy: 0.9067
loss: 0.3173	 accuracy: 0.9067
loss: 0.3127	 accuracy: 0.91
loss: 0.3084	 accuracy: 0.91<jupyter_text>结果展示，无需填写代码<jupyter_code>plt.scatter(C1[:, 0], C1[:, 1], c='b', marker='+')
plt.scatter(C2[:, 0], C2[:, 1], c='g', marker='o')
plt.scatter(C3[:, 0], C3[:, 1], c='r', marker='*')

x = np.arange(0., 10., 0.1)
y = np.arange(0., 10., 0.1)

X, Y = np.meshgrid(x, y)
inp = np.array(list(zip(X.reshape(-1), Y.reshape(-1))), dtype=np.float32)
print(inp.shape)
Z = model(inp)
Z = np.argmax(Z, axis=1)
Z = Z.reshape(X.shape)
plt.contour(X,Y,Z)
plt.show()<jupyter_output>(10000, 2)
{'fraction_non_alphanumeric': 0.20357322596879718, 'fraction_numerical': 0.07398087569199799, 'mean_word_length': 4.216535433070866, 'pattern_counts': {'":': 0, '<': 15, '<?xml version=': 0, '>': 15, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '4', 'dir': 'jupyter-structured-clean-dedup', 'id': '199787', 'license': 'Apache-2.0', 'n_tokens_mistral': 1967, 'n_tokens_neox': 1849, 'n_words': 357, 'path': 'assignments/nndl-exercise/chap3_softmax_regression/softmax_regression-exercise.ipynb', 'repo_name': 'Cannizza-zzk/DL_review', 'size': '3974'}