<jupyter_start><jupyter_text>Chapter20 深層学習のチューニング - **20.1 ハイパーパラメータ**    - **20.1.1 ハイパーパラメータ**- **20.2 ネットワーク構造**    - **20.2.1 ネットワーク構造**- **20.3 ドロップアウト**    - **2v.3.1 ドロップアウト**- **20.4 活性化関数**    - **20.4.1 活性化関数**    - **20.4.2 シグモイド関数**    - **2v.4.3 ReLU**- **20.5 損失関数**    - **20.5.1 損失関数**    - **20.5.2 二乗誤差**    - **20.5.3 クロスエントロピー誤差**- **20.6 最適化関数**    - **20.6.1 最適化関数**- **20.7 学習率**    - **20.7.1 学習率**- **20.8 ミニバッチ学習**    - **20.8.1 ミニバッチ学習**- **20.9 反復学習**    - **20.9.1 反復学習**- **20.10 添削問題**  ●20.1 ハイパーパラメータ  〇20.1.1 ハイパーパラメータ<jupyter_code>import numpy as np
import matplotlib.pyplot as plt
from keras.datasets import mnist
from keras.layers import Activation, Dense, Dropout
from keras.models import Sequential, load_model
from keras import optimizers
from keras.utils.np_utils import to_categorical

(X_train, y_train), (X_test, y_test) = mnist.load_data()

X_train = X_train.reshape(X_train.shape[0], 784)[:6000]
X_test = X_test.reshape(X_test.shape[0], 784)[:1000]
y_train = to_categorical(y_train)[:6000]
y_test = to_categorical(y_test)[:1000]

model = Sequential()
model.add(Dense(256, input_dim=784))
# ハイパーパラメータ：活性化関数
model.add(Activation("sigmoid"))
# ハイパーパラメータ：隠れ層の数、隠れ層のチャンネル数
model.add(Dense(128))
model.add(Activation("sigmoid"))
# ハイパーパラメータ：ドロップアウトする割合（rate）
model.add(Dropout(rate=0.5))
model.add(Dense(10))
model.add(Activation("softmax"))

# ハイパーパラメータ：学習率（Ir）
sgd = optimizers.SGD(lr=0.01)

# ハイパーパラメータ：最適化関数（optimizer）
# ハイパーパラメータ：誤差関数（loss）
model.compile(optimizer=sgd, loss="categorical_crossentropy", metrics=["accuracy"])

# ハイパーパラメータ：バッチサイズ（batch_size）
# ハイパーパラメータ：エポック数（epochs）
model.fit(X_train, y_train, batch_size=32, epochs=10, verbose=1)

score = model.evaluate(X_test, y_test, verbose=0)
print("evaluate loss: {0[0]}\nevaluate acc: {0[1]}".format(score))<jupyter_output><empty_output><jupyter_text>**リスト20.1：ハイパーパラメータの例**  ●20.2 ネットワーク構造  〇20.2.1 ネットワーク構造   □問題<jupyter_code>import numpy as np
import matplotlib.pyplot as plt
from keras.datasets import mnist
from keras.layers import Activation, Dense, Dropout
from keras.models import Sequential, load_model
from keras import optimizers
from keras.utils.np_utils import to_categorical

(X_train, y_train), (X_test, y_test) = mnist.load_data()

X_train = X_train.reshape(X_train.shape[0], 784)[:6000]
X_test = X_test.reshape(X_test.shape[0], 784)[:1000]
y_train = to_categorical(y_train)[:6000]
y_test = to_categorical(y_test)[:1000]

model = Sequential()
model.add(Dense(256, input_dim=784))
model.add(Activation("sigmoid"))

def funcA():
    model.add(Dense(128))
    model.add(Activation("sigmoid"))

def funcB():
    model.add(Dense(128))
    model.add(Activation("sigmoid"))
    model.add(Dense(128))
    model.add(Activation("sigmoid"))
    model.add(Dense(128))
    model.add(Activation("sigmoid"))

def funcC():
    model.add(Dense(1568))
    model.add(Activation("sigmoid"))

# 2 つをコメントアウトしてください
# ---------------------------
funcA()
funcB()
funcC()
# ---------------------------

model.add(Dropout(rate=0.5))
model.add(Dense(10))
model.add(Activation("softmax"))

sgd = optimizers.SGD(lr=0.1)

model.compile(optimizer=sgd, loss="categorical_crossentropy", metrics=["accuracy"])

model.fit(X_train, y_train, batch_size=32, epochs=3, verbose=1)

score = model.evaluate(X_test, y_test, verbose=0)
print("evaluate loss: {0[0]}\nevaluate acc: {0[1]}".format(score))<jupyter_output><empty_output><jupyter_text>**リスト20.2：問題**  □解答例<jupyter_code>import numpy as np
import matplotlib.pyplot as plt
from keras.datasets import mnist
from keras.layers import Activation, Dense, Dropout
from keras.models import Sequential, load_model
from keras import optimizers
from keras.utils.np_utils import to_categorical

(X_train, y_train), (X_test, y_test) = mnist.load_data()

X_train = X_train.reshape(X_train.shape[0], 784)[:6000]
X_test = X_test.reshape(X_test.shape[0], 784)[:1000]
y_train = to_categorical(y_train)[:6000]
y_test = to_categorical(y_test)[:1000]

model = Sequential()
model.add(Dense(256, input_dim=784))
model.add(Activation("sigmoid"))

def funcA():
    model.add(Dense(128))
    model.add(Activation("sigmoid"))

def funcB():
    model.add(Dense(128))
    model.add(Activation("sigmoid"))
    model.add(Dense(128))
    model.add(Activation("sigmoid"))
    model.add(Dense(128))
    model.add(Activation("sigmoid"))

def funcC():
    model.add(Dense(1568))
    model.add(Activation("sigmoid"))

# 2 つをコメントアウトしてください
#---------------------------
funcA()
#funcB()
#funcC()
#---------------------------

model.add(Dropout(rate=0.5))
model.add(Dense(10))
model.add(Activation("softmax"))

sgd = optimizers.SGD(lr=0.1)

model.compile(optimizer=sgd, loss="categorical_crossentropy", metrics=["accuracy"])

model.fit(X_train, y_train, batch_size=32, epochs=3, verbose=1)

score = model.evaluate(X_test, y_test, verbose=0)
print("evaluate loss: {0[0]}\nevaluate acc: {0[1]}".format(score))<jupyter_output>Using TensorFlow backend.<jupyter_text>**リスト20.3：解答例**  ●20.3 ドロップアウト  〇20.3.1 ドロップアウト  □問題<jupyter_code>import numpy as np
import matplotlib.pyplot as plt
from keras.datasets import mnist
from keras.layers import Activation, Dense, Dropout
from keras.models import Sequential, load_model
from keras import optimizers
from keras.utils.np_utils import to_categorical

(X_train, y_train), (X_test, y_test) = mnist.load_data()

X_train = X_train.reshape(X_train.shape[0], 784)[:6000]
X_test = X_test.reshape(X_test.shape[0], 784)[:1000]
y_train = to_categorical(y_train)[:6000]
y_test = to_categorical(y_test)[:1000]

model = Sequential()
model.add(Dense(256, input_dim=784))
model.add(Activation("sigmoid"))
model.add(Dense(128))
model.add(Activation("sigmoid"))

# ---------------------------
# ここにコードを記述してください   
# ---------------------------

model.add(Dense(10))
model.add(Activation("softmax"))

sgd = optimizers.SGD(lr=0.1)

model.compile(optimizer=sgd, loss="categorical_crossentropy", metrics=["accuracy"])

history = model.fit(X_train, y_train, batch_size=32, epochs=5, verbose=1, validation_data=(X_test, y_test))

#acc、val_accのプロット
plt.plot(history.history["acc"], label="acc", ls="-", marker="o")
plt.plot(history.history["val_acc"], label="val_acc", ls="-", marker="x")
plt.ylabel("accuracy")
plt.xlabel("epoch")
plt.legend(loc="best")
plt.show()<jupyter_output><empty_output><jupyter_text>**リスト20.4：問題**  □解答例<jupyter_code>import numpy as np
import matplotlib.pyplot as plt
from keras.datasets import mnist
from keras.layers import Activation, Dense, Dropout
from keras.models import Sequential, load_model
from keras import optimizers
from keras.utils.np_utils import to_categorical

(X_train, y_train), (X_test, y_test) = mnist.load_data()

X_train = X_train.reshape(X_train.shape[0], 784)[:6000]
X_test = X_test.reshape(X_test.shape[0], 784)[:1000]
y_train = to_categorical(y_train)[:6000]
y_test = to_categorical(y_test)[:1000]

model = Sequential()
model.add(Dense(256, input_dim=784))
model.add(Activation("sigmoid"))
model.add(Dense(128))
model.add(Activation("sigmoid"))
# ---------------------------
# ここにコードを記述してください
model.add(Dropout(rate=0.5))
# ---------------------------
model.add(Dense(10))
model.add(Activation("softmax"))

sgd = optimizers.SGD(lr=0.1)

model.compile(optimizer=sgd, loss="categorical_crossentropy", metrics=["accuracy"])

history = model.fit(X_train, y_train, batch_size=32, epochs=5, verbose=1, validation_data=(X_test, y_test))

#acc、val_accのプロット
plt.plot(history.history["acc"], label="acc", ls="-", marker="o")
plt.plot(history.history["val_acc"], label="val_acc", ls="-", marker="x")
plt.ylabel("accuracy")
plt.xlabel("epoch")
plt.legend(loc="best")
plt.show()<jupyter_output>Train on 6000 samples, validate on 1000 samples
Epoch 1/5
6000/6000 [==============================] - 1s 186us/step - loss: 1.7093 - acc: 0.4225 - val_loss: 1.0756 - val_acc: 0.7270
Epoch 2/5
6000/6000 [==============================] - 1s 108us/step - loss: 1.0370 - acc: 0.6692 - val_loss: 0.8398 - val_acc: 0.7610
Epoch 3/5
6000/6000 [==============================] - 1s 108us/step - loss: 0.8752 - acc: 0.7247 - val_loss: 0.7204 - val_acc: 0.8060
Epoch 4/5
6000/6000 [==============================] - 1s 109us/step - loss: 0.8071 - acc: 0.7425 - val_loss: 0.7205 - val_acc: 0.7860
Epoch 5/5
6000/6000 [==============================] - 1s 111us/step - loss: 0.7462 - acc: 0.7618 - val_loss: 0.6435 - val_acc: 0.8310<jupyter_text>**リスト20.5：解答例（出力結果は誌面と異なります）**  ●20.4 活性化関数  〇20.4.1 活性化関数 この項にサンプルはありません  〇20.4.2 シグモイド関数 この項にサンプルはありません  〇20.4.3 ReLU関数 この項にサンプルはありません  ●20.5 損失関数  〇20.5.1 損失関数 この項にサンプルはありません  〇20.5.2 二乗誤差 この項にサンプルはありません  〇20.5.3 クロスエントロピー誤差 この項にサンプルはありません  ●20.6 最適化関数  〇20.6.1 最適化関数 この項にサンプルはありません  ●20.7 学習率  ○20.7.1 学習率 **問題**<jupyter_code>import numpy as np
import matplotlib.pyplot as plt
from keras.datasets import mnist
from keras.layers import Activation, Dense, Dropout
from keras.models import Sequential, load_model
from keras import optimizers
from keras.utils.np_utils import to_categorical

(X_train, y_train), (X_test, y_test) = mnist.load_data()

X_train = X_train.reshape(X_train.shape[0], 784)[:6000]
X_test = X_test.reshape(X_test.shape[0], 784)[:1000]
y_train = to_categorical(y_train)[:6000]
y_test = to_categorical(y_test)[:1000]

model = Sequential()
model.add(Dense(256, input_dim=784))
model.add(Activation("sigmoid"))
model.add(Dense(128))
model.add(Activation("sigmoid"))
model.add(Dropout(rate=0.5))
model.add(Dense(10))
model.add(Activation("softmax"))


def funcA():
    global lr
    lr = 0.01

def funcB():
    global lr
    lr = 0.1

def funcC():
    global lr
    lr = 1.0

# 2 行をコメントアウトして学習率を決めてください
# ---------------------------
funcA()
funcB()
funcC()
# ---------------------------

sgd = optimizers.SGD(lr=lr)

model.compile(optimizer=sgd, loss="categorical_crossentropy", metrics=["accuracy"])

model.fit(X_train, y_train, batch_size=32, epochs=3, verbose=1)

score = model.evaluate(X_test, y_test, verbose=0)
print("evaluate loss: {0[0]}\nevaluate acc: {0[1]}".format(score))<jupyter_output><empty_output><jupyter_text>**リスト20.6：問題**  □解答例<jupyter_code>import numpy as np
import matplotlib.pyplot as plt
from keras.datasets import mnist
from keras.layers import Activation, Dense, Dropout
from keras.models import Sequential, load_model
from keras import optimizers
from keras.utils.np_utils import to_categorical

(X_train, y_train), (X_test, y_test) = mnist.load_data()

X_train = X_train.reshape(X_train.shape[0], 784)[:6000]
X_test = X_test.reshape(X_test.shape[0], 784)[:1000]
y_train = to_categorical(y_train)[:6000]
y_test = to_categorical(y_test)[:1000]

model = Sequential()
model.add(Dense(256, input_dim=784))
model.add(Activation("sigmoid"))
model.add(Dense(128))
model.add(Activation("sigmoid"))
model.add(Dropout(rate=0.5))
model.add(Dense(10))
model.add(Activation("softmax"))


def funcA():
    global lr
    lr = 0.01

def funcB():
    global lr
    lr = 0.1

def funcC():
    global lr
    lr = 1.0

# 2 行をコメントアウトして学習率を決めてください
# ---------------------------
# funcA()
funcB()
# funcC()
# ---------------------------

sgd = optimizers.SGD(lr=lr)

model.compile(optimizer=sgd, loss="categorical_crossentropy", metrics=["accuracy"])

model.fit(X_train, y_train, batch_size=32, epochs=3, verbose=1)

score = model.evaluate(X_test, y_test, verbose=0)
print("evaluate loss: {0[0]}\nevaluate acc: {0[1]}".format(score))<jupyter_output>Epoch 1/3
6000/6000 [==============================] - 1s 188us/step - loss: 1.7546 - acc: 0.4112
Epoch 2/3
6000/6000 [==============================] - 1s 126us/step - loss: 1.0164 - acc: 0.6782
Epoch 3/3
6000/6000 [==============================] - 1s 128us/step - loss: 0.8506 - acc: 0.7350
evaluate loss: 0.7203813996315003
evaluate acc: 0.807<jupyter_text>**リスト20.7：解答例（出力結果は誌面と異なります）**  ●20.8 ミニバッチ学習  ○2.8.1 ミニバッチ学習  □問題<jupyter_code>import numpy as np
import matplotlib.pyplot as plt
from keras.datasets import mnist
from keras.layers import Activation, Dense, Dropout
from keras.models import Sequential, load_model
from keras import optimizers
from keras.utils.np_utils import to_categorical

(X_train, y_train), (X_test, y_test) = mnist.load_data()

X_train = X_train.reshape(X_train.shape[0], 784)[:6000]
X_test = X_test.reshape(X_test.shape[0], 784)[:1000]
y_train = to_categorical(y_train)[:6000]
y_test = to_categorical(y_test)[:1000]

model = Sequential()
model.add(Dense(256, input_dim=784))
model.add(Activation("sigmoid"))
model.add(Dense(128))
model.add(Activation("sigmoid"))
model.add(Dropout(rate=0.5))
model.add(Dense(10))
model.add(Activation("softmax"))

sgd = optimizers.SGD(lr=0.1)

model.compile(optimizer=sgd, loss="categorical_crossentropy", metrics=["accuracy"])

def funcA():
    global batch_size
    batch_size = 16

def funcB():
    global batch_size
    batch_size = 32

def funcC():
    global batch_size
    batch_size = 64

# 2 行をコメントアウトして batch_size を決めてください
# ---------------------------
# batch_size: 16
funcA()
# batch_size: 32
funcB()
# batch_size: 64
funcC()
# ---------------------------

model.fit(X_train, y_train, batch_size=batch_size, epochs=3, verbose=1)

score = model.evaluate(X_test, y_test, verbose=0)
print("evaluate loss: {0[0]}\nevaluate acc: {0[1]}".format(score))<jupyter_output><empty_output><jupyter_text>**リスト20.8：問題**  □解答例<jupyter_code>import numpy as np
import matplotlib.pyplot as plt
from keras.datasets import mnist
from keras.layers import Activation, Dense, Dropout
from keras.models import Sequential, load_model
from keras import optimizers
from keras.utils.np_utils import to_categorical

(X_train, y_train), (X_test, y_test) = mnist.load_data()

X_train = X_train.reshape(X_train.shape[0], 784)[:6000]
X_test = X_test.reshape(X_test.shape[0], 784)[:1000]
y_train = to_categorical(y_train)[:6000]
y_test = to_categorical(y_test)[:1000]

model = Sequential()
model.add(Dense(256, input_dim=784))
model.add(Activation("sigmoid"))
model.add(Dense(128))
model.add(Activation("sigmoid"))
model.add(Dropout(rate=0.5))
model.add(Dense(10))
model.add(Activation("softmax"))

sgd = optimizers.SGD(lr=0.1)

model.compile(optimizer=sgd, loss="categorical_crossentropy", metrics=["accuracy"])

def funcA():
    global batch_size
    batch_size = 16

def funcB():
    global batch_size
    batch_size = 32

def funcC():
    global batch_size
    batch_size = 64

# 2 行をコメントアウトして batch_size を決めてください
# ---------------------------
# batch_size: 16
# funcA()import numpy as np
import matplotlib.pyplot as plt
from keras.datasets import mnist
from keras.layers import Activation, Dense, Dropout
from keras.models import Sequential, load_model
from keras import optimizers
from keras.utils.np_utils import to_categorical

(X_train, y_train), (X_test, y_test) = mnist.load_data()

X_train = X_train.reshape(X_train.shape[0], 784)[:6000]
X_test = X_test.reshape(X_test.shape[0], 784)[:1000]
y_train = to_categorical(y_train)[:6000]
y_test = to_categorical(y_test)[:1000]

model = Sequential()
model.add(Dense(256, input_dim=784))
model.add(Activation("sigmoid"))
model.add(Dense(128))
model.add(Activation("sigmoid"))
model.add(Dropout(rate=0.5))
model.add(Dense(10))
model.add(Activation("softmax"))

sgd = optimizers.SGD(lr=0.1)

model.compile(optimizer=sgd, loss="categorical_crossentropy", metrics=["accuracy"])

def funcA():
    global batch_size
    batch_size = 16

def funcB():
    global batch_size
    batch_size = 32

def funcC():
    global batch_size
    batch_size = 64

# 2 行をコメントアウトして batch_size を決めてください
# ---------------------------
# batch_size: 16
# funcA()
# batch_size: 32
# funcB()
# batch_size: 64
funcC()
# ---------------------------

model.fit(X_train, y_train, batch_size=batch_size, epochs=3, verbose=1)

score = model.evaluate(X_test, y_test, verbose=0)
print("evaluate loss: {0[0]}\nevaluate acc: {0[1]}".format(score))<jupyter_output>Epoch 1/3
6000/6000 [==============================] - 1s 147us/step - loss: 1.8868 - acc: 0.3663
Epoch 2/3
6000/6000 [==============================] - 1s 96us/step - loss: 1.1446 - acc: 0.6557
Epoch 3/3
6000/6000 [==============================] - 1s 83us/step - loss: 0.8448 - acc: 0.7617
evaluate loss: 0.6808103404045105
evaluate acc: 0.829<jupyter_text>**リスト20.9：解答例（出力結果は誌面と異なります）**   ●20.9 反復学習  ○20.9.1 反復学習  □問題<jupyter_code>import numpy as np
import matplotlib.pyplot as plt
from keras.datasets import mnist
from keras.layers import Activation, Dense, Dropout
from keras.models import Sequential, load_model
from keras import optimizers
from keras.utils.np_utils import to_categorical

(X_train, y_train), (X_test, y_test) = mnist.load_data()

X_train = X_train.reshape(X_train.shape[0], 784)[:1500]
X_test = X_test.reshape(X_test.shape[0], 784)[:6000]
y_train = to_categorical(y_train)[:1500]
y_test = to_categorical(y_test)[:6000]

model = Sequential()
model.add(Dense(256, input_dim=784))
model.add(Activation("sigmoid"))
model.add(Dense(128))
model.add(Activation("sigmoid"))
# ここではDropoutを使いません
#model.add(Dropout(rate=0.5))
model.add(Dense(10))
model.add(Activation("softmax"))

sgd = optimizers.SGD(lr=0.1)

model.compile(optimizer=sgd, loss="categorical_crossentropy", metrics=["accuracy"])

def funcA():
    global epochs
    epochs = 5

def funcB():
    global epochs
    epochs = 10

def funcC():
    global epochs
    epochs = 60

# 2 行をコメントアウトしてエポック数を決めてください
# ---------------------------
# epochs: 5
funcA()
# epochs: 10
funcB()
# epochs: 60
funcC()
# ---------------------------

history = model.fit(X_train, y_train, batch_size=32, epochs=epochs, verbose=1, validation_data=(X_test, y_test))

#acc、val_accのプロット
plt.plot(history.history["acc"], label="acc", ls="-", marker="o")
plt.plot(history.history["val_acc"], label="val_acc", ls="-", marker="x")
plt.ylabel("accuracy")
plt.xlabel("epoch")
plt.legend(loc="best")
plt.show()

score = model.evaluate(X_test, y_test, verbose=0)
print("evaluate loss: {0[0]}\nevaluate acc: {0[1]}".format(score))<jupyter_output><empty_output><jupyter_text>**リスト20.10：問題**  □解答例<jupyter_code>import numpy as np
import matplotlib.pyplot as plt
from keras.datasets import mnist
from keras.layers import Activation, Dense, Dropout
from keras.models import Sequential, load_model
from keras import optimizers
from keras.utils.np_utils import to_categorical

(X_train, y_train), (X_test, y_test) = mnist.load_data()

X_train = X_train.reshape(X_train.shape[0], 784)[:1500]
X_test = X_test.reshape(X_test.shape[0], 784)[:6000]
y_train = to_categorical(y_train)[:1500]
y_test = to_categorical(y_test)[:6000]

model = Sequential()
model.add(Dense(256, input_dim=784))
model.add(Activation("sigmoid"))
model.add(Dense(128))
model.add(Activation("sigmoid"))
# ここではDropoutを使いません
#model.add(Dropout(rate=0.5))
model.add(Dense(10))
model.add(Activation("softmax"))

sgd = optimizers.SGD(lr=0.1)

model.compile(optimizer=sgd, loss="categorical_crossentropy", metrics=["accuracy"])

def funcA():
    global epochs
    epochs = 5

def funcB():
    global epochs
    epochs = 10

def funcC():
    global epochs
    epochs = 60

# 2 行をコメントアウトしてエポック数を決めてください
# ---------------------------
# epochs: 5
# funcA()
# epochs: 10
funcB()
# epochs: 60
# funcC()
# ---------------------------

history = model.fit(X_train, y_train, batch_size=32, epochs=epochs, verbose=1, validation_data=(X_test, y_test))

#acc、val_accのプロット
plt.plot(history.history["acc"], label="acc", ls="-", marker="o")
plt.plot(history.history["val_acc"], label="val_acc", ls="-", marker="x")
plt.ylabel("accuracy")
plt.xlabel("epoch")
plt.legend(loc="best")
plt.show()

score = model.evaluate(X_test, y_test, verbose=0)
print("evaluate loss: {0[0]}\nevaluate acc: {0[1]}".format(score))<jupyter_output>Train on 1500 samples, validate on 6000 samples
Epoch 1/10
1500/1500 [==============================] - 1s 497us/step - loss: 2.0076 - acc: 0.3667 - val_loss: 1.7191 - val_acc: 0.5618
Epoch 2/10
1500/1500 [==============================] - 0s 310us/step - loss: 1.4583 - acc: 0.6667 - val_loss: 1.3321 - val_acc: 0.6878
Epoch 3/10
1500/1500 [==============================] - 0s 252us/step - loss: 1.0898 - acc: 0.7707 - val_loss: 1.0424 - val_acc: 0.7233
Epoch 4/10
1500/1500 [==============================] - 0s 254us/step - loss: 0.8779 - acc: 0.7993 - val_loss: 0.9073 - val_acc: 0.7435
Epoch 5/10
1500/1500 [==============================] - 0s 247us/step - loss: 0.7484 - acc: 0.8300 - val_loss: 0.8399 - val_acc: 0.7772
Epoch 6/10
1500/1500 [==============================] - 0s 248us/step - loss: 0.6766 - acc: 0.8520 - val_loss: 0.7846 - val_acc: 0.7737
Epoch 7/10
1500/1500 [==============================] - 0s 237us/step - loss: 0.6222 - acc: 0.8573 - val_loss: 0.7558 - val_acc: 0.7782
[...]<jupyter_text>**リスト20.11：解答例（出力結果は誌面と異なります）**　　  ●添削問題  □問題<jupyter_code>import numpy as np
import matplotlib.pyplot as plt
from keras.datasets import mnist
from keras.layers import Activation, Dense, Dropout
from keras.models import Sequential, load_model
from keras import optimizers
from keras.utils.np_utils import to_categorical

(X_train, y_train), (X_test, y_test) = mnist.load_data()

X_train = X_train.reshape(X_train.shape[0], 784)[:6000]
X_test = X_test.reshape(X_test.shape[0], 784)[:1000]
y_train = to_categorical(y_train)[:6000]
y_test = to_categorical(y_test)[:1000]

#---------------------------
model = Sequential()
model.add(Dense(256, input_dim=784))
model.add(Activation("sigmoid"))
model.add(Dense(128))
model.add(Activation("sigmoid"))
model.add(Dropout(rate=0.5))
model.add(Dense(10))
model.add(Activation("softmax"))

sgd = optimizers.SGD(lr=0.1)
model.compile(optimizer=sgd, loss="categorical_crossentropy", metrics=["accuracy"])

model.fit(X_train, y_train, batch_size=10, epochs=5, verbose=1)
score = model.evaluate(X_test, y_test, verbose=0)
print("evaluate loss: {0[0]}\nevaluate acc: {0[1]}".format(score))
#---------------------------<jupyter_output><empty_output><jupyter_text>**リスト20.12：問題**  □解答例<jupyter_code>import numpy as np
import matplotlib.pyplot as plt
from keras.datasets import mnist
from keras.layers import Activation, Dense, Dropout
from keras.models import Sequential, load_model
from keras import optimizers
from keras.utils.np_utils import to_categorical

(X_train, y_train), (X_test, y_test) = mnist.load_data()

X_train = X_train.reshape(X_train.shape[0], 784)[:6000]
X_test = X_test.reshape(X_test.shape[0], 784)[:1000]
y_train = to_categorical(y_train)[:6000]
y_test = to_categorical(y_test)[:1000]

#---------------------------
model = Sequential()
model.add(Dense(256, input_dim=784))
model.add(Activation("sigmoid"))
model.add(Dense(128))
model.add(Activation("sigmoid"))
model.add(Dropout(rate=0.5))
model.add(Dense(10))
model.add(Activation("softmax"))

sgd = optimizers.SGD(lr=0.1)
model.compile(optimizer=sgd, loss="categorical_crossentropy", metrics=["accuracy"])

model.fit(X_train, y_train, batch_size=96, epochs=5, verbose=1)
score = model.evaluate(X_test, y_test, verbose=0)
print("evaluate loss: {0[0]}\nevaluate acc: {0[1]}".format(score))
#---------------------------<jupyter_output>Epoch 1/5
6000/6000 [==============================] - 1s 110us/step - loss: 2.0149 - acc: 0.2995
Epoch 2/5
6000/6000 [==============================] - 0s 81us/step - loss: 1.3310 - acc: 0.5955
Epoch 3/5
6000/6000 [==============================] - 0s 74us/step - loss: 0.9829 - acc: 0.7138
Epoch 4/5
6000/6000 [==============================] - 0s 73us/step - loss: 0.7943 - acc: 0.7763: 0s - loss: 0.8733 - acc
Epoch 5/5
6000/6000 [==============================] - 0s 65us/step - loss: 0.6871 - acc: 0.8143
evaluate loss: 0.595065972328186
evaluate acc: 0.857
{'fraction_non_alphanumeric': 0.20696991617377983, 'fraction_numerical': 0.07237989872771372, 'mean_word_length': 4.928859737638748, 'pattern_counts': {'":': 0, '<': 47, '<?xml version=': 0, '>': 47, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '13', 'dir': 'jupyter-structured-clean-dedup', 'id': '394522', 'license': 'CC0-1.0', 'n_tokens_mistral': 10998, 'n_tokens_neox': 9853, 'n_words': 1713, 'path': 'notebooks/ShinsouGakushu_sample/Chapter20-Sample.ipynb', 'repo_name': 'BlueMoon55/jekyll_python', 'size': '23501'}