<jupyter_start><jupyter_text>用 NumPy 就能辨識數字<jupyter_code>from PIL import Image
import numpy as np<jupyter_output><empty_output><jupyter_text>先下載 MNIST 資料<jupyter_code>import os
import urllib
from urllib.request import urlretrieve
dataset = 'mnist.pkl.gz'
def reporthook(a,b,c):
    print("\rdownloading: %5.1f%%"%(a*b*100.0/c), end="")
    
if not os.path.isfile(dataset):
        origin = "https://github.com/mnielsen/neural-networks-and-deep-learning/raw/master/data/mnist.pkl.gz"
        print('Downloading data from %s' % origin)
        urlretrieve(origin, dataset, reporthook=reporthook)
import gzip
import pickle
with gzip.open(dataset, 'rb') as f:
    train_set, validation_set, test_set = pickle.load(f, encoding='latin1')<jupyter_output><empty_output><jupyter_text>Q先看看這些資料是什麼吧！<jupyter_code>%run -i q_see_mnist_data.py<jupyter_output>train_set[0]:      shape=(50000, 784)	 dtype=float32
train_set[1]:      shape=(50000,)	 dtype=int64
validation_set[0]: shape=(10000, 784)	 dtype=float32
validation_set[1]: shape=(10000,)	 dtype=int64
test_set[0]:       shape=(10000, 784)	 dtype=float32
test_set[1]:       shape=(10000,)	 dtype=int64<jupyter_text>Supervised Learning類比：類比：中文房間<jupyter_code>train_X, train_y = train_set
test_X, test_y = test_set<jupyter_output><empty_output><jupyter_text>看一下 MNIST 的 y 是什麼<jupyter_code># 訓練資料， y 的前 20 筆
train_y[:20]<jupyter_output><empty_output><jupyter_text>看一下 MNIST 的 X<jupyter_code>from IPython.display import display
def showX(X):
    int_X = (X*255).clip(0,255).astype('uint8')
    # N*784 -> N*28*28 -> 28*N*28 -> 28 * 28N
    int_X_reshape = int_X.reshape(-1,28,28).swapaxes(0,1).reshape(28,-1)
    display(Image.fromarray(int_X_reshape))
# 訓練資料， X 的前 20 筆
showX(train_X[:20])<jupyter_output><empty_output><jupyter_text>先從簡單的方法開始笨方法，直接比較，找最接近的圖。先試試看使用方差好了  Q 計算 u, v 方差```pythonu = train_X[0]v = train_X[1]```<jupyter_code>%run -i q_square_error.py<jupyter_output>86.9492
86.9491830207<jupyter_text>Q 試著* 顯示 test_X[0]* 在 `train_X` 中找出最像 `test_X[0]` 的圖片編號* display 那張最圖片* 然後查看對應的 `train_y`* 看看 test_y[0]<jupyter_code>%run -i q_find_nn_0.py<jupyter_output><empty_output><jupyter_text>Q拿前面10/100個 `test_X` 做同樣的事情，然後統計一準確度。<jupyter_code>%run -i q_find_nn_10.py<jupyter_output>test_X[0]<jupyter_text>Q如果 train_X 只有 500 筆資料呢？利用 reshaping,  broadcasting 技巧， 算出對 `test_X[:100]` 的準確度！Hint: `np.expand_dims` 來取代 `np.reshape`<jupyter_code># !可能會用掉太多記憶體!
#%run -i q_small_data.py
# accuracy: 85%<jupyter_output><empty_output><jupyter_text>Q用其他距離函數? e.g.  `np.abs(...).sum()`  改用來內積取代方差$$\begin{align*}\left\Vert \mathbf{u}-\mathbf{v}\right\Vert ^{2} & =\left(\mathbf{u}-\mathbf{v}\right)\cdot\left(\mathbf{u}-\mathbf{v}\right)\\ & =\left\Vert \mathbf{u}\right\Vert ^{2}-2\mathbf{u}\cdot\mathbf{v}+\left\Vert \mathbf{v}\right\Vert ^{2}\\\end{align*}$$<jupyter_code># 資料 normalize
train_X  = train_X / np.linalg.norm(train_X, axis=1, keepdims=True)
test_X  = test_X / np.linalg.norm(test_X, axis=1, keepdims=True)
# 矩陣乘法 == 大量計算內積
A = test_X @ train_X.T
print(A.shape)
A.argmax(axis=1)
predict_y = train_y[A.argmax(axis=1)]
# 測試資料， X 的前 20 筆
showX(test_set[0][:20])
# 猜測的 Y 前20筆
predict_y[:20]
#測試資料的 y 前 20 筆
test_y[:20]
# 正確率
(predict_y == test_y).mean()<jupyter_output><empty_output><jupyter_text>用 PCA 降低維度<jupyter_code>from sklearn.decomposition import PCA
pca = PCA(n_components=60)
train_X = pca.fit_transform(train_set[0])
test_X = pca.transform(test_set[0])
train_X.shape
train_X /= np.linalg.norm(train_X, axis=1, keepdims=True)
test_X /= np.linalg.norm(test_X, axis=1, keepdims=True)
# 矩陣乘法
A = test_X @ train_X.T
predict_y = train_y[A.argmax(axis=1)]
# 正確率
(predict_y == test_y).mean()<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.16284779050736498, 'fraction_numerical': 0.04719039825422804, 'mean_word_length': 6.393145161290323, 'pattern_counts': {'":': 0, '<': 46, '<?xml version=': 0, '>': 49, 'https://': 1, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 2, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '12', 'dir': 'jupyter-structured-clean-dedup', 'id': '16320', 'license': 'Apache-2.0', 'n_tokens_mistral': 1907, 'n_tokens_neox': 1872, 'n_words': 244, 'path': 'Basic_ML/01-From NumPy to MNIST.ipynb', 'repo_name': 'tjwei/taishin20190827', 'size': '3666'}