<jupyter_start><jupyter_text>Data<jupyter_code>points = [[0.71, 0.18, 2.50, 0.45, 0.03, 
           0.12, 0.30, 2.65, 0.90, 0.46], 
          [0.14, 0.23, 2.30, 0.17, 0.44,
           0.24, 0.03, 2.10, 0.92, 0.33]]
data = np.array(points).T
data.shape
plt.plot(data[:, 0], data[:, 1], 'o')
plt.grid()<jupyter_output><empty_output><jupyter_text>Move data using mean<jupyter_code>x = data[:, 0]
y = data[:, 1]
x_bar = round(x.mean(), 2)
x_bar
y_bar = round(y.mean(), 2)
y_bar
scaled_x = x - x_bar
scaled_x
scaled_y = y - y_bar
scaled_y
plt.plot(scaled_x, scaled_y, 'o')
plt.grid()<jupyter_output><empty_output><jupyter_text>Covariance matrix $cov(x, y)=cov(y,x)=\Sigma_{1}^{10}\frac{(x_i-\bar{x})(y_i-\bar{y})}{10-1}$<jupyter_code>cov_xy = round(np.sum(scaled_x * scaled_y) / 9, 5)
cov_xy<jupyter_output><empty_output><jupyter_text>$cov(x, x)=var(x)=\Sigma_{1}^{10}\frac{(x_i-\bar{x})(x_i-\bar{x})}{10-1}$<jupyter_code>var_x = round(np.sum(scaled_x * scaled_x) / 9, 5)
var_x<jupyter_output><empty_output><jupyter_text>$cov(y, y)=var(y)=\Sigma_{1}^{10}\frac{(y_i-\bar{y})(y_i-\bar{y})}{10-1}$<jupyter_code>var_y = round(np.sum(scaled_y * scaled_y) / 9, 5)
var_y
np.cov(x, y, ddof=1)<jupyter_output><empty_output><jupyter_text>Eigenvectors of covariance matrix<jupyter_code>e_vals, e_vecs = np.linalg.eig(np.cov(x, y, ddof=1))
e_vals
e_vecs<jupyter_output><empty_output><jupyter_text>첫 번째 eigenvect를 선택한다.<jupyter_code>e_vecs[:, 0]<jupyter_output><empty_output><jupyter_text>Points on principal component<jupyter_code>scaled_data = np.c_[scaled_x, scaled_y]
scaled_data<jupyter_output><empty_output><jupyter_text>Scaled data를 principal components에 투영시키면 새로운 points가 생성된다.<jupyter_code>new_points = np.dot(scaled_data, e_vecs[:, 0])
new_points<jupyter_output><empty_output><jupyter_text>Visualization<jupyter_code>origin = [0], [0]
plt.plot(scaled_x, scaled_y, 'o')
plt.quiver(*origin, e_vecs[:, 0], e_vecs[:, 1], color=['r','b','g'], scale=3)
plt.grid()
# 두 eigenvector의 내적값은 0이다. 즉 90도 이다.
np.dot(e_vecs[:, 0], e_vecs[:, 1])<jupyter_output><empty_output><jupyter_text>Sklearn을 사용하여 결과를 비교해보자.<jupyter_code>from sklearn.decomposition import PCA
pca = PCA(n_components=2)
pca.fit(data)<jupyter_output><empty_output><jupyter_text>Covariance matrix<jupyter_code>pca.get_covariance()<jupyter_output><empty_output><jupyter_text>Pricipal components위로 projection된 데이터<jupyter_code>pca.transform(data)<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.18726746589499793, 'fraction_numerical': 0.0434063662670525, 'mean_word_length': 7.52112676056338, 'pattern_counts': {'":': 0, '<': 53, '<?xml version=': 0, '>': 53, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '13', 'dir': 'jupyter-structured-clean-dedup', 'id': '266252', 'license': 'MIT', 'n_tokens_mistral': 1239, 'n_tokens_neox': 1182, 'n_words': 148, 'path': 'machine-learning-lecture-notes-master/Lecture19_PCA_By_Hands.ipynb', 'repo_name': 'm1016m/NN', 'size': '2419'}