<jupyter_start><jupyter_text>利用PCA来简化数据**降维（dimensionality reduction）**在低维下，数据更容易处理。我们在应用机器学习算法之前，必须先识别出其相关的特征。 降维技术* **PCA（Principal Component Analysis）主成分分析**将原坐标系转换，第一个坐标轴选择方差最大的方向，第二个坐标系选择与第一个坐标系正交（orthogonal）且方差最大的方向，以此类推。* **FA（Factor Analysis）因子分析**假设数据是隐变量（latent variable）和某些噪声的线性组合。* **ICA（Independent Component Analysis）独立成分分析**假设数据是有N个数据源生成的，这些数据源之间相互独立。  在Numpy中实现PCA伪代码：        去除平均值        计算协方差矩阵        计算协方差矩阵的特征值和特征向量        将特征值从大到小排序        保留最上面的N个特征向量        将数据转换到上述N个特征向量构建的新空间中<jupyter_code>from numpy import *
def load_dataset(filename, delim='\t'):
    fr = open(filename)
    str_arr = [line.strip().split(delim) for line in fr.readlines()]
    data_arr = [list(map(float, line)) for line in str_arr]
    return mat(data_arr)
datamat = load_dataset('testSet.txt')
datamat
def pca(datamat, topNfeat=9999999):
    mean_val = mean(datamat, axis=0)
    mean_removed = datamat - mean_val
    covmat = cov(mean_removed, rowvar=0)
    print('covmat:\n',covmat)
    # covmat = (mean_removed.T * mean_removed) / len(mean_removed[:, 0])
    eig_vals, eig_vec = linalg.eig(covmat)
    print('eig_vals:\n',eig_vals)
    eig_vals_index = argsort(eig_vals)
    eig_vals_index = eig_vals_index[:-(topNfeat+1):-1]
    print('eig_vals_index:\n',eig_vals_index)
    reg_eig_vec = eig_vec[:, eig_vals_index]
    print('reg_eig_vec:\n',reg_eig_vec)
    low_datamat = mean_removed * reg_eig_vec
    reconmat = (low_datamat * reg_eig_vec.T) + mean_val
    return low_datamat, reconmat
lowmat, reconmat = pca(datamat, 1)
shape(lowmat)
import matplotlib.pyplot as plt
fig = plt.figure()
ax = plt.subplot(111)
# print('datamat[:, 1].T.A:\n',datamat[:, 1].T.A)
ax.scatter(datamat[:, 0].T.A[0], datamat[:, 1].T.A[0], marker='s', s=10)
ax.scatter(reconmat[:, 0].T.A[0], reconmat[:, 1].T.A[0], marker='o', s=10)
lowmat, reconmat = pca(datamat, 2)
fig.clf()
ax = plt.subplot(111)
ax.scatter(datamat[:, 0].T.A[0], datamat[:, 1].T.A[0], marker='s', s=10)
ax.scatter(reconmat[:, 0].T.A[0], reconmat[:, 1].T.A[0], marker='o', s=10)<jupyter_output><empty_output><jupyter_text>示例：利用PCA对半导体制造数据降维  处理数据中的缺失值* 使用可用特征的均值填补* 使用特殊值来填补，如0，-1* 忽略有缺失值的样本* 使用相似样本的均值填补* 使用另外的机器学习预测缺失值在这里，我们用平均值代替。<jupyter_code>def replace_NaN_with_mean():
    datamat = load_dataset('secom.data', ' ')
    numfeat = shape(datamat)[1]
    for i in range(numfeat):
#         print('~isnan(datamat[:, i]).A:\n', ~isnan(datamat[:, i]))
        mean_val = mean(datamat[nonzero(~isnan(datamat[:, i]).A)[0], i])
        datamat[nonzero(isnan(datamat[:, i]).A)[0], i] = mean_val
    return datamat
datamat = replace_NaN_with_mean()
mean_val = mean(datamat, axis=0)
mean_removed = datamat - mean_val
covmat = cov(mean_removed, rowvar=0)
eig_val, eig_vec = linalg.eig(covmat)
eig_val
eig_val_index = argsort(eig_val)
eig_val_index = eig_val_index[::-1]#reverse
sorted_eig_vals = eig_val[eig_val_index]
total = sum(sorted_eig_vals)
varPercentage = sorted_eig_vals / total * 100

fig = plt.figure()
ax = fig.add_subplot(111)
ax.plot(range(1, 21), varPercentage[:20], marker='^')
plt.xlabel('Principal Component Number')
plt.ylabel('Percentage of Variance')<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.15977084659452578, 'fraction_numerical': 0.020687460216422662, 'mean_word_length': 5.6308016877637135, 'pattern_counts': {'":': 0, '<': 9, '<?xml version=': 0, '>': 9, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 1, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '2', 'dir': 'jupyter-structured-clean-dedup', 'id': '489683', 'license': 'MIT', 'n_tokens_mistral': 1536, 'n_tokens_neox': 1618, 'n_words': 202, 'path': 'PCA/PCA.ipynb', 'repo_name': 'Dream-Fu/ML_In_Action', 'size': '3142'}