<jupyter_start><jupyter_text>PCA for Large Datasets   In this lab, you will calculate the covariance matrix for datasets with a large number of samples; then, you will use PCA to transform the datase 1- Preparation Download eigenvectors and eigenvalues:<jupyter_code>! wget https://s3-api.us-geo.objectstorage.softlayer.net/cf-courses-data/CognitiveClass/meet_up/12.02.2020/eigenvectors.pt
! wget https://s3-api.us-geo.objectstorage.softlayer.net/cf-courses-data/CognitiveClass/meet_up/12.02.2020/eigenvalues.pt
!pip install Pillow==6.2.2<jupyter_output><empty_output><jupyter_text>We'll need the following libraries:<jupyter_code># Import the libraries we need for this lab

# Using the following line code to install the torchvision library
# !conda install -y torchvision

import torch 
import torch.nn as nn
import torchvision.transforms as transforms
import torchvision.datasets as dsets
import matplotlib.pylab as plt
import numpy as np<jupyter_output><empty_output><jupyter_text>Use the following function to visualize data:<jupyter_code># Display data

def show_data(data_sample,y,raw_image=True):
    if raw_image:
        plt.imshow(data_sample[0].numpy().reshape(16, 16), cmap='gray')
    else:
        plt.imshow(data_sample[0].numpy().reshape(16, 16))
    plt.title('y = ' + str(y.item()))<jupyter_output><empty_output><jupyter_text>PCA transform class.<jupyter_code>class transform(object):
    def __init__(self, eigenvalues,eigenvectors):
        self.eigenvalues=eigenvalues
        self.eigenvectors=eigenvectors
        #calculate the diagonal matrix of eigenvalues
        dim=eigenvalues[:,0].shape[0]
        diag=torch.eye(dim)
        dim=eigenvalues[:,0].shape[0]
        self.diag=torch.eye(dim)
        
        for n,eigenvalue in enumerate(eigenvalues[:,0]):
            self.diag[n,n]=(eigenvalue+0.01)**(0.5)
            
        
        self.Qin=torch.inverse(diag)
    def PCA(self,X):
        X_hat=torch.mm(X[0].view(1,-1),self.eigenvectors)
        return X_hat<jupyter_output><empty_output><jupyter_text>The image is a rectangular  tensor<jupyter_code>IMAGE_SIZE = 16
composed = transforms.Compose([transforms.Resize((IMAGE_SIZE, IMAGE_SIZE)), transforms.ToTensor()])<jupyter_output><empty_output><jupyter_text>2- Load Data  Load the training dataset by setting the parameters train to True and convert it to a tensor by placing a transform object in the argument transform.<jupyter_code># Create and print the training dataset

train_dataset = dsets.MNIST(root='./data', train=True, download=True, transform=composed )
validation_dataset = dsets.MNIST(root='./data', train=False, download=True, transform=composed)<jupyter_output><empty_output><jupyter_text>the image is a rectangler tensors   We can covert the tensor to a 1D tensor of vector and perform PCA or ZCA.  In this cell will calculate the Covariance Matrix and save it. We do it in a way that we don't need to store all the samples in memory. This takes some time so you can load the results in the next cell. dim=train_dataset[0][0].shape[1]*train_dataset[0][0].shape[2]mean=torch.zeros((1,dim))C=torch.zeros((dim,dim))N_samples=len(train_dataset)for n in range(N_samples):    mean=mean+train_dataset[0][0].view(-1,1)    mean=mean/N_samplesfor n in range(N_samples):    x=train_dataset[0][0].view(1,-1)    x=x-mean        C+=torch.mm(torch.t(x),x)    C=C/N_sampleseigenvalues,eigenvectors=torch.eig(C,True)torch.save(eigenvalues, 'eigenvalues.pt') torch.save(eigenvectors, 'eigenvectors.pt') Load eigenvalues and eigenvectors.<jupyter_code>eigenvalues=torch.load('eigenvalues.pt')
eigenvectors=torch.load('eigenvectors.pt')
show_data(train_dataset[0][0],train_dataset[0][1])<jupyter_output><empty_output><jupyter_text>PCA transform object<jupyter_code>transform=transform(eigenvalues,eigenvectors)<jupyter_output><empty_output><jupyter_text>We can calculate the PCA transform of the image. We can see the transform for each digit looks similar.   Try changing the variable select_number to change the number.<jupyter_code>j=0
#select_number from 0 -9
select_number= 2
for i,(X,y) in enumerate(train_dataset):
    if y.item()==select_number:
        j+=1
        Xhat=transform.PCA(X)
        show_data(torch.log(Xhat+1),y=y,raw_image=False)
        plt.show()
    if j==5:
        break<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.11347846225104215, 'fraction_numerical': 0.017369152385363594, 'mean_word_length': 5.134943181818182, 'pattern_counts': {'":': 0, '<': 37, '<?xml version=': 0, '>': 37, 'https://': 2, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 3, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '9', 'dir': 'jupyter-structured-clean-dedup', 'id': '436426', 'license': 'MIT', 'n_tokens_mistral': 1497, 'n_tokens_neox': 1438, 'n_words': 327, 'path': 'edx-ibm-dl-pytorch/9.5.3_PCA_image.ipynb', 'repo_name': 'deehzee/colabs', 'size': '4318'}