<jupyter_start><jupyter_text>Nyström Kernel PCRIn this notebook we will go through an example of how to use kernel PCA with the Nyström method. We will load some data, calculate the Nyström PCA solution and compare it to kernel ridge regression with the Nyström method. Finally we illustrate the results through some plots. DataLet's load the example dataset we will use. We will use the Boston housing dataset.<jupyter_code>import pandas as pd
from sklearn.datasets import load_boston

data = load_boston()

X = data['data']

y = data['target']

pd.DataFrame(X).describe()<jupyter_output><empty_output><jupyter_text>RegressionLet's run Nyström kernel PCR and compare with standard linear regression and kernel ridge regression.<jupyter_code>from nystrompca import NystromKPCR

from sklearn.linear_model import LinearRegression
from sklearn.kernel_ridge import KernelRidge
from sklearn.model_selection import train_test_split


X_train, X_test, y_train, y_test = train_test_split(X, y)

lin_reg = LinearRegression()
lin_reg.fit(X_train, y_train)

print(f"Linear regression R-squared:\t{lin_reg.score(X_test, y_test):.3f}", )


krr = KernelRidge(kernel='rbf', gamma=1/10000)
krr.fit(X_train, y_train)

print(f"Kernel ridge R-squared:\t\t{krr.score(X_test, y_test):.3f}", )


nystrom_kpcr = NystromKPCR(sigma=100, n_components=100, m_subset=200, scale=False)
nystrom_kpcr.fit(X_train, y_train)

print(f"Nyström KPCR R-squared:\t\t{nystrom_kpcr.score(X_test, y_test):.3f}")<jupyter_output>Linear regression R-squared:	0.746
Kernel ridge R-squared:		0.591
Nyström KPCR R-squared:		0.531<jupyter_text>Nyström KPCR does better than standard kernel ridge regression in terms of the R-squared in this example. For this simple data set with mostly linear relationships in the data a regular linear regression often does best. Recall that an R-squared of 0 implies that a method does as well as just predicting the mean of the target variables. PlotsNow we plot the actual versus predicted target values for the two models.<jupyter_code>import seaborn as sns
import matplotlib.pyplot as plt

%matplotlib inline

sns.set(style='darkgrid')
sns.set_style({'font.family':'serif', 'font.serif':'Times New Roman'})

sns.scatterplot(x=y_test, y=nystrom_kpcr.predict(X_test).squeeze())
sns.scatterplot(x=y_test, y=krr.predict(X_test))

plt.legend(['Nyström KPCR','Kernel Ridge'])
plt.title("Regression comparison")
plt.xlabel("Actual target")
plt.ylabel("Predicted target");<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.09412711182622686, 'fraction_numerical': 0.01246983105390185, 'mean_word_length': 6.401785714285714, 'pattern_counts': {'":': 0, '<': 12, '<?xml version=': 0, '>': 12, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '3', 'dir': 'jupyter-structured-clean-dedup', 'id': '270702', 'license': 'Apache-2.0', 'n_tokens_mistral': 916, 'n_tokens_neox': 824, 'n_words': 250, 'path': 'notebook_examples/nystrom_kernel_pcr.ipynb', 'repo_name': 'fredhallgren/nystrompca', 'size': '2486'}