<jupyter_start><jupyter_text>Constructing multiple views to classify singleview dataAs demonstrated in "Asymmetric bagging and random subspace for support vectormachines-based relevance feedback in image retrieval" (Dacheng 2006), in highdimensional data it can be useful to subsample the features and constructmultiple classifiers on each subsample whose individual predictions arecombined using majority vote. This is akin to bagging but concerns thefeatures rather than samples and is how random forests are ensembledfrom individual decision trees. Here, we apply Linear Discriminant Analysis(LDA) to a high dimensional image classification problem and demonstratehow subsampling features can help when the sample size is relatively low.A variety of possible subsample dimensions are considered, and for each thenumber of classifiers (views) is chosen such that their product is equal tothe number of features in the singleview data.Two subsampling methods are applied. The random subspace method simply selectsa random subset of the features. The random Gaussian projection method createsnew features by sampling random multivariate Gaussian vectors used to projectthe original features. The latter method can potentially help in complicatedsettings where combinations of features better capture informative relations.It is clear that subsampling features in this setting leads to improvedout-of-sample accuracy, most likely as it reduces overfitting to the largenumber of raw features. This is confirmed as the accuracy seems to peakaround when the number of features is equal to the number of samples, at whichpoint overfitting becomes possible.<jupyter_code># Author: <NAME>
# License: MIT

import numpy as np
import matplotlib.pyplot as plt
from sklearn.preprocessing import StandardScaler
from sklearn.discriminant_analysis import LinearDiscriminantAnalysis
from sklearn.pipeline import make_pipeline
from sklearn.model_selection import cross_val_score, ShuffleSplit
from sklearn.datasets import fetch_olivetti_faces
from mvlearn.compose import RandomSubspaceMethod, RandomGaussianProjection, \
    ViewClassifier

# Load the singleview Olivevetti faces dataset from sklearn
X, y = fetch_olivetti_faces(return_X_y=True)

# The data has 4096 features. The following subspace dimensions are used
dims = [16, 64, 256, 1024]

# We are interested in when the low sample size, high dimensionality setting
train_size = 0.2
rsm_scores = []
rgp_scores = []

# Initialze cross validation
splitter = ShuffleSplit(n_splits=5, train_size=train_size, random_state=0)

# Compute singleview score, using all dimensions
singleview_clf = make_pipeline(StandardScaler(), LinearDiscriminantAnalysis())
singleview_scores = cross_val_score(singleview_clf, X, y, cv=splitter)

# For each dimension, we compute scores for a multiview classifier
for dim in dims:
    n_views = int(X.shape[1] / dim)

    rsm_clf = make_pipeline(
            StandardScaler(),
            RandomSubspaceMethod(n_views=n_views, subspace_dim=dim),
            ViewClassifier(LinearDiscriminantAnalysis())
        )
    rsm_scores.append(cross_val_score(rsm_clf, X, y, cv=splitter))

    rgp_clf = make_pipeline(
        StandardScaler(),
        RandomGaussianProjection(n_views=n_views, n_components=dim),
        ViewClassifier(LinearDiscriminantAnalysis())
    )
    rgp_scores.append(cross_val_score(rgp_clf, X, y, cv=splitter))

# The results are plotted
fig, ax = plt.subplots()
ax.axvline(X.shape[0] * train_size, ls=':', c='grey',
           label='Number of training samples')
ax.axhline(np.mean(singleview_scores), ls='--', c='grey',
           label='LDA singleview score')
ax.errorbar(
    dims, np.mean(rsm_scores, axis=1),
    yerr=np.std(rsm_scores, axis=1), label='LDA o Random Subspace')
ax.errorbar(
    dims, np.mean(rgp_scores, axis=1),
    yerr=np.std(rgp_scores, axis=1), label='LDA o Random Gaussian Projection')
ax.set_xlabel('Number of subsampled dimensions')
ax.set_ylabel('Score')
plt.title('Classification accuracy using constructed multiview data')
plt.legend()
plt.show()<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.06593406593406594, 'fraction_numerical': 0.007081807081807082, 'mean_word_length': 5.4503937007874015, 'pattern_counts': {'":': 0, '<': 6, '<?xml version=': 0, '>': 6, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '1', 'dir': 'jupyter-structured-clean-dedup', 'id': '10100', 'license': 'MIT', 'n_tokens_mistral': 1174, 'n_tokens_neox': 1109, 'n_words': 419, 'path': '_downloads/3e284c0b2569d71a0b84598b36c8d3c6/plot_multiview_construction.ipynb', 'repo_name': 'mvlearn/mvlearn.github.io', 'size': '4100'}