<jupyter_start><jupyter_text>**STK-IN4300, <NAME>**This assignment with code might also be found at my **[github](https://github.com/simehaa/stk-in4300/tree/master/oblig1)**. Exercise 1Contrast the ability of a lasso and a ridge regression model to predict the Duke CAD index (CADi) from the gene expression. AlgorithmThe data file which is to be analysed contains 110 patients **with** Coronary Artery Disease.The X-data contains 22283 various gene expressions, $X\in \mathbb{R}^{110 \times 22,283}$.The y-data contains the true CADi value , $y \in \mathbb{R}^{110 \times 1}$.I have written a python code, which uses the packages from Scikit-learn to perform Ridge and LASSO regression. The script to solve the exercise is presented. I will provide explainatory information between the code blocks as well as comments in the actual code. To begin, I will do the necessary imports:<jupyter_code>%matplotlib inline
import matplotlib.pyplot as plt # for plotting
import pandas as pd # for reading csv file
import numpy as np # array/linear algebra library
import os # convenient when reading/writing files to other directories
from sklearn.linear_model import LassoCV, RidgeCV # Lasso and ridge with cross validation
from sklearn.model_selection import train_test_split # simple split of train and test set
from sklearn.preprocessing import scale # scaling of the data with both mean and std<jupyter_output><empty_output><jupyter_text>Definition of functionsThe two following functions are meant to easily read a file in the **./data/** directory and to plot/save figures to the **./figures/** directory.<jupyter_code>def read_file(filename):
    """Read a csv data file located in the ./data/ directory
    Return design matrix X and y-data"""
    DATA_DIR = "./data"
    if not os.path.exists(DATA_DIR):
        os.mkdir(DATA_DIR)
    DATA_FILE = os.path.join(DATA_DIR,filename)
    df = pd.read_csv(DATA_FILE).values
    # 0th column contains file names
    y = np.array(df[:,1]) # 1st column is y-data
    X = np.array(df[:,2:]) # remaining columns is the X-data
    return X, y


def save_fig(y_test, y_pred, score, fig_id):
    """Save a matplotlib figure to the ./figure/ directory.
    If/else statements ask the user if they wish to overwrite an
    already existing figure."""
    # Plot
    l = y_test.shape[0]
    x = np.linspace(1,l,l)
    # plot of y_pred vs. y_test:
    plt.plot(y_test, y_pred, 'ro', label=fr"prediction, $R^2 = ${score:2.2f}")
    # plot of the ideal linear case
    min = np.min(y_test)
    max = np.max(y_test)
    plt.plot([min,max],[min,max])
    # plot options
    plt.xlabel(r"$y$")
    plt.ylabel(r"$\hat{y}$")
    plt.legend()
    plt.grid()
    # Saving of figure
    FIGURE_DIR = "./figures"
    if not os.path.exists(FIGURE_DIR):
        os.mkdir(FIGURE_DIR)
    FIGURE_FILE = os.path.join(FIGURE_DIR, fig_id)
    if os.path.exists(FIGURE_FILE):
        overwrite = str(input("The file \"" + FIGURE_FILE + \
            "\" already exists, do you wish to overwrite it [y/n]?\n"))
        if overwrite == "y":
            plt.savefig(FIGURE_FILE, format="png")
            plt.close()
            print("Figure was overwritten.")
        else:
            print("Figure was not saved.")
    else:
        plt.savefig(FIGURE_FILE, format="png")
        plt.close()
    return None<jupyter_output><empty_output><jupyter_text>The purpose of the main function is to use scikit-learn to perform Ridge regression and LASSO regression. Note that X and y are scaled so that both mean = 0 and std = 1.<jupyter_code>def main():
    """
    Info:
    Perform Ridge and Lasso regression on gene data. Try to predict
    The output which is the Duke CAD index
    (Coronary Artery Disease index).

    Notes:
    Data file "data_E1.csv" contains the 110 pasients with case:
    * The first column contains file IDs (not necessary for analysis)
    * the second column contains the y data: true CADi
    * the remaining columns contains 22283 features
    """
    # Read data from file
    X, y = read_file("data_E1.csv")
    # Scale data with both mean and std: e.g. y = y/std(y) - mean(y)
    std = np.std(y)
    mean = np.mean(y)
    X = scale(X)
    y = scale(y)
    X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.20)
    
    # Ridge (fit_intercept=False assumes that data is already scaled)
    clf_r = RidgeCV(fit_intercept=False).fit(X_train, y_train)
    y_pred_r = clf_r.predict(X_test)
    score_r = clf_r.score(X_test, y_test)
    y_test_r = y_test*std + mean
    y_pred_r = y_pred_r*std + mean
    # Plot and save figure
    save_fig(y_test_r, y_pred_r, score_r, "ridge.png")
   
    # Lasso (fit_intercept=False assumes that data is already scaled
    # Note: cv=5 is the default of a future version of scikit-learn
    clf_l = LassoCV(fit_intercept=False, cv=5).fit(X_train, y_train)
    y_pred_l = clf_l.predict(X_test)
    score_l = clf_l.score(X_test, y_test)
    y_test_l = y_test*std + mean
    y_pred_l = y_pred_l*std + mean
    # Plot and save figure
    save_fig(y_test_l, y_pred_l, score_l, "lasso.png")<jupyter_output><empty_output><jupyter_text>Run the main function:<jupyter_code>if __name__ == "__main__":
    np.random.seed(1) # affect the random splitting of train/test set: same seed obtain the same results each run
    main() # call the main function where the regressions are performed

# Test run, information about versions:
# python 3.7.3
# scikit-learn 0.21.3
# numpy 1.17.2
# pandas 0.25.1
# matplotlib 3.1.1
"""
$ python main.py
The file "./figures/ridge.png" already exists, do you wish to overwrite it [y/n]?
y
Figure was overwritten.
The file "./figures/lasso.png" already exists, do you wish to overwrite it [y/n]?
y
Figure was overwritten.
"""<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.10227075749696654, 'fraction_numerical': 0.013000520020800831, 'mean_word_length': 3.4763382467028703, 'pattern_counts': {'":': 2, '<': 18, '<?xml version=': 0, '>': 18, 'https://': 1, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '4', 'dir': 'jupyter-structured-clean-dedup', 'id': '464185', 'license': 'MIT', 'n_tokens_mistral': 1968, 'n_tokens_neox': 1882, 'n_words': 643, 'path': 'stk-in4300/oblig1/Mandatory Assignment 1.ipynb', 'repo_name': 'simehaa/MachineLearning', 'size': '5775'}