<jupyter_start><jupyter_text>Midterm test and practice session  1. Questions.Please, answer the following questions briefly. Two or three sentences with main idea would be enough.Do not use external resourses in this part, please. Answer with you own words. If you forgot something, don't worry, we will discuss it later.  1.0. Please, formulate the supervised learning problem statement._Your answer here_  1.1.What are regression and classification problems. What’s the difference?_Your answer here_  1.2.Write down the linear model for regression problem in matrix notation. What is Mean Squared Error (MSE) loss function? How can it be expressed?_Your answer here_  1.3.What is the gradient of a function? How is it being used in optimization?_Your answer here_  1.4.Write down gradient descent step for linear model and MSE for one-dimensional case._Your answer here_  1.5.What is validation? Cross validation?_Your answer here_  1.6.What is regularization? How does L1 regularization differ from L2 for linear models?_Your answer here_  1.7.What are precision and recall metrics?_Your answer here_  1.8.What is bagging? What is the main idea beneath it?_Your answer here_  2. Tackling Machine Learning problems in the wildNow you will work with real data in classification problem. Your goal it to solve (with some quality) and make some conclusions. It's quite similar to the `assignment0_02`. You may use external resources here.  2.0 Reading the dataToday we work with the [dataset](https://archive.ics.uci.edu/ml/datasets/Statlog+%28Vehicle+Silhouettes%29), describing different cars for multiclass ($k=4$) classification problem. The data is available below.<jupyter_code># If on colab, uncomment the following lines

# !wget https://raw.githubusercontent.com/neychev/harbour_ml2020/master/assignments/assignment_Midterm/car_data.csv
import pandas as pd
import numpy as np
import matplotlib.pyplot as plt
from sklearn.model_selection import train_test_split

dataset = pd.read_csv('car_data.csv', delimiter=',', header=None).values
data = dataset[:, :-1].astype(int)
target = dataset[:, -1]

print(data.shape, target.shape)

X_train, X_test, y_train, y_test = train_test_split(data, target, test_size=0.35)
print(X_train.shape, y_train.shape, X_test.shape, y_test.shape)<jupyter_output>(846, 19) (846,)
(549, 19) (549,) (297, 19) (297,)<jupyter_text>To get some insights about the dataset, `pandas` might be used. The `train` part is transformed to `pd.DataFrame` below.<jupyter_code>X_train_pd = pd.DataFrame(X_train)

# First 15 rows of our dataset.
X_train_pd.head(15)<jupyter_output><empty_output><jupyter_text>Methods `describe` and `info` deliver some useful information.<jupyter_code>X_train_pd.describe()
np.unique(y_train)
X_train_pd.info()<jupyter_output><class 'pandas.core.frame.DataFrame'>
RangeIndex: 549 entries, 0 to 548
Data columns (total 19 columns):
0     549 non-null int64
1     549 non-null int64
2     549 non-null int64
3     549 non-null int64
4     549 non-null int64
5     549 non-null int64
6     549 non-null int64
7     549 non-null int64
8     549 non-null int64
9     549 non-null int64
10    549 non-null int64
11    549 non-null int64
12    549 non-null int64
13    549 non-null int64
14    549 non-null int64
15    549 non-null int64
16    549 non-null int64
17    549 non-null int64
18    549 non-null int64
dtypes: int64(19)
memory usage: 81.6 KB<jupyter_text>2.1. Data preprocessing* Make some transformations of the dataset (if necessary). Briefly explain the transformations<jupyter_code>from sklearn.preprocessing import StandardScaler
### YOUR CODE HERE<jupyter_output><empty_output><jupyter_text>2.2. PCA: explained variance plot* Apply the PCA to the train part of the data. Build the explaided variance plot (like in assignment 2).<jupyter_code>from sklearn.decomposition import PCA

### YOUR CODE HERE<jupyter_output><empty_output><jupyter_text>2.3. PCA trasformation* Select the appropriate number of components. Use `fit` and `transform` methods to transform the `train` and `test` parts.<jupyter_code>### YOUR CODE HERE<jupyter_output><empty_output><jupyter_text>2.4. Logistic regression* Find the optimal hyperparameters for logistic regression using cross-validation (e.g. `GridSearchCV`). You can vary only a parameter for `l2` regularization (e.g 5 different values).* Then build a ROC curve for this classifier (`sklearn.metrics.roc_curve`). Estimate the model quality with appropriate metrics (which will you use?)_Note: be careful with preprocessing (like scaling/PCA) to avoid data leaks._<jupyter_code>from sklearn.linear_model import LogisticRegression

### YOUR CODE HERE<jupyter_output><empty_output><jupyter_text>2.5. Decision tree* Now train a desicion tree on the same data. Find optimal tree depth (`max_depth`) using cross-validation (again, checking 5 variants would be fine).* Measure the model quality using the same metrics you used above.<jupyter_code>from sklearn.tree import DecisionTreeClassifier

# YOUR CODE HERE<jupyter_output><empty_output><jupyter_text>2.6. Random ForestNow we will work with the Random Forest (its `sklearn` implementation).* Vary the number of trees (from 5 to 50 with step 5) and build the plot in axes "model accuracy" - "number of trees".* What is the optimal number of trees you've got?<jupyter_code>from sklearn.ensemble import RandomForestClassifier

# YOUR CODE HERE<jupyter_output><empty_output><jupyter_text>2.7. Bonus part: "learning curve"* Split the training data into 10 equal (almost) parts. Then train the models from above (Logistic regression, Desicion Tree, Random Forest) with optimal hyperparameters you have selected on 1 part, 2 parts (combined, so the train size in increased by 2 times), 3 parts and so on.* Build a plot of accuracy and f1-score (on `test` part) varying the `train` dataset size (so the axes will be metric - dataset size.* Analyse the final plot. Can you make any conlusions using it?<jupyter_code># YOUR CODE HERE<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.07871914609739826, 'fraction_numerical': 0.0395263509006004, 'mean_word_length': 5.100712105798576, 'pattern_counts': {'":': 0, '<': 40, '<?xml version=': 0, '>': 40, 'https://': 2, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '10', 'dir': 'jupyter-structured-clean-dedup', 'id': '32910', 'license': 'MIT', 'n_tokens_mistral': 1944, 'n_tokens_neox': 1759, 'n_words': 728, 'path': 'assignments/assignment_Midterm/assignment_midterm.ipynb', 'repo_name': 'neychev/harbour_ml2020', 'size': '5996'}