<jupyter_start><jupyter_text>***Chapter 2 - Conceptual Exercises***  **1(a)** The flexible statistical learning method would perform better than the inflexible method because witha large sample size and inflexible method would perform poorly not generalizing the behavior of the points.**1(b)** The flexible statistical learning method would perform worse than the inflexible method due to the small amount of given observations. With a flexible mode, you can run the risk of overfitting your data.**1(c)** The flexible statistical learning method would perform better because flexible methods generally perform better with predictors and response that are highly non-linear.**1(d)** The flexible statistical learning method would perform better than the inflexible method due to the high varianceof the error terms in the dataset. An inflexible method would not be able to fit the data making it difficult to fit the noise and error terms. **2(a)** This is a regression problem and we are interested in inference of how factors affect the CEO salary. The n (sample size) is 500 and p (predictors/features/variables) is 4.**2(b)** This is a classification problem and we are interested in the prediction of its success/failure. The n (sample size) is 20 and p (predictors/features/variables) is 14.**2(c)** This is a regression problem and we are interested in the prediction of weekly changes. The n (sample size) is 52 and p (predictors/features/variables) is 4.<jupyter_code>#3a
import numpy as np
import matplotlib.pyplot as plt
from matplotlib.image import imread

# On your computer, you will have to copy/paste the full path and assign it to the variable data_dir

data_dir = #FULL PATH HERE - DON'T FORGET QUOTES AND MAKE SURE YOU USE DOUBLE BACK SLASH FOR EVERY SLASH YOU SEE
plt.imshow(imread(data_dir))
plt.xticks([]) 
plt.yticks([]) 
plt.xlabel('Flexibility')
plt.ylabel('MSE')
plt.show()

# Hope you enjoyed my shaky Picasso drawing attempt :)<jupyter_output><empty_output><jupyter_text>**3(b)** The five curves displayed shows the test mse represented in the color black, the train mse represented in the color grey, the variance represented in the color orange, the bias represented in the color light blue, and the irreducible error represented in the color of burgundy. The test mse has this shape because by following the curve we can say that the true f is non-linear due to the fact that the curve of the models increases after it hits its lowest point. The training mse also supports this fact because as the flexibility increases we see that the mse decreases with each model. The variance as seen increases due to the flexiblity of the model and the bias shows a sharp decrease also supporting the fact that the true f is non-linear. Another point shown in the graph is the bias-variance trade-off with the variance increasing and the bias decreasing. More flexible methods usually result in a low bias. **4(a)**  1. Classification would be useful in predicting heart failure. The predictors could involve age, sex, smoking, whether they have high blood pressure, and whether they have diabetes. The response would be predict yes or no for heart failure. This problem would be a prediction classification problem because it takes on a binary response format.2. Classification would be useful in predicting a certain type of flower species. The predictors could involve sepal length, sepal width, petal length, and petal width. The response is predicting the type of flower species. This problem would be a prediction classification problem because it predicts categorical labels.3. Classification would be useful in predicting whether a mushroom is poisonous or not. The predictors could involve odor, gill size, cap shape, cap size, etc.. The response would be a prediction of yes or no in determing whether the mushroom is poisonous or not. This problem would be a prediction classification problem because it takes on a binary response format.**4(b)** 1. Regression is useful when it comes to housing predictions. The predictors could involve number of bedrooms, house size, number of bathrooms, location, pool etc. The response would be the housing price. This problem would be a prediction regression problem because we are expecting a quantitative value.2. Regression is useful when it comes to prediction a player's salary. The predictors could involve height, weight, position, years in the league, etc.. This problem would be a prediction regression problem because we are expecting a quantitative value.3. Regression is useful when it comes to predicting the life expectancy of a person. The predictors could involve whether the country is developed/developing, country, gdp, etc.. This problem would be a prediction regression problem because we are expecting a quantitative value.**4(c)** 1. Market segmentation from the given ccomapny data about different types of customers.2. Categorize whether a country is developed/developing based on socioeconomic factors, health, etc.3. You can identify different types of flowers based on the characteristics or attributes of their predictors. **5** The advantages of using a flexible versus a less flexible approach when using regression/classification is that when your using a flexible model the interpretability of the model decreases the more flexible the model. When using a less flexible approach, the interpretability of the model increases the higher the less flexibe the model. Cases where you would prefer a flexible approach over a less flexible approach is if the interpretability of the model does not matter and you are mainly focued on the results or predictions. Cases where you would prefer a less flexible model over a flexible model is if you are mainly focused on inference or the interpretability of the model. **6** The differences between a parametric approach and a non-parametric approach is that a parametric approach makes an assumption about the true form of f and a non-parametric approach does not make assumptions about the true form of f. The advantages of using a parametric approach is that you can simplify the model by estimating parameters with fewer observations opposed to a non-parametric function that attempt to fit a arbitrary function f with many observations. Disadvantage in using a parametric approach is you could obtain the true functional form f and overfit the data. Disadvantages in using a non-parametric approach is that it does reduce the problem of estimating parameters and it requires a very large number of observations to get the true functional form of f.<jupyter_code># Import numpy and panda library
import numpy as np
import pandas as pd
#7
df = pd.DataFrame(data = [[0,3,0,'Red'],[2,0,0,'Red'],[0,1,3,'Red'],[0,1,2,'Green'],[-1,0,1,'Green'],[1,1,1,'Red']],index=np.arange(1,7),columns=['X1','X2','X3','Y'])
df
a = df[['X1','X2','X3']]
# 8(a)
# Adding the calculated Euclidean Distance to the dataframe
df['distance'] = np.sqrt(a.apply(lambda x: (x**2).sum(),axis=1)).round(2)
df<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.05445263754963131, 'fraction_numerical': 0.00935904707884288, 'mean_word_length': 5.069707401032702, 'pattern_counts': {'":': 0, '<': 9, '<?xml version=': 0, '>': 9, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '2', 'dir': 'jupyter-structured-clean-dedup', 'id': '236716', 'license': 'MIT', 'n_tokens_mistral': 1705, 'n_tokens_neox': 1604, 'n_words': 1044, 'path': 'Chapter 2/ConceptualExercises.ipynb', 'repo_name': 'kwilson-tech/ISL', 'size': '7052'}