<jupyter_start><jupyter_text>_Lambda School Data Science, Unit 2_  Linear Models Sprint ChallengeTo demonstrate mastery on your Sprint Challenge, do all the required, numbered instructions in this notebook.To earn a score of "3", also do all the stretch goals.You are permitted and encouraged to do as much data exploration as you want.  Part 1, Classification- 1.1. Do train/test split. Arrange data into X features matrix and y target vector- 1.2. Use scikit-learn to fit a logistic regression model- 1.3. Report classification metric: accuracy Part 2, Regression- 2.1. Begin with baselines for regression- 2.2. Do train/validate/test split- 2.3. Arrange data into X features matrix and y target vector- 2.4. Do one-hot encoding- 2.5. Use scikit-learn to fit a linear regression or ridge regression model- 2.6. Report validation MAE and $R^2$ Stretch Goals, Regression- Make at least 2 visualizations to explore relationships between features and target. You may use any visualization library- Try at least 3 feature combinations. You may select features manually, or automatically- Report validation MAE and $R^2$ for each feature combination you try- Report test MAE and $R^2$ for your final model- Print or plot the coefficients for the features in your model<jupyter_code>import plotly.express as px
from sklearn.metrics import r2_score
# If you're in Colab...
import sys
if 'google.colab' in sys.modules:
    !pip install category_encoders==2.*
    !pip install pandas-profiling==2.*
    !pip install plotly==4.*<jupyter_output>Requirement already satisfied: category_encoders==2.* in /usr/local/lib/python3.6/dist-packages (2.2.2)
Requirement already satisfied: numpy>=1.14.0 in /usr/local/lib/python3.6/dist-packages (from category_encoders==2.*) (1.18.5)
Requirement already satisfied: scipy>=1.0.0 in /usr/local/lib/python3.6/dist-packages (from category_encoders==2.*) (1.4.1)
Requirement already satisfied: patsy>=0.5.1 in /usr/local/lib/python3.6/dist-packages (from category_encoders==2.*) (0.5.1)
Requirement already satisfied: scikit-learn>=0.20.0 in /usr/local/lib/python3.6/dist-packages (from category_encoders==2.*) (0.22.2.post1)
Requirement already satisfied: statsmodels>=0.9.0 in /usr/local/lib/python3.6/dist-packages (from category_encoders==2.*) (0.10.2)
Requirement already satisfied: pandas>=0.21.1 in /usr/local/lib/python3.6/dist-packages (from category_encoders==2.*) (1.0.5)
Requirement already satisfied: six in /usr/local/lib/python3.6/dist-packages (from patsy>=0.5.1->category_encoders==2.*) (1.15[...]<jupyter_text>Part 1, Classification: Predict Blood Donations 🚑Our dataset is from a mobile blood donation vehicle in Taiwan. The Blood Transfusion Service Center drives to different universities and collects blood as part of a blood drive.The goal is to predict whether the donor made a donation in March 2007, using information about each donor's history.Good data-driven systems for tracking and predicting donations and supply needs can improve the entire supply chain, making sure that more patients get the blood transfusions they need.<jupyter_code>import pandas as pd

donors = pd.read_csv('https://archive.ics.uci.edu/ml/machine-learning-databases/blood-transfusion/transfusion.data')
assert donors.shape == (748,5)

donors = donors.rename(columns={
    'Recency (months)': 'months_since_last_donation', 
    'Frequency (times)': 'number_of_donations', 
    'Monetary (c.c. blood)': 'total_volume_donated', 
    'Time (months)': 'months_since_first_donation', 
    'whether he/she donated blood in March 2007': 'made_donation_in_march_2007'
})<jupyter_output><empty_output><jupyter_text>Notice that the majority class (did not donate blood in March 2007) occurs about 3/4 of the time. This is the accuracy score for the "majority class baseline" (the accuracy score we'd get by just guessing the majority class every time).<jupyter_code>donors['made_donation_in_march_2007'].value_counts(normalize=True)
donors['made_donation_in_march_2007'].value_counts(normalize=True).plot(kind='bar', )
###Data Exploration
import numpy as np
import plotly as py
donors.columns
donors.head()
donors.isnull().sum()
###Looking at the correlation of the different features to each other. 
donor_corr = donors.corr()
donor_corr
import matplotlib.pyplot as plt
import seaborn as sns
plt.figure(figsize=(12, 8))
sns.heatmap(donor_corr);
donor_corr['made_donation_in_march_2007'].sort_values(ascending=False)<jupyter_output><empty_output><jupyter_text>It appears, when taking a look at the correlations between people who made donations in march, and the other features in the dataset, I see that 'number of donations' and 'total volume donated', both have the highest correlation with the target feature. Interestingly, they both are the same value.   1.1. Do train/test split. Arrange data into X features matrix and y target vectorDo these steps in either order.Use scikit-learn's train/test split function to split randomly. (You can include 75% of the data in the train set, and hold out 25% for the test set, which is the default.)<jupyter_code>###Import SKlearn
from sklearn.model_selection import train_test_split
from sklearn.linear_model import LinearRegression
from sklearn import metrics
###Declare target feature
y = donors['made_donation_in_march_2007']
###Declare features
x = donors[['number_of_donations','total_volume_donated','months_since_last_donation']]
###Split data for train and test with test size at 25% of our data
x_train, x_test, y_train, y_test = train_test_split(x, y, test_size=0.25, random_state=70)
print(x_train.head())
print(y_train.head())
print(x_test.head())
print(y_test.head())<jupyter_output>number_of_donations  total_volume_donated  months_since_last_donation
253                    3                   750                          11
74                     2                   500                           2
458                    1                   250                          21
523                   10                  2500                           1
69                     9                  2250                           1
253    1
74     1
458    0
523    1
69     0
Name: made_donation_in_march_2007, dtype: int64
     number_of_donations  total_volume_donated  months_since_last_donation
688                    3                   750                          14
695                    7                  1750                          17
268                    7                  1750                           4
531                    4                  1000                           2
588                    1                   250                           2
688    [...]<jupyter_text>1.2. Use scikit-learn to fit a logistic regression modelYou may use any number of features<jupyter_code>###Import LogisticRegression
from sklearn.linear_model import LogisticRegression
from sklearn.metrics import accuracy_score
donors_log_reg = LogisticRegression(solver='lbfgs')
donors_log_reg.fit(x_train, y_train)<jupyter_output><empty_output><jupyter_text>1.3. Report classification metric: accuracyWhat is your model's accuracy on the test set?Don't worry if your model doesn't beat the majority class baseline. That's okay!_"The combination of some data and an aching desire for an answer does not ensure that a reasonable answer can be extracted from a given body of data."_ —[<NAME>](https://en.wikiquote.org/wiki/John_Tukey)(Also, if we used recall score instead of accuracy score, then your model would almost certainly beat the baseline. We'll discuss how to choose and interpret evaluation metrics throughout this unit.)<jupyter_code>###Majority Class Baseline Accuracy
majority_class = y_train.mode()
y_pred = [majority_class] * len(y_train)
print(len(y_train))
accuracy_score(y_train, y_pred)
print('Validation Accuracy', donors_log_reg.score(x_test, y_test))<jupyter_output>Validation Accuracy 0.7486631016042781<jupyter_text>Part 2, Regression: Predict home prices in Ames, Iowa 🏠You'll use historical housing data. ***There's a data dictionary at the bottom of the notebook.*** Run this code cell to load the dataset:<jupyter_code>from category_encoders import OneHotEncoder as ce
from sklearn.linear_model import LinearRegression
from sklearn.metrics import mean_absolute_error
from sklearn.feature_selection import SelectKBest
from sklearn.metrics import mean_squared_error
import pandas as pd
URL = 'https://drive.google.com/uc?export=download&id=1522WlEW6HFss36roD_Cd9nybqSuiVcCK'
homes = pd.read_csv(URL)
assert homes.shape == (2904, 47)
homes.head()
homes.isnull().sum()
homes_corr = homes.corr()
homes_corr
plt.figure(figsize=(12, 8))
sns.heatmap(homes_corr);
###This helps me decide which numerical features make the most sense to use. 
homes_corr['SalePrice'].sort_values(ascending=False)
corr_home = homes_corr['SalePrice'].sort_values(ascending=False)
### This is displaying the correlation between the these features (numerical)
### and the target 'SalePrice'
list_corrs=corr_home.to_list
list_corrs
fig = px.scatter(homes, x='Gr_Liv_Area',y='SalePrice',trendline='ols')
fig.show()<jupyter_output><empty_output><jupyter_text>2.1. Begin with baselinesWhat is the Mean Absolute Error and R^2 score for a mean baseline? (You can get these estimated scores using all your data, before splitting it.)<jupyter_code>from sklearn.metrics import mean_absolute_error
### To find baseline, we start with the mean of the target, which is 'SalePrice' 
target = 'SalePrice'
y_train = train[target]
y_test = test[target]
### This is creating the baseline 
print('This is the Mean Baseline with 0 Features')
baseline = y_train.mean()
print (baseline)
y_predict = [baseline] * len(y_train)
### Mean absolute error of Baseline
mae_baseline = mean_absolute_error(y_train, y_predict)
print('Mean Absolute Error of Baseline:',mae_baseline)
 baseline_r2_score = r2_score(y_train, y_predict)
print("Validation R-squared is:", baseline_r2_score, '\n')<jupyter_output>Validation R-squared is: 0.0<jupyter_text>2.2. Do train/validate/test splitTrain on houses sold in the years 2006 - 2008. (1,920 rows)Validate on house sold in 2009. (644 rows)Test on houses sold in 2010. (340 rows)<jupyter_code>train = homes[(homes['Yr_Sold'] >= 2006) & (homes['Yr_Sold']<= 2008)].reset_index()
val   = homes[homes['Yr_Sold'] == 2009].reset_index()
test  = homes[homes['Yr_Sold'] == 2010].reset_index()
###Validating shapes of data
print(train.shape)
print(val.shape)
print(test.shape)<jupyter_output>(1920, 48)
(644, 48)
(340, 48)<jupyter_text>2.3. Arrange data into X features matrix and y target vectorSelect at least one numeric feature and at least one categorical feature.Otherwise, you may choose whichever features and however many you want.<jupyter_code>###Define features and target
features = ['Overall_Qual','Gr_Liv_Area','1st_Flr_SF','Year_Built','Full_Bath',         
            'TotRms_AbvGrd','House_Style',
            'Year_Built','Lot_Area','Gr_Liv_Area',] ### < - - - Features were found through corr and the dict at the bottom
target = 'SalePrice'

X_train = train[features]
y_train = train[target]
X_val = val[features]
y_val = val[target]
X_test = test[features]
y_test = test[target]

X_train.shape, y_train.shape, X_val.shape, y_val.shape<jupyter_output><empty_output><jupyter_text>2.4. Do one-hot encodingEncode your categorical feature(s).<jupyter_code>import category_encoders as ce
from sklearn.preprocessing import StandardScaler
def dummy_getter(df,columns_dummy):
  dummy = pd.get_dummies(df, columns=columns_dummy)
  return dummy
cat_features = ['House_Style']
X_train_encoded = dummy_getter(X_train,cat_features)
X_val_encoded   = dummy_getter(X_val,cat_features)
X_test_encoded = dummy_getter(X_test, cat_features)
X_train_encoded.head()<jupyter_output><empty_output><jupyter_text>2.5. Use scikit-learn to fit a linear regression or ridge regression modelFit your model.<jupyter_code>from sklearn.linear_model import LinearRegression
###Instantiate Model
model = LinearRegression(normalize=True)
###Fit model to data
model.fit(X_train_encoded, y_train,)
###Encode training data
y_pred_train = model.predict(X_train_encoded)
###Test Accuracy against validation set from 2009
y_pred_val = model.predict(X_val_encoded)<jupyter_output><empty_output><jupyter_text>2.6. Report validation MAE and $R^2$What is your model's Mean Absolute Error and $R^2$ score on the validation set? (You are not graded on how high or low your validation scores are.)<jupyter_code>from sklearn.metrics import r2_score
### I am using SKlearn r2_score to get the r squared score for my linear regression
print('Validation MAE:', mean_absolute_error(y_val, y_pred_val), '\n')
print("Validation R-squared is:", r2_score(y_val, y_pred_val), '\n')
### Baseline Values
print('Baseline MAE:', mae_baseline)
print('Baseline r2:', baseline_r2_score)<jupyter_output>Baseline MAE: 58502.9394639757
Baseline r2: 0.0<jupyter_text>Stretch Goals, Regression- Make at least 2 visualizations to explore relationships between features and target. You may use any visualization library- Try at least 3 feature combinations. You may select features manually, or automatically- Report validation MAE and $R^2$ for each feature combination you try- Report test MAE and $R^2$ for your final model- Print or plot the coefficients for the features in your model<jupyter_code><jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.09073893838436775, 'fraction_numerical': 0.03196825634498765, 'mean_word_length': 4.414673692744223, 'pattern_counts': {'":': 0, '<': 54, '<?xml version=': 0, '>': 61, 'https://': 3, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 22, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '14', 'dir': 'jupyter-structured-clean-dedup', 'id': '342799', 'license': 'MIT', 'n_tokens_mistral': 4394, 'n_tokens_neox': 3998, 'n_words': 1336, 'path': 'DS_Sprint_Challenge_5.ipynb', 'repo_name': 'JoshuaHaga/DS-Unit-2-Linear-Models', 'size': '13361'}