<jupyter_start><jupyter_text>scikit-learnStrona biblioteki: [https://scikit-learn.org](https://scikit-learn.org)  Dokumentacja/User Guide: [https://scikit-learn.org/stable/user_guide.html](https://scikit-learn.org/stable/user_guide.html)Podstawowa biblioteka do uczenia maszynowego w języku Python.Aby zainstalować bibliotekę scikit-learn, użyj polecenia poniżej:```!pip install scikit-learn```Aby zaktualizować do najnowszej wersji bibliotekę scikit-learn, użyj polecenia poniżej:```!pip install --upgrade scikit-learn```Kurs stworzony w oparciu o wersję `0.22.1` Spis treści:1. [Import bibliotek](0)2. [Wczytanie danych](1)3. [Utworzenie kopii danych](2)4. [Eksplorcja i wstępne przygotowanie danych](3)5. [Podział danych na zbiór treningowy i testowy](4)6. [Regresja liniowa](5)7. [Dobór cech modelu - eliminacja wsteczna](6)8. [Eliminacja wsteczna - automatyzacja](7)9. [Zapisanie końcowego modelu](8)   Import bibliotek<jupyter_code>import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
import seaborn as sns
import sklearn

sns.set()
np.random.seed(42)
np.set_printoptions(precision=4, suppress=True)
sklearn.__version__<jupyter_output><empty_output><jupyter_text>Wczytanie danych<jupyter_code>df_raw = pd.read_csv('https://storage.googleapis.com/esmartdata-courses-files/ml-course/insurance.csv')
df_raw.head()<jupyter_output><empty_output><jupyter_text>Utworzenie kopii danych<jupyter_code>df = df_raw.copy()
df.info()<jupyter_output><class 'pandas.core.frame.DataFrame'>
RangeIndex: 1338 entries, 0 to 1337
Data columns (total 7 columns):
age         1338 non-null int64
sex         1338 non-null object
bmi         1338 non-null float64
children    1338 non-null int64
smoker      1338 non-null object
region      1338 non-null object
charges     1338 non-null float64
dtypes: float64(2), int64(2), object(3)
memory usage: 73.3+ KB<jupyter_text>Eksplorcja i wstępne przygotowanie danych<jupyter_code>df[df.duplicated()]
df[df['charges'] == 1639.5631]
df = df.drop_duplicates()
df.info()
cat_cols = [col for col in df.columns if df[col].dtype == 'O']
cat_cols
for col in cat_cols:
    df[col] = df[col].astype('category')
df.info()
df.describe().T
df.describe(include=['category']).T
df.isnull().sum()
df.sex.value_counts()
df.sex.value_counts().plot(kind='pie')
df.smoker.value_counts()
df.region.value_counts()
df.charges.plot(kind='hist', bins=30)
import plotly.express as px

px.histogram(df, x='charges', width=700, height=400, nbins=50, facet_col='smoker', facet_row='sex')
px.histogram(df, x='smoker', facet_col='sex', color='sex', width=700, height=400)
df_dummies = pd.get_dummies(df, drop_first=True)
df_dummies
corr = df_dummies.corr()
corr
sns.set(style="white")
mask = np.zeros_like(corr, dtype=np.bool)
mask[np.triu_indices_from(mask)] = True
f, ax = plt.subplots(figsize=(8, 6))
cmap = sns.diverging_palette(220, 10, as_cmap=True)
sns.heatmap(corr, mask=mask, cmap=cmap, vmax=.3, center=0,
            square=True, linewidths=.5, cbar_kws={"shrink": .5})
df_dummies.corr()['charges'].sort_values(ascending=False)
sns.set()
df_dummies.corr()['charges'].sort_values()[:-1].plot(kind='barh')
data = df_dummies.copy()
target = data.pop('charges')
data.head()
target.head()<jupyter_output><empty_output><jupyter_text>Podział danych na zbiór treningowy i testowy<jupyter_code>from sklearn.model_selection import train_test_split

X_train, X_test, y_train, y_test = train_test_split(data, target, test_size=0.2)

print(f'X_trian shape: {X_train.shape}')
print(f'X_test shape: {X_test.shape}')
print(f'y_trian shape: {y_train.shape}')
print(f'y_test shape: {y_test.shape}')<jupyter_output>X_trian shape: (1069, 8)
X_test shape: (268, 8)
y_trian shape: (1069,)
y_test shape: (268,)<jupyter_text>Regresja liniowa<jupyter_code>from sklearn.linear_model import LinearRegression

regressor = LinearRegression()
regressor.fit(X_train, y_train)
print(f'R2 score: {regressor.score(X_test, y_test):.4f}')
y_pred = regressor.predict(X_test)
y_pred[:10]
y_true = y_test.copy()
predictions = pd.DataFrame(data={'y_true': y_true, 'y_pred': y_pred})
predictions['error'] = predictions['y_true'] -  predictions['y_pred']
predictions.head()
predictions.error.plot(kind='hist', bins=30)
from sklearn.metrics import mean_absolute_error

mae = mean_absolute_error(y_true, y_pred)
print(f'MAE wynosi: {mae:.2f}')
regressor.intercept_
regressor.coef_
data.columns<jupyter_output><empty_output><jupyter_text>Dobór cech modelu - eliminacja wsteczna<jupyter_code>import statsmodels.api as sm

X_train_ols = X_train.copy()
X_train_ols = X_train_ols.values
X_train_ols = sm.add_constant(X_train_ols)
X_train_ols
ols = sm.OLS(endog=y_train, exog=X_train_ols).fit()
predictors = ['const'] + list(X_train.columns)
print(ols.summary(xname=predictors))
X_selected = X_train_ols[:, [0, 1, 2, 3, 5, 6, 7, 8]]
predictors.remove('sex_male')

ols = sm.OLS(endog=y_train, exog=X_selected).fit()
print(ols.summary(xname=predictors))
X_selected = X_train_ols[:, [0, 1, 2, 3, 5, 7, 8]]
predictors.remove('region_northwest')

ols = sm.OLS(endog=y_train, exog=X_selected).fit()
print(ols.summary(xname=predictors))
X_selected = X_train_ols[:, [0, 1, 2, 3, 5, 7]]
predictors.remove('region_southwest')

ols = sm.OLS(endog=y_train, exog=X_selected).fit()
print(ols.summary(xname=predictors))
X_selected = X_train_ols[:, [0, 1, 2, 3, 5]]
predictors.remove('region_southeast')

ols = sm.OLS(endog=y_train, exog=X_selected).fit()
print(ols.summary(xname=predictors))<jupyter_output>OLS Regression Results                            
==============================================================================
Dep. Variable:                charges   R-squared:                       0.729
Model:                            OLS   Adj. R-squared:                  0.728
Method:                 Least Squares   F-statistic:                     716.3
Date:                Mon, 27 Jan 2020   Prob (F-statistic):          5.56e-300
Time:                        11:06:45   Log-Likelihood:                -10832.
No. Observations:                1069   AIC:                         2.167e+04
Df Residuals:                    1064   BIC:                         2.170e+04
Df Model:                           4                                         
Covariance Type:            nonrobust                                         
==============================================================================
                 coef    std err          t      P>|[...]<jupyter_text>Eliminacja wsteczna - automatyzacja<jupyter_code>X_train_numpy = X_train.values
X_train_numpy = sm.add_constant(X_train_numpy)
num_vars = len(X_train_numpy[0])

predictors = ['const'] + list(X_train.columns)
sl = 0.05

for i in range(0, num_vars):
    ols = sm.OLS(endog=y_train, exog=X_train_numpy).fit()
    max_pval = max(ols.pvalues.astype('float'))
    if max_pval > sl:
        for j in range(0, num_vars - i):
            if ols.pvalues[j].astype('float') == max_pval:
                X_train_numpy = np.delete(X_train_numpy, j, axis=1)
                predictors.remove(predictors[j])

print(ols.summary(xname=predictors))<jupyter_output>OLS Regression Results                            
==============================================================================
Dep. Variable:                charges   R-squared:                       0.729
Model:                            OLS   Adj. R-squared:                  0.728
Method:                 Least Squares   F-statistic:                     716.3
Date:                Mon, 27 Jan 2020   Prob (F-statistic):          5.56e-300
Time:                        11:11:49   Log-Likelihood:                -10832.
No. Observations:                1069   AIC:                         2.167e+04
Df Residuals:                    1064   BIC:                         2.170e+04
Df Model:                           4                                         
Covariance Type:            nonrobust                                         
==============================================================================
                 coef    std err          t      P>|[...]<jupyter_text>Zapisanie końcowego modelu<jupyter_code>ols.save('model.pickle')<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.1617826617826618, 'fraction_numerical': 0.03504273504273504, 'mean_word_length': 3.3545986177565124, 'pattern_counts': {'":': 1, '<': 34, '<?xml version=': 0, '>': 37, 'https://': 5, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 6, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '9', 'dir': 'jupyter-structured-clean-dedup', 'id': '620630', 'license': 'MIT', 'n_tokens_mistral': 3171, 'n_tokens_neox': 2963, 'n_words': 542, 'path': 'supervised/02_regression/04_backward_elimination.ipynb', 'repo_name': 'krakowiakpawel9/machine-learning-bootcamp', 'size': '8190'}