<jupyter_start><jupyter_text>1. Get some data<jupyter_code>df = pd.read_csv("./data/CaliforniaHousing/cal_housing.data", header=None)
df.head()
headers = ['longitude',
            'latitude',
            'housingMedianAge',
            'totalRooms',
            'totalBedrooms',
            'population',
            'households',
            'medianIncome',
            'medianHouseValue']
df.columns = headers
df.head()
df.describe()
df.info()<jupyter_output><class 'pandas.core.frame.DataFrame'>
RangeIndex: 20640 entries, 0 to 20639
Data columns (total 9 columns):
 #   Column            Non-Null Count  Dtype  
---  ------            --------------  -----  
 0   longitude         20640 non-null  float64
 1   latitude          20640 non-null  float64
 2   housingMedianAge  20640 non-null  float64
 3   totalRooms        20640 non-null  float64
 4   totalBedrooms     20640 non-null  float64
 5   population        20640 non-null  float64
 6   households        20640 non-null  float64
 7   medianIncome      20640 non-null  float64
 8   medianHouseValue  20640 non-null  float64
dtypes: float64(9)
memory usage: 1.4 MB<jupyter_text>2. Create train and test sets<jupyter_code>from sklearn.model_selection import train_test_split
y = df['medianHouseValue']
X = df.iloc[:,:-1]
X_train, X_test, y_train, y_test = train_test_split(X, y, train_size=0.8, random_state=1234)
X_train.shape
X_test.shape<jupyter_output><empty_output><jupyter_text>3. Choosing the (baseline) model<jupyter_code>from sklearn.linear_model import LinearRegression
lr = LinearRegression()
lr.fit(X_train, y_train)
lr.intercept_
lr.coef_
sum(X_train.iloc[0,:]*lr.coef_)+lr.intercept_
y_train[0]
y_preds = lr.predict(X_train)
y_preds[0]
training_score = lr.score(X_train, y_train)
training_score
test_score = lr.score(X_test, y_test)
test_score<jupyter_output><empty_output><jupyter_text>Note: Scaling- Usually necessary for methods that estimate coefficients (linear regression, SVM, neural networks).- Algorithms like decision trees (random forest, XGBoost) may not need it.- **Gotcha:** Careful to not amplify noise by accident (spectrometry, time series). Normalization might make things worse.<jupyter_code>from sklearn.preprocessing import StandardScaler, MinMaxScaler<jupyter_output><empty_output><jupyter_text>- Standard scaler: $x \to \frac{x-mean(x)}{std(x)}$- Min max scaler: $x \to \frac{x-min(x)}{max(x)-min(x)}$<jupyter_code>scl = StandardScaler()
X_train_scl = scl.fit_transform(X_train)
X_train.head()
X_train_scl[:5,:]<jupyter_output><empty_output><jupyter_text>**Important:** Transformations (scaling, feature extraction, etc) should be `fit_transform` on the *training* set and `transform` on the *test* set. That means, we apply the transformation created with the train set, never on the whole the data.<jupyter_code>X_test_scl = scl.transform(X_test)
X_test_scl[:5,:]<jupyter_output><empty_output><jupyter_text>Pipelines<jupyter_code>from sklearn.pipeline import make_pipeline
pipe = make_pipeline(MinMaxScaler(), LinearRegression())
pipe.fit(X_train, y_train)
pipe.score(X_train, y_train)
pipe.predict(X_train)
y_preds<jupyter_output><empty_output><jupyter_text>pipes = [pipe_lr, pipe_svm, pipe_custom]for pipe in pipes:    pipe.score()  4. Model validation and decide next steps<jupyter_code>training_score
test_score<jupyter_output><empty_output><jupyter_text>**Parity plot**: Predicted values vs Observed values (usually in test set).<jupyter_code>observed = y_test.values
predicted = pipe.predict(X_test)
observed
predicted
minval = min(min(observed), min(predicted))
maxval = max(max(observed), max(predicted))
import numpy as np
ref = np.linspace(minval, maxval, 100)
plt.scatter(observed, predicted)
plt.plot(ref, ref, 'r')
plt.xlim(minval, maxval)
plt.ylim(minval, maxval)
plt.xlabel("Observed")
plt.ylabel("Predicted")<jupyter_output><empty_output><jupyter_text>Learning curve<jupyter_code>from sklearn.model_selection import learning_curve
train_sizes, train_scores, test_scores = learning_curve(lr, X_train, y_train
                                                        , cv=5
                                                        , train_sizes=np.linspace(0.1, 1.0, 10))
train_scores # Error on each cross-validation fold ()
train_sizes
train_scores_cv = np.mean(train_scores, axis=1)
test_scores_cv = np.mean(test_scores, axis=1)
plt.plot(train_sizes, train_scores_cv, label='Training score', color='r')
plt.plot(train_sizes, test_scores_cv, label='Testing score', color='b')
plt.legend(loc='best')
plt.xlabel("# training examples")
plt.ylabel("R2 score")<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.11700739451935624, 'fraction_numerical': 0.02566333188342758, 'mean_word_length': 4.561064087061669, 'pattern_counts': {'":': 0, '<': 41, '<?xml version=': 0, '>': 41, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 4, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '10', 'dir': 'jupyter-structured-clean-dedup', 'id': '256724', 'license': 'MIT', 'n_tokens_mistral': 1691, 'n_tokens_neox': 1576, 'n_words': 344, 'path': '01 Sklearn supervised learning workflow.ipynb', 'repo_name': 'jpmaldonado/np-advanced-ml-python', 'size': '4598'}