<jupyter_start><jupyter_text>Using `partial_fit()` with grid search`GridSearchCV` doesn't have a `partial_fit()` method, but we can use `GridSearchCV` to find the best initial hyperparameters for our models before moving to `partial_fit()`. Setup<jupyter_code>%matplotlib inline
import matplotlib.pyplot as plt
import numpy as np
import pandas as pd

import sqlite3

with sqlite3.connect('../../ch_11/logs/logs.db') as conn:
    logs_2018 = pd.read_sql(
        'SELECT * FROM logs WHERE datetime BETWEEN "2018-01-01" AND "2019-01-01";', 
        conn, parse_dates=['datetime'], index_col='datetime'
    )
    hackers_2018 = pd.read_sql(
        'SELECT * FROM attacks WHERE start BETWEEN "2018-01-01" AND "2019-01-01";', 
        conn, parse_dates=['start', 'end']
    ).assign(
        duration=lambda x: x.end - x.start, 
        start_floor=lambda x: x.start.dt.floor('min'),
        end_ceil=lambda x: x.end.dt.ceil('min')
    )<jupyter_output><empty_output><jupyter_text>Get training and testing sets<jupyter_code>def get_X(log, day):
    """
    Get data we can use for the X
    
    Parameters:
        - log: The logs dataframe
        - day: A day or single value we can use as a datetime index slice
    
    Returns: 
        A `pandas.DataFrame` object
    """
    return pd.get_dummies(log.loc[day].assign(
        failures=lambda x: 1 - x.success
    ).query('failures > 0').resample('1min').agg(
        {'username': 'nunique', 'failures': 'sum'}
    ).dropna().rename(
        columns={'username': 'usernames_with_failures'}
    ).assign(
        day_of_week=lambda x: x.index.dayofweek, 
        hour=lambda x: x.index.hour
    ).drop(columns=['failures']), columns=['day_of_week', 'hour'])

def get_y(datetimes, hackers, resolution='1min'):
    """
    Get data we can use for the y (whether or not a hacker attempted a log in during that time).
    
    Parameters:
        - datetimes: The datetimes to check for hackers
        - hackers: The dataframe indicating when the attacks started and stopped
        - resolution: The granularity of the datetime. Default is 1 minute.
        
    Returns:
        `pandas.Series` of Booleans.
    """
    date_ranges = hackers.apply(
        lambda x: pd.date_range(x.start_floor, x.end_ceil, freq=resolution), 
        axis=1
    )
    dates = pd.Series(dtype='object')
    for date_range in date_ranges:
        dates = pd.concat([dates, date_range.to_series()])
    return datetimes.isin(dates)

def get_X_y(log, day, hackers):
    """
    Get the X, y data to build a model with.
    
    Parameters:
        - log: The logs dataframe
        - day: A day or single value we can use as a datetime index slice
        - hackers: The dataframe indicating when the attacks started and stopped
        
    Returns:
        X, y tuple where X is a `pandas.DataFrame` object
        and y is a `pandas.Series` object
    """
    X = get_X(log, day)
    y = get_y(X.reset_index().datetime, hackers)
    return X, y<jupyter_output><empty_output><jupyter_text>Isolate January and February 2018:<jupyter_code>X_jan, y_jan = get_X_y(logs_2018, '2018-01', hackers_2018)
X_feb, y_feb = get_X_y(logs_2018, '2018-02', hackers_2018)<jupyter_output><empty_output><jupyter_text>Step 1: Run `GridSearchCV` to build a Passive Aggressive ClassifierTrain on January 2018.<jupyter_code>%%capture
from sklearn.linear_model import PassiveAggressiveClassifier
from sklearn.model_selection import GridSearchCV
from sklearn.preprocessing import StandardScaler
from ml_utils.partial_fit_pipeline import PartialFitPipeline

pipeline = PartialFitPipeline([
    ('scale', StandardScaler()), 
    ('pa', PassiveAggressiveClassifier(random_state=0, max_iter=1000, tol=1e-3))
])

search_space = {
    'pa__C': [0.01, 0.1, 1, 10],
    'pa__fit_intercept': [True, False],
    'pa__class_weight': ['balanced', None]
}

pa_grid = GridSearchCV(pipeline, search_space, cv=5, scoring='f1_macro').fit(X_jan, y_jan)<jupyter_output><empty_output><jupyter_text>Examine the best parameters after the grid search:<jupyter_code>pa_grid.best_params_<jupyter_output><empty_output><jupyter_text>Evaluate the model's initial performance (on February)<jupyter_code>preds = pa_grid.predict(X_feb)
pd.Series(preds).value_counts()
from sklearn.metrics import classification_report

print(classification_report(y_feb, preds))
from ml_utils.classification import confusion_matrix_visual

confusion_matrix_visual(y_feb, preds, class_labels=[False, True])<jupyter_output><empty_output><jupyter_text>Step 2: Store the best estimator for `partial_fit()` usage later<jupyter_code>pa = pa_grid.best_estimator_<jupyter_output><empty_output><jupyter_text>Step 3: Update the model with the February 2018 data<jupyter_code>pa.partial_fit(X_feb, y_feb)<jupyter_output><empty_output><jupyter_text>Step 4: Evaluate model on March through June 2018 data<jupyter_code>X_mar, y_mar = get_X_y(logs_2018, '2018-03', hackers_2018)
X_q2, y_q2 = get_X_y(logs_2018, '2018-Q2', hackers_2018)
X_eval = pd.concat([X_mar, X_q2])
y_eval = pd.concat([y_mar, y_q2])
preds = pa.predict(X_eval)<jupyter_output><empty_output><jupyter_text>Get metrics on the predictions:<jupyter_code>from sklearn.metrics import classification_report

print(classification_report(y_eval, preds))
from ml_utils.classification import confusion_matrix_visual

confusion_matrix_visual(y_eval, preds, class_labels=[False, True])<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.11115214180206795, 'fraction_numerical': 0.026957163958641065, 'mean_word_length': 4.31078431372549, 'pattern_counts': {'":': 0, '<': 41, '<?xml version=': 0, '>': 42, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '10', 'dir': 'jupyter-structured-clean-dedup', 'id': '210599', 'license': 'MIT', 'n_tokens_mistral': 1966, 'n_tokens_neox': 1807, 'n_words': 452, 'path': 'solutions/ch_11/exercise_4.ipynb', 'repo_name': 'piotrv/DA-Pandas-Stefanie-Molin', 'size': '5416'}