<jupyter_start><jupyter_text>SolutionsSimulation results will be different unless you use the seed. Check that your strategy for completing the exercises is similar to the sample solutions here, in that case. Exercise 1Simulate December 2018 using the seed of 27.<jupyter_code>!python3 ../../ch_08/simulate.py \
    -s 27 \
    -u ../../ch_08/user_data/user_base.txt \
    -i ../../ch_08/user_data/user_ips.json \
    -l dec_2018_log.csv \
    -hl dec_2018_attacks.csv \
    31 "2018-12-01"<jupyter_output>[INFO] [ simulate.py ] Simulating 31.0 days...
[INFO] [ simulate.py ] Saving logs
[INFO] [ simulate.py ] All done!<jupyter_text>Imports for Remaining Exercises<jupyter_code>%matplotlib inline
import matplotlib.pyplot as plt
import numpy as np
import pandas as pd
import seaborn as sns<jupyter_output><empty_output><jupyter_text>Exercise 2Find the number of unique usernames, attempts, successes, failures, and success/failure rates per IP address.<jupyter_code>dec_log = pd.read_csv('dec_2018_log.csv', parse_dates=True, index_col='datetime')

log_aggs = dec_log.assign(
    failures=lambda x: np.invert(x.success)
).groupby('source_ip').agg(
    {'username': 'nunique', 'success': 'sum', 'failures': 'sum'}
).assign(
    attempts=lambda x: x.success + x.failures,
    success_rate=lambda x: x.success / x.attempts,
    failure_rate=lambda x: 1 - x.success_rate
).dropna().reset_index()

log_aggs.head()<jupyter_output><empty_output><jupyter_text>Exercise 3Create two subplots with failures versus attempts on the left and failure rate versus distinct usernames on the right. Draw a decision boundary for what you see. Be sure to color by whether or not it is a hacker IP address.<jupyter_code>is_attack_ip = log_aggs.source_ip.isin(
    pd.read_csv('dec_2018_attacks.csv').source_ip
)

fig, axes = plt.subplots(1, 2, figsize=(10, 3))

for ax, (x, y) in zip(axes, (('attempts', 'failures'), ('username', 'failure_rate'))):
    ax = sns.scatterplot(
        x=log_aggs[x], 
        y=log_aggs[y], 
        hue=is_attack_ip,
        ax=ax
    )
    ax.set_title(f'{y.title()} vs. {x.title()}')

# boundaries
axes[0].plot([0, 80], [80, 0], 'r--')
axes[1].axhline(0.5, color='red', linestyle='--')<jupyter_output><empty_output><jupyter_text>Exercise 4Build a rule-based criteria using percent difference from the median that flags an IP address if failures and attempts are 5 times the median OR if distinct usernames is 5 times the median.<jupyter_code>hourly_ip_logs = dec_log.assign(
    failures=lambda x: np.invert(x.success)
).groupby('source_ip').resample('1H').agg(
    {'username': 'nunique', 'success': 'sum', 'failures': 'sum'}
).assign(
    attempts=lambda x: x.success + x.failures,
    success_rate=lambda x: x.success / x.attempts,
    failure_rate=lambda x: 1 - x.success_rate
).dropna().reset_index()

hourly_ip_logs.head()<jupyter_output><empty_output><jupyter_text>Function from chapter for getting baselines:<jupyter_code>def get_baselines(hourly_ip_logs, func, *args, **kwargs):
    """
    Calculate hourly bootstrapped statistic per column.
    
    Parameters:
        - hourly_ip_logs: Data to sample from.
        - func: Statistic to calculate.
        - args: Additional positional arguments for `func`
        - kwargs: Additional keyword arguments for `func`
    
    Returns:
        `pandas.DataFrame` of hourly bootstrapped statistics
    """
    if isinstance(func, str):
        func = getattr(pd.DataFrame, func)

    return hourly_ip_logs.assign(
        hour=lambda x: x.datetime.dt.hour
    ).groupby('hour').apply(
        lambda x: x.sample(10, random_state=0, replace=True).pipe(func, *args, **kwargs, numeric_only=True)
    )<jupyter_output><empty_output><jupyter_text>Get baseline:<jupyter_code>medians = get_baselines(hourly_ip_logs, 'median')<jupyter_output><empty_output><jupyter_text>Flag if both failures and attempts are 5 times higher than the median or if usernames tried is 5 times higher than the median:<jupyter_code>flagged_ips = hourly_ip_logs.assign(
    hour=lambda x: x.datetime.dt.hour
).join(
    medians, on='hour', rsuffix='_median'
).assign(
    flag_median=lambda x: np.logical_or(
        np.logical_and(
            x.failures_median * 5 <= x.failures,
            x.attempts_median * 5 <= x.attempts
        ), x.username_median * 5 <= x.username
    )
).query('flag_median').source_ip.drop_duplicates()<jupyter_output><empty_output><jupyter_text>Exercise 5Calculate metrics to evaluate how well the ensemble method performed. We can use the `evaluate()` function from the chapter:<jupyter_code>def evaluate(alerted_ips, attack_ips, log_ips):
    """
    Calculate true positives (TP), false positives (FP), 
    true negatives (TN), and false negatives (FN) for 
    IP addresses flagged as suspicious.
    
    Parameters:
        - alerted_ips: `pandas.Series` of flagged IP addresses
        - attack_ips: `pandas.Series` of attacker IP addresses
        - log_ips: `pandas.Series` of all IP addresses seen
    
    Returns:
        Tuple of form (TP, FP, TN, FN)
    """
    tp = alerted_ips.isin(attack_ips).sum()
    tn = np.invert(np.isin(log_ips[~log_ips.isin(alerted_ips)].unique(), attack_ips)).sum()
    fp = np.invert(
        np.isin(log_ips[log_ips.isin(alerted_ips)].unique(), attack_ips)
    ).sum()
    fn = np.invert(
        np.isin(log_ips[log_ips.isin(attack_ips)].unique(), alerted_ips)
    ).sum()
    return tp, fp, tn, fn<jupyter_output><empty_output><jupyter_text>Next, we make a partial to store the attacker IP addreses and the unique IP addresses in the logs:<jupyter_code># make this easier to call
from functools import partial
scores = partial(
    evaluate, 
    attack_ips=pd.read_csv('dec_2018_attacks.csv').source_ip, 
    log_ips=dec_log.source_ip.drop_duplicates()
)<jupyter_output><empty_output><jupyter_text>We can evaluate the performance with the `classification_stats()` function from the chapter:<jupyter_code>def classification_stats(tp, fp, tn, fn):
    """Calculate metrics"""
    return {
        'FPR': fp / (fp + tn),
        'FDR': fp / (fp + tp),
        'FNR': fn / (fn + tp),
        'FOR': fn / (fn + tn)
    }<jupyter_output><empty_output><jupyter_text>Performance is decent:<jupyter_code>classification_stats(*scores(flagged_ips))<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.12370806169502306, 'fraction_numerical': 0.012879631101923994, 'mean_word_length': 4.07258064516129, 'pattern_counts': {'":': 0, '<': 51, '<?xml version=': 0, '>': 48, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '12', 'dir': 'jupyter-structured-clean-dedup', 'id': '253798', 'license': 'MIT', 'n_tokens_mistral': 2242, 'n_tokens_neox': 2115, 'n_words': 536, 'path': 'solutions/ch_08/solutions.ipynb', 'repo_name': 'piotrv/DA-Pandas-Stefanie-Molin', 'size': '6289'}