<jupyter_start><jupyter_text>The testing dataset would be loaded and used for validation on model performance.<jupyter_code>import pandas as pd
import numpy as np

import matplotlib.pyplot as plt
import matplotlib.colors as mcolors
import seaborn as sns

from numpy import mean
from numpy import std

from sklearn.linear_model import LogisticRegression
from sklearn.metrics import classification_report

import joblib<jupyter_output><empty_output><jupyter_text>Load testing dataset into notebook. Observe its shape.<jupyter_code>occu = pd.read_csv('datatest.txt')
print(occu.shape)
occu.head(2)<jupyter_output>(2665, 7)<jupyter_text>Observe a general picture on the data of target and features<jupyter_code>def snapshot(df):
    n_missing = pd.DataFrame(df.isnull().sum(), columns = ['n_missing'])
    pct_missing = pd.DataFrame(round(df.isnull().sum() / df.shape[0], 2), columns = ['pct_missing'])
    dtype = pd.DataFrame(df.dtypes, columns = ['dtype'])
    n_unique = []
    for col in df.columns:
        n_unique.append(df[col].nunique())    
    return pd.DataFrame(n_unique, index = df.columns, columns = ['n_unique']).join(dtype).join(n_missing).join(pct_missing)

snapshot = snapshot(occu)
snapshot
occu['date'] = pd.to_datetime(occu['date'])
occu.set_index(keys = 'date', drop = True, inplace = True)
occu['Occupancy'] = occu['Occupancy'].astype(str)<jupyter_output><empty_output><jupyter_text>Observe the distribution of target data in the testing dataset.<jupyter_code>def plot_single_categorical(df, col):
    plt.figure(figsize = (4, 4))
    plt.bar(x = df[col].value_counts().index, height = df[col].value_counts(), color = mcolors.TABLEAU_COLORS)
    sns.despine(top = True)
    n_level = df[col].nunique()
    for x_coor in range(n_level):
        plt.annotate(df[col].value_counts().iloc[x_coor], 
                     xy = (x_coor, 
                           df[col].value_counts().iloc[x_coor] + df[col].value_counts().iloc[0]/50))
    plt.xticks(rotation = 0)
    plt.grid()
    plt.title(col)
    plt.show()
    
plot_single_categorical(occu, 'Occupancy')<jupyter_output><empty_output><jupyter_text>The bar graph above shows that the target of testing dataset is unbalanced with more samples with label '0' than samples with label '1'.<jupyter_code>X = occu.drop('Occupancy', axis = 1)
y = occu['Occupancy'].astype(int)<jupyter_output><empty_output><jupyter_text>Load the model stored in 'joblib' file into this notebook.<jupyter_code>logi = joblib.load('logi.joblib')<jupyter_output><empty_output><jupyter_text>Since the model was trained with the feature 'Light' alone, only that feature would be used to predict targets in the testing datset.<jupyter_code>X[['Light']]
y_pred_logi = logi.predict(X[['Light']])
print(classification_report(y, y_pred_logi))<jupyter_output>precision    recall  f1-score   support

           0       1.00      0.97      0.98      1693
           1       0.95      1.00      0.97       972

    accuracy                           0.98      2665
   macro avg       0.97      0.98      0.98      2665
weighted avg       0.98      0.98      0.98      2665
{'fraction_non_alphanumeric': 0.11844222722883811, 'fraction_numerical': 0.02510460251046025, 'mean_word_length': 3.8335925349922237, 'pattern_counts': {'":': 0, '<': 27, '<?xml version=': 0, '>': 27, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 4, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '7', 'dir': 'jupyter-structured-clean-dedup', 'id': '58544', 'license': 'MIT', 'n_tokens_mistral': 1105, 'n_tokens_neox': 1065, 'n_words': 243, 'path': 'occupancy_test.ipynb', 'repo_name': 'chao05/Prediction-for-Room-Occupancy', 'size': '3107'}