<jupyter_start><jupyter_text>Handle Missing Data<jupyter_code>missing_data = df_weather.isnull().sum().sort_values(ascending=False)
percent = (df_weather.isnull().sum()/df_weather.isnull().count()).sort_values(ascending=False)
missing_data = pd.concat([missing_data, percent], axis=1, keys=['Total_missing', 'Percent'])
print(missing_data)<jupyter_output>Total_missing   Percent
Sunshine               69835  0.480098
Evaporation            62790  0.431665
Cloud3pm               59358  0.408071
Cloud9am               55888  0.384216
Pressure9am            15065  0.103568
Pressure3pm            15028  0.103314
WindDir9am             10566  0.072639
WindGustDir            10326  0.070989
WindGustSpeed          10263  0.070555
Humidity3pm             4507  0.030984
WindDir3pm              4228  0.029066
Temp3pm                 3609  0.024811
RainTomorrow            3267  0.022460
RainToday               3261  0.022419
Rainfall                3261  0.022419
WindSpeed3pm            3062  0.021050
Humidity9am             2654  0.018246
WindSpeed9am            1767  0.012148
Temp9am                 1767  0.012148
MinTemp                 1485  0.010209
MaxTemp                 1261  0.008669
Location                   0  0.000000
Date                       0  0.000000<jupyter_text>There are 4 columns around 40% missing data. So, we need to handle that.Lets find the categorical and continuous value. Then we'll use the mean/mode value for these missing value.<jupyter_code>category=[]
contin = []

for i in df_weather.columns:
    if df_weather[i].dtype =="object":
        category.append(i)
        
    else:
        contin.append(i)

print("Categorical:",category)
print("Continuous:", contin)
# filling the missing values for continuous variables with mean
df_weather["MinTemp"]= df_weather["MinTemp"].fillna(df_weather["MinTemp"].mean())
df_weather["MaxTemp"]= df_weather["MaxTemp"].fillna(df_weather["MaxTemp"].mean())
df_weather["Evaporation"]= df_weather["Evaporation"].fillna(df_weather["Evaporation"].mean())
df_weather["Sunshine"]= df_weather["Sunshine"].fillna(df_weather["Sunshine"].mean())
df_weather["WindGustSpeed"]= df_weather["WindGustSpeed"].fillna(df_weather["WindGustSpeed"].mean())
df_weather["Rainfall"]= df_weather["Rainfall"].fillna(df_weather["Rainfall"].mean())
df_weather["WindSpeed9am"]= df_weather["WindSpeed9am"].fillna(df_weather["WindSpeed9am"].mean())
df_weather["WindSpeed3pm"]= df_weather["WindSpeed3pm"].fillna(df_weather["WindSpeed3pm"].mean())
df_weather["Humidity9am"]= df_weather["Humidity9am"].fillna(df_weather["Humidity9am"].mean())
df_weather["Humidity3pm"]= df_weather["Humidity3pm"].fillna(df_weather["Humidity3pm"].mean())
df_weather["Pressure9am"]= df_weather["Pressure9am"].fillna(df_weather["Pressure9am"].mean())
df_weather["Pressure3pm"]= df_weather["Pressure3pm"].fillna(df_weather["Pressure3pm"].mean())
df_weather["Cloud9am"]= df_weather["Cloud9am"].fillna(df_weather["Cloud9am"].mean())
df_weather["Cloud3pm"]= df_weather["Cloud3pm"].fillna(df_weather["Cloud3pm"].mean())
df_weather["Temp9am"]= df_weather["Temp9am"].fillna(df_weather["Temp9am"].mean())
df_weather["Temp3pm"]= df_weather["Temp3pm"].fillna(df_weather["Temp3pm"].mean())
#Filling the missing values for continuous variables with mode
df_weather['RainToday']=df_weather['RainToday'].fillna(df_weather['RainToday'].mode()[0])
df_weather['RainTomorrow']=df_weather['RainTomorrow'].fillna(df_weather['RainTomorrow'].mode()[0])
df_weather['WindDir9am'] = df_weather['WindDir9am'].fillna(df_weather['WindDir9am'].mode()[0])
df_weather['WindGustDir'] = df_weather['WindGustDir'].fillna(df_weather['WindGustDir'].mode()[0])
df_weather['WindDir3pm'] = df_weather['WindDir3pm'].fillna(df_weather['WindDir3pm'].mode()[0])<jupyter_output><empty_output><jupyter_text>Label Encoding<jupyter_code>from sklearn.preprocessing import LabelEncoder
le = LabelEncoder()
df_weather['RainTomorrow'] = df_weather['RainTomorrow'].map({'Yes': 1, 'No': 0})
df_weather['RainToday'] = df_weather['RainToday'].map({'Yes': 1, 'No': 0})
#encoding remaining columns
df_weather["Location"] = le.fit_transform(df_weather["Location"])
df_weather["WindDir9am"]= le.fit_transform(df_weather["WindDir9am"])
df_weather["WindDir3pm"]= le.fit_transform(df_weather["WindDir3pm"])
df_weather["WindGustDir"] = le.fit_transform(df_weather["WindGustDir"])<jupyter_output><empty_output><jupyter_text>Feature EngineeringNow we'll parse the date and extract year and month.<jupyter_code>df_weather['Date'] = pd.to_datetime(df_weather['Date'])

df_weather['Year'] = df_weather['Date'].dt.year
df_weather['Month'] = df_weather['Date'].dt.month
df_weather['Day'] = df_weather['Date'].dt.day

df_weather.drop('Date', axis=1, inplace = True)
df_weather = df_weather[['Year','Month','Day', 'Location', 'MinTemp', 'MaxTemp', 'Rainfall', 'Evaporation',
       'Sunshine', 'WindGustDir', 'WindGustSpeed', 'WindDir9am', 'WindDir3pm',
       'WindSpeed9am', 'WindSpeed3pm', 'Humidity9am', 'Humidity3pm',
       'Pressure9am', 'Pressure3pm', 'Cloud9am', 'Cloud3pm', 'Temp9am',
       'Temp3pm', 'RainToday', 'RainTomorrow']]<jupyter_output><empty_output><jupyter_text>Correlation between the attributesAfter finding the cor() value we'll drop the highly correlated attributes.<jupyter_code>plt.figure(figsize=(15,15))
sns.heatmap(df_weather.corr(), cmap="crest",annot=True)
plt.xticks(rotation=90)
plt.show()<jupyter_output><empty_output><jupyter_text>MaxTemp and Temp3pm highly correlated.Temp3pm and Temp9am highly correlated.Humidity9am and Humidity3pm highly correlated.MaxTemp and Temp9am highly correlated.MinTemp and Temp9am highly correlated.MinTemp and Temp3pm highly correlated.<jupyter_code>df_weather=df_weather.drop(['Temp3pm','Temp9am','Humidity9am'],axis=1)
df_weather.columns<jupyter_output><empty_output><jupyter_text>Normalize data<jupyter_code># # Standardizing data
# from sklearn import preprocessing
# r_scaler = preprocessing.MinMaxScaler()
# r_scaler.fit(df_weather)
# df_weather_norm = pd.DataFrame(r_scaler.transform(df_weather), index=df_weather.index, columns=df_weather.columns)<jupyter_output><empty_output><jupyter_text>Train-test Split<jupyter_code>x = df_weather.drop('RainTomorrow', axis=1)
y = df_weather['RainTomorrow']
x
from sklearn.preprocessing import StandardScaler

sc = StandardScaler()
x = sc.fit_transform(x)
# Split into test and train
from sklearn.model_selection import train_test_split
x_train, x_test, y_train, y_test = train_test_split(x, y, test_size=0.25, random_state=6)<jupyter_output><empty_output><jupyter_text>Model<jupyter_code>from sklearn.ensemble import RandomForestClassifier
from lightgbm import LGBMClassifier
from xgboost import XGBClassifier
from sklearn.metrics import accuracy_score, confusion_matrix,classification_report,roc_curve, roc_auc_score<jupyter_output><empty_output><jupyter_text>XGB Model<jupyter_code>xgbc = XGBClassifier(objective='binary:logistic')
xgbc.fit(x_train,y_train)
y_predxgb = xgbc.predict(x_test)
report = classification_report(y_test, y_predxgb)
print(report)
print("XGBoost Model Accuracy:",accuracy_score(y_test,y_predxgb)*100,"%")
cm = confusion_matrix(y_test, y_predxgb)
sns.heatmap(cm, annot=True,cmap="Blues")
plt.title("Confusion Matrix for XGBoost Model")
plt.show()<jupyter_output>precision    recall  f1-score   support

           0       0.87      0.95      0.91     28301
           1       0.73      0.48      0.58      8064

    accuracy                           0.85     36365
   macro avg       0.80      0.71      0.74     36365
weighted avg       0.84      0.85      0.83     36365

XGBoost Model Accuracy: 84.56207892204043 %<jupyter_text>Random Forest<jupyter_code>classifier = RandomForestClassifier(n_estimators=300)
classifier.fit(x_train, y_train)

y_predrf = classifier.predict(x_test)
report2 = classification_report(y_test, y_predrf)
print(report2)
print("Random Forest Model Accuracy:",accuracy_score(y_test,y_predrf)*100,"%")
cm2 = confusion_matrix(y_test, y_predrf)
sns.heatmap(cm2, annot=True,cmap="Blues")
plt.title("Confusion Matrix for Random Forest Classifier Model")
plt.show()<jupyter_output>precision    recall  f1-score   support

           0       0.87      0.95      0.91     28301
           1       0.76      0.51      0.61      8064

    accuracy                           0.86     36365
   macro avg       0.82      0.73      0.76     36365
weighted avg       0.85      0.86      0.84     36365

Random Forest Model Accuracy: 85.54929190155369 %<jupyter_text>LightGBM<jupyter_code>lightgbmc = LGBMClassifier(n_estimators=500,max_depth=6)
lightgbmc.fit(x_train, y_train)
y_pred_lgbm = lightgbmc.predict(x_test)
report3 = classification_report(y_test, y_pred_lgbm)
print(report3)
print("LightGBM Model Accuracy:",accuracy_score(y_test,y_pred_lgbm)*100,"%")
cm3 = confusion_matrix(y_test, y_pred_lgbm)
sns.heatmap(cm3, annot=True,cmap="Blues")
plt.title("Confusion Matrix for LightGBM Model")
plt.show()<jupyter_output>precision    recall  f1-score   support

           0       0.88      0.95      0.91     28301
           1       0.75      0.56      0.64      8064

    accuracy                           0.86     36365
   macro avg       0.81      0.75      0.78     36365
weighted avg       0.85      0.86      0.85     36365

LightGBM Model Accuracy: 85.97002612402035 %<jupyter_text>Neural Network<jupyter_code>import keras
from keras.models import Sequential
from keras.layers import Dense
from keras.callbacks import ModelCheckpoint, EarlyStopping
es = EarlyStopping(monitor='accuracy', mode='min', verbose=1, patience=3)
nn = Sequential()

nn.add(Dense(units = 60,kernel_initializer='uniform',activation = 'relu',input_dim = 21))
nn.add(Dense(units = 30,kernel_initializer='uniform',activation = 'relu'))
nn.add(Dense(units = 10,kernel_initializer='uniform',activation = 'relu'))
nn.add(Dense(units = 1,activation='sigmoid',kernel_initializer='uniform'))
nn.compile(optimizer = 'adam',loss = 'binary_crossentropy',metrics = ['accuracy'])
nn.fit(x_train,y_train,epochs = 20,batch_size=20,callbacks=[es])
y_pred_nn = nn.predict(x_test)
y_train_pred = nn.predict(x_train)
y_pred_nn = np.where(y_pred_nn > 0.5, 1, 0)
#y_pred_nn = nn.predict(x_test)
report4 = classification_report(y_test, y_pred_nn)
print(report4)
print("NN Model Accuracy:",accuracy_score(y_test,y_pred_nn)*100,"%")
cm4 = confusion_matrix(y_test, y_pred_nn)
sns.heatmap(cm4, annot=True,cmap="Blues")
plt.title("Confusion Matrix for NN Model")
plt.show()<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.1300152846771112, 'fraction_numerical': 0.06238058846006878, 'mean_word_length': 5.076030179918746, 'pattern_counts': {'":': 1, '<': 49, '<?xml version=': 0, '>': 50, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 15, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '13', 'dir': 'jupyter-structured-clean-dedup', 'id': '176928', 'license': 'MIT', 'n_tokens_mistral': 4168, 'n_tokens_neox': 3635, 'n_words': 602, 'path': 'Weather Prediction/weather.ipynb', 'repo_name': 'asifabdullah-git/NybSys-Project', 'size': '10468'}