<jupyter_start><jupyter_text>Team Slytherins     <NAME>, <NAME>, <NAME>  Net Hourly Electrical Power Output Prediction in a Combined Cycle Power Plant  Dataset AttributesThe dataset is open source, found  here, and contains **9568** data points collected from a *Combined Cycle Power Plant* over 6 years (2006-2011), when the power plant was set to work with full load.Features consist of hourly average ambient variables, namely- Temperature (T) in °C, - Ambient Pressure (AP) in milibar, - Relative Humidity (RH) in percentage, and - Exhaust Vacuum (V) in cm HgThe target is to predict the net hourly electrical energy output (EP) of the plant in MegaWatts.  Step I: Importing required modules<jupyter_code>import numpy as np
import pandas as pd
import seaborn as sns
from sklearn.svm import SVR
from matplotlib import pyplot as plt
from sklearn.preprocessing import MinMaxScaler
from sklearn.linear_model import LinearRegression
from sklearn.neighbors import KNeighborsRegressor
from sklearn.ensemble import RandomForestRegressor
from sklearn.metrics import r2_score, mean_squared_error, mean_absolute_error
from sklearn.model_selection import train_test_split, cross_val_predict, cross_val_score<jupyter_output><empty_output><jupyter_text>Step II: Importing dataset, exploratory data analysis  Reading the dataset<jupyter_code>df = pd.read_csv("ccpp.csv")<jupyter_output><empty_output><jupyter_text>Printing first five data samples<jupyter_code>df.head()<jupyter_output><empty_output><jupyter_text>Printing dataset properties<jupyter_code>df.info()<jupyter_output><class 'pandas.core.frame.DataFrame'>
RangeIndex: 9568 entries, 0 to 9567
Data columns (total 5 columns):
 #   Column  Non-Null Count  Dtype  
---  ------  --------------  -----  
 0   AT      9568 non-null   float64
 1   V       9568 non-null   float64
 2   AP      9568 non-null   float64
 3   RH      9568 non-null   float64
 4   PE      9568 non-null   float64
dtypes: float64(5)
memory usage: 373.9 KB<jupyter_text>Generating descriptive statistics for the dataset<jupyter_code>df.describe()<jupyter_output><empty_output><jupyter_text>Sketching density distribution<jupyter_code>sns.pairplot(df, diag_kind='kde', height=2, aspect=1)
plt.show()<jupyter_output><empty_output><jupyter_text>Visualizing *Linearity Trend* of independent variables with dependent variable<jupyter_code>fig = plt.figure(figsize = (12,3))

for i in range(0, df.shape[1]-1):
    ax = fig.add_subplot(1, 4, i+1)
    sns.regplot(x=df.columns[i], y="PE", data=df, line_kws={"color": "red"})

plt.tight_layout()
plt.savefig('pairplot.jpg',dpi=1200, bbox_inches='tight')

plt.show()<jupyter_output><empty_output><jupyter_text>Visualizing correlation<jupyter_code>fig = plt.figure(figsize = (8, 6))
sns.heatmap(df.corr(), annot=True, cmap='Blues')
plt.show()<jupyter_output><empty_output><jupyter_text>Step III: Preprocessing  Extracting features and targets<jupyter_code>X = df.iloc[:, :-1].values                   # features
y = df.iloc[:, -1].values.reshape(-1, 1)     # target<jupyter_output><empty_output><jupyter_text>Scaling the features in the range of [0, 1]<jupyter_code>scaler = MinMaxScaler(feature_range = (0,1))
X = scaler.fit_transform(X)<jupyter_output><empty_output><jupyter_text>Splitting the data into Training set and Test set<jupyter_code># train test ratio - 70:30
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size = 0.3, random_state = 0)

# Print data dimensions
print("Training set: \nFeatures:", np.shape(X_train), "Targets:", np.shape(y_train))
print("\nTest set: \nFeatures:", np.shape(X_test), "Targets:", np.shape(y_test))<jupyter_output>Training set: 
Features: (6697, 4) Targets: (6697, 1)

Test set: 
Features: (2871, 4) Targets: (2871, 1)<jupyter_text>Step IV: Building Machine Learning Models  Multiple Linear Regression model<jupyter_code># Create ML model instance
MLR = LinearRegression()

# Model fitting
MLR.fit(X_train, y_train)

# Making predictions
y_pred_MLR = MLR.predict(X_test)<jupyter_output><empty_output><jupyter_text>Support Vector Regression model<jupyter_code># Create ML model instance
svr = SVR()

# Model fitting
svr.fit(X_train, y_train.ravel())

# Making predictions
y_pred_svr = svr.predict(X_test).reshape(-1, 1)<jupyter_output><empty_output><jupyter_text>Random Forest Regression Model<jupyter_code># Create ML model instance
RF = RandomForestRegressor(n_estimators = 10, random_state = 0)

# Model fitting
RF.fit(X_train, y_train.ravel())

# Making predictions
y_pred_RF = RF.predict(X_test).reshape(-1, 1)<jupyter_output><empty_output><jupyter_text>K-Nearest Neighbors Regression Model<jupyter_code># Create ML model instance
knn = KNeighborsRegressor()

# Model fitting
knn.fit(X_train, y_train)

# Making predictions
y_pred_knn = knn.predict(X_test)<jupyter_output><empty_output><jupyter_text>Step V: Performance Evaluation  Actual vs. Predicted Results<jupyter_code># Combine all true and predicted targets
results = np.concatenate((y_test, y_pred_MLR, y_pred_svr, y_pred_RF, y_pred_knn), axis = 1)

# Create a dataframe
R = pd.DataFrame(results, columns = ['Actual', 'MLR', 'SVR', 'RF', 'KNN'])

# Print first 10 values
R.head(10)<jupyter_output><empty_output><jupyter_text>Model performance<jupyter_code># ML models
models = ['Multiple Linear Regression', 'Support Vector Regression', 
          'Random Forest Regression', 'K-Nearest Neighbors Regression']
model_instance = [MLR, svr, RF, knn]

# Predictions drawn
y_pred = [y_pred_MLR, y_pred_svr, y_pred_RF, y_pred_knn]

# Performance evaluation loop
for i in range(0, len(models)):
    print(models[i])
    print("Mean Absolute Error = ", mean_absolute_error(y_test, y_pred[i]))
    print("Mean Squared Error = ", mean_squared_error(y_test, y_pred[i]))
    print("Root Mean Squared Error = ", (mean_squared_error(y_test, y_pred[i]))**0.5)
    print("R-Squared = ", r2_score(y_test, y_pred[i]), '\n')<jupyter_output>Multiple Linear Regression
Mean Absolute Error =  3.598266124779354
Mean Squared Error =  20.368383002885913
Root Mean Squared Error =  4.513134498647909
R-Squared =  0.9304112159477682 

Support Vector Regression
Mean Absolute Error =  3.196791269513175
Mean Squared Error =  17.357916342781074
Root Mean Squared Error =  4.16628327682853
R-Squared =  0.9406965053728933 

Random Forest Regression
Mean Absolute Error =  2.585423894113549
Mean Squared Error =  12.234531647509579
Root Mean Squared Error =  3.497789537337771
R-Squared =  0.9582006003776485 

K-Nearest Neighbors Regression
Mean Absolute Error =  2.7958739115290845
Mean Squared Error =  14.039791547196105
Root Mean Squared Error =  3.746970982966922
R-Squared =  0.9520329118920363<jupyter_text>Step VI: 10-fold Cross-Validation<jupyter_code># Initialize an array of zeros to store predicted values
CV_pred = np.zeros((np.shape(y)[0], 4))

for i in range(0, len(models)):
    # Cross validated predictions
    CV_pred[:, i] = cross_val_predict(model_instance[i], df, y.ravel(), cv = 10)
    
    # Evaluation metrics
    cv_MAE = cross_val_score(model_instance[i], X, y.ravel(), scoring = 'neg_mean_absolute_error', cv=10)
    cv_MSE = cross_val_score(model_instance[i], X, y.ravel(), scoring = 'neg_mean_squared_error', cv=10)
    cv_RMSE = abs(cv_MSE)**0.5
    cv_r2score = cross_val_score(model_instance[i], X, y.ravel(), scoring = 'r2', cv=10)
    
    # Printed averaged results
    print(models[i])
    print("Average MAE: ", abs(cv_MAE.mean()))
    print("Average MSE: ", abs(cv_MSE.mean()))
    print("Average RMSE: ", cv_RMSE.mean())
    print("Average R-squared: ", cv_r2score.mean(), '\n')<jupyter_output>Multiple Linear Regression
Average MAE:  3.62786188243413
Average MSE:  20.793875111034232
Average RMSE:  4.556585045760783
Average R-squared:  0.9285245252500409 

Support Vector Regression
Average MAE:  3.157806073053007
Average MSE:  17.360371462326846
Average RMSE:  4.163614805460648
Average R-squared:  0.9403123521770169 

Random Forest Regression
Average MAE:  2.4329963826331418
Average MSE:  11.81313189916504
Average RMSE:  3.4312832280722008
Average R-squared:  0.9593942187651756 

K-Nearest Neighbors Regression
Average MAE:  2.688650416223992
Average MSE:  13.944028127522373
Average RMSE:  3.7312458913672515
Average R-squared:  0.952062398647179<jupyter_text>Actual power output vs CV-predictions<jupyter_code>cv_result =  pd.DataFrame(np.concatenate((y, CV_pred), axis=1), columns = ['Actual', 'MLR', 'SVR', 'RF', 'KNN'])

# Print first 10 values
cv_result.head(10)<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.11171277221895233, 'fraction_numerical': 0.08063566804002355, 'mean_word_length': 5.695035460992908, 'pattern_counts': {'":': 1, '<': 77, '<?xml version=': 0, '>': 77, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '19', 'dir': 'jupyter-structured-clean-dedup', 'id': '517498', 'license': 'Apache-2.0', 'n_tokens_mistral': 3568, 'n_tokens_neox': 3037, 'n_words': 731, 'path': 'electrical power output prediction in a CCPP.ipynb', 'repo_name': 'rimshasaeed/Net-Hourly-Electrical-Power-Prediction-via-Regression', 'size': '8515'}