<jupyter_start><jupyter_text>I lacked the domain knowledge of thermodynamics. So, I tried the possible statistical transformations and feature engineering that I would try when I don't understand features.1. I have normalised the target column.2. I have introduced polynomial features with the intention of explaining the target variable better from the features that explained the target variable better. But, I got better performance with polynomial features from all the 4 independent features.3. I have also tried different transformations of the data. ranging from boxcox transformation to standardising features to the same sclale. but, nothing gave significant performance improvement.4. I have tried ensemble of multiple algorithms with different hyperparametes, but wierdly the simplest one worked the best. Clearly, presence of a bit of domain knowledge would have hugely helped.<jupyter_code>import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
%matplotlib inline
import seaborn as sns
sns.set_style('darkgrid')
!pip install ppscore
import ppscore as pps
train_data = pd.read_csv('/content/drive/My Drive/do not delete/power plant energy output prediction/Train.csv')
test_data = pd.read_csv('/content/drive/My Drive/do not delete/power plant energy output prediction/Test.csv')
train_data.info()
full_data = pd.concat([train_data, test_data])
full_data.reset_index(drop=True, inplace=True)
full_data
# renaming columns for better understanding.
full_data.rename(mapper={'AT':'temperature', 'V':'exhaust_vacuum',
                         'AP':'ambient_pressure', 'RH':'relative_humidity',
                         'PE':'power_estimated'},
                 axis='columns', inplace=True)

full_data
full_data.loc[:9567,'power_estimated'] = np.log1p(full_data.loc[:9567,'power_estimated'])
for col in ['temperature', 'exhaust_vacuum', 'ambient_pressure', 'relative_humidity']:
    full_data.loc[:,col] = np.log(full_data.loc[:,col])
full_data
from sklearn.preprocessing import PolynomialFeatures
pf = PolynomialFeatures(degree=3, interaction_only=False,  
                        include_bias=False)
pf_one = pf.fit_transform(full_data.loc[:, full_data.columns != 'power_estimated'])
pf_one
pf_one.shape
poly_features = pd.DataFrame(data=pf_one, columns=['pf_col_'+str(i) for i in range(34)])
poly_features
full_data_new = pd.concat([full_data, poly_features], join='outer', axis=1)
full_data_new.describe()<jupyter_output><empty_output><jupyter_text>modelling<jupyter_code>train_new = full_data_new.loc[:9_567,:]
test_new = full_data_new.loc[9_568:,full_data_new.columns != 'power_estimated']
train_data.shape, train_new.shape, test_data.shape, test_new.shape
from sklearn.model_selection import train_test_split

X_tr, X_val, y_tr, y_val = train_test_split(train_new.loc[:, train_new.columns != 'power_estimated'],
                                            train_new.loc[:,'power_estimated'],
                                            test_size=0.05, random_state=100, shuffle=True)

X_tr.shape, X_val.shape, y_tr.shape, y_val.shape<jupyter_output><empty_output><jupyter_text>XG Boost<jupyter_code>xgb_one = XGBRegressor(max_depth=6, learning_rate=0.01, n_estimators=10_000, colsample_bytree=0.85, seed=100)
xgb_one.fit(X=X_tr, y=y_tr, eval_set=[(X_val, y_val)], eval_metric='rmse', early_stopping_rounds=500, verbose=100)
xgb_predict_one = xgb_one.predict(data=test_new, validate_features=True)
xgb_predict_one = np.expm1(xgb_predict_one)
xgb_predict_one = pd.Series(xgb_predict_one)
xgb_predict_one = xgb_predict_one.rename('PE')
xgb_predict_one
xgb_predict_one.to_csv(path_or_buf='/content/drive/My Drive/do not delete/power plant energy output prediction/three_xgb_predict_with_pf_unrounded.csv', index=False, float_format='%.7f')<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.10063224446786091, 'fraction_numerical': 0.014488935721812434, 'mean_word_length': 5.546551724137931, 'pattern_counts': {'":': 0, '<': 13, '<?xml version=': 0, '>': 13, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 4, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '3', 'dir': 'jupyter-structured-clean-dedup', 'id': '439158', 'license': 'MIT', 'n_tokens_mistral': 1247, 'n_tokens_neox': 1195, 'n_words': 288, 'path': 'fourth_attempt_best_result.ipynb', 'repo_name': 'naveen-9697/53rd-place-solution-for-Power-Plant-Energy-Output-Prediction-Hackathon', 'size': '3796'}