<jupyter_start><jupyter_text>General informationIn this we work with the data from ventilators connected to a sedated patient's lung. Our goal is to simulate the ventilator and correctly predict the airway pressure in the respiratory circuit during the breath.<jupyter_code># libraries
import pandas as pd
import matplotlib.pyplot as plt
%matplotlib inline
import seaborn as sns
import numpy as np
from sklearn.linear_model import LinearRegression
from xgboost import XGBRegressor<jupyter_output><empty_output><jupyter_text>Data loading and overview<jupyter_code>train = pd.read_csv('train.csv')
test = pd.read_csv('test.csv')
train.head()
# number of data points & number of features train dataset
train.shape
# getting some information about thye dataset
train.info()
# checking for missing values
train.isnull().sum()
train.describe()<jupyter_output><empty_output><jupyter_text>The dataset is quite large - more than 6 lac rows. There are 2 categorical features (R and C), the id of a row, the id of a breath, timestamp are continuous variables.<jupyter_code>fig, ax = plt.subplots(figsize = (12, 8))
plt.subplot(2, 2, 1)
sns.countplot(x='R', data=train)
plt.title('Counts of R in train');
plt.subplot(2, 2, 2)
sns.countplot(x='R', data=test)
plt.title('Counts of R in test');
plt.subplot(2, 2, 3)
sns.countplot(x='C', data=train)
plt.title('Counts of C in train');
plt.subplot(2, 2, 4)
sns.countplot(x='C', data=test)
plt.title('Counts of C in test');<jupyter_output><empty_output><jupyter_text>R and C categorical variables have a similar distribution in train and test data. Now, let's have a look at one of the series in the data.<jupyter_code>fig, ax1 = plt.subplots(figsize = (12, 8))

breath_1 = train.loc[train['breath_id'] == 1]
ax2 = ax1.twinx()

ax1.plot(breath_1['time_step'], breath_1['pressure'], 'r-', label='pressure')
ax1.plot(breath_1['time_step'], breath_1['u_in'], 'g-', label='u_in')
ax2.plot(breath_1['time_step'], breath_1['u_out'], 'b-', label='u_out')

ax1.set_xlabel('Timestep')

ax1.legend(loc=(1.1, 0.8))
ax2.legend(loc=(1.1, 0.7))
plt.show()<jupyter_output><empty_output><jupyter_text>This is quite interesting: we can see that at first the pressure (our target) is rising and then, after the u_out becomes equal to 1, it has an abrupt drop.<jupyter_code># STORE PREDICTION RELEVANT COLUMNS
test_dataset_prediction_columns = test['id']<jupyter_output><empty_output><jupyter_text>Drop Irrelevant ColumnsIn the given input dataset there are 1 column that can be removed as follows: id<jupyter_code># Drop IRRELEVANT COLUMNS
train.drop(['id'], axis=1, inplace=True)
test.drop(['id'], axis=1, inplace=True)<jupyter_output><empty_output><jupyter_text>Target ColumnThe target column is the value which we need to predict. Therefore, we need to detach the target columns in prediction. Note that if we don't drop this fields, it will generate a model with high accuracy on training and worst accuracy on test (because the value in test dataset is Null). Here is the list of target column: pressure<jupyter_code>#DETACH TARGET
feature_train = train.drop(['pressure'], axis=1)
target_train =train['pressure']
feature_test = test<jupyter_output><empty_output><jupyter_text>Training Model and PredictionFirst, we will train a model based on preprocessed values of training data set. Second, let's predict test values based on the trained model  Linear Regression<jupyter_code>model=LinearRegression()
model.fit(feature_train, target_train) 
y_pred = model.predict(feature_test)
Prediction = pd.DataFrame(columns=['id'], data=test_dataset_prediction_columns)
Prediction['pressure'] = y_pred
Prediction.head()<jupyter_output><empty_output><jupyter_text>XGBoost RegressorWe will use XGBoost Regressor which is constructing a gradient boosting model.<jupyter_code># MODEL
model = XGBRegressor()
model.fit(feature_train, target_train) 
y_pred = model.predict(feature_test)<jupyter_output><empty_output><jupyter_text>Prediction FileWe have to maintain the target columns in "Final_Prediction.csv" which will be submitted as our prediction results.<jupyter_code>Prediction = pd.DataFrame(columns=['id'], data=test_dataset_prediction_columns)
Prediction['pressure'] = y_pred
Prediction.head()
# save prediction file
Prediction.to_csv("Final_Prediction.csv", index=False)<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.10897287271041038, 'fraction_numerical': 0.011592858798979828, 'mean_word_length': 7.048507462686567, 'pattern_counts': {'":': 0, '<': 41, '<?xml version=': 0, '>': 41, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '10', 'dir': 'jupyter-structured-clean-dedup', 'id': '577841', 'license': 'MIT', 'n_tokens_mistral': 1465, 'n_tokens_neox': 1395, 'n_words': 434, 'path': 'Google Brain- Ventilator Pressure Prediction.ipynb', 'repo_name': 'GeekyR/Ventilator-Pressure-Prdiction', 'size': '4313'}