<jupyter_start><jupyter_text>Flight Price Prediction<jupyter_code>import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
import seaborn as sns

sns.set()<jupyter_output><empty_output><jupyter_text>Importing Dataset 1. Since data is in form of excel file we have to use pandas read_excel to load the data2. After loading it is important to check the complete information of data as it can indication many of the hidden infomation such as null values in a column or a row3. Check whether any null values are there or not. if it is present then following can be done, 1. Imputing data using Imputation method in sklearn 2. Filling NaN values with mean, median and mode using fillna() method4. Describe data --> which can give statistical analysis<jupyter_code>train_data = pd.read_excel("Data_Train.xlsx")
pd.set_option("display.max_columns",None)
train_data.head()
train_data.info()
train_data["Duration"].value_counts()
train_data.shape
train_data.dropna(inplace=True)
train_data.info()
train_data.isnull().sum()<jupyter_output><empty_output><jupyter_text>Exploratory Data Analysis From description we can see that Date_of_Journey is a object data type,Therefore, we have to convert this datatype into timestamp to use this column properly for predictionFor this we require pandas to_datetime to convert object data type to datetime dtype.**.dt.day method will extract only day of that date**\**.dt.month method will extract only month of that date**<jupyter_code>train_data["Journey_day"] =pd.to_datetime(train_data["Date_of_Journey"],format="%d/%m/%Y").dt.day
train_data["Journey_month"] =pd.to_datetime(train_data["Date_of_Journey"],format="%d/%m/%Y").dt.month
train_data.sample(5)<jupyter_output><empty_output><jupyter_text>since we have converted converted Date_of_Journey column into integers, Now we can drop as it is of no use<jupyter_code>train_data.drop(["Date_of_Journey"], axis = 1, inplace = True)
# Departure time is when a plane leaves the gate.
# Similar to the Date_of_Journey we can extract values from Dep_Time

#Extracting Hours 
train_data["Dep_hour"] = pd.to_datetime(train_data["Dep_Time"]).dt.hour

# Extracting Minutes 
train_data["Dep_min"] = pd.to_datetime(train_data["Dep_Time"]).dt.minute

# Now we can drop Dep_Time as it of no use
train_data.drop(["Dep_Time"], axis = 1, inplace = True)
train_data.head()
# Arrival time is when the plane pulls up to the gate 
# Similar to Date_of_Journey we can extract values from Arrival_Time

#Extracting Hours 
train_data["Arrival_hour"] = pd.to_datetime(train_data["Arrival_Time"]).dt.hour

# Extracting Minutes 
train_data["Arrival_min"] = pd.to_datetime(train_data["Arrival_Time"]).dt.minute

# Now we can drop Dep_Time as it of no use
train_data.drop(["Arrival_Time"], axis = 1, inplace = True)
train_data.head()
# Time taken by plane to reach destination is called Duration
# It is the differnce betwwen Departure Time and Arrival time


# Assigning and converting Duration column into list
duration = list(train_data["Duration"])

for i in range(len(duration)):
    if len(duration[i].split()) != 2:    # Check if duration contains only hour or mins
        if "h" in duration[i]:
            duration[i] = duration[i].strip() + " 0m"   # Adds 0 minute
        else:
            duration[i] = "0h " + duration[i]           # Adds 0 hour

duration_hours = []
duration_mins = []
for i in range(len(duration)):
    duration_hours.append(int(duration[i].split(sep = "h")[0]))    # Extract hours from duration
    duration_mins.append(int(duration[i].split(sep = "m")[0].split()[-1]))   # Extracts only minutes from duration
# Adding duration_hours and duration_mins list to train_data dataframe

train_data["Duration_hours"] = duration_hours
train_data["Duration_mins"] =duration_mins
train_data.head()
train_data.drop(["Duration"], axis =1, inplace=True)
train_data.head()<jupyter_output><empty_output><jupyter_text>Handling Categorical DataOne can find many ways to handle categorical data. Some of the categorical data are 1. NOMINAL DATA which is data that are not in any order, We can use OneHotEncodet to handle this data2. ORDINAL DATA which is data that are in order. We can use LabelEncoder to handle this data<jupyter_code>train_data["Airline"].value_counts()
# From grap we can see that Jet Airways Business have the highest Price
# Apart from the first Airlines almost all are having similar median

# Airline vs price 
sns.catplot(y= "Price", x = "Airline", data = train_data.sort_values("Price", ascending = False), kind="boxen", height = 6, aspect = 3)
plt.show()
# As Airline is Nominal Categorical data, we will perform OneHotEncoder

Airline = train_data[["Airline"]]
Airline = pd.get_dummies(Airline, drop_first =True)

Airline.head()
train_data["Source"].value_counts()
# Compare Source and Price
# We can see some outliers in Bangalore while the others place doe not too different

sns.catplot(y = "Price", x= "Source", data = train_data.sort_values("Price", ascending = False), kind="boxen", height = 6, aspect = 3)
# As source is Nominal categorical data, we will perform OneHotEncoding 
# Bangalore Source can be representated by OOOO
Source =train_data[["Source"]]

Source =pd.get_dummies(Source, drop_first=True)

Source.head()
train_data["Destination"].value_counts()
# As Destination is Nominal categorical data, we will perform OneHotEncoding 

Destination =train_data[["Destination"]]

Destination =pd.get_dummies(Destination, drop_first=True)

Destination.head()
train_data["Route"]
# Additional_Info contains almost 80% no_info
# Route and Total_Stops are related to each other

train_data.drop(["Route","Additional_Info"],axis=1, inplace=True)
train_data["Total_Stops"].value_counts()
# As this is case of Ordinal Categorical data, we can perform LabelEncoder
# Here values are assigned with corresponding keys

train_data.replace({"non-stop":0, "1 stop": 1, "2 stops": 2, "3 stops":3, "4 stops": 4}, inplace=True)
train_data.head()
# Concatenate dataframe that consist of train_data, Airline, Source,  and Destination
data_train =pd.concat([train_data,Airline,Source,Destination], axis=1)
data_train.head()
data_train.drop(["Airline","Source", "Destination"], axis=1, inplace=True)
data_train.head()
data_train.shape<jupyter_output><empty_output><jupyter_text>Test Set<jupyter_code>test_data =pd.read_excel(r"Test_set.xlsx")
test_data.head()
# Preprocessing

print("Test data Info")
print("-"*75)
print(test_data.info())

print()
print()

print("Null values :")
print("-"*75)
test_data.dropna(inplace = True)
print(test_data.isnull().sum())

# EDA

# Date_of_Journey
test_data["Journey_day"] = pd.to_datetime(test_data.Date_of_Journey, format="%d/%m/%Y").dt.day
test_data["Journey_month"] = pd.to_datetime(test_data["Date_of_Journey"], format = "%d/%m/%Y").dt.month
test_data.drop(["Date_of_Journey"], axis = 1, inplace = True)

# Dep_Time
test_data["Dep_hour"] = pd.to_datetime(test_data["Dep_Time"]).dt.hour
test_data["Dep_min"] = pd.to_datetime(test_data["Dep_Time"]).dt.minute
test_data.drop(["Dep_Time"], axis = 1, inplace = True)

# Arrival_Time
test_data["Arrival_hour"] = pd.to_datetime(test_data.Arrival_Time).dt.hour
test_data["Arrival_min"] = pd.to_datetime(test_data.Arrival_Time).dt.minute
test_data.drop(["Arrival_Time"], axis = 1, inplace = True)

# Duration
duration = list(test_data["Duration"])

for i in range(len(duration)):
    if len(duration[i].split()) != 2:    # Check if duration contains only hour or mins
        if "h" in duration[i]:
            duration[i] = duration[i].strip() + " 0m"   # Adds 0 minute
        else:
            duration[i] = "0h " + duration[i]           # Adds 0 hour

duration_hours = []
duration_mins = []
for i in range(len(duration)):
    duration_hours.append(int(duration[i].split(sep = "h")[0]))    # Extract hours from duration
    duration_mins.append(int(duration[i].split(sep = "m")[0].split()[-1]))   # Extracts only minutes from duration

# Adding Duration column to test set
test_data["Duration_hours"] = duration_hours
test_data["Duration_mins"] = duration_mins
test_data.drop(["Duration"], axis = 1, inplace = True)


# Categorical data

print("Airline")
print("-"*75)
print(test_data["Airline"].value_counts())
Airline = pd.get_dummies(test_data["Airline"], drop_first= True)

print()

print("Source")
print("-"*75)
print(test_data["Source"].value_counts())
Source = pd.get_dummies(test_data["Source"], drop_first= True)

print()

print("Destination")
print("-"*75)
print(test_data["Destination"].value_counts())
Destination = pd.get_dummies(test_data["Destination"], drop_first = True)

# Additional_Info contains almost 80% no_info
# Route and Total_Stops are related to each other
test_data.drop(["Route", "Additional_Info"], axis = 1, inplace = True)

# Replacing Total_Stops
test_data.replace({"non-stop": 0, "1 stop": 1, "2 stops": 2, "3 stops": 3, "4 stops": 4}, inplace = True)

# Concatenate dataframe --> test_data + Airline + Source + Destination
data_test = pd.concat([test_data, Airline, Source, Destination], axis = 1)

data_test.drop(["Airline", "Source", "Destination"], axis = 1, inplace = True)

print()
print()

print("Shape of test data : ", data_test.shape)



data_test.head()<jupyter_output><empty_output><jupyter_text>Feature SelectionFinding out the best feature which will contribute and have good relation with target variable.Following are some of the feature selection methods,1. **heatmap**2. **feature_importance_**3. **SelectKBest**<jupyter_code>data_train.shape
data_train.columns
X = data_train.loc[:, ['Total_Stops','Journey_day', 'Journey_month', 'Dep_hour',
       'Dep_min', 'Arrival_hour', 'Arrival_min', 'Duration_hours',
       'Duration_mins', 'Airline_Air India', 'Airline_GoAir', 'Airline_IndiGo',
       'Airline_Jet Airways', 'Airline_Jet Airways Business',
       'Airline_Multiple carriers',
       'Airline_Multiple carriers Premium economy', 'Airline_SpiceJet',
       'Airline_Trujet', 'Airline_Vistara', 'Airline_Vistara Premium economy',
       'Source_Chennai', 'Source_Delhi', 'Source_Kolkata', 'Source_Mumbai',
       'Destination_Cochin', 'Destination_Delhi', 'Destination_Hyderabad',
       'Destination_Kolkata', 'Destination_New Delhi']]

X.head()
y = data_train.iloc[:,1]
y.head()
#Find correlation between Independent(X) and dependent attributes(y)

plt.figure(figsize = (18,18))

sns.heatmap(train_data.corr(),annot= True, cmap = "coolwarm")

plt.show()
# Important feature using ExtraTreeRegressor

from sklearn.ensemble import ExtraTreesRegressor
selection =ExtraTreesRegressor()
selection.fit(X,y)
#ExtraTreesRegressor is used to choose the importan feature for the prediction
print(selection.feature_importances_)
# plot graph of important feature for better visualization

plt.figure(figsize =(12,8))
feat_importances =pd.Series(selection.feature_importances_,index =X.columns)
feat_importances.nlargest(20).plot(kind="barh")
plt.show()<jupyter_output><empty_output><jupyter_text>Fitting model using Random Forest1. Split dataset into train and test set in order to prediction w.r.t X_test2. If needed do scaling of data    * Scaling is not done in Random forest3. Import model4. Fit the data5. Predict w.r.t X_test6. In regression check **RSME** Score7. Plot graph<jupyter_code>from sklearn.model_selection import train_test_split
X_train, X_test,y_train,y_test =train_test_split(X,y,test_size = 0.2, random_state = 42)
from sklearn.ensemble import RandomForestRegressor
reg_rf = RandomForestRegressor()
reg_rf.fit(X_train,y_train)
y_pred =reg_rf.predict(X_test)

reg_rf.score(X_train,y_train)
reg_rf.score(X_test,y_test)
sns.displot(y_test-y_pred)
plt.show()
plt.scatter(y_test,y_pred,alpha =0.5,color="DarkBlue")
plt.xlabel("y_test")
plt.ylabel("y_pred")
plt.show()
from sklearn import metrics

print("MAE:" , metrics.mean_absolute_error(y_test,y_pred))
print("MSE:" , metrics.mean_squared_error(y_test,y_pred))
print("RMSE:" , np.sqrt(metrics.mean_squared_error(y_test,y_pred)))
metrics.r2_score(y_test, y_pred)<jupyter_output><empty_output><jupyter_text>Hyperparameter Tuning* Choose following method for hyperparameter tuning    1. **RandomizedSearchCV** --> Fast    2. **GridSearchCV*** Assign hyperparameters in form of dictionery* Fit the model* Check best paramters and best score<jupyter_code>from sklearn.model_selection import RandomizedSearchCV
#Randomized Search CV

# Number of trees in random forest
n_estimators = [int(x) for x in np.linspace(start = 100, stop = 1200, num = 12)]
# Number of features to consider at every split
max_features = ['auto', 'sqrt']
# Maximum number of levels in tree
max_depth = [int(x) for x in np.linspace(5, 30, num = 6)]
# Minimum number of samples required to split a node
min_samples_split = [2, 5, 10, 15, 100]
# Minimum number of samples required at each leaf node
min_samples_leaf = [1, 2, 5, 10]
# Create the random grid

random_grid = {'n_estimators': n_estimators,
               'max_features': max_features,
               'max_depth': max_depth,
               'min_samples_split': min_samples_split,
               'min_samples_leaf': min_samples_leaf}
# Random search of parameters, using 5 fold cross validation, 
# search across 100 different combinations
rf_random = RandomizedSearchCV(estimator = reg_rf, param_distributions = random_grid,scoring='neg_mean_squared_error', n_iter = 10, cv = 5, verbose=2, random_state=42, n_jobs = 1)
rf_random.fit(X_train,y_train)
rf_random.best_params_
prediction = rf_random.predict(X_test)
plt.figure(figsize =(8,8))
sns.displot(y_test-prediction)
plt.show()
plt.scatter(y_test,prediction,alpha =0.5,color="DarkBlue")
plt.xlabel("y_test")
plt.ylabel("y_pred")
plt.show()
print("MAE:" , metrics.mean_absolute_error(y_test,prediction))
print("MSE:" , metrics.mean_squared_error(y_test,prediction))
print("RMSE:" , np.sqrt(metrics.mean_squared_error(y_test,prediction)))<jupyter_output>MAE: 1164.6509617554095
MSE: 4041334.9589260807
RMSE: 2010.307180240393<jupyter_text>Save the model to reuse it again<jupyter_code>import pickle
# open a file, where you ant to store the data
file = open('flight_rf.pkl', 'wb')

# dump information to that file
pickle.dump(rf_random, file)

model = open('flight_rf.pkl','rb')
forest = pickle.load(model)
y_prediction =forest.predict(X_test)
metrics.r2_score(y_test,y_prediction)<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.12055101041885183, 'fraction_numerical': 0.013775260471295714, 'mean_word_length': 5.979990239141045, 'pattern_counts': {'":': 10, '<': 40, '<?xml version=': 0, '>': 43, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '10', 'dir': 'jupyter-structured-clean-dedup', 'id': '20021', 'license': 'MIT', 'n_tokens_mistral': 4992, 'n_tokens_neox': 4752, 'n_words': 1269, 'path': 'Flight_Price_Prediction.ipynb', 'repo_name': 'naiborhujosua/Flight-Price-Prediction', 'size': '14301'}