<jupyter_start><jupyter_text>Problem statement:The problem statement revolves around the need to predict the forest cover type (the predominant kind of tree cover) from strictly cartographic variables (as opposed to remotely sensed data).It includes four wilderness areas located in the Roosevelt National Forest of northern Colorado. These areas represent forests with minimal human-caused disturbances, so that existing forest cover types are more a result of ecological processes rather than forest management practices.The study area includes four wilderness areas located in the Roosevelt National Forest of northern Colorado. Each observation is a 30m x 30m patch. You are asked to predict an integer classification for the forest cover type. The seven types are:1 - Spruce/Fir2 - Lodgepole Pine3 - Ponderosa Pine4 - Cottonwood/Willow5 - Aspen6 - Douglas-fir7 - Krummholz About the datasetA zipped file containing following items is given:train.csvThe data file train.csv contains the 9072 instances with the 56 features including the target feature.test.csvThe datafile test.csv contains the 6048instances with the 55 features excluding the target feature.sample_submission.csvExplained under the Submission sub-headingFCDataDictionary.csvThe file contains data dictionary(Dictionary explaining what each feature of the dataset means) of the forest cover datasetforest_cover_student_template.ipynbA template notebook explaining the task breakdown to solve the given problem statement (Learners are recommended to use it) SubmissionAfter training of the model on train.csv data, learner has to predict the target feature of the test.csv data using the trained model. The learner has to then submit a csv file with the predicted feature.Sample submission file(sample_submission.csv) is given to you as reference to the format expected when you submitEvaluation metricsFor this particular dataset we are using accuracy_score as the evaluation metric.Submissions will be evaluated based on Accuracy score OutcomesThe main objective of this task is to provide you with an open field where you can practise and work your way with a dataset end to end without any restrictions from our side. So feel free to try out different preprocessing and feature selection techniques and apply different models of your choice and tune them until you arrive at your best solution. In this project, you will apply the following concepts:EDA & Data PreprocessingFeature SelectionModel Building and hyperparameter tuningAfter completing this project, you will have a better understanding of how to how to work on a dataset end to end and the different strategies you can use to increase the evaluation metric. Skills Covered:Hyperparmeter Tuning, Ensemble Methods, RF  Load the dataset- Load the train data and using all your knowledge of pandas try to explore the different statistical properties of the dataset.<jupyter_code>accuracy_score(y_true, y_pred, normalize=True, sample_weight=None)
def load_sb():
    df = pd.read_csv('sample_submission.csv')
#     print(df.info())
    return df

df = load_sb()
df.head()
# Code starts here
#Loading the Train data
def load_train():
    df = pd.read_csv('train.csv')
#     print(df.info())
    return df

#Loading the Test data
def load_test():
    df = pd.read_csv('test.csv')
#     print(df.info())
    return df

#Loading the sample_submission file
def load_sb():
    df = pd.read_csv('sample_submission.csv')
#     print(df.info())
    return df



train = load_train()
# train.head()
test = load_test()
# test.head()
# Storing the Id column
Id = test[['Id']]
test.drop('Id',1,inplace=True)

sb = load_sb()
# sb.head()

#Initial Random Forest Model fitting with 1000 estimators. Note below, that the labels lie in the last column, Cover Type. 
rf=RandomForestClassifier(n_estimators=1000, bootstrap=True, oob_score=True) 
rf.fit(df.ix[:,:-1].values, df.ix[:,-1:].values.ravel()) 
print("Initial Traincsv score: %.2f" %rf.score(df.ix[:,:-1].values, df.ix[:,-1:].values.ravel()))

X = df.iloc[:,:-1]
y = df.iloc[:,-1]
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.3, random_state=6)
#Initial Random Forest Model fitting with 1000 estimators. Note below, that the labels lie in the last column, Cover Type. 
rf=RandomForestClassifier(n_estimators=1000, bootstrap=True, oob_score=True) 
rf.fit(df.ix[:,:-1].values, df.ix[:,-1:].values.ravel()) 
print("Initial Traincsv score: %.2f" %rf.score(df.ix[:,:-1].values, df.ix[:,-1:].values.ravel()))
print('Train Accuracy:\n',rf.score(X_train,y_train))

accuracy_score()
# Code ends here.
#Loading the Train data
def load_train():
    df = pd.read_csv('train.csv')
#     print(df.info())
    return df

#Loading the Test data
def load_test():
    df = pd.read_csv('test.csv')
#     print(df.info())
    return df

# #Loading the sample_submission file
# def load_sb():
#     df = pd.read_csv('sample_submission.csv')
# #     print(df.info())
#     return df



train = load_train()
# train.head()
test = load_test()
# test.head()
# Storing the Id column
Id = test[['Id']]
test.drop('Id',1,inplace=True)

# sb = load_sb()
# sb.head()
#removing ID column
train.drop(columns=['Id'],axis=1,inplace=True)
# train.head()

def rff(df):
    global rf
    #Initial Random Forest Model fitting with 1000 estimators. Note below, that the labels lie in the last column, Cover Type. 
    rf=RandomForestClassifier(n_estimators=1000, bootstrap=True, oob_score=True) 
    rf.fit(df.ix[:,:-1].values, df.ix[:,-1:].values.ravel()) 
    print("Initial Traincsv score: %.2f" %rf.score(df.ix[:,:-1].values, df.ix[:,-1:].values.ravel()))

    

rff(train)



# Preprocessed Test File
y_pred = rf.predict(test)
# print('Accuracy: ',accuracy_score(y_pred,))

#predicting on test file
y_pred = pd.DataFrame(y_pred,columns=['Cover_Type']) 

submission = pd.concat([Id,y_pred['Cover_Type']],1)
submission.to_csv('sample_submission.csv',index=False)<jupyter_output>Initial Traincsv score: 1.00<jupyter_text>EDA & Data Preprocessing- The target is the only categorical feature consisting of 7 classes and all the other features are continuous. - Check out the best plots for plotting between categorical target and continuous features and try making some inferences from these plots.<jupyter_code># Code starts here



train.ix[:,:10].hist(figsize=(16,10), bins=50) 
plt.show() 





# Code ends here<jupyter_output><empty_output><jupyter_text>Feature Selection and Model building- Separate the features and target and then split the train data into train and validation set.- Apply different models of your choice and  then predict on the validation data and find the `accuracy_score` for this prediction.- Try improving upon the `accuracy_score` using different feature selection techniques like wrapper methods, PCA and try using hyperparameter tuning to find the parameters that give the best accuracy.<jupyter_code># Code Starts here


#Always call fit on the estimator before invoking this method. 
def importances(estimator, col_array, title):
    # Calculate the feature ranking - Top 10 
    importances = estimator.feature_importances_ 
    indices = np.argsort(importances)[::-1] 
    print ("%s Top 20 Important Features\n" %title)
    for f in range(20): 
        print("%d. %s (%f)" % (f + 1, col_array.columns[indices[f]], importances[indices[f]])) 
        #Mean Feature Importance 
    print("\nMean Feature Importance %.6f" %np.mean(importances))
    #Plot the feature importances of the forest 
    indices=indices[:10] 
    plt.figure() 
    plt.title(title+" Top 10 Feature importances") 
    plt.bar(range(10), importances[indices], 
                color="gr", align="center") 
    plt.xticks(range(10), col_array.columns[indices], fontsize=14, rotation=90) 
    plt.xlim([-1, 10]) 
    plt.show() 
        
#Call the method we just created to display the feature importances 
importances(rf, train, "Cover Type (Initial RF)")





# Code ends here<jupyter_output><empty_output><jupyter_text>Prediction on the test data and creating the sample submission file.- Load the test data and store the `Id` column in a separate variable.- Perform the same operations on the test data that you have performed on the train data.- Create the submission file as a `csv` file consisting of the `Id` column from the test data and your prediction as the second column.<jupyter_code># Code starts here
#Loading the Train data
def load_train():
    df = pd.read_csv('train.csv')
#     print(df.info())
    return df

#Loading the Test data
def load_test():
    df = pd.read_csv('test.csv')
#     print(df.info())
    return df

#Loading the sample_submission file
def load_sb():
    df = pd.read_csv('sample_submission.csv')
#     print(df.info())
    return df

train = load_train()
# train.head()
test = load_test()
# test.head()
sb = load_sb()
# sb.head()

X = df.iloc[:,:-1]
y = df.iloc[:,-1]
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.3, random_state=6)
#Initial Random Forest Model fitting with 1000 estimators. Note below, that the labels lie in the last column, Cover Type. 
rf=RandomForestClassifier(n_estimators=1000, bootstrap=True, oob_score=True) 
rf.fit(df.ix[:,:-1].values, df.ix[:,-1:].values.ravel()) 
print("Initial Traincsv score: %.2f" %rf.score(df.ix[:,:-1].values, df.ix[:,-1:].values.ravel()))
print('Train Accuracy:\n',rf.score(X_train,y_train))


# Code ends here.<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.10368247904064523, 'fraction_numerical': 0.013265414411546217, 'mean_word_length': 4.694259818731118, 'pattern_counts': {'":': 0, '<': 16, '<?xml version=': 0, '>': 16, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '4', 'dir': 'jupyter-structured-clean-dedup', 'id': '533950', 'license': 'MIT', 'n_tokens_mistral': 2916, 'n_tokens_neox': 2748, 'n_words': 1033, 'path': 'forest_cover_student_template.ipynb', 'repo_name': 'anajikadam17/forest-cover-prediction', 'size': '9423'}