<jupyter_start><jupyter_text>Forest Cover Deep Learning ModelIn this project, we will take raw, non-scaled and non-preprocessed, datafrom the US Geological Survery (USGS) and the US Forest Service (USFS).The USFS Region 2 cartographic data is details forest cover type in a 30x30 meter cell, and has the following covertypes:- Spruce/Fir- Lodgepole Pine- Ponderosa Pine- Cottonwood/Willow- Aspen- Douglas-fir- KrummholzIndependent variables were from the USGS and USFS and are arranged in binary columns of data.It contains qualitative independent variables e.g. wilderness areas and soil type.Overall, the data encompasses four wilderness areas in the Roosevelt National Forestin Colorado. These areas are comprised of forests with minimal human-caused disturbances,which is lets us focus on change as a result of the environment, rather than forest management practices. Objectives- Develop classifiers for this classification problem.- Utilize TensorFlow (tf) with Keras in the build.- Tune the hyperparameters and optimize performance of the model.  ImportsWe'll be using pandas, numpy, sklearn, tensorflow, and matplotlib, seaborn, and sweetviz for graphing.<jupyter_code>import pandas as pd
import numpy as np
import os

from sklearn.preprocessing import StandardScaler
from sklearn.model_selection import train_test_split
from sklearn.metrics import confusion_matrix
from sklearn.metrics import classification_report
import tensorflow as tf
from tensorflow import keras
from tensorflow.keras import layers
from tensorflow.keras.callbacks import EarlyStopping
import matplotlib.pyplot as plt
%matplotlib inline
import seaborn as sns
import sweetviz as sv<jupyter_output><empty_output><jupyter_text>First we'll suppress verbose logging from tf.<jupyter_code># Disable warnings
tf.get_logger().setLevel('ERROR')<jupyter_output><empty_output><jupyter_text>Then we'll check for GPU availability and run off that if possible.<jupyter_code># disable logging
os.environ["TF_CPP_MIN_LOG_LEVEL"] = '2'

# checks for GPU and CUDA, if not available defaults to CPU.
try:
    print("Num GPUs Available: ", len(tf.config.list_physical_devices('GPU')))
    if len(tf.config.list_physical_devices('GPU')) < 1:
        os.environ['CUDA_VISIBLE_DEVICES'] = '-1'
    
except: 
    os.environ['CUDA_VISIBLE_DEVICES'] = '-1'<jupyter_output><empty_output><jupyter_text>Helper FunctionsLet's define some helper functions! We need to accomplish the following:- Clean our data- Construct our model- Plot out our data/models We'll clean our data, ensuring that it is normalized. Let's also split it into training and test sets.<jupyter_code>def clean_data(fresh_df):
    '''
    Prepare data the data for ML/DL
    - separates features from class variables
    - splits into training and testing dataset
    - scales the numerical data

    param: a dataframe of the input data
    output: x_train_normalized, x_test_normalized, y_train, y_test
    '''

    # getting the values from the df
    fresh_data = fresh_df.values
    # grabbing all rows and columns into X, y
    x, y = fresh_data[:, :-1], fresh_data[:, -1]

    # Split into train and test set
    x_train, x_test, y_train, y_test = train_test_split(x, y, test_size=0.20, random_state=42, stratify=y)

    # normalizing the data with StandardScaler
    scaler = StandardScaler()
    x_train_normalized = scaler.fit_transform(x_train)
    x_test_normalized = scaler.transform(x_test)

    return x_train_normalized, x_test_normalized, y_train, y_test<jupyter_output><empty_output><jupyter_text>Now we can construct our model.<jupyter_code>def construct_model(num_features):
    '''
    Build the model architecture (and compile it).
    input: the number of features
    output: Keras object
    '''
    classifier = keras.Sequential()

    # adding the input layer
    # defining a depth of 64 with relu activation
    classifier.add(layers.Dense(64, input_dim=num_features, activation='relu'))
    # adding dropout layers to prevent over-fitting
    classifier.add(layers.Dropout(0.3))

    # adding the hidden layer
    classifier.add(layers.Dense(32, activation='relu'))
    # adding dropout layers to prevent over-fitting
    classifier.add(layers.Dropout(0.3))

    # adding the output layer
    classifier.add(layers.Dense(8, activation='softmax'))

    # compiling the model, using adam, sparse_categorical_crossentropy (because ints), and accuracy metrics
    classifier.compile(
        optimizer='adam',
        loss='sparse_categorical_crossentropy',
        metrics=['accuracy']
    )
    return classifier<jupyter_output><empty_output><jupyter_text>Finally, we can look at our metrics!We define our confusion matrix and heatmap plot here.<jupyter_code>def heatmap_plot(class_names, y_pred, y_test):
    '''
    Creates a confusion matrix and plots it as a heatmap.
    input: class names, y-predicted, y-test (ground-truth)
    output: a PNG file of the heatmap.
    '''
    # making the matrix
    matrix = confusion_matrix(y_test, y_pred)
    fig, ax = plt.subplots(figsize=(15, 15))
    # using seaborn to plot the heatmap of the matrix
    heatmap = sns.heatmap(
        matrix,
        fmt='g',
        cmap='Blues',
        annot=True,
        ax=ax
    )
    # setting the labels and ticks
    ax.set_xlabel('Predicted class')
    ax.set_ylabel('True class')
    ax.set_title('Confusion Matrix')
    ax.xaxis.set_ticklabels(class_names)
    ax.yaxis.set_ticklabels(class_names)

    # Saving the heatmap to a file in base directory
    heatmapfig = heatmap.get_figure()
    heatmapfig.savefig(f'confusion_matrix.png')<jupyter_output><empty_output><jupyter_text>And our accuracy vs. loss history plot here.<jupyter_code>def history_plot(history, param):
    """
    Shows model performance over several epochs.
    """
    if param == 'acc':
        plt.plot(history.history['accuracy'])
        plt.plot(history.history['val_accuracy'])
        plt.title('model accuracy')
        plt.ylabel('Accuracy')
        plt.xlabel('Epoch')
        plt.legend(['Train', 'Vval'], loc='upper left')
        plt.show()

    elif param == 'loss':
        plt.plot(history.history['loss'])
        plt.plot(history.history['val_loss'])
        plt.title('model loss')
        plt.ylabel('Loss')
        plt.xlabel('Epoch')
        plt.legend(['Train', 'Val'], loc='upper right')
        plt.show()<jupyter_output><empty_output><jupyter_text>Implementing Our ModelNow that we have our helpers defined, let's run our model.<jupyter_code>def main():
    # bringing in the data and creating a df
    infile = 'cover_data.csv'
    df = pd.read_csv(infile)

    # Exploratory Data Analysis (EDA)
    my_report = sv.analyze(df)
    my_report.show_html()

    # creating our cols and features
    cols = df.columns.tolist()
    features, label = cols[:-1], cols[-1]

    # cleaning up our data
    x_train, x_test, y_train, y_test = clean_data(df)

    # Building the DL model
    n_features = len(features)
    model = construct_model(n_features)

    print("Summary report of Keras classifier:")
    model.summary()


    num_epochs = 100
    batch_size = 1024

    # early stopping to make sure we don't overrun
    earlystop_callback = EarlyStopping(monitor='val_accuracy', min_delta=0.0001, patience=3)

    # fitting our model
    history = model.fit(x_train,
                        y_train,
                        epochs=num_epochs,
                        batch_size=batch_size,
                        callbacks=[earlystop_callback],
                        validation_split=0.1,
                        verbose=1
                        )

    # plotting our accuracy and loss
    history_plot(history, 'acc')
    history_plot(history, 'loss')

    # scoring our model
    score = model.evaluate(x_test, y_test, verbose=0)
    print(f'Test loss: {score[0]}')
    print(f'Test accuracy: {score[1]}')

    y_pred = model.predict(x_test)

    # converting the predictions into discrete values
    y_pred = np.argmax(y_pred, axis=1)
    class_names = ['Spruce/Fir', 'Lodgepole Pine', 'Ponderosa Pine', 'Cottonwood/Willow', 'Aspen', 'Douglas-fir', 'Krummholz']
    print(classification_report(y_test, y_pred, target_names=class_names))

    # plotting the heatmap
    heatmap_plot(class_names, y_pred, y_test)
if __name__ == '__main__':
    main()<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.092012012012012, 'fraction_numerical': 0.006366366366366366, 'mean_word_length': 3.6959954878736605, 'pattern_counts': {'":': 0, '<': 34, '<?xml version=': 0, '>': 33, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 1, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '8', 'dir': 'jupyter-structured-clean-dedup', 'id': '634333', 'license': 'MIT', 'n_tokens_mistral': 2624, 'n_tokens_neox': 2497, 'n_words': 750, 'path': 'Data Analysis/Machine Learning/Deep Learning/forest_cover.ipynb', 'repo_name': 'DrakeOpensource/Portfolio', 'size': '8325'}