title timeseries classification from scratch hfawazhttps github comhfawaz date created 20200721 last modified 20210716 description training a timeseries classifier from scratch on the forda dataset from the ucruea archive accelerator gpu introduction this example shows how to do timeseries classification from scratch starting from raw csv timeseries files on disk we demonstrate the workflow on the forda dataset from the ucruea archivehttps www cs ucr edu7eeamonntimeseriesdata2018 setup load the data the forda dataset dataset description the dataset we are using here is called forda the data comes from the ucr archive the dataset contains 3601 training instances and another 1320 testing instances each timeseries corresponds to a measurement of engine noise captured by a motor sensor for this task the goal is to automatically detect the presence of a specific issue with the engine the problem is a balanced binary classification task the full description of this dataset can be found herehttp www jwichard depublicationsfordpaper pdf read the tsv data we will use the fordatrain file for training and the fordatest file for testing the simplicity of this dataset allows us to demonstrate effectively how to use convnets for timeseries classification in this file the first column corresponds to the label visualize the data here we visualize one timeseries example for each class in the dataset standardize the data our timeseries are already in a single length 500 however their values are usually in various ranges this is not ideal for a neural network in general we should seek to make the input values normalized for this specific dataset the data is already znormalized each timeseries sample has a mean equal to zero and a standard deviation equal to one this type of normalization is very common for timeseries classification problems see bagnall et al 2016https link springer comarticle10 1007s1061801604839 note that the timeseries data used here are univariate meaning we only have one channel per timeseries example we will therefore transform the timeseries into a multivariate one with one channel using a simple reshaping via numpy this will allow us to construct a model that is easily applicable to multivariate time series finally in order to use sparsecategoricalcrossentropy we will have to count the number of classes beforehand now we shuffle the training set because we will be using the validationsplit option later when training standardize the labels to positive integers the expected labels will then be 0 and 1 build a model we build a fully convolutional neural network originally proposed in this paperhttps arxiv orgabs1611 06455 the implementation is based on the tf 2 version provided herehttps github comhfawazdl4tsc the following hyperparameters kernelsize filters the usage of batchnorm were found via random search using kerastunerhttps github comkerasteamkerastuner train the model evaluate model on test data plot the model s training and validation loss we can see how the training accuracy reaches almost 0 95 after 100 epochs however by observing the validation accuracy we can see how the network still needs training until it reaches almost 0 97 for both the validation and the training accuracy after 200 epochs beyond the 200th epoch if we continue on training the validation accuracy will start decreasing while the training accuracy will continue on increasing the model starts overfitting introduction this example shows how to do timeseries classification from scratch starting from raw csv timeseries files on disk we demonstrate the workflow on the forda dataset from the ucr uea archive https www cs ucr edu 7eeamonn time_series_data_2018 setup load the data the forda dataset dataset description the dataset we are using here is called forda the data comes from the ucr archive the dataset contains 3601 training instances and another 1320 testing instances each timeseries corresponds to a measurement of engine noise captured by a motor sensor for this task the goal is to automatically detect the presence of a specific issue with the engine the problem is a balanced binary classification task the full description of this dataset can be found here http www j wichard de publications fordpaper pdf read the tsv data we will use the forda_train file for training and the forda_test file for testing the simplicity of this dataset allows us to demonstrate effectively how to use convnets for timeseries classification in this file the first column corresponds to the label visualize the data here we visualize one timeseries example for each class in the dataset standardize the data our timeseries are already in a single length 500 however their values are usually in various ranges this is not ideal for a neural network in general we should seek to make the input values normalized for this specific dataset the data is already z normalized each timeseries sample has a mean equal to zero and a standard deviation equal to one this type of normalization is very common for timeseries classification problems see bagnall et al 2016 https link springer com article 10 1007 s10618 016 0483 9 note that the timeseries data used here are univariate meaning we only have one channel per timeseries example we will therefore transform the timeseries into a multivariate one with one channel using a simple reshaping via numpy this will allow us to construct a model that is easily applicable to multivariate time series finally in order to use sparse_categorical_crossentropy we will have to count the number of classes beforehand now we shuffle the training set because we will be using the validation_split option later when training standardize the labels to positive integers the expected labels will then be 0 and 1 build a model we build a fully convolutional neural network originally proposed in this paper https arxiv org abs 1611 06455 the implementation is based on the tf 2 version provided here https github com hfawaz dl 4 tsc the following hyperparameters kernel_size filters the usage of batchnorm were found via random search using kerastuner https github com keras team keras tuner train the model evaluate model on test data plot the model s training and validation loss we can see how the training accuracy reaches almost 0 95 after 100 epochs however by observing the validation accuracy we can see how the network still needs training until it reaches almost 0 97 for both the validation and the training accuracy after 200 epochs beyond the 200th epoch if we continue on training the validation accuracy will start decreasing while the training accuracy will continue on increasing the model starts overfitting
import keras
import numpy as np
import matplotlib.pyplot as plt




def readucr(filename):
    data = np.loadtxt(filename, delimiter="\t")
    y = data[:, 0]
    x = data[:, 1:]
    return x, y.astype(int)


root_url = "https://raw.githubusercontent.com/hfawaz/cd-diagram/master/FordA/"

x_train, y_train = readucr(root_url + "FordA_TRAIN.tsv")
x_test, y_test = readucr(root_url + "FordA_TEST.tsv")



classes = np.unique(np.concatenate((y_train, y_test), axis=0))

plt.figure()
for c in classes:
    c_x_train = x_train[y_train == c]
    plt.plot(c_x_train[0], label="class " + str(c))
plt.legend(loc="best")
plt.show()
plt.close()



x_train = x_train.reshape((x_train.shape[0], x_train.shape[1], 1))
x_test = x_test.reshape((x_test.shape[0], x_test.shape[1], 1))



num_classes = len(np.unique(y_train))



idx = np.random.permutation(len(x_train))
x_train = x_train[idx]
y_train = y_train[idx]



y_train[y_train == -1] = 0
y_test[y_test == -1] = 0




def make_model(input_shape):
    input_layer = keras.layers.Input(input_shape)

    conv1 = keras.layers.Conv1D(filters=64, kernel_size=3, padding="same")(
        input_layer
    )
    conv1 = keras.layers.BatchNormalization()(conv1)
    conv1 = keras.layers.ReLU()(conv1)

    conv2 = keras.layers.Conv1D(filters=64, kernel_size=3, padding="same")(
        conv1
    )
    conv2 = keras.layers.BatchNormalization()(conv2)
    conv2 = keras.layers.ReLU()(conv2)

    conv3 = keras.layers.Conv1D(filters=64, kernel_size=3, padding="same")(
        conv2
    )
    conv3 = keras.layers.BatchNormalization()(conv3)
    conv3 = keras.layers.ReLU()(conv3)

    gap = keras.layers.GlobalAveragePooling1D()(conv3)

    output_layer = keras.layers.Dense(num_classes, activation="softmax")(gap)

    return keras.models.Model(inputs=input_layer, outputs=output_layer)


model = make_model(input_shape=x_train.shape[1:])
keras.utils.plot_model(model, show_shapes=True)



epochs = 500
batch_size = 32

callbacks = [
    keras.callbacks.ModelCheckpoint(
        "best_model.keras", save_best_only=True, monitor="val_loss"
    ),
    keras.callbacks.ReduceLROnPlateau(
        monitor="val_loss", factor=0.5, patience=20, min_lr=0.0001
    ),
    keras.callbacks.EarlyStopping(monitor="val_loss", patience=50, verbose=1),
]
model.compile(
    optimizer="adam",
    loss="sparse_categorical_crossentropy",
    metrics=["sparse_categorical_accuracy"],
)
history = model.fit(
    x_train,
    y_train,
    batch_size=batch_size,
    epochs=epochs,
    callbacks=callbacks,
    validation_split=0.2,
    verbose=1,
)



model = keras.models.load_model("best_model.keras")

test_loss, test_acc = model.evaluate(x_test, y_test)

print("Test accuracy", test_acc)
print("Test loss", test_loss)



metric = "sparse_categorical_accuracy"
plt.figure()
plt.plot(history.history[metric])
plt.plot(history.history["val_" + metric])
plt.title("model " + metric)
plt.ylabel(metric, fontsize="large")
plt.xlabel("epoch", fontsize="large")
plt.legend(["train", "val"], loc="best")
plt.show()
plt.close()