<jupyter_start><jupyter_text>Recurrent Neural Networks ModelsIn this notebook, we show an example of how RNNs can be used with darts.If you are new to darts, we recommend you first follow the `darts-intro.ipynb` notebook.<jupyter_code>%load_ext autoreload
%autoreload 2
%matplotlib inline
import torch
import torch.nn as nn
import torch.optim as optim
import numpy as np
import pandas as pd
import shutil
from sklearn.preprocessing import MinMaxScaler
from tqdm import tqdm_notebook as tqdm

from torch.utils.tensorboard import SummaryWriter
import matplotlib.pyplot as plt

from darts import TimeSeries
from darts.preprocessing import ScalerWrapper
from darts.models import RNNModel, ExponentialSmoothing
from darts.metrics import mape
from darts.utils.statistics import check_seasonality, plot_acf

import warnings
warnings.filterwarnings("ignore")
import logging
logging.disable(logging.CRITICAL)<jupyter_output><empty_output><jupyter_text>Some hyper-parameters:<jupyter_code># Number of previous time stamps taken into account.
SEQ_LENGTH = 12
# Number of features in last hidden state
HIDDEN_SIZE = 25
# number of output time-steps to predict
OUTPUT_LEN = 1
# Number of stacked rnn layers.
NUM_LAYERS = 1<jupyter_output><empty_output><jupyter_text>Air Passenger Example<jupyter_code># Read data:
df = pd.read_csv('AirPassengers.csv', delimiter=",")
series = TimeSeries.from_dataframe(df, 'Month', '#Passengers')

# Create training and validation sets:
train, val = series.split_after(pd.Timestamp('19590101'))

# Normalize the time series (note: we avoid fitting the transformer on the validation set)
transformer = ScalerWrapper()
train_transformed = transformer.fit_transform(train)
val_transformed = transformer.transform(val)
series_transformed = transformer.transform(series)<jupyter_output><empty_output><jupyter_text>Let's train an LSTM neural net. For using vanilla RNN or GRU instead, replace `'LSTM'` by `'RNN'` or `'GRU'`, respectively.<jupyter_code>my_model = RNNModel(
    model='LSTM',
    output_length=OUTPUT_LEN,
    hidden_size=HIDDEN_SIZE,
    n_rnn_layers=NUM_LAYERS,
    input_length=SEQ_LENGTH,
    batch_size=16,
    n_epochs=400,
    optimizer_kwargs={'lr': 1e-3}, 
    model_name='Air_RNN', log_tensorboard=True
)
my_model.fit(train_transformed, val_transformed, verbose=True)  # 107<jupyter_output><empty_output><jupyter_text>Look at predictions on the validation setUse the "current" model - i.e., the model at the end of the training procedure:<jupyter_code>def eval_model(model):
    pred_series = model.predict(n=26)
    plt.figure(figsize=(8,5))
    series_transformed.plot(label='actual')
    pred_series.plot(label='forecast')
    plt.title('MAPE: {:.2f}%'.format(mape(pred_series, val_transformed)))
    plt.legend();
    
eval_model(my_model)<jupyter_output><empty_output><jupyter_text>Use the best model obtained over training, according to validation loss:<jupyter_code>best_model = RNNModel.load_from_checkpoint(model_name='Air_RNN', best=True)
eval_model(best_model)<jupyter_output>loading model_best_290.pth.tar<jupyter_text>BacktestingLet's backtest an `RNN` model, with a forecast horizon of 6 months:<jupyter_code>from darts.backtesting import backtest_forecasting

my_model = RNNModel(
    model='RNN',
    output_length=OUTPUT_LEN,
    hidden_size=HIDDEN_SIZE,
    n_rnn_layers=NUM_LAYERS,
    input_length=SEQ_LENGTH,
    batch_size=32,
    n_epochs=200,
    optimizer_kwargs={'lr': 1e-3},
    model_name='Air_RNN'
)

# Perform the actual backtest
backtest_series = backtest_forecasting(series_transformed, my_model, pd.Timestamp('19590101'), 
                                       fcast_horizon_n=6, verbose=True)
plt.figure(figsize=(8,5))
series_transformed.plot(label='actual', lw=2)
backtest_series.plot(label='backtest', lw=2)
plt.legend()
plt.title('Backtest, starting Jan 1959, with a 6-months horizon');
print('MAPE: {:.2f}%'.format(mape(transformer.inverse_transform(series_transformed), 
                                 transformer.inverse_transform(backtest_series))))<jupyter_output>MAPE: 7.95%<jupyter_text>Trials with a longer output length<jupyter_code>my_model_gru = RNNModel(
    model='GRU',
    output_length=OUTPUT_LEN*4,
    input_length=SEQ_LENGTH,
    hidden_size=HIDDEN_SIZE,
    n_rnn_layers=NUM_LAYERS,
    batch_size=64,
    n_epochs=1500,
    model_name='Air_GRU_out12',
    log_tensorboard=True
)
my_model_gru.fit(train_transformed, val_series=val_transformed, verbose=True)<jupyter_output><empty_output><jupyter_text>Result of training Test with horizon=28, and feeding predictions<jupyter_code>pred_series = my_model_gru.predict(n=28)
series_transformed.plot()
pred_series.plot()<jupyter_output><empty_output><jupyter_text>Monthly sunspotLet's now try a more challenging time series; that of the monthly number of sunspots since 1749. First, we build the time series from the data, and check its periodicity.<jupyter_code>df2 = pd.read_csv('monthly-sunspots.csv', delimiter=",")
series_sunspot = TimeSeries.from_dataframe(df2, 'Month', 'Sunspots')

series_sunspot.plot()
check_seasonality(series_sunspot, max_lag=240)
plot_acf(series_sunspot, 125, max_lag=240) # ~11 years seasonality
train_sp, val_sp = series_sunspot.split_after(pd.Timestamp('19401001'))

transformer_sunspot = ScalerWrapper()
train_sp_transformed = transformer_sunspot.fit_transform(train_sp)
val_sp_transformed = transformer_sunspot.transform(val_sp)
series_sp_transformed = transformer.transform(series_sunspot)
SEQ_LENGTH = 125
HIDDEN_SIZE = 10
OUTPUT_LEN = 10
NUM_LAYERS = 1
my_model_sun = RNNModel(
    model='GRU',
    output_length=OUTPUT_LEN,
    input_length=SEQ_LENGTH,
    hidden_size=HIDDEN_SIZE,
    n_rnn_layers=NUM_LAYERS,
    batch_size=64,
    n_epochs=300,
    model_name='sun_GRU',
    nr_epochs_val_period=1,
    optimizer_kwargs={'lr': 1e-3},
    log_tensorboard=True
)

my_model_sun.fit(train_sp_transformed, val_series=val_sp_transformed, verbose=True)<jupyter_output><empty_output><jupyter_text>For the sake of comparison, let's also fit an exponential smoothing model:<jupyter_code>my_model_ets = ExponentialSmoothing()
my_model_ets.fit(train_sp_transformed)
# Compute the predictions with the two models (the RNN can be somewhat slow)
pred_series = my_model_sun.predict(550)
pred_series_ets = my_model_ets.predict(550)
val_sp_transformed.plot(label='actual')
pred_series.plot(label='our RNN')
pred_series_ets.plot(label='ETS')
plt.legend();<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.09720930232558139, 'fraction_numerical': 0.019069767441860466, 'mean_word_length': 6.527421236872812, 'pattern_counts': {'":': 0, '<': 43, '<?xml version=': 0, '>': 43, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 2, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '11', 'dir': 'jupyter-structured-clean-dedup', 'id': '199666', 'license': 'Apache-2.0', 'n_tokens_mistral': 2326, 'n_tokens_neox': 2164, 'n_words': 456, 'path': 'examples/RNN-examples.ipynb', 'repo_name': 'guillaumeraille/darts', 'size': '6450'}