<jupyter_start><jupyter_text>Deep learning examples     This notebooks contains examples with neural network models.**Table of Contents*** [Creating TSDataset](chapter1)  * [Architecture](chapter2)* [Testing models](chapter3)    * [DeepAR](section_3_1)    * [TFT](section_3_2)    * [Simple Model](section_3_3)<jupyter_code>import torch
import random

import pandas as pd
import numpy as np

from etna.datasets.tsdataset import TSDataset
from etna.transforms import DateFlagsTransform
from etna.transforms import LagTransform
from etna.transforms import LinearTrendTransform
from etna.metrics import SMAPE, MAPE, MAE
from etna.model_selection import TimeSeriesCrossValidation
from etna.analysis import plot_backtest
from etna.models import SeasonalMovingAverageModel

import warnings
warnings.filterwarnings("ignore")<jupyter_output><empty_output><jupyter_text>1. Creating TSDataset  We are going to take transformed [Household Electric Power Consumption] dataset. Let's load and look at it.<jupyter_code>original_df = pd.read_csv("data/example_dataset.csv")
original_df.head()<jupyter_output><empty_output><jupyter_text>Our library works with the spacial data structure TSDataset. Let's create it as it was done in "Get started" notebook.<jupyter_code>df = TSDataset.to_dataset(original_df)
ts = TSDataset(df, freq="D")
ts.head(5)<jupyter_output><empty_output><jupyter_text>2. Architecture  Our library uses [PyTorch Forecasting](https://github.com/jdb78/pytorch-forecasting) to work with time series neural networks. To include it in our current architecture we use `PytorchForecastingTransform` class.Let's look at it closer.<jupyter_code>from etna.transforms import PytorchForecastingTransform
?PytorchForecastingTransform<jupyter_output><empty_output><jupyter_text>```python"""Init signature:PytorchForecastingTransform(    max_encoder_length: int = 30,    min_encoder_length: int = None,    min_prediction_idx: int = None,    min_prediction_length: int = None,    max_prediction_length: int = 1,    static_categoricals: List[str] = [],    static_reals: List[str] = [],    time_varying_known_categoricals: List[str] = [],    time_varying_known_reals: List[str] = [],    time_varying_unknown_categoricals: List[str] = [],    time_varying_unknown_reals: List[str] = [],    variable_groups: Dict[str, List[int]] = {},    dropout_categoricals: List[str] = [],    constant_fill_strategy: Dict[str, Union[str, float, int, bool]] = {},    allow_missings: bool = True,    lags: Dict[str, List[int]] = {},    add_relative_time_idx: bool = True,    add_target_scales: bool = True,    add_encoder_length: Union[bool, str] = True,    target_normalizer: Union[pytorch_forecasting.data.encoders.TorchNormalizer, pytorch_forecasting.data.encoders.NaNLabelEncoder, pytorch_forecasting.data.encoders.EncoderNormalizer, str, List[Union[pytorch_forecasting.data.encoders.TorchNormalizer, pytorch_forecasting.data.encoders.NaNLabelEncoder, pytorch_forecasting.data.encoders.EncoderNormalizer]], Tuple[Union[pytorch_forecasting.data.encoders.TorchNormalizer, pytorch_forecasting.data.encoders.NaNLabelEncoder, pytorch_forecasting.data.encoders.EncoderNormalizer]]] = 'auto',    categorical_encoders: Dict[str, pytorch_forecasting.data.encoders.NaNLabelEncoder] = None,    scalers: Dict[str, Union[sklearn.preprocessing._data.StandardScaler, sklearn.preprocessing._data.RobustScaler, pytorch_forecasting.data.encoders.TorchNormalizer, pytorch_forecasting.data.encoders.EncoderNormalizer]] = {},)Docstring:      Transform for models from PytorchForecasting library.Init docstring:Parameters for TimeSeriesDataSet object.Reference---------https://github.com/jdb78/pytorch-forecasting/blob/v0.8.5/pytorch_forecasting/data/timeseries.pyL117"""``` We can see a pretty scary signature, but don't panic, we will look at the most important parameters. * `time_varying_known_reals` &mdash; known real values that change across the time (real regressors), now it it necessary to add "time_idx" variable to the list;* `time_varying_unknown_reals` &mdash; our real value target, set it to `["target"]`;* `max_prediction_length` &mdash; our horizon for forecasting;* `max_encoder_length` &mdash; length of past context to use;* `static_categoricals` &mdash; static categorical values, for example, if we use multiple segments it can be some its characteristics including identifier: "segment";* `time_varying_known_categoricals` &mdash; known categorical values that change across the time (categorical regressors);* `target_normalizer` &mdash; class for normalization targets across different segments. Our library currently supports these models:* [DeepAR](https://arxiv.org/abs/1704.04110),* [TFT](https://arxiv.org/abs/1912.09363).  3. Testing models  In this section we will test our models on example.  3.1 DeepAR  Before training let's fix seeds for reproducibility.<jupyter_code>torch.manual_seed(42)
random.seed(42)
np.random.seed(42)<jupyter_output><empty_output><jupyter_text>Creating transforms for DeepAR.<jupyter_code>from pytorch_forecasting.data import GroupNormalizer

HORIZON = 7

transform_date = DateFlagsTransform(day_number_in_week=True, day_number_in_month=False, out_column="regressor_dateflag")
num_lags = 10
transform_lag = LagTransform(in_column="target", lags=[HORIZON+i for i in range(num_lags)], out_column="regressor_target_lag")
lag_columns = [f"regressor_target_lag_{HORIZON+i}" for i in range(num_lags)]

transform_deepar = PytorchForecastingTransform(
    max_encoder_length=HORIZON,
    max_prediction_length=HORIZON,
    time_varying_known_reals=["time_idx"]+lag_columns,
    time_varying_unknown_reals=["target"],
    time_varying_known_categoricals=["regressor_dateflag_day_number_in_week"],
    target_normalizer=GroupNormalizer(groups=["segment"]),
)<jupyter_output><empty_output><jupyter_text>Now we are going to start backtest.<jupyter_code>from etna.models.nn import DeepARModel


model_deepar = DeepARModel(max_epochs=150, learning_rate=[0.01], gpus=0, batch_size=64)
metrics = [SMAPE(), MAPE(), MAE()]

tscv_deepar = TimeSeriesCrossValidation(
    model=model_deepar, horizon=HORIZON, metrics=metrics, n_folds=3, n_jobs=1
)
results_deepar = tscv_deepar.backtest(ts, transforms=[transform_lag, transform_date, transform_deepar])<jupyter_output>[Parallel(n_jobs=1)]: Using backend SequentialBackend with 1 concurrent workers.
GPU available: False, used: False
TPU available: False, using: 0 TPU cores
IPU available: False, using: 0 IPUs

  | Name                   | Type                   | Params
------------------------------------------------------------------
0 | loss                   | NormalDistributionLoss | 0     
1 | logging_metrics        | ModuleList             | 0     
2 | embeddings             | MultiEmbedding         | 35    
3 | rnn                    | LSTM                   | 2.2 K 
4 | distribution_projector | Linear                 | 22    
------------------------------------------------------------------
2.3 K     Trainable params
0         Non-trainable params
2.3 K     Total params
0.009     Total estimated model params size (MB)<jupyter_text>Let's compare results across different segments.<jupyter_code>results_deepar[0]<jupyter_output><empty_output><jupyter_text>To summarize it we will take mean value of SMAPE metric because it is scale tolerant.<jupyter_code>score = results_deepar[0]["SMAPE"].mean()
print(f"Average SMAPE for DeepAR: {score:.3f}")<jupyter_output>Average SMAPE for DeepAR: 10.592<jupyter_text>Visualize results.<jupyter_code>plot_backtest(results_deepar[1], ts, history_len=20)<jupyter_output><empty_output><jupyter_text>3.2 TFT  Let's move to the next model.<jupyter_code>torch.manual_seed(42)
random.seed(42)
np.random.seed(42)
transform_date = DateFlagsTransform(day_number_in_week=True, day_number_in_month=False, out_column="regressor_dateflag")
num_lags = 10
transform_lag = LagTransform(in_column="target", lags=[HORIZON+i for i in range(num_lags)], out_column="regressor_target_lag")
lag_columns = [f"regressor_target_lag_{HORIZON+i}" for i in range(num_lags)]

transform_tft = PytorchForecastingTransform(
    max_encoder_length=HORIZON,
    max_prediction_length=HORIZON,
    time_varying_known_reals=["time_idx"],
    time_varying_unknown_reals=["target"],
    time_varying_known_categoricals=["regressor_dateflag_day_number_in_week"],
    static_categoricals=["segment"],
    target_normalizer=GroupNormalizer(groups=["segment"]),
)
from etna.models.nn import TFTModel

model_deepar = TFTModel(max_epochs=200, learning_rate=[0.01], gpus=0, batch_size=64)

tscv_tft = TimeSeriesCrossValidation(
    model=model_deepar, horizon=HORIZON, metrics=metrics, n_folds=3, n_jobs=1
)
results_tft = tscv_tft.backtest(ts, transforms=[transform_lag, transform_date, transform_tft])
results_tft[0]
score = results_tft[0]["SMAPE"].mean()
print(f"Average SMAPE for TFT: {score:.3f}")
plot_backtest(results_tft[1], ts, history_len=20)<jupyter_output><empty_output><jupyter_text>3.3 Simple model  For comparison let's train a much more simpler model.<jupyter_code>model_sma = SeasonalMovingAverageModel(window=5, seasonality=7)

tscv_sma = TimeSeriesCrossValidation(
    model=model_sma, horizon=HORIZON, metrics=metrics, n_folds=3, n_jobs=1
)
linear_trend_transform = LinearTrendTransform(in_column='target')
results_sma = tscv_sma.backtest(ts, transforms=[linear_trend_transform])
results_sma[0]
score = results_sma[0]["SMAPE"].mean()
print(f"Average SMAPE for Seasonal MA: {score:.3f}")
plot_backtest(results_sma[1], ts, history_len=20)<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.1152353989724232, 'fraction_numerical': 0.01488937821117752, 'mean_word_length': 5.881673881673882, 'pattern_counts': {'":': 0, '<': 47, '<?xml version=': 0, '>': 47, 'https://': 4, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 2, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '12', 'dir': 'jupyter-structured-clean-dedup', 'id': '206095', 'license': 'Apache-2.0', 'n_tokens_mistral': 3281, 'n_tokens_neox': 3114, 'n_words': 648, 'path': 'examples/NN_examples.ipynb', 'repo_name': 'Carlosbogo/etna', 'size': '9537'}