<jupyter_start><jupyter_text>We see that the forecast captures the primary data trend rather nicely.  We can add the left-out test data to the chart above by uncommenting the remaining pyplot stanza and rerunning to get an idea of how well it has been fit, and observe that the delta is acceptably small.** As a pragmatic note: **  It can often be wise to not include all historical data when training your model; this may result in overfitting if the behavior of the data changes subtlely.  Finding a balance between "enough data to capture the trend" and "not enough to overfit" is an important question to ask, and should be re-evaluated over time as the underlying data changes.  A notebook (such as this one!) can help for easily evaluating that manually, and can give inspiration for how this process could be automated, potentially using blob storage as well for persistance of model metadata.<jupyter_code>## Use Azure Storage to iterate and automate

Now that we have a promising model, we need to ensure it does not regress. A common pattern for this is to store the hyperparameters and model outcomes in a persisted data store such as Azure Storage. We'll reuse the client that we used earlier. The credential you are using will need the "Blob Data Writer" permission to write back to Azure Storage.

First, let's capture some metrics to denote the current state of the model:
from sklearn.metrics import mean_squared_error

# Root mean squared error, a common method for observing the delta between a forecast and the source-of-truth.
rmse = numpy.sqrt(mean_squared_error(test, forecast))
# Akaike's information criterion, a measure that also folds the "simplicity" of the model into the score.
aic = model.aic()
# We should also record what data we were "up to" when we made this forecast.
forecast_time = test.index[-1]
# And finally let's get the parameters of the model
params = model.params()
Now, store these in an Azure Storage blob. This same information could be stored in any store; for example, CosmosDB, Table storage, or an SQL database.
from azure.core.exceptions import ResourceExistsError

# The container that will hold the output data
output_container_name = 'REPLACE_ME'

# Create the container if it doesn't exist
try:
    storage_client.create_container(output_container_name)
except ResourceExistsError:
    print("Warning: Container already exists")

# Upload data to a blob
import json
blob_client = storage_client.get_blob_client(output_container_name, str(forecast_time))
try:
    blob_client.upload_blob(json.dumps({
        'rmse': rmse,
        'aic': aic,
        'forecast_time': str(forecast_time),
        'params', list(params)
    }))
except ResourceExistsError:
    print("Warning: Blob already exists")<jupyter_output><empty_output><jupyter_text>We can then take steps similar to fetching the initial log data above to pull the model logs for inspection; for instance, to watch fora regression in model performance:<jupyter_code>container = storage_client.get_container_client(output_container_name)
blob_list = container.list_blobs()
for blob in blob_list:
    body = json.loads(container.download_blob(blob.name).readall().decode('utf8'))
    print(body)<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.05873261205564142, 'fraction_numerical': 0.0006182380216383308, 'mean_word_length': 4.926739926739927, 'pattern_counts': {'":': 0, '<': 9, '<?xml version=': 0, '>': 9, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 1, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '2', 'dir': 'jupyter-structured-clean-dedup', 'id': '366572', 'license': 'MIT', 'n_tokens_mistral': 865, 'n_tokens_neox': 833, 'n_words': 421, 'path': 'images/posts/ForecastingInVSCodeWithBlob.ipynb', 'repo_name': 'AlexGhiondea/azure-sdk', 'size': '3235'}