<jupyter_start><jupyter_text>Monitor[![Github](https://img.shields.io/github/stars/lab-ml/labml?style=social)](https://github.com/lab-ml/labml)[![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/lab-ml/labml/blob/master/guides/monitor.ipynb)[![Docs](https://img.shields.io/badge/labml-docs-blue)](https://docs.labml.ai/api/monit.html)<jupyter_code>%%capture
!pip install labml<jupyter_output><empty_output><jupyter_text>Iterators & EnumeratorsYou can use [`labml.monit.iterate`](https://docs.labml.ai/api/monit.htmllabml.monit.iterate) and [`labml.monit.enum`](https://docs.labml.ai/api/monit.htmllabml.monit.enum) with any iterable object.In this example we use a PyTorch `DataLoader`.<jupyter_code># Create a data loader for illustration
import time

import torch
from torchvision import datasets, transforms

from labml import logger, monit, lab, tracker

test_loader = torch.utils.data.DataLoader(
        datasets.MNIST(lab.get_data_path(),
                       train=False,
                       download=True,
                       transform=transforms.Compose([
                           transforms.ToTensor(),
                           transforms.Normalize((0.1307,), (0.3081,))
                       ])),
        batch_size=32, shuffle=True)
for data, target in monit.iterate("Test", test_loader):
    time.sleep(0.01)
for i, (data, target) in monit.enum("Test", test_loader):
    time.sleep(0.01)<jupyter_output><empty_output><jupyter_text>SectionsSections let you monitor time taken for different tasks and also helps keep the code clean by separating different blocks of code.<jupyter_code>with monit.section("Load data"):
    # code to load data
    time.sleep(2)
with monit.section("Load saved model"):
    time.sleep(1)
    monit.fail()<jupyter_output><empty_output><jupyter_text>You can also show progress while a section is running<jupyter_code>with monit.section("Train", total_steps=100):
    for i in range(100):
        time.sleep(0.1)
        # Multiple training steps in the inner loop
        monit.progress(i)<jupyter_output><empty_output><jupyter_text>LoopThis can be used for the training loop. The :func:loop keeps track of the time taken and time remaining for the loop.:func:labml.tracker.save outputs the current status along with global step.<jupyter_code>for step in monit.loop(range(0, 400)):
    tracker.save()
tracker.set_global_step(0)<jupyter_output><empty_output><jupyter_text>You can manually increment global step too.<jupyter_code>for step in monit.loop(range(0, 400)):
    tracker.add_global_step(5)
    tracker.save()<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.12570356472795496, 'fraction_numerical': 0.01425891181988743, 'mean_word_length': 4.078095238095238, 'pattern_counts': {'":': 0, '<': 25, '<?xml version=': 0, '>': 25, 'https://': 8, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 1, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '6', 'dir': 'jupyter-structured-clean-dedup', 'id': '356682', 'license': 'MIT', 'n_tokens_mistral': 910, 'n_tokens_neox': 865, 'n_words': 176, 'path': 'guides/monitor.ipynb', 'repo_name': 'vishalbelsare/labml', 'size': '2665'}