<jupyter_start><jupyter_text>LitMatter DeepChem* This notebook shows how to speed up [DeepChem](https://github.com/deepchem/deepchem) model training on [MoleculeNet](https://arxiv.org/abs/1703.00564) datasets using the LitMatter template.  * In this example, we train a simple DeepChem `TorchModel` on the Tox21 dataset.* The training workflow shown here can be scaled to hundreds of GPUs by changing a single keyword argument!<jupyter_code>%load_ext autoreload
%autoreload 2
import torch
import deepchem as dc

import pytorch_lightning as pl
from pytorch_lightning.callbacks import ModelCheckpoint
from pytorch_lightning import (LightningDataModule, LightningModule, Trainer,
                               seed_everything)<jupyter_output><empty_output><jupyter_text>Load a `LitMolNet` datasetAny MolNet dataset from `deepchem.molnet` can be used with LitMatter. The specific MolNet dataset and any pre-processing steps can be defined in `data.LitMolNet`<jupyter_code>from lit_data.molnet_data import LitMolNet

dm = LitMolNet(loader=dc.molnet.load_tox21, batch_size=16)
dm.prepare_data()
dm.setup()<jupyter_output>RDKit WARNING: [11:14:30] WARNING: not removing hydrogen atom without neighbors
[11:14:30] WARNING: not removing hydrogen atom without neighbors
RDKit WARNING: [11:14:41] WARNING: not removing hydrogen atom without neighbors
[11:14:41] WARNING: not removing hydrogen atom without neighbors<jupyter_text>Instantiate a `LitDeepChem` modelAny `deepchem.models.torch_models.TorchModel` can be used with LitMatter. Here, we'll write our own custom base model in PyTorch and make a `TorchModel`.<jupyter_code>from lit_models.deepchem_models import LitDeepChem

base_model = torch.nn.Sequential(
torch.nn.Linear(1024, 256),
    torch.nn.ReLU(),
    torch.nn.Linear(256, 12),
)

torch_model = dc.models.TorchModel(base_model, loss=torch.nn.MSELoss())

model = LitDeepChem(torch_model, lr=1e-2)<jupyter_output>Exception ignored in: <function Model.__del__ at 0x7f028f6c6550>
Traceback (most recent call last):
  File "/home/gridsan/NA30490/.conda/envs/litmatter/lib/python3.8/site-packages/deepchem/models/models.py", line 61, in __del__
    shutil.rmtree(self.model_dir)
  File "/home/gridsan/NA30490/.conda/envs/litmatter/lib/python3.8/shutil.py", line 709, in rmtree
    onerror(os.lstat, path, sys.exc_info())
  File "/home/gridsan/NA30490/.conda/envs/litmatter/lib/python3.8/shutil.py", line 707, in rmtree
    orig_st = os.lstat(path)
FileNotFoundError: [Errno 2] No such file or directory: '/state/partition1/slurm_tmp/48690281.0.0/tmp60c4fwyb'<jupyter_text>Train the modelSimply change the `Trainer` flags as desired for multi-gpu and multi-node training.<jupyter_code>trainer = Trainer(gpus=-1,  # use all available GPUs on each node
#                   num_nodes=1,  # change to number of available nodes
#                  accelerator='ddp',
                 max_epochs=5,
                 )
trainer.fit(model, datamodule=dm)<jupyter_output>/home/gridsan/NA30490/.conda/envs/litmatter/lib/python3.8/site-packages/pytorch_lightning/core/datamodule.py:469: LightningDeprecationWarning: DataModule.setup has already been called, so it will not be called again. In v1.6 this behavior will change to always call DataModule.setup.
  rank_zero_deprecation(
LOCAL_RANK: 0 - CUDA_VISIBLE_DEVICES: [GPU-13636255-d3c9-b0ac-83c7-b25c82e0dbc5]
Set SLURM handle signals.

  | Name    | Type       | Params
---------------------------------------
0 | model   | Sequential | 265 K 
1 | loss_fn | MSELoss    | 0     
---------------------------------------
265 K     Trainable params
0         Non-trainable params
265 K     Total params
1.062     Total estimated model params size (MB)
/home/gridsan/NA30490/.conda/envs/litmatter/lib/python3.8/site-packages/pytorch_lightning/callbacks/model_checkpoint.py:617: UserWarning: Checkpoint directory /home/gridsan/NA30490/litmatter_dev/litmatter/lightning_logs/version_48690281/checkpoints exists and is not empt[...]
{'fraction_non_alphanumeric': 0.11150353178607467, 'fraction_numerical': 0.04641775983854692, 'mean_word_length': 4.162760416666667, 'pattern_counts': {'":': 0, '<': 15, '<?xml version=': 0, '>': 15, 'https://': 2, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '4', 'dir': 'jupyter-structured-clean-dedup', 'id': '362296', 'license': 'MIT', 'n_tokens_mistral': 1443, 'n_tokens_neox': 1316, 'n_words': 329, 'path': 'LitDeepChem.ipynb', 'repo_name': 'ncfrey/litmatter', 'size': '3964'}