<jupyter_start><jupyter_text>Train Example: We wil predict:- `n_in`: 5 images- `n_out`: 5 images  - `n_obj`: up to 3 objects<jupyter_code>Path.cwd()
DATA_PATH = Path.cwd()/'data'
class ImageStack(Transform):
    order=89
    def __init__(self, dim=1): store_attr()
    
    def encodes(self, x:ImageSeq): return torch.stack(x, dim=self.dim)
    def decodes(self, x:ImageSeq): return torch.unbind(x, dim=self.dim)
ds = MovingMNIST(DATA_PATH, n_in=5, n_out=5, n_obj=[1,2], th=None)
train_tl = TfmdLists(range(12500), ImageTupleTransform(ds))
valid_tl = TfmdLists(range(100), ImageTupleTransform(ds))
dls = DataLoaders.from_dsets(train_tl, valid_tl, bs=64,
                             after_batch=[Normalize.from_stats(*mnist_stats)]).cuda()
mse_loss = StackLoss(MSELossFlat(axis=1))
metrics = []<jupyter_output><empty_output><jupyter_text>Left: Input, Right: Target<jupyter_code>dls.show_batch()
b = dls.one_batch()
explode_types(b)
x,y = b 
x[0].shape<jupyter_output><empty_output><jupyter_text>Baseline `StackUnstack` takes cares of stacking the list of images into a fat tensor, and unstacking them at the end, we will need to modify our loss function to take a list of tensors as input and target.<jupyter_code>model = StackUnstack(Seq2Seq(5, ch_out=1), dim=1).cuda()<jupyter_output><empty_output><jupyter_text>As the `ImageSeq` is a `tuple` of images, we will need to stack them to compute loss.<jupyter_code>learn = Learner(dls, model, loss_func=mse_loss, metrics=metrics, cbs=[EncoderLoss(mse_loss)])<jupyter_output><empty_output><jupyter_text>I have a weird bug that if I use `nn.LeakyReLU` after doing `learn.lr_find()` the model does not train (the loss get stucked).<jupyter_code>learn.lr_find()
learn.fit_one_cycle(10, 1e-3)
p,t = learn.get_preds()<jupyter_output><empty_output><jupyter_text>As you can see, the results is a list of 5 tensors with 100 samples each.<jupyter_code>len(p), p[0].shape
def show_res(t, idx, argmax=False):
    if argmax:
        im_seq = ImageSeq.create([t[i][idx].argmax(0).unsqueeze(0) for i in range(5)], TensorMask)
    else:
        im_seq = ImageSeq.create([t[i][idx] for i in range(5)])
    im_seq.show(figsize=(8,4));
k = random.randint(0,99)
show_res(t,k)
show_res(p,k, False)<jupyter_output><empty_output><jupyter_text>Teacher Forcing<jupyter_code>model = StackUnstack(Seq2Seq(5, ch_out=1, ks=3, n_layers=1, debug=False), dim=1).cuda()
learn = Learner(dls, model, loss_func=mse_loss, metrics=metrics, 
                cbs=[TeacherForcing(4), EncoderLoss(mse_loss, alpha=1)])
learn.lr_find()
learn.fit_one_cycle(12, 1e-3)
p,t = learn.get_preds()<jupyter_output><empty_output><jupyter_text>As you can see, the results is a list of 5 tensors with 100 samples each.<jupyter_code>len(p), p[0].shape
def show_res(t, idx, argmax=False):
    if argmax:
        im_seq = ImageSeq.create([t[i][idx].argmax(0).unsqueeze(0) for i in range(5)], TensorMask)
    else:
        im_seq = ImageSeq.create([t[i][idx] for i in range(5)])
    im_seq.show(figsize=(8,4));
k = random.randint(0,99)
show_res(t,k)
show_res(p,k, False)<jupyter_output><empty_output><jupyter_text>PhyCell<jupyter_code>model = StackUnstack(Seq2SeqPhy(5, ch_out=1, ks=3, n_layers=3, debug=False), dim=1).cuda()
learn = Learner(dls, model, loss_func=mse_loss, metrics=metrics, 
                cbs=[TeacherForcing(6), EncoderLoss(mse_loss, alpha=1)])
learn.lr_find()
learn.fit_one_cycle(12, 1e-3)
p,t = learn.get_preds()<jupyter_output><empty_output><jupyter_text>As you can see, the results is a list of 5 tensors with 100 samples each.<jupyter_code>len(p), p[0].shape
def show_res(t, idx, argmax=False):
    if argmax:
        im_seq = ImageSeq.create([t[i][idx].argmax(0).unsqueeze(0) for i in range(5)], TensorMask)
    else:
        im_seq = ImageSeq.create([t[i][idx] for i in range(5)])
    im_seq.show(figsize=(8,4));
k = random.randint(0,99)
show_res(t,k)
show_res(p,k, False)<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.14679601736022466, 'fraction_numerical': 0.024508552463620117, 'mean_word_length': 5.802083333333333, 'pattern_counts': {'":': 0, '<': 41, '<?xml version=': 0, '>': 41, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '10', 'dir': 'jupyter-structured-clean-dedup', 'id': '393334', 'license': 'Apache-2.0', 'n_tokens_mistral': 1611, 'n_tokens_neox': 1543, 'n_words': 318, 'path': '03_train_seq2seq.ipynb', 'repo_name': 'shawnwang-tech/moving_mnist', 'size': '3917'}