For all our experiments, we keep the DNC architecture similar to the ones used in experiments of [1]}. We use a LSTM controller with 256 units, an external memory size of length 256, width 64, and 4 read heads, and a batch size of 1. We use the same CAE used by [2]}, and give our latent state representation (the \(z\)  vector) 32 dimensions. For our RL experiments, the output of the DNC is of 34 dimensions, where 32 dimensions are for the next predicted latent state representation, 1 for the predicted reward, and 1 for the predicted probability that a rollout is done. We use the Adam optimizer with a learning rate of \(\alpha =1e-4\) . The model is trained with Truncated Backpropagation Through Time using a sequence length of 64.
