Now that we have conclusively demonstrated the successful disentanglement of our learned embedding space, we next perform an ablation analysis of our system. To this end, we explore the contributions of 3 parts of our model. For this experiment we additionally report the imitation accuracy (percentage of actions that agree with the expert) for comparison against [1]}.
As shown in table REF , we first removed the current state observation branch (no \(O_t\) ). With only the gated VAE structure, our model performs similarly to the SOTA model (CPV-FULL[1]}).
Introducing the current state observation branch improves performance significantly by giving the agent a more direct observation of it's current state.
In the next experiment, we remove the random sampling from the latent distribution, and instead simply take the mean latent embedding. We refer to this as Fixed Sampling (FS).
This offers a small improvement in all metrics.
Finally, we introduce the dynamic loss (DL) weighting scheme proposed in equation REF . This approach provides further improvement in task performance and completion speed.
Adjusting the proportions of the loss functions according to the training mode, improves training stability. However, this also reduces the training speed, as the learning happens less aggressively.
It is interesting to note that although imitation accuracy goes down for the DL model, task success and completion speed go up. This indicates that our agent may now be outperforming the expert.
