We visualize the learned lossy representations in the real world
using t-SNE [1]} in Fig. REF , comparing them to a representation learned via a VQ-VAE model [2]}, as used by PTP [3]}.
Image observations in the target scene are sampled from the offline dataset and encoded using our state encoder and VQ-VAE, respectively. Each encoded representation is plotted as a grey dot. From 3 different tasks, we sample 9 trajectories with varied camera poses, lighting conditions, objects, and motions. We subsample the trajectories every 5 time steps, then plot the encoded trajectories overlaid on the t-SNE plots, with the number showing the start state of each one. Different colors indicate different trajectories and trajectories of the same semantic meaning are marked by similar colors.
