Network Architectures.
We illustrate the network architectures of the state encoder, the policy, the value network, the Q-network, and the affordance model in Fig. REF . The state encoder projects the images into 128-dimensional representations. The state encoder is implemented with two 128-dimensional fully-connected (FC) layers on top of a VQ-VAE [1]} backbone. The VQ-VAE backbone is trained on the offline dataset using the reconstruction loss as in [1]} and its weights are fixed during the offline pre-training and online fine-tuning in FLAP. The state encoder outputs the mean \(\mu _z\)  and standard deviation \(\sigma _z\)  for sampling the lossy representation \(z\) .
The policy network, the value network, and the Q-network are implemented with 128-dimensional FC layers.
Inputs to these networks are concatenated and then fed to the FC layers.
The policy network produces the mean \(\mu _a\)  and a state-independent standard deviation \(\sigma _a\)  is jointly learned to form the action distribution.
The affordance model operates in an 8-dimensional latent space,
and both its encoder and decoder are implemented by 128-dimensional FC layers.
The affordance model is trained to predict subgoals of \(\Delta t = 30\)  in simulation and \(\Delta t = 20\)  in the real world using the training protocol from [3]}.
<FIGURE>