To capture the task-relevant information of states and goals, we learn a parametric state encoder \(\phi (z | s)\)  to project both states and goals from the high-dimensional space (
meaning:NTF . e.g catcode:NTF a e.g. e.g., RGB images) \(\mathcal {S}\)  into a learned representation space \(\mathcal {Z}\) . We use \(z_t\)  and \(z_g\)  to denote the representation of the state \(s_t\)  and the goal \(g\) , respectively. In the learned representation space, we construct the goal-conditioned policy \(\pi (a | z_t, z_g)\) , as well as the value function \(V(z_t, z_g)\)  and Q-function \(Q(z_t, z_g, a)\) . Both \(V\)  and \(Q\)  are used for training the policy via the offline RL algorithm, which in our prototype is IQL [1]}, and \(V\)  is also used for selecting subgoals in the planner, which will be discussed in Sec. REF . When training \(\phi \) , \(\pi \) , \(V\) , and \(Q\)  to optimize the goal-reaching objective described in Sec. , \(\phi \)  would need to extract sufficient information for selecting the action to transit from \(s_t\)  to \(g\)  and estimating the required discounted number of steps.
