Planning with affordance models.
When learning to solve long-horizon tasks, subgoals can significantly improve performance by breaking down the original problem into easier short-horizon tasks.
We can use sampling-based planning to find a suitable sequence of \(K\)  subgoals \(\hat{s}_{1:K}\) , which samples multiple candidate sequences and chooses the optimal plan based on a cost function.
In high-dimensional state spaces, such a planning process can be computationally intractable, since most sampled candidates will be unlikely to form a reasonable plan.
To facilitate planning, we would need to focus on sampling subgoal candidates that are realistic and reachable from the current state.
Following the prior work [1]}, [2]}, we use affordance models to capture the distribution of reachable future states and recursively propose subgoals conditioned on the initial state of each task.
The affordance model can be defined as a generative model \(m(s^{\prime } | s, u)\) , where \(u\)  is the latent representation that captures the information of the transition from the current state \(s\)  to the goal \(s^{\prime }\) .
It can be trained in a conditional variational autoencoder (CVAE) [3]} paradigm using the encoder \(q(u | s, s^{\prime })\)  to estimate the evidence lower bound (ELBO) [4]}.
