In order for this relabeling procedure not to bias our policy towards behavior that is overly-optimistic about the user's responses, it is necessary that the distribution of these responses in our dataset does not depend on the portion of the context that is relabeled. For example, relabeling the table of available flights for a flight booking task should generally be reasonable, because the user is usually unaware of the flight table. On the other hand, relabeling the desired flight would not make sense, since the user's utterances are strongly depend on this. To provide another example, in a bargaining task [1]}, the agent might fail to obtain the desired item and instead get an item of lesser value. But relabeling with a context that assigns a higher value to the item received would not lead to a reasonable example, since the agent mainly received this item as a result of the user's responses rather than as a result of their own bargaining skill.
