To analyze how promising these switching approaches are for detecting that we should plug in the planning-based predictor instead of using the default end-to-end one, we need data where tail events and distribution shift occur. While tail events will happen for most training distributions we would encounter, distribution shift is something we decided to purposefully introduce in a controlled way. We use a real driving data set (the INTERACTION data [1]}), and design three experiments that use different train and test sets to probe at different types of shift – from introducing noise to the input, to testing on a new exit from a roundabout where we didn't have data (new reference paths), all the way to switching to an entirely new map.
