When choosing the function class for these learned predictors, high capacity models are very appealing. Recent progress has shown that we can train deep neural networks end-to-end to go from a history of raw state information or even raw sensor data to a distribution over predicted trajectories for a human, implicitly or explicitly extracting relevant features, identifying potential targets in the scene, computing trajectories for each, and assessing their relative likelihoods [1]}, [2]}, [3]}, [4]}. Such models dominate the leaderboards in benchmarks for motion prediction (or "forecasting", as it is sometimes referred to) like Argoverse [5]} or INTERACTION [6]}. They free us from specifying what features might be important or identifying a "theory of mind" for how humans make decisions. Their capacity enables them to represent subtle nuances of human behavior, like people's implicit proxemics preferences, risk aversion level, or anything else that influences where humans go that would be otherwise very challenging to explicitly write down.
