A lot of recent work in autonomous driving has been focused on designing end-to-end learning networks for the task of driving using input images [1]}, [2]}, [3]}, [4]}, [5]}. Most works model this as an end-to-end regression problem of arriving at the control values using the input pixel values directly. Clearly, humans utilize a much richer hierarchical task decomposition pipeline while carrying out visuomotor tasks like driving. Rather than arriving at the exact throttle, brake and steering percentages directly from high-dimensional pixel values, our brains first decompose the image into recognizable objects and their intuitive states in the scene [6]}, [7]}, then use visual abstractions along with past experience to come up with approximately optimal high level decisions or action primitives for the given state, and lastly use these action primitives to arrive at the exact motor commands to be executed [8]}, [9]}, [10]}, [11]}. Such kind of hierarchical structure allows us to tackle hard problems like driving by decomposing them into multiple sub-tasks allowing for better generalization.
