Multi-task learning (MTL) research shows the joint training of auxiliary related side-tasks along with the main task enhances the training performance [1]}, [2]}. MTL in neural networks [3]} has been successfully demonstrated in many tasks previously including text-to-speech conversion [4]}, natural language processing [5]}, speech processing [6]} and computer vision [7]}, [8]}. In the field of sequential decision making, [9]} demonstrate MTL for 3D game playing, [10]} and [11]} demonstrate MTL in 3D maze navigation task whereas [12]} utilize the MTL framework for autonomous driving. Instead of employing future control outputs as auxiliary tasks as shown by [12]}, in this work we employ action and visual abstractions to guide the driving behavior.
