
This paper considers an end-to-end imitation learning framework for steering autonomous driving platforms that take raw sensor inputs and predict the steering command. yellowWe consider deep learning framework since it allows for the automatic extraction of data features that are furthermore useful for a particular learning task and potentially more scalable to real-life systems. Alternative non-deep learning techniques for multi-sensor data fusion for autonomous driving were investigated in [1]}, [2]}, [3]}, [4]}. The functional mapping between inputs and outputs in the end-to-end setting was realized in past works using fully-connected networks trained on synthetic data [5]} or convolutional neural networks (CNNs) trained on real recorded data without [6]}, [7]}, [8]} and with [9]}, [7]} augmentation. All of the above mentioned works rely only on a single camera sensor at inference. The conditional imitation learning approach [11]} relies on a single camera and uses camera images and navigation commands to steer the vehicle through road intersections in desired directions. This work was further extended [12]} to use both camera images and maps to generate a probability distribution over possible steering commands. Finally, [13]} proposes a reconfigurable network that uses images captured from three front-facing cameras as inputs to navigate the autonomous driving car in an indoor environment. There are only a few works proposing end-to-end learning frameworks accommodating multiple sensing modalities such as LiDARs and cameras [14]}, [15]}, [16]}, [17]}. Our work concerns precisely such setting and engages the gating mechanism for relevant sensor selection unlike existing approaches.
