Federated Deep Learning Meets Autonomous Vehicle Perception: Design and Verification
Realizing human-like perception is a challenge in open driving scenarios due
to corner cases and visual occlusions. To gather knowledge of rare and occluded
instances, federated learning empowered connected autonomous vehicle (FLCAV)
has been proposed, which leverages vehicular networks to establish federated
deep neural networks (DNNs) from distributed data captured by vehicles and road
sensors. Without the need of data aggregation, FLCAV preserves privacy while
reducing communication and annotation costs compared with conventional
centralized learning. However, it is challenging to determine the network
resources and road sensor poses for multi-stage training with multi-modal
datasets in multi-variant scenarios. This article presents networking and
training frameworks for FLCAV perception. Multi-layer graph resource allocation
and vehicle-road pose contrastive methods are proposed to address the network
management and sensor pose problems, respectively. We also develop CarlaFLCAV,
a software platform that implements the above system and methods. Experimental
results confirm the superiority of the proposed techniques compared with
various benchmarks.