Look, Radiate, and Learn: Self-supervised Localisation via Radio-Visual Correspondence
Next generation cellular networks will implement radio sensing functions
alongside customary communications, thereby enabling unprecedented worldwide
sensing coverage outdoors. Deep learning has revolutionised computer vision but
has had limited application to radio perception tasks, in part due to lack of
systematic datasets and benchmarks dedicated to the study of the performance
and promise of radio sensing. To address this gap, we present MaxRay: a
synthetic radio-visual dataset and benchmark that facilitate precise target
localisation in radio. We further propose to learn to localise targets in radio
without supervision by extracting self-coordinates from radio-visual
correspondence. We use such self-supervised coordinates to train a radio
localiser network. We characterise our performance against a number of
state-of-the-art baselines. Our results indicate that accurate radio target
localisation can be automatically learned from paired radio-visual data without
labels, which is highly relevant to empirical data. This opens the door for
vast data scalability and may prove key to realising the promise of robust
radio sensing atop a unified perception-communication cellular infrastructure.
Dataset will be hosted on IEEE DataPort.