Integrating LEO Satellites and Multi UAV Reinforcement Learning for Hybrid FSO/RF Non Terrestrial Networks
A mega constellation of low altitude earth orbit (LEO) satellites (SATs) and burgeoning unmanned aerial vehicles (UAVs) are promising enablers for high speed and long distance communications in beyond fifth generation (5G) systems. Integrating SATs and UAVs within a non terrestrial network (NTN) in this article we investigate the problem of forwarding packets between two faraway ground terminals through SAT and UAV relays using either millimeter wave (mmWave) radio frequency (RF) or free space optical (FSO) link. Towards maximizing the communication efficiency, the real time associations with orbiting SATs and the moving trajectories of UAVs should be optimized with suitable FSO/RF links, which is challenging due to the time varying network topology and a huge number of possible control actions. To overcome the difficulty, we lift this problem to multi agent deep reinforcement learning (MARL) with a novel action dimensionality reduction technique. Simulation results corroborate that our proposed SAT UAV integrated scheme achieves 1.99x higher end to end sum throughput compared to a benchmark scheme with fixed ground relays. While improving the throughput, our proposed scheme also aims to reduce the UAV control energy, yielding 2.25x higher energy efficiency than a baseline method only maximizing the throughput. Lastly, thanks to utilizing hybrid FSO/RF links, the proposed scheme achieves up to 62.56x higher peak throughput and 21.09x higher worst case throughput than the cases utilizing either RF or FSO links, highlighting the importance of co designing SAT UAV associations, UAV trajectories, and hybrid FSO/RF links in beyond 5G NTNs.
ArXiv