Robust Semantic Communications with Masked VQ-VAE Enabled Codebook
Although semantic communications have exhibited satisfactory performance for
a large number of tasks, the impact of semantic noise and the robustness of the
systems have not been well investigated. Semantic noise refers to the
misleading between the intended semantic symbols and received ones, thus cause
the failure of tasks. In this paper, we first propose a framework for the
robust end-to-end semantic communication systems to combat the semantic noise.
In particular, we analyze sample-dependent and sample-independent semantic
noise. To combat the semantic noise, the adversarial training with weight
perturbation is developed to incorporate the samples with semantic noise in the
training dataset. Then, we propose to mask a portion of the input, where the
semantic noise appears frequently, and design the masked vector
quantized-variational autoencoder (VQ-VAE) with the noise-related masking
strategy. We use a discrete codebook shared by the transmitter and the receiver
for encoded feature representation. To further improve the system robustness,
we develop a feature importance module (FIM) to suppress the noise-related and
task-unrelated features. Thus, the transmitter simply needs to transmit the
indices of these important task-related features in the codebook. Simulation
results show that the proposed method can be applied in many downstream tasks
and significantly improve the robustness against semantic noise with remarkable
reduction on the transmission overhead.