Recently, deep learning supplemented communication systems have shown the potential to further enhance the system's performance [1]}, [2]}, [3]}, [4]}, [5]}. In particular, for the detection and decoding algorithms, the research on autoencoders [6]} and the NN optimization schemes which transform the FGs into NN systems [7]}, [8]}, has drawn significant interests.
For the autoencoders, an NN system with multiple layers is employed and trained to overcome the issues such as multipath interferences and signal distortions [6]}.
However, since the connection among the multiple layers of the NN model does not rely on the mathematical models of the channel, the NN usually needs a large number of training samples. Generally, more than \(2^K\)  samples are needed to converge to a good performance [10]}, where \(K\)  is the information sequence length.
On the other hand, the "unfolded" NN detection or decoding algorithms take advantage of the well-developed channel models [7]}, which lead to specific NN connections. However, the flexibility of the NN designs is neglected. Such an NN can only optimize the performance based on a specific graph, which may not lead to the globally optimized performance.
Other than the autoencoders, NN aided algorithms for Turbo equalization systems have also been reported in the literature.
In [12]}, a deep NN (DNN) was proposed for Turbo equalization by assuming that the log-likelihood ratios (LLRs) of the extrinsic information from the decoder follow a consistent Gaussian distribution with a variance computed by the inverse \(J\) -function as introduced in [13]}.
However, the DNN in [12]} is constructed by a fully-connected network that has inputs of both the decoder's extrinsic information and the channel information, which indicates that the extrinsic information from the decoder is fed into the NN separately from the channel information. Consequently, this may largely increase the number of neurons in the NN, leading to a high training complexity.
