Following the setup of DNN in V2X communications [1]}, [2]}, a five-layer DNN is utilized to construct the neural network for DQL and DDPG algorithms, in which three hidden layers include 500, 250, and 120 neurons, respectively. The rectified linear unit (ReLU) is used as the activation function for the hidden layers used in both DQN and DDPG. Sigmoid is used as the activation function for the output layer of the actor network of DDPG to scale the output into the transmission power region. The adaptive moment estimation method (Adam) is used to update the network parameters [3]}. The payload \(B\)  for each V2V pair is 1060 bytes unless specified. At the beginning of each payload transmission, the remaining load equals to the payload for each V2V pairs. Based on a trial-and-error approach, we consider to use 0.001 as the learning rate of the DQN unit and use 0.0001 and 0.001 as the learning rates of the actor and critic networks in DDPG, respectively. The numbers of samples in mini-batches used for the training of the proposed DRL algorithm and the proposed meta DRL algorithm are 2000 and 100, respectively. For the meta training, the number of tasks in each batch is 100, and the number of batches is 200. The learning rates used in the meta learning are set as \(\hat{\alpha }=0.01\) , \(\hat{\beta }_c=0.01\) , and \(\hat{\beta }_a=0.001\) . The discount factor is \(\gamma =0.5\) , and the update frequency factor (in (18) and (19)) of the target network in DDPG is \(\tau =0.001\) . All simulation results are generated by using PyTorch 1.4.0 on the Python 3.6 platform.
