For the implementation of the DRL attacker, we used the OpenAI Gym, and gym-malware tool kits [1]}, [2]}. Keras-rl and Keras-rl2 libraries were used for the selection of the DQN agent. Unlike a typical DRL algorithm, we execute a new training session in every batch of the RELEVAGAN training where the weights of the neural network are not reset to ensure that the agent is learning in each batch iteration of RELEVAGAN. The reason for keeping the weights is that we can not estimate the number of training iterations that would e traverse the whole batch of botnet samples for generating manipulations. Hence, we reinitialise the session after each batch keeping the agent's neural network unchanged. A single training session in each batch iteration lasts until the following two cases appear:
