We consider the medium game set and hard game set as different experiments. We split the medium game set into 200 training games / 40 validation games / 40 testing games, and the hard game set into 300 / 60 / 60. We follow the default setting of  [1]} to conduct reinforcement learning. We set the step limit of an episode as 50 for training and 100 for validation / testing. We set the subtask time limit \(\xi =5\) . For each episode, we sample a game from the training set to interact with. We train the models for 100,000 episodes. The models are optimized via Double DQN (epsilon decays from 1.0 to 0.1 in 20,000 episodes, Adam optimizer with a learning rate of 0.001) with Pritorized Experience Replay (replay buffer size 500,000). For every 1,000 training episodes, we validate the model and report the testing performance.
