After each local search procedure the neural network \(f_\theta \)  is trained during \(N\)  epochs on the new dataset \(\mathcal {D}\)  using Adam optimizer [1]} with initial learning rate \(lr=0.001\)  and batches of size \(b=100\)  in order to minimize the mean square error loss (MSE) between the outputs and the targets. In order to speed up the training, prior to the non linearity we apply a batch normalization layer [2]}, adapted to keep the invariant property of the network. For the layer \(j\)  of the network, the ouput of the invariant batch normalization layer is:
\(y = \frac{x - \mathbb {E}[x]}{ \sqrt{\text{Var}[x] + \epsilon }} * \gamma + \mu \) 
