The explanation for the ineffectiveness of EWC probably comes from the derivation into the final equation of the regularization loss term. From the theoretical analysis [1]}, EWC originates from replacing the log posterior \(\log p(\theta |T_1)\)  with its Taylor expansion form, that requires the optimal value \(\theta ^*\)  during optimizing the model for the data \(T_1\) . The SGD algorithm is not guaranteed to achieve the exact optimal value (averaging checkpoints often yield better model parameters)
