The approximation is further "approximated" by the fact that the Hessian in the Taylor expansion is approximated by the diagonal of the Fisher Information matrix. Furthermore, the prior is also assumed to be a zero-mean isometric Gaussian [1]} which is rather a simple assumption [2]}. From such approximation, it is understandable that EWC might be only effective when the new task/data is somewhat close to the original task which is unlikely in language learning.
