We can also experimentally find that the simple solution of storing a fixed number of buffer with the same size of each frame [1]} is effective enough to serve as a replayed experience of zero level-set observations. Fig. REF , REF  and the supplementary material demonstrate that training a network without replayed buffer leads to catastrophic forgetting.
