Selecting Useful Knowledge from Previous Tasks for Future Learning in a Single Network
Continual learning can learn new tasks incrementally while avoiding catastrophic forgetting. Recent work has shown that packing multiple tasks into a single network incrementally by iterative pruning and re training network is a promising method. We build upon this idea and propose an improved version of PackNet. Specifically, we propose a novel gradient based threshold method to reuse the knowledge of the previous tasks selectively when learning new tasks. Our experiments on a variety of classification tasks and different network architectures demonstrate that our method obtains competitive results when compared to PackNet.
2020 25th International Conference on Pattern Recognition (ICPR)