We compare our method with other recent expandable networks continual learning approaches PG[1]}, PN[2]}, L2G. We evaluate each compared approach by considering average test accuracy on all the tasks, model size and training time. Model size is measured via the number of model parameters after training all the tasks. We evaluate on Permuted Mnist and Split-CIFAR100 tasks, which are the only tasks common among these approaches.
Fig. REF  shows the results.
We omit comparisons to regularization based approaches to continual learning [3]}, [4]}, as they have been shown earlier to perform worse against the expansion based methods .
Our approach (L2BG) has comparable average accuracy with previous approaches.
Compared with Learn-to-grow (L2G), L2BG uses more parameters to obtain similar performance.
L2G performs expensive optimization over all layer expansions, and hence is able to obtain more compact architectures.
In contrast, we employ a cheaper, approximate RSA-based conflict model to expand a subset of layers greedily, which leads to higher layer expansion.
In future, we plan to explore other representation similarity techniques to compute improved conflict models [5]} to help us compute smaller networks.
While L2BG does not improve performance on these tasks significantly, we argue that its advantages are orthogonal.
The network architectures obtained by L2BG reflect similarity between different tasks and different layers thereof. Further, L2BG enables exploring the performance-size trade-offs (cf. Sec. REF ), which is not possible with previous expansion based approaches.
<FIGURE>