Computing the Conflict Model. Computing a precise, layer-wise conflict model between tasks involves characterizing interference precisely [1]}, [2]}, [3]}, which is intractable [4]}.
Several approaches to approximate interference are possible but expensive, e.g., we may identify the overlap of important parameters for both tasks using the Fisher Matrix [5]} or try different combinations of layer expansions with re-training and evaluation.
Instead, we compute a low-cost, approximate conflict model, by correlating the layer-wise features of an independently trained task-net \({\mathcal {G}}^{\downarrow }_t\)  for \(t\)  (see Sec. ) and the current task-net \(\mathcal {G}_b\)  for binding candidate \(b\) .
We use the current (fully-trained) task-net \(\mathcal {G}_b\) , instead of say, a pre-trained \({\mathcal {G}}^{\downarrow }_b\) , to ensure that the conflict model is computed in context of the growing network.
When a task \(t\)  arrives, we first train an individual network \({\mathcal {G}}^{\downarrow }_t\)  for \(t\) .
Then, we sample data from validation set \(\mathcal {V}_t\) , feed to both \(\mathcal {G}_b\)  and \({\mathcal {G}}^{\downarrow }_t\)  and collect post-activations after each layer of the networks.
Finally, using the representational similarity analysis (RSA) algorithm [6]}, [7]}, we compute correlation scores between pairwise post-activations of mapped layers \(l\in L\) , for every sample in \(\mathcal {V}_t\) .
The conflict score \({\mathcal {C}}_{t,b}(l)\)  = \(\rho _l\)  where \(\rho _l\in [-1,1]\)  denotes the RSA dissimilarity score for \(l\) .
Our implementation follows the original RSA algorithm [6]} and requires space quadratic in the size of layer post-activations.
Further, we normalize \({\mathcal {C}}\)  to be in the range \((0,1)\) .
