Given a coefficient \(\delta _t\)  for task \(t\) , we first compute a nucleus [1]} of \(\tilde{{\mathcal {C}}}\)  under \(\delta _t\) : the smallest possible set of layers, whose cumulative probability under \(\tilde{{\mathcal {C}}}\)  exceeds the bound \(\delta _t\) .
We then expand all the layers in the nucleus to obtain a new task-net \(\mathcal {G}_t\) , which is re-trained using the training dataset \(\mathcal {D}_t\)  for t.
Training \(\mathcal {G}_t\)  inside the joint network \(\mathcal {G}\)  enables forward information transfer from bound task \(b\)  (and its transitive bindings), and often improves the task performance on \(t\) .
Note that with a single coefficient \(\delta _t\)  per task \(t\) , we can span the complete expansion space for \(t\) : from no expansion, full sharing (\(\delta _t \sim 0\) ) to expanding all the layers (\(\delta _t \sim 1\) ).
This capability comes with a caveat that all expansions must abide with the layer ordering given by the conflict distribution \(\tilde{{\mathcal {C}}}\) .
We summarize our Bind-Grow-Step algorithm in Algorithm .
