Analysis of Branch Specialization and its Application in Image Decomposition
Branched neural networks have been used extensively for a variety of tasks.
Branches are sub-parts of the model that perform independent processing
followed by aggregation. It is known that this setting induces a phenomenon
called Branch Specialization, where different branches become experts in
different sub-tasks. Such observations were qualitative by nature. In this
work, we present a methodological analysis of Branch Specialization. We explain
the role of gradient descent in this phenomenon. We show that branched
generative networks naturally decompose animal images to meaningful channels of
fur, whiskers and spots and face images to channels such as different
illumination components and face parts.