@inproceedings{
zhao2023role,
title={Role Taxonomy of Units in Deep Neural Networks},
author={Yang Zhao and Hao Zhang and Xiuyuan Hu},
booktitle={UniReps:  the First Workshop on Unifying Representations in Neural Models},
year={2023},
url={https://openreview.net/forum?id=GpBQYrTEPG}
}
Identifying the role of network units in deep neural networks (DNNs) is critical in many aspects including giving understandings on the mechanisms of DNNs and building basic connections between deep learning and neuroscience. However, there remains unclear on which roles the units in DNNs with different generalization ability could present. To this end, we give role taxonomy of units in DNNs, where units are categorized into four types in terms of their functional preference on separately the training set and testing set. We show that ratios of the four categories are highly associated with the generalization ability of DNNs from two distinct perspectives, based on which we give signs of DNNs with well generalization.
Role Taxonomy of Units in Deep Neural Networks
Yang Zhao Hao Zhang Xiuyuan Hu