@inproceedings{
xu2024infinitedimensional,
title={Infinite-Dimensional Feature Interaction},
author={Chenhui Xu and Fuxun Yu and Maoliang Li and Zihao Zheng and Zirui Xu and Jinjun Xiong and Xiang Chen},
booktitle={The Thirty-eighth Annual Conference on Neural Information Processing Systems},
year={2024},
url={https://openreview.net/forum?id=xO9GHdmK76}
}
The past neural network design has largely focused on feature \textit{representation space} dimension and its capacity scaling (e.g., width, depth), but overlooked the feature \textit{interaction space} scaling. 
 Recent advancements have shown shifted focus towards element-wise multiplication to facilitate higher-dimensional feature interaction space for better information transformation. Despite this progress, multiplications predominantly capture low-order interactions, thus remaining confined to a finite-dimensional interaction space. To transcend this limitation, classic kernel methods emerge as a promising solution to engage features in an infinite-dimensional space. We introduce InfiNet, a model architecture that enables feature interaction within an infinite-dimensional space created by RBF kernel. Our experiments reveal that InfiNet achieves new state-of-the-art, owing to its capability to leverage infinite-dimensional interactions, significantly enhancing model performance.
Infinite-Dimensional Feature Interaction
Chenhui Xu Fuxun Yu Maoliang Li Zihao Zheng Zirui Xu Jinjun Xiong Xiang Chen