In this paper, we propose a novel approach for generating
randomized, single-hidden-layer networks based on insights from
the fly brain that allow for increasing the efficiency of these
simple architectures. In particular, we introduce KCNet, a
single-hidden-layer neural network with sparse and randomized
binary weights that mimics how odor stimuli are encoded in the
insect brain's high-dimensional KC representation. In addition, we also
define a dynamic optimization method based on the straight-through
gradient estimator [1]}, which allows the model
to dynamically explore a more efficient set of inputs. We first apply
our model to odorant-perception tasks, which determines how the
perceptual characteristics of odorants can relate to the structural
properties of the molecules. We demonstrate that our model achieves
better performance compared to the existing data-driven approaches,
such as XGBoost. Furthermore, to demonstrate the proposed model's
versatility, we also address image-classification tasks on image
benchmark datasets, including MNIST, Fashion-MNIST, and EMNIST. We show
that without the need for data augmentation or convolutional layers,
KCNet can achieve reasonable performance.
