Building upon previous work [1]}, [2]}, [3]}, [4]}, [5]}, we propose a non-linear sparse coding factor analysis model based on Bayesian nonparametrics. The model employs non-linear “multiplexer” neural net that encodes latent binary vectors \(z_n \in \lbrace 0,1\rbrace ^K\)  to sparse latent variables \(\xi _n \in \mathbb {R}_{+}^{M}\)  . The network has the capacity to non-linearly explore the large parameter space over the latent encodings \(\xi _n\) . In addition, given a factorial distribution over \(z_n\) , the network can learn the correlation structure and approximate non-factorial distributions over the latent codes \(\xi _n\)  at the deepest layer. This allows the model to generate better samples than traditional linear factor models with factorial prior over the latents. By defining a sparse beta-Bernoulli process prior on the \(z_n\) , the model learns the optimal size \(K\) . Despite the non-linearity of the model, the parameters of the model are still interpretable as the interaction between the factor loading matrix \(\Phi \)  and the outputs of the multiplexer network \(\xi _n\)  is a linear operation. We propose a stochastic MAP-EM algorithm with a “selective” M step for efficient scalable inference in this model. [5]}.
