We can maximize \(\mathcal {L}_{\pi }(z,\theta )\)  w.r.t. \(\theta \) , which includes both the neural net parameters \(\theta _{\xi }\)  and the dictionary \(\theta _{\Phi }\)  by stochastic optimization using ADAM [1]}. First order gradient methods with moment estimates such as ADAM, can implicitly take into account the rate of change of natural parameters \((a_k , b_k)\)  for \(q(\pi )\)  when optimizing the neural net parameters \(\theta _{\xi }\)  and the dictionary \(\theta _{\Phi }\) . The full sparse coding algorithm is outlined in Alg. (REF ).
