
Complex Training Process: Training of Deep Neural Network (DNN) and optimization by tuning parameters is an NP-complete problem where the convergence of the optimization process is not guaranteed [1]}, [2]}, [3]}. Therefore, it is assumed that training of DNN is very difficult [4]} especially in the case of HSI when a large number of parameters need to be adjusted/tuned. However, the convergence task becomes somehow easier due to the advancement of various optimization techniques for deep CNNs. Among stochastic gradient descent (SGD) [5]} and its momentum version (SGDM) [6]}, RMSProp [7]}, Adam [8]}, AdamW [9]}, diffGrad [10]}, RAdam [11]}, gradient centralization (GC) [12]}, AngularGrad [13]}, respectively are the successful CNN optimization techniques and widely used in any classification problems.

Limited Availability of Training Data: As discussed above, supervised DNN requires a considerably large amount of training data otherwise their tendency to overfit increases significantly [14]} leads to the Hughes phenomena. The high dimensional characteristic of HSI coupled with a small amount of labeled training data makes the DNNs ineffective for HSIC as it demands a lot of adjustments during the training phase [15]}.

Model's Interpretability: The training procedure of DNNs is difficult to interpret and understand. The black box kind of nature is considered as a potential weakness of DNNs and may affect the design decisions of the optimization process. Although, a lot of work has been done to interpret the model's internal dynamics.

High Computational Burden: One of the main challenges of DNN is dealing with a big amount of data that involves increased memory bandwidth, high computational cost, and storage consumption [16]}. However, advanced processing techniques like parallel and distributed architectures [17]}, [18]} and high-performance computing (HPC) [19]} make it possible for DNNs to process large amounts of data.

Training Accuracy Degradation: It is assumed that deeper networks extract more rich features from data [20]}, however, this is not true for all systems to achieve higher accuracy by simply adding more layers. Because by increasing the network’s depth, the problem of exploding or vanishing gradient becomes more prominent [21]} and affects the convergence of the model [20]}.

