A Survey on the New Generation of Deep Learning in Image Processing <s> 1) DEEP FOREST <s> In this paper, we propose gcForest, a decision tree ensemble approach with performance highly competitive to deep neural networks in a broad range of tasks. In contrast to deep neural networks which require great effort in hyper-parameter tuning, gcForest is much easier to train; even when it is applied to different data across different domains in our experiments, excellent performance can be achieved by almost same settings of hyper-parameters. The training process of gcForest is efficient, and users can control training cost according to computational resource available. The efficiency may be further enhanced because gcForest is naturally apt to parallel implementation. Furthermore, in contrast to deep neural networks which require largescale training data, gcForest can work well even when there are only small-scale training data. <s> BIB001 </s> A Survey on the New Generation of Deep Learning in Image Processing <s> 1) DEEP FOREST <s> In this study, the super-resolution convolutional neural network (SRCNN) scheme, which is the emerging deep-learning-based super-resolution method for enhancing image resolution in chest CT images, was applied and evaluated using the post-processing approach. For evaluation, 89 chest CT cases were sampled from The Cancer Imaging Archive. The 89 CT cases were divided randomly into 45 training cases and 44 external test cases. The SRCNN was trained using the training dataset. With the trained SRCNN, a high-resolution image was reconstructed from a low-resolution image, which was down-sampled from an original test image. For quantitative evaluation, two image quality metrics were measured and compared to those of the conventional linear interpolation methods. The image restoration quality of the SRCNN scheme was significantly higher than that of the linear interpolation methods (p < 0.001 or p < 0.05). The high-resolution image reconstructed by the SRCNN scheme was highly restored and comparable to the original reference image, in particular, for a ×2 magnification. These results indicate that the SRCNN scheme significantly outperforms the linear interpolation methods for enhancing image resolution in chest CT images. The results also suggest that SRCNN may become a potential solution for generating high-resolution CT images from standard CT images. <s> BIB002
The multi-grained cascades Forest (gcForest or Deep Forest BIB001 ), which is a novel decision tree ensemble method, can work well even when there are only small-scale training data. This ensemble method can generate a deep forest (see in Fig.18) , with a cascade structure which enables gcForest to do representation learning. Also, the main advantage of the deep forest is that the number of cascade levels can be adaptively determined such that the model complexity can be automatically set. First of all, one of the essential concepts in gcForest is the forest, which can regard as an ensemble classifier consisting of multiple decision trees. For example, suppose there are three classes, then each of the forests will produce a three-dimensional class vector. Moreover, the class vector of sample x can obtain by averaging across all trees class vector for this forest. Here, the number of trees in the forest is a hyperparameter. Further, one can apply different types of forests to encourage diversity, and diversity is crucial for an ensemble framework. For example, gcForest uses two completely random tree forests and two random forests, in which each completely-random tree forest contains 500 completely random trees, and each random forest also includes 500 trees. Therefore, for the cascade forest structure, we have the following illustrations in Fig.19 . Here y is the final prediction for input feature vector x. Concatenate operations can be used to avoid excessive attenuation of information with the deepening of the level. Further, inspired by the local receptive field of CNN, sliding windows are used to enhance cascade forest. Then we have scanned for input feature vector. For example, suppose there are 100 dimension raw features x and a window size of 10 features is used in Fig.20 , then 91 local feature vectors are produced by sliding the window, and the dimension of each local feature vectors is 10, these instances extracted from the same size of windows will be used to train a completelyrandom tree forest and a random forest, and then the class vectors are generated and concatenated as transformed features. Finally, we can construct the overall procedure of gcForest by the above cascade forest structure and scanning. That is, given a feature vector x, corresponding transformed feature representation can obtain by multi-grained scanning procedure, in other words, scanning with multi-sliding window size; and then go through the cascade till the last level, the final prediction will obtain via an average and maximum operation. More importantly, hyperparameters of gcForest include four parts, ones are the number of trees in a forest, second is the number of the different forest for diversity and the number of the forest for scanning, third is the number of levels for cascade forest structure, and fourth is the number of sliding window size. The same with traditional deep neural networks, gcForest can also achieve highly competitive generalization performance. Further, gcForest can also be regarded as a definite attempt of deep learning of non-neural networks. 2) ADMM-NET ADMM-Net, which is a novel deep network architecture, can derive from the iterative procedures in ADMM algorithm  ,  . It is application motivation is that it improves the current magnetic resonance imaging system in reconstruction accuracy and speed. As a starting point, we introduce ADMM algorithm. Assume x is an image to be reconstructed, and y is the observed image or under-sampled image, then we have the following generalized compressed sensing problem, where A F is a measurement matrix and is an undersampling matrix, F is a Fourier transform. D l denotes a transform matrix for a filtering operation, g(·) is a regularization function derived from the data prior, such as sparse prior. This optimization problem can be solved efficiently by the ADMM algorithm. Concretely, we use the norm equation for simplicity, and its augmented Lagrangian function is: where β = [β 1 , · · · , β L ] and z = [z 1 , · · · , z L ]. and ρ = [ρ 1 , · · · , ρ L ] are penalty parameters. Further, Eq.(52) can be solved the following three sub-problems: Substitute A = F into Eq.(53), then the three sub-problems have the following solutions: where n denotes n-th iteration, S(·) is a non-linear shrinkage function with parameters λ l /ρ l , and η l is an update rate for updating the multiplier. Here X (n) , Z (n) and β (n) denotes three types of solutions. Based on the above iterative algorithm in the ADMM algorithm, Basic ADMM-Net can design by the following steps; the first is that generalizes X (n) to reconstruction layer, the second is that decomposes Z (n) to convolution layer and non-linear transform layer, the third is that extends β (n) to multiplier update layer. Concretely, for reconstruction layer, we have rewritten as, where H (n) l is the l-th filter with size of w f × w f in the iteration stage n, and replace the fixed D l in Eq. BIB002 . For convolution layer C (n) , we have D (n) l ⊗ x (n) replace D l x (n) , and l = 1, 2, · · · , L. For non-linear transform layer N (n) , the output of this layer is defined as, where S(·) is a piecewise linear function determined by a set of control points. Finally, for multiplier update layer β (n) R , we have where η (n) l is a parameter to be learned. Compare to convolution flow in CNNs, here new version convolution flow in stage n can be represented as R → · · · In this novel deep architecture, its purpose to learn parameters includes H (n) l and ρ (n) l for reconstruction layer, D (n) l for convolution layer, and η (n) l in multiplier update layer. Obviously, immediate reconstruction result at each stage can be visualized under each reconstruction layer.