Deep learning algorithms have achieved excellent performance lately in a wide range of fields (e.g., computer version). However, a severe challenge faced by deep learning is the high dependency on hyper-parameters. The algorithm results may fluctuate dramatically under the different configuration of hyper-parameters. Addressing the above issue, this paper presents an efficient Orthogonal Array Tuning Method (OATM) for deep learning hyper-parameter tuning. We describe the OATM approach in five detailed steps and elaborate on it using two widely used deep neural network structures (Recurrent Neural Networks and Convolutional Neural Networks). The proposed method is compared to the state-of-the-art hyper-parameter tuning methods including manually (e.g., grid search and random search) and automatically (e.g., Bayesian Optimization) ones. The experiment results state that OATM can significantly save the tuning time compared to the state-of-the-art methods while preserving the satisfying performance.
Apart from the aforementioned methods, the orthogonal array based hyper-parameter tuning already used in a range of research areas such as mechanical engineering and electrical engineering. J.A @cite_6 applied orthogonal array based approach to optimize the cutting parameters in the end milling. S.S. @cite_10 optimized wire electrical discharge machining (WEDM) process parameters by orthogonal array method. The traditional methods are not suited for deep learning algorithms while the effectiveness of OATM has been demonstrated in many research topics. Intuitively, we adopt OATM for deep learning hyper-parameter tuning. To our best knowledge, our work is the first batch of studies in this area.
@cite_10 @cite_6
2075314456 2084841915 1579064899 2042992924
Wire electrical discharge machining (WEDM) is extensively used in machining of conductive materials when precision is of prime importance. Rough cutting operation in WEDM is treated as a challenging one because improvement of more than one machining performance measures viz. met al removal rate (MRR), surface finish (SF) and cutting width (kerf) are sought to obtain a precision work. Using Taguchi’s parameter design, significant machining parameters affecting the performance measures are identified as discharge current, pulse duration, pulse frequency, wire speed, wire tension, and dielectric flow. It has been observed that a combination of factors for optimization of each performance measure is different. In this study, the relationship between control factors and responses like MRR, SF and kerf are established by means of nonlinear regression analysis, resulting in a valid mathematical model. Finally, genetic algorithm, a popular evolutionary approach, is employed to optimize the wire electrical discharge machining process with multiple objectives. The study demonstrates that the WEDM process parameters can be adjusted to achieve better met al removal rate, surface finish and cutting width simultaneously. Abstract In this study, the Taguchi method is used to find the optimal cutting parameters for surface roughness in turning. The orthogonal array, the signal-to-noise ratio, and analysis of variance are employed to study the performance characteristics in turning operations of AISI 1030 steel bars using TiN coated tools. Three cutting parameters namely, insert radius, feed rate, and depth of cut, are optimized with considerations of surface roughness. Experimental results are provided to illustrate the effectiveness of this approach. We introduce a means of automating machine learning (ML) for big data tasks, by performing scalable stochastic Bayesian optimisation of ML algorithm parameters and hyper-parameters. More often than not, the critical tuning of ML algorithm parameters has relied on domain expertise from experts, along with laborious hand-tuning, brute search or lengthy sampling runs. Against this background, Bayesian optimisation is finding increasing use in automating parameter tuning, making ML algorithms accessible even to non-experts. However, the state of the art in Bayesian optimisation is incapable of scaling to the large number of evaluations of algorithm performance required to fit realistic models to complex, big data. We here describe a stochastic, sparse, Bayesian optimisation strategy to solve this problem, using many thousands of noisy evaluations of algorithm performance on subsets of data in order to effectively train algorithms for big data. We provide a comprehensive benchmarking of possible sparsification strategies for Bayesian optimisation, concluding that a Nystrom approximation offers the best scaling and performance for real tasks. Our proposed algorithm demonstrates substantial improvement over the state of the art in tuning the parameters of a Gaussian Process time series prediction task on real, big data. We present an auto-tuning system for optimizing I O performance of HDF5 applications and demonstrate its value across platforms, applications, and at scale. The system uses a genetic algorithm to search a large space of tunable parameters and to identify effective settings at all layers of the parallel I O stack. The parameter settings are applied transparently by the auto-tuning system via dynamically intercepted HDF5 calls. To validate our auto-tuning system, we applied it to three I O benchmarks (VPIC, VORPAL, and GCRM) that replicate the I O activity of their respective applications. We tested the system with different weak-scaling configurations (128, 2048, and 4096 CPU cores) that generate 30 GB to 1 TB of data, and executed these configurations on diverse HPC platforms (Cray XE6, IBM BG P, and Dell Cluster). In all cases, the auto-tuning framework identified tunable parameters that substantially improved write performance over default system settings. We consistently demonstrate I O write speedups between 2x and 100x for test configurations.