For intelligence and learning materials in human learning (HL), we identify their counterparts in machine learning as model capacity and data. We are interested in asking: for learning skills in HL, do they have counterparts in ML as well? Can machines be equipped with effective learning skills as humans are? In this paper, we aim to address these questions. We propose a general framework – Skillearn, which draws inspiration from humans' learning skills and formulates them into machines' learning skills (MLS). These MLS are leveraged to train better ML models. In Skillearn, there are one or multiple learner models, each with one or multiple sets of learnable parameters such as weight parameters, architectures, hyperparameters, etc. Different learners interact with each other through interaction functions. The learning of all learners is organized into multiple stages, each involving a subset of learners. The stages have an order, but they are performed end-to-end in a multi-level optimization framework where latter stages influence earlier stages and vice versa. We develop a unified optimization algorithm for solving the multi-level optimization problem in Skillearn. In two case studies, we apply Skillearn to formalize two learning skills of humans – learning by passing tests (LPT) and interleaving learning (IL) – into machines' learning skills (MLS) and leverage these MLS for neural architecture search [1]}, [2]}, [3]}. In LPT, a tester model dynamically creates tests with increasing levels of difficulty to evaluate a testee model; the testee continuously improves its architecture by passing however difficult tests created by the tester. In IL, a set of models collaboratively learn a data encoder in an interleaving fashion: the encoder is trained by model 1 for a while, then passed to model 2 for further training, then model 3, and so on; after trained by all models, the encoder returns back to model 1 and is trained again, then moving to model 2, 3, etc. This process repeats for multiple rounds. Experiments on various datasets demonstrate that ML models trained by these two learning skills achieve significantly better performance.
