Machine learning algorithms attempt to learn to solve classification problems (e.g. recognizing which handwritten digit has been written from an image) or regression problems (e.g. estimating the price of a financial asset based on various financial variables) by training on a large database of examples where the correct answer is given. They are then typically tested on unknown examples where the correct answer is withheld.
There are a variety of existing machine learning algorithms. The currently leading classes of algorithms for classification include those based on neural networks (particularly popular nowadays are so-called “deep belief networks” (DBNs) trained using the back-propagation algorithm), support vector machines (SVMs) and K-nearest neighbor (KNN) algorithms. Versions of all of these algorithms for regression problems also exist, although they are sometimes less natural.
The performance of a machine learning algorithm can be assessed in several ways. The most popular approach is to assess based on the average error an algorithm makes on a set of test examples, after very many training examples have been presented. Unfortunately, the best machine learning algorithms, as assessed according to this perspective, are extremely slow to train. They typically need very large training corpuses, and the training time can be as long as several CPU-years, which means that clusters of computers and/or GPU acceleration often need to be employed. Existing state-of-the-art machine learning algorithms also “learn” in a way that seems unnatural compared to human learners—for example, if presented with a training example and its correct answer, they may fail to give the correct answer even if the very same example is used as a test query immediately afterward, and only learn the correct answer after thousands of presentations of the example. Also, they tend to perform very poorly if only a small number of training examples are presented, and they often require that training examples be presented in large batches. Finally, some machine learning algorithms become progressively slower to respond to test queries as they learn from larger training sets.