The idea of training multiple local classifiers on portions of the dataset has been extensively studied for specific classifiers [1]}, [2]}, [3]}, [4]}, [5]}, [6]}, [7]}, [8]}.
For instance,
[9]} presents a novel locality-sensitive support vector machine (LSSVM) for the image retrieval problem. Locally Linear SVM [10]} has a smooth decision boundary and bounded curvature. Its authors show how functions defining the classifier can be approximated using any local coding scheme.
[11]} proposes Mixed SVMs which partition the feature space into subregions of linearly separable data points and learn a Linear SVM for each of these regions. [12]} proposes Clustered Support Vector Machines (CSVMs) which tackles the problem in a divide-and-conquer manner, thus avoiding the need for computationally expensive kernel SVMs. CSVM groups the data into several clusters, after which it trains a linear SVM in each cluster to separate the data locally. Additionally, it imposes a global regularization, that aligns the weight vectors of each locally linear SVM with a global weight vector.
