This work has been funded in part by a research grant from Science Foundation Ireland (SFI) under Grant Number SFI 12 RC 2289 (INSIGHT) and by Enterprise Ireland and the IDA under the Technology Centre Programme [Grant TC-2012-009]
An economic, scalable machine learning system and process perform document (concept) classification with high accuracy using large topic schemes, including large hierarchical topic schemes. One or more highly relevant classification topics is suggested for a-given document (concept) to be classified. The invention includes training and concept classification processes. The invention also provides methods that may be used as part of the training and or concept classification processes, including: a method of scoring the relevance of features in training concepts, a method of ranking concepts based on relevance score, and a method of voting on topics associated with an input concept. In a preferred embodiment, the invention is applied to the legal (case law) domain, classifying legal concepts (rules of law) according to a proprietary legal topic classification scheme (a hierarchical scheme of areas of law). Document management is not often handled appropriately by organisations, if at all. Despite that, and despite the lack of structure in documents, organisations must face regulations that require owning a document collection with semantic content. The technique based on taxonomies and folksonomies can easily produce an adequate semantic classification for documents. It requires an adequate setup among the domain experts that apply it. The approach we propose uses Lean Kanban to coordinate the phases of definition, validation and implementation of taxonomies and folksonomies. It helps organisations to create a semantic classification of existing document resources, making them ready to be used in ways that were not possible before. At the same time, it helps to improve the quality of work of the organisation itself, adding speed to document search.