natural language toolkit classifiers c 20012023 nltk project edward loper edlopergmail com url https www nltk org for license information see license txt classes and interfaces for labeling tokens with category labels or class labels typically labels are represented with strings such as health or sports classifiers can be used to perform a wide range of classification tasks for example classifiers can be used to classify documents by topic to classify ambiguous words by which word sense is intended to classify acoustic signals by which phoneme they represent to classify sentences by their features in order to decide which category label is appropriate for a given token classifiers examine one or more features of the token these features are typically chosen by hand and indicate which aspects of the token are relevant to the classification decision for example a document classifier might use a separate feature for each word recording how often that word occurred in the document featuresets the features describing a token are encoded using a featureset which is a dictionary that maps from feature names to feature values feature names are unique strings that indicate what aspect of the token is encoded by the feature examples include prevword for a feature whose value is the previous word and containswordlibrary for a feature that is true when a document contains the word library feature values are typically booleans numbers or strings depending on which feature they describe featuresets are typically constructed using a feature detector also known as a feature extractor a feature detector is a function that takes a token and sometimes information about its context as its input and returns a featureset describing that token for example the following feature detector converts a document stored as a list of words to a featureset describing the set of words included in the document define a feature detector function def documentfeaturesdocument return dict containswords w true for w in document feature detectors are typically applied to each token before it is fed to the classifier classify each gutenberg document from nltk corpus import gutenberg for fileid in gutenberg fileids doctest skip doc gutenberg wordsfileid doctest skip printfileid classifier classifydocumentfeaturesdoc doctest skip the parameters that a feature detector expects will vary depending on the task and the needs of the feature detector for example a feature detector for word sense disambiguation wsd might take as its input a sentence and the index of a word that should be classified and return a featureset for that word the following feature detector for wsd includes features describing the left and right contexts of the target word def wsdfeaturessentence index featureset for i in rangemax0 index3 index featureset leftcontexts sentencei true for i in rangeindex maxindex3 lensentence featureset rightcontexts sentencei true return featureset training classifiers most classifiers are built by training them on a list of handlabeled examples known as the training set training sets are represented as lists of featuredict label tuples natural language toolkit classifiers c 2001 2023 nltk project edward loper edloper gmail com url https www nltk org for license information see license txt classes and interfaces for labeling tokens with category labels or class labels typically labels are represented with strings such as health or sports classifiers can be used to perform a wide range of classification tasks for example classifiers can be used to classify documents by topic to classify ambiguous words by which word sense is intended to classify acoustic signals by which phoneme they represent to classify sentences by their features in order to decide which category label is appropriate for a given token classifiers examine one or more features of the token these features are typically chosen by hand and indicate which aspects of the token are relevant to the classification decision for example a document classifier might use a separate feature for each word recording how often that word occurred in the document featuresets the features describing a token are encoded using a featureset which is a dictionary that maps from feature names to feature values feature names are unique strings that indicate what aspect of the token is encoded by the feature examples include prevword for a feature whose value is the previous word and contains word library for a feature that is true when a document contains the word library feature values are typically booleans numbers or strings depending on which feature they describe featuresets are typically constructed using a feature detector also known as a feature extractor a feature detector is a function that takes a token and sometimes information about its context as its input and returns a featureset describing that token for example the following feature detector converts a document stored as a list of words to a featureset describing the set of words included in the document define a feature detector function def document_features document return dict contains word s w true for w in document feature detectors are typically applied to each token before it is fed to the classifier classify each gutenberg document from nltk corpus import gutenberg for fileid in gutenberg fileids doctest skip doc gutenberg words fileid doctest skip print fileid classifier classify document_features doc doctest skip the parameters that a feature detector expects will vary depending on the task and the needs of the feature detector for example a feature detector for word sense disambiguation wsd might take as its input a sentence and the index of a word that should be classified and return a featureset for that word the following feature detector for wsd includes features describing the left and right contexts of the target word def wsd_features sentence index featureset for i in range max 0 index 3 index featureset left context s sentence i true for i in range index max index 3 len sentence featureset right context s sentence i true return featureset training classifiers most classifiers are built by training them on a list of hand labeled examples known as the training set training sets are represented as lists of featuredict label tuples
from nltk.classify.api import ClassifierI, MultiClassifierI
from nltk.classify.decisiontree import DecisionTreeClassifier
from nltk.classify.maxent import (
    BinaryMaxentFeatureEncoding,
    ConditionalExponentialClassifier,
    MaxentClassifier,
    TypedMaxentFeatureEncoding,
)
from nltk.classify.megam import call_megam, config_megam
from nltk.classify.naivebayes import NaiveBayesClassifier
from nltk.classify.positivenaivebayes import PositiveNaiveBayesClassifier
from nltk.classify.rte_classify import RTEFeatureExtractor, rte_classifier, rte_features
from nltk.classify.scikitlearn import SklearnClassifier
from nltk.classify.senna import Senna
from nltk.classify.textcat import TextCat
from nltk.classify.util import accuracy, apply_features, log_likelihood
from nltk.classify.weka import WekaClassifier, config_weka