porter stemmer this is the porter stemming algorithm it follows the algorithm presented in porter m an algorithm for suffix stripping program 14 3 1980 130137 with some optional deviations that can be turned on or off with the mode argument to the constructor martin porter the algorithm s inventor maintains a web page about the algorithm at https www tartarus orgmartinporterstemmer which includes another python implementation and other implementations in many languages a word stemmer based on the porter stemming algorithm porter m an algorithm for suffix stripping program 14 3 1980 130137 see https www tartarus orgmartinporterstemmer for the homepage of the algorithm martin porter has endorsed several modifications to the porter algorithm since writing his original paper and those extensions are included in the implementations on his website additionally others have proposed further improvements to the algorithm including nltk contributors there are thus three modes that can be selected by passing the appropriate constant to the class constructor s mode attribute porterstemmer originalalgorithm an implementation that is faithful to the original paper note that martin porter has deprecated this version of the algorithm martin distributes implementations of the porter stemmer in many languages hosted at https www tartarus orgmartinporterstemmer and all of these implementations include his extensions he strongly recommends against using the original published version of the algorithm only use this mode if you clearly understand why you are choosing to do so porterstemmer martinextensions an implementation that only uses the modifications to the algorithm that are included in the implementations on martin porter s website he has declared porter frozen so the behaviour of those implementations should never change porterstemmer nltkextensions default an implementation that includes further improvements devised by nltk contributors or taken from other modified implementations found on the web for the best stemming you should use the default nltkextensions version however if you need to get the same results as either the original algorithm or one of martin porter s hosted versions for compatibility with an existing implementation or dataset you can use one of the other modes instead modes the stemmer can be instantiated in this is a table of irregular forms it is quite short but still reflects the errors actually drawn to martin porter s attention over a 20 year period returns true if wordi is a consonant false otherwise a consonant is defined in the paper as follows a consonant in a word is a letter other than a e i o or u and other than y preceded by a consonant the fact that the term consonant is defined to some extent in terms of itself does not make it ambiguous so in toy the consonants are t and y and in syzygy they are s z and g if a letter is not a consonant it is a vowel cvsequence construct a string of c s and v s representing whether each character in stem is a consonant or a vowel e g falafel becomes cvcvcvc architecture becomes vcccvcvccvcv for i in rangelenstem if self isconsonantstem i cvsequence c else cvsequence v count the number of vc occurrences which is equivalent to the number of vc occurrences in porter s reduced form in the docstring above which is in turn equivalent to m return cvsequence countvc def haspositivemeasureself stem return self measurestem 0 def containsvowelself stem implements condition d from the paper returns true if word ends with a double consonant implements condition o from the paper from the paper o the stem ends cvc where the second c is not w x or y e g wil hop replaces suffix of word with replacement assert word endswithsuffix given word doesn t end with given suffix if suffix return word replacement else return word lensuffix replacement def applyrulelistself word rules for rule in rules suffix replacement condition rule if suffix d and self endsdoubleconsonantword stem word 2 if condition is none or conditionstem return stem replacement else don t try any further rules return word if word endswithsuffix stem self replacesuffixword suffix if condition is none or conditionstem return stem replacement else don t try any further rules return word return word def step1aself word this nltkonly rule extends the original algorithm so that flies fli but dies die etc if self mode self nltkextensions if word endswithies and lenword 4 return self replacesuffixword ies ie return self applyrulelist word sses ss none sses ss ies i none ies i ss ss none ss ss s none s def step1bself word this nltkonly block extends the original algorithm so that spied spi but died die etc if self mode self nltkextensions if word endswithied if lenword 4 return self replacesuffixword ied ie else return self replacesuffixword ied i m0 eed ee if word endswitheed stem self replacesuffixword eed if self measurestem 0 return stem ee else return word rule2or3succeeded false for suffix in ed ing if word endswithsuffix intermediatestem self replacesuffixword suffix if self containsvowelintermediatestem rule2or3succeeded true break if not rule2or3succeeded return word return self applyrulelist intermediatestem at ate none at ate bl ble none bl ble iz ize none iz ize d and not l or s or z single letter d intermediatestem1 lambda stem intermediatestem1 not in l s z m1 and o e e lambda stem self measurestem 1 and self endscvcstem def step1cself word def nltkconditionstem return lenstem 1 and self isconsonantstem lenstem 1 def originalconditionstem return self containsvowelstem return self applyrulelist word y i nltkcondition if self mode self nltkextensions else originalcondition def step2self word if self mode self nltkextensions instead of applying the alli al rule after abli per the published algorithm instead we apply it first and if it succeeds run the result through step2 again if word endswithalli and self haspositivemeasure self replacesuffixword alli return self step2self replacesuffixword alli al blirule bli ble self haspositivemeasure ablirule abli able self haspositivemeasure rules ational ate self haspositivemeasure tional tion self haspositivemeasure enci ence self haspositivemeasure anci ance self haspositivemeasure izer ize self haspositivemeasure ablirule if self mode self originalalgorithm else blirule alli al self haspositivemeasure entli ent self haspositivemeasure eli e self haspositivemeasure ousli ous self haspositivemeasure ization ize self haspositivemeasure ation ate self haspositivemeasure ator ate self haspositivemeasure alism al self haspositivemeasure iveness ive self haspositivemeasure fulness ful self haspositivemeasure ousness ous self haspositivemeasure aliti al self haspositivemeasure iviti ive self haspositivemeasure biliti ble self haspositivemeasure if self mode self nltkextensions rules appendfulli ful self haspositivemeasure the l of the logi log rule is put with the stem so that short stems like geo theo etc work like archaeo philo etc rules append logi log lambda stem self haspositivemeasureword 3 if self mode self martinextensions rules appendlogi log self haspositivemeasure return self applyrulelistword rules def step3self word return self applyrulelist word icate ic self haspositivemeasure ative self haspositivemeasure alize al self haspositivemeasure iciti ic self haspositivemeasure ical ic self haspositivemeasure ful self haspositivemeasure ness self haspositivemeasure def step4self word measuregt1 lambda stem self measurestem 1 return self applyrulelist word al measuregt1 ance measuregt1 ence measuregt1 er measuregt1 ic measuregt1 able measuregt1 ible measuregt1 ant measuregt1 ement measuregt1 ment measuregt1 ent measuregt1 m1 and s or t ion ion lambda stem self measurestem 1 and stem1 in s t ou measuregt1 ism measuregt1 ate measuregt1 iti measuregt1 ous measuregt1 ive measuregt1 ize measuregt1 def step5aself word note that martin s test vocabulary and reference implementations are inconsistent in how they handle the case where two rules both refer to a suffix that matches the word to be stemmed but only the condition of the second one is true earlier in step2b we had the rules m0 eed ee v ed but the examples in the paper included feedfeed even though v is true for fe and therefore the second rule alone would map feedfe however in this case we need to handle the consecutive rules differently and try both conditions obviously the second rule here would be redundant otherwise martin s paper makes no explicit mention of the inconsistency you have to infer it from the examples for this reason we can t use applyrulelist here if word endswithe stem self replacesuffixword e if self measurestem 1 return stem if self measurestem 1 and not self endscvcstem return stem return word def step5bself word return self applyrulelist word ll l lambda stem self measureword 1 1 def stemself word tolowercasetrue stem word lower if tolowercase else word if self mode self nltkextensions and word in self pool return self poolstem if self mode self originalalgorithm and lenword 2 with this line strings of length 1 or 2 don t go through the stemming process although no mention is made of this in the published algorithm return stem stem self step1astem stem self step1bstem stem self step1cstem stem self step2stem stem self step3stem stem self step4stem stem self step5astem stem self step5bstem return stem def reprself return porterstemmer def demo from nltk import stem from nltk corpus import treebank stemmer stem porterstemmer orig stemmed for item in treebank fileids 3 for word tag in treebank taggedwordsitem orig appendword stemmed appendstemmer stemword convert the results to a string and wordwrap them results joinstemmed results re subr 70s r1n results rstrip convert the original to a string and word wrap it original joinorig original re subr 70s r1n original rstrip print the results printoriginal center70 replace replace printoriginal printresults center70 replace replace printresults print 70 a word stemmer based on the porter stemming algorithm porter m an algorithm for suffix stripping program 14 3 1980 130 137 see https www tartarus org martin porterstemmer for the homepage of the algorithm martin porter has endorsed several modifications to the porter algorithm since writing his original paper and those extensions are included in the implementations on his website additionally others have proposed further improvements to the algorithm including nltk contributors there are thus three modes that can be selected by passing the appropriate constant to the class constructor s mode attribute porterstemmer original_algorithm an implementation that is faithful to the original paper note that martin porter has deprecated this version of the algorithm martin distributes implementations of the porter stemmer in many languages hosted at https www tartarus org martin porterstemmer and all of these implementations include his extensions he strongly recommends against using the original published version of the algorithm only use this mode if you clearly understand why you are choosing to do so porterstemmer martin_extensions an implementation that only uses the modifications to the algorithm that are included in the implementations on martin porter s website he has declared porter frozen so the behaviour of those implementations should never change porterstemmer nltk_extensions default an implementation that includes further improvements devised by nltk contributors or taken from other modified implementations found on the web for the best stemming you should use the default nltk_extensions version however if you need to get the same results as either the original algorithm or one of martin porter s hosted versions for compatibility with an existing implementation or dataset you can use one of the other modes instead modes the stemmer can be instantiated in this is a table of irregular forms it is quite short but still reflects the errors actually drawn to martin porter s attention over a 20 year period returns true if word i is a consonant false otherwise a consonant is defined in the paper as follows a consonant in a word is a letter other than a e i o or u and other than y preceded by a consonant the fact that the term consonant is defined to some extent in terms of itself does not make it ambiguous so in toy the consonants are t and y and in syzygy they are s z and g if a letter is not a consonant it is a vowel returns the measure of stem per definition in the paper from the paper a consonant will be denoted by c a vowel by v a list ccc of length greater than 0 will be denoted by c and a list vvv of length greater than 0 will be denoted by v any word or part of a word therefore has one of the four forms cvcv c cvcv v vcvc c vcvc v these may all be represented by the single form c vcvc v where the square brackets denote arbitrary presence of their contents using vc m to denote vc repeated m times this may again be written as c vc m v m will be called the measure of any word or word part when represented in this form the case m 0 covers the null word here are some examples m 0 tr ee tree y by m 1 trouble oats trees ivy m 2 troubles private oaten orrery construct a string of c s and v s representing whether each character in stem is a consonant or a vowel e g falafel becomes cvcvcvc architecture becomes vcccvcvccvcv count the number of vc occurrences which is equivalent to the number of vc occurrences in porter s reduced form in the docstring above which is in turn equivalent to m returns true if stem contains a vowel else false implements condition d from the paper returns true if word ends with a double consonant implements condition o from the paper from the paper o the stem ends cvc where the second c is not w x or y e g wil hop replaces suffix of word with replacement applies the first applicable suffix removal rule to the word takes a word and a list of suffix removal rules represented as 3 tuples with the first element being the suffix to remove the second element being the string to replace it with and the final element being the condition for the rule to be applicable or none if the rule is unconditional don t try any further rules don t try any further rules implements step 1a from an algorithm for suffix stripping from the paper sses ss caresses caress ies i ponies poni ties ti ss ss caress caress s cats cat this nltk only rule extends the original algorithm so that flies fli but dies die etc sses ss ies i ss ss s implements step 1b from an algorithm for suffix stripping from the paper m 0 eed ee feed feed agreed agree v ed plastered plaster bled bled v ing motoring motor sing sing if the second or third of the rules in step 1b is successful the following is done at ate conflat ed conflate bl ble troubl ed trouble iz ize siz ed size d and not l or s or z single letter hopp ing hop tann ed tan fall ing fall hiss ing hiss fizz ed fizz m 1 and o e fail ing fail fil ing file the rule to map to a single letter causes the removal of one of the double letter pair the e is put back on at bl and iz so that the suffixes ate ble and ize can be recognised later this e may be removed in step 4 this nltk only block extends the original algorithm so that spied spi but died die etc m 0 eed ee at ate bl ble iz ize d and not l or s or z single letter m 1 and o e implements step 1c from an algorithm for suffix stripping from the paper step 1c v y i happy happi sky sky this has been modified from the original porter algorithm so that y i is only done when y is preceded by a consonant but not if the stem is only a single consonant i e c and not c y i so happy happi but enjoy enjoy etc this is a much better rule formerly enjoy enjoi and enjoyment enjoy step 1c is perhaps done too soon but with this modification that no longer really matters also the removal of the contains_vowel z condition means that spy fly try stem to spi fli tri and conflate with spied tried flies implements step 2 from an algorithm for suffix stripping from the paper step 2 m 0 ational ate relational relate m 0 tional tion conditional condition rational rational m 0 enci ence valenci valence m 0 anci ance hesitanci hesitance m 0 izer ize digitizer digitize m 0 abli able conformabli conformable m 0 alli al radicalli radical m 0 entli ent differentli different m 0 eli e vileli vile m 0 ousli ous analogousli analogous m 0 ization ize vietnamization vietnamize m 0 ation ate predication predicate m 0 ator ate operator operate m 0 alism al feudalism feudal m 0 iveness ive decisiveness decisive m 0 fulness ful hopefulness hopeful m 0 ousness ous callousness callous m 0 aliti al formaliti formal m 0 iviti ive sensitiviti sensitive m 0 biliti ble sensibiliti sensible instead of applying the alli al rule after a bli per the published algorithm instead we apply it first and if it succeeds run the result through step2 again the l of the logi log rule is put with the stem so that short stems like geo theo etc work like archaeo philo etc implements step 3 from an algorithm for suffix stripping from the paper step 3 m 0 icate ic triplicate triplic m 0 ative formative form m 0 alize al formalize formal m 0 iciti ic electriciti electric m 0 ical ic electrical electric m 0 ful hopeful hope m 0 ness goodness good implements step 4 from an algorithm for suffix stripping step 4 m 1 al revival reviv m 1 ance allowance allow m 1 ence inference infer m 1 er airliner airlin m 1 ic gyroscopic gyroscop m 1 able adjustable adjust m 1 ible defensible defens m 1 ant irritant irrit m 1 ement replacement replac m 1 ment adjustment adjust m 1 ent dependent depend m 1 and s or t ion adoption adopt m 1 ou homologou homolog m 1 ism communism commun m 1 ate activate activ m 1 iti angulariti angular m 1 ous homologous homolog m 1 ive effective effect m 1 ize bowdlerize bowdler the suffixes are now removed all that remains is a little tidying up m 1 and s or t ion implements step 5a from an algorithm for suffix stripping from the paper step 5a m 1 e probate probat rate rate m 1 and not o e cease ceas note that martin s test vocabulary and reference implementations are inconsistent in how they handle the case where two rules both refer to a suffix that matches the word to be stemmed but only the condition of the second one is true earlier in step2b we had the rules m 0 eed ee v ed but the examples in the paper included feed feed even though v is true for fe and therefore the second rule alone would map feed fe however in this case we need to handle the consecutive rules differently and try both conditions obviously the second rule here would be redundant otherwise martin s paper makes no explicit mention of the inconsistency you have to infer it from the examples for this reason we can t use _apply_rule_list here implements step 5a from an algorithm for suffix stripping from the paper step 5b m 1 and d and l single letter controll control roll roll param to_lowercase if to_lowercase true the word always lowercase with this line strings of length 1 or 2 don t go through the stemming process although no mention is made of this in the published algorithm a demonstration of the porter stemmer on a sample from the penn treebank corpus convert the results to a string and word wrap them convert the original to a string and word wrap it print the results
__docformat__ = "plaintext"

import re

from nltk.stem.api import StemmerI


class PorterStemmer(StemmerI):
    

    
    NLTK_EXTENSIONS = "NLTK_EXTENSIONS"
    MARTIN_EXTENSIONS = "MARTIN_EXTENSIONS"
    ORIGINAL_ALGORITHM = "ORIGINAL_ALGORITHM"

    def __init__(self, mode=NLTK_EXTENSIONS):
        if mode not in (
            self.NLTK_EXTENSIONS,
            self.MARTIN_EXTENSIONS,
            self.ORIGINAL_ALGORITHM,
        ):
            raise ValueError(
                "Mode must be one of PorterStemmer.NLTK_EXTENSIONS, "
                "PorterStemmer.MARTIN_EXTENSIONS, or "
                "PorterStemmer.ORIGINAL_ALGORITHM"
            )

        self.mode = mode

        if self.mode == self.NLTK_EXTENSIONS:
            
            
            
            irregular_forms = {
                "sky": ["sky", "skies"],
                "die": ["dying"],
                "lie": ["lying"],
                "tie": ["tying"],
                "news": ["news"],
                "inning": ["innings", "inning"],
                "outing": ["outings", "outing"],
                "canning": ["cannings", "canning"],
                "howe": ["howe"],
                "proceed": ["proceed"],
                "exceed": ["exceed"],
                "succeed": ["succeed"],
            }

            self.pool = {}
            for key in irregular_forms:
                for val in irregular_forms[key]:
                    self.pool[val] = key

        self.vowels = frozenset(["a", "e", "i", "o", "u"])

    def _is_consonant(self, word, i):
        
        if word[i] in self.vowels:
            return False
        if word[i] == "y":
            if i == 0:
                return True
            else:
                return not self._is_consonant(word, i - 1)
        return True

    def _measure(self, stem):
        r
        cv_sequence = ""

        
        
        
        
        for i in range(len(stem)):
            if self._is_consonant(stem, i):
                cv_sequence += "c"
            else:
                cv_sequence += "v"

        
        
        
        return cv_sequence.count("vc")

    def _has_positive_measure(self, stem):
        return self._measure(stem) > 0

    def _contains_vowel(self, stem):
        
        for i in range(len(stem)):
            if not self._is_consonant(stem, i):
                return True
        return False

    def _ends_double_consonant(self, word):
        
        return (
            len(word) >= 2
            and word[-1] == word[-2]
            and self._is_consonant(word, len(word) - 1)
        )

    def _ends_cvc(self, word):
        
        return (
            len(word) >= 3
            and self._is_consonant(word, len(word) - 3)
            and not self._is_consonant(word, len(word) - 2)
            and self._is_consonant(word, len(word) - 1)
            and word[-1] not in ("w", "x", "y")
        ) or (
            self.mode == self.NLTK_EXTENSIONS
            and len(word) == 2
            and not self._is_consonant(word, 0)
            and self._is_consonant(word, 1)
        )

    def _replace_suffix(self, word, suffix, replacement):
        
        assert word.endswith(suffix), "Given word doesn't end with given suffix"
        if suffix == "":
            return word + replacement
        else:
            return word[: -len(suffix)] + replacement

    def _apply_rule_list(self, word, rules):
        
        for rule in rules:
            suffix, replacement, condition = rule
            if suffix == "*d" and self._ends_double_consonant(word):
                stem = word[:-2]
                if condition is None or condition(stem):
                    return stem + replacement
                else:
                    
                    return word
            if word.endswith(suffix):
                stem = self._replace_suffix(word, suffix, "")
                if condition is None or condition(stem):
                    return stem + replacement
                else:
                    
                    return word

        return word

    def _step1a(self, word):
        
        
        
        if self.mode == self.NLTK_EXTENSIONS:
            if word.endswith("ies") and len(word) == 4:
                return self._replace_suffix(word, "ies", "ie")

        return self._apply_rule_list(
            word,
            [
                ("sses", "ss", None),  
                ("ies", "i", None),  
                ("ss", "ss", None),  
                ("s", "", None),  
            ],
        )

    def _step1b(self, word):
        
        
        
        if self.mode == self.NLTK_EXTENSIONS:
            if word.endswith("ied"):
                if len(word) == 4:
                    return self._replace_suffix(word, "ied", "ie")
                else:
                    return self._replace_suffix(word, "ied", "i")

        
        if word.endswith("eed"):
            stem = self._replace_suffix(word, "eed", "")
            if self._measure(stem) > 0:
                return stem + "ee"
            else:
                return word

        rule_2_or_3_succeeded = False

        for suffix in ["ed", "ing"]:
            if word.endswith(suffix):
                intermediate_stem = self._replace_suffix(word, suffix, "")
                if self._contains_vowel(intermediate_stem):
                    rule_2_or_3_succeeded = True
                    break

        if not rule_2_or_3_succeeded:
            return word

        return self._apply_rule_list(
            intermediate_stem,
            [
                ("at", "ate", None),  
                ("bl", "ble", None),  
                ("iz", "ize", None),  
                
                
                (
                    "*d",
                    intermediate_stem[-1],
                    lambda stem: intermediate_stem[-1] not in ("l", "s", "z"),
                ),
                
                (
                    "",
                    "e",
                    lambda stem: (self._measure(stem) == 1 and self._ends_cvc(stem)),
                ),
            ],
        )

    def _step1c(self, word):
        

        def nltk_condition(stem):
            
            return len(stem) > 1 and self._is_consonant(stem, len(stem) - 1)

        def original_condition(stem):
            return self._contains_vowel(stem)

        return self._apply_rule_list(
            word,
            [
                (
                    "y",
                    "i",
                    nltk_condition
                    if self.mode == self.NLTK_EXTENSIONS
                    else original_condition,
                )
            ],
        )

    def _step2(self, word):
        

        if self.mode == self.NLTK_EXTENSIONS:
            
            
            
            if word.endswith("alli") and self._has_positive_measure(
                self._replace_suffix(word, "alli", "")
            ):
                return self._step2(self._replace_suffix(word, "alli", "al"))

        bli_rule = ("bli", "ble", self._has_positive_measure)
        abli_rule = ("abli", "able", self._has_positive_measure)

        rules = [
            ("ational", "ate", self._has_positive_measure),
            ("tional", "tion", self._has_positive_measure),
            ("enci", "ence", self._has_positive_measure),
            ("anci", "ance", self._has_positive_measure),
            ("izer", "ize", self._has_positive_measure),
            abli_rule if self.mode == self.ORIGINAL_ALGORITHM else bli_rule,
            ("alli", "al", self._has_positive_measure),
            ("entli", "ent", self._has_positive_measure),
            ("eli", "e", self._has_positive_measure),
            ("ousli", "ous", self._has_positive_measure),
            ("ization", "ize", self._has_positive_measure),
            ("ation", "ate", self._has_positive_measure),
            ("ator", "ate", self._has_positive_measure),
            ("alism", "al", self._has_positive_measure),
            ("iveness", "ive", self._has_positive_measure),
            ("fulness", "ful", self._has_positive_measure),
            ("ousness", "ous", self._has_positive_measure),
            ("aliti", "al", self._has_positive_measure),
            ("iviti", "ive", self._has_positive_measure),
            ("biliti", "ble", self._has_positive_measure),
        ]

        if self.mode == self.NLTK_EXTENSIONS:
            rules.append(("fulli", "ful", self._has_positive_measure))

            
            
            
            rules.append(
                ("logi", "log", lambda stem: self._has_positive_measure(word[:-3]))
            )

        if self.mode == self.MARTIN_EXTENSIONS:
            rules.append(("logi", "log", self._has_positive_measure))

        return self._apply_rule_list(word, rules)

    def _step3(self, word):
        
        return self._apply_rule_list(
            word,
            [
                ("icate", "ic", self._has_positive_measure),
                ("ative", "", self._has_positive_measure),
                ("alize", "al", self._has_positive_measure),
                ("iciti", "ic", self._has_positive_measure),
                ("ical", "ic", self._has_positive_measure),
                ("ful", "", self._has_positive_measure),
                ("ness", "", self._has_positive_measure),
            ],
        )

    def _step4(self, word):
        
        measure_gt_1 = lambda stem: self._measure(stem) > 1

        return self._apply_rule_list(
            word,
            [
                ("al", "", measure_gt_1),
                ("ance", "", measure_gt_1),
                ("ence", "", measure_gt_1),
                ("er", "", measure_gt_1),
                ("ic", "", measure_gt_1),
                ("able", "", measure_gt_1),
                ("ible", "", measure_gt_1),
                ("ant", "", measure_gt_1),
                ("ement", "", measure_gt_1),
                ("ment", "", measure_gt_1),
                ("ent", "", measure_gt_1),
                
                (
                    "ion",
                    "",
                    lambda stem: self._measure(stem) > 1 and stem[-1] in ("s", "t"),
                ),
                ("ou", "", measure_gt_1),
                ("ism", "", measure_gt_1),
                ("ate", "", measure_gt_1),
                ("iti", "", measure_gt_1),
                ("ous", "", measure_gt_1),
                ("ive", "", measure_gt_1),
                ("ize", "", measure_gt_1),
            ],
        )

    def _step5a(self, word):
        
        
        
        
        
        
        
        
        
        
        
        
        
        
        
        
        
        
        if word.endswith("e"):
            stem = self._replace_suffix(word, "e", "")
            if self._measure(stem) > 1:
                return stem
            if self._measure(stem) == 1 and not self._ends_cvc(stem):
                return stem
        return word

    def _step5b(self, word):
        
        return self._apply_rule_list(
            word, [("ll", "l", lambda stem: self._measure(word[:-1]) > 1)]
        )

    def stem(self, word, to_lowercase=True):
        
        stem = word.lower() if to_lowercase else word

        if self.mode == self.NLTK_EXTENSIONS and word in self.pool:
            return self.pool[stem]

        if self.mode != self.ORIGINAL_ALGORITHM and len(word) <= 2:
            
            
            
            return stem

        stem = self._step1a(stem)
        stem = self._step1b(stem)
        stem = self._step1c(stem)
        stem = self._step2(stem)
        stem = self._step3(stem)
        stem = self._step4(stem)
        stem = self._step5a(stem)
        stem = self._step5b(stem)

        return stem

    def __repr__(self):
        return "<PorterStemmer>"


def demo():
    

    from nltk import stem
    from nltk.corpus import treebank

    stemmer = stem.PorterStemmer()

    orig = []
    stemmed = []
    for item in treebank.fileids()[:3]:
        for (word, tag) in treebank.tagged_words(item):
            orig.append(word)
            stemmed.append(stemmer.stem(word))

    
    results = " ".join(stemmed)
    results = re.sub(r"(.{,70})\s", r"\1\n", results + " ").rstrip()

    
    original = " ".join(orig)
    original = re.sub(r"(.{,70})\s", r"\1\n", original + " ").rstrip()

    
    print("-Original-".center(70).replace(" ", "*").replace("-", " "))
    print(original)
    print("-Results-".center(70).replace(" ", "*").replace("-", " "))
    print(results)
    print("*" * 70)