Part of Speech Taggers for Morphologically Rich Indian Languages: A Survey <s> Results Analysis <s> This paper describes our work on building Part-of-Speech (POS) tagger for Bengali. We have use Hidden Markov Model (HMM) and Maximum Entropy (ME) based stochastic taggers. Bengali is a morphologically rich language and our taggers make use of morphological and contextual information of the words. Since only a small labeled training set is available (45,000 words), simple stochastic approach does not yield very good results. In this work, we have studied the effect of using a morphological analyzer to improve the performance of the tagger. We find that the use of morphology helps improve the accuracy of the tagger especially when less amount of tagged corpora are available. <s> BIB001 </s> Part of Speech Taggers for Morphologically Rich Indian Languages: A Survey <s> Results Analysis <s> Part of speech (POS) tagging is the task of labeling each word in a sentence with its appropriate syntactic category called part of speech. POS tagging is a very important preprocessing task for language processing activities. This paper reports about task of POS tagging for Bengali using support vector machine (SVM). The POS tagger has been developed using a tagset of 26 POS tags, defined for the Indian languages. The system makes use of the different contextual information of the words along with the variety of features that are helpful in predicting the various POS classes. The POS tagger has been trained, and tested with the 72,341, and 20 K wordforms, respectively. Experimental results show the effectiveness of the proposed SVM based POS tagger with an accuracy of 86.84%. Results show that the lexicon, named entity recognizer and different word suffixes are effective in handling the unknown word problems and improve the accuracy of the POS tagger significantly. Comparative evaluation results have demonstrated that this SVM based system outperforms the three existing systems based on the hidden markov model (HMM), maximum entropy (ME) and conditional random field (CRF). <s> BIB002
The performance of the POS tagger system in terms of accuracy is evaluated using SVMTeval. Initially, when the size of the lexicon is small the tagger achieves low accuracy. The following table shows the accuracy of POS tagger: BIB001 . Support vector machine based tagger was proposed in the year 2008 BIB002 . Both these tagging are explained in the following sections. The accuracy of any Part of Speech tagger is measured in terms of the accuracy i.e. the percentage of words which are accurately tagged by the tagger. This is defined as belows: For evaluation of the proposed tagger, a corpus having texts from different genres were used. The outcome was manually evaluated to mark the correct and incorrect tag assignments. 25,006 words collected randomly from an 8 million corpus of Punjabi were manually evaluated and are grouped into five genres. Table 4 are based on the present state of our POS tagger having around 40 handwritten disambiguation rules and the tagset having around 630 tags. Total 503 tags of the possible 630 tags were found at least once in the 8 million words corpus of Punjabi Based on the data presented in table 4, the following different accuracy measures were calculated: Accuracy 2 = ------(10) Accuracy 4 = ------ (12) Accuracy achieved by the proposed tagger based on the Table 4 for these accuracy measures are: From the results it is found that the accuracy of 80.29% including unknown words and 88.86% excluding unknown words was achieved by the proposed tagger. The results obtained from the three proposed taggers are summarized in the following Table: 