Part of Speech Taggers for Morphologically Rich Indian Languages: A Survey <s> Support Vector Machine based tagging <s> We present conditional random fields , a framework for building probabilistic models to segment and label sequence data. Conditional random fields offer several advantages over hidden Markov models and stochastic grammars for such tasks, including the ability to relax strong independence assumptions made in those models. Conditional random fields also avoid a fundamental limitation of maximum entropy Markov models (MEMMs) and other discriminative Markov models based on directed graphical models, which can be biased towards states with few successor states. We present iterative parameter estimation algorithms for conditional random fields and compare the performance of the resulting models to HMMs and MEMMs on synthetic and natural-language data. <s> BIB001 </s> Part of Speech Taggers for Morphologically Rich Indian Languages: A Survey <s> Support Vector Machine based tagging <s> We apply Support Vector Machines (SVMs) to identify English base phrases (chunks). SVMs are known to achieve high generalization performance even with input data of high dimensional feature spaces. Furthermore, by the Kernel principle, SVMs can carry out training with smaller computational overhead independent of their dimensionality. We apply weighted voting of 8 SVMs-based systems trained with distinct chunk representations. Experimental results show that our approach achieves higher accuracy than previous approaches. <s> BIB002 </s> Part of Speech Taggers for Morphologically Rich Indian Languages: A Survey <s> Support Vector Machine based tagging <s> In this paper we report our work on building a POS tagger for a morphologically rich language- Hindi. The theme of the research is to vindicate the stand that- if morphology is strong and harnessable, then lack of training corpora is not debilitating. We establish a methodology of POS tagging which the resource disadvantaged (lacking annotated corpora) languages can make use of. The methodology makes use of locally annotated modestly-sized corpora (15,562 words), exhaustive morpohological analysis backed by high-coverage lexicon and a decision tree based learning algorithm (CN2). The evaluation of the system was done with 4-fold cross validation of the corpora in the news domain (www.bbc.co.uk/hindi). The current accuracy of POS tagging is 93.45% and can be further improved. <s> BIB003 </s> Part of Speech Taggers for Morphologically Rich Indian Languages: A Survey <s> Support Vector Machine based tagging <s> Lexicon development and Part of Speech (POS) tagging are very important for almost all Natural Language Processing(NLP) application areas. The rapid development of these resources and tools using machine learning techniques for less computerized languages requires appropriately tagged corpus. A tagged Bengali news corpus has been developed from the web archive of a widely read Bengali newspaper. This corpus is then used for lexicon development and POS tagging. Tagged Bengali News Corpus Development Newspaper is a huge source of readily available documents. A tagged corpus has been developed from the web archive of a very well known and widely read Bengali News Paper. The development of the tagged Bengali news corpus includes language resource acquisition using a web crawler, language resource creation which includes HTML file cleaning and code conversion, as well as language resource annotation that involves defining a tag set and subsequent tagging of the news corpus. Code conversion is necessary to convert the dynamic fonts used in the newspaper into the standard Indian Standard Code for Information Interchange (ISCII) form, which can be processed for various text processing tasks. At present, the corpus contains 34 million wordforms and it is available in both ISCII and UTF-8 formats. A news corpus, whether in Bengali or in any other language has different parts like title, date, reporter, location, body etc. To identify these parts in a news corpus, the following tagset has been defined: header (Header of the news document), title (Headline of the news document), t1 (1st headline of the title), t2 (2nd headline of the title), date (Date of the news document), bd (Bengali date), day (Day), ed (English date), reporter (Reporter-name), agency (Agency providing news), location (the news location), body (Body of the news document), p (Paragraph), table (information in tabular form), tc (Table Column), and tr (Table row). Lexicon Development from the Corpus The tagged Bengali news corpus has been used to develop a Bengali lexicon that is a list of Bengali root words derived Copyright c © 2007, American Association for Artificial Intelligence (www.aaai.org). All rights reserved. from the corpus along with its basic part of speech information. An unsupervised learning method has been used for the lexicon development. No extensive knowledge about the language is required except the knowledge of the different inflections that can appear with the different words in Bengali. In Bengali, there are five different parts of speech namely: noun, pronoun, verb, adjective and indeclinable (postpositions, conjunctions, and interjections). Noun, verb and adjective are the open class of part of speech for Bengali. Initially, all the words (inflected and uninflected) are extracted from the tagged corpus and added to the database. A list of inflections that may appear with the noun words is kept and at present the list has 27 entries. In Bengali, the verbs can be organized into 20 different groups according to their spelling patterns and the different inflections that can be attached to them. Original word-form of a verb word often changes when any suffix is attached to the verb. At present, there are 214 different entries in the verb inflection list. Noun and verb words are tagged by looking at their inflections. Some inflections may be common to both nouns and verbs. In these cases, more than one root words will be generated for a wordform. The POS ambiguity is resolved by checking the number of occurrences of these possible root words along with the POS tags as derived from the other wordforms. Pronoun and indeclinable are basically closed class of part of speech in Bengali and these are added to the lexicon manually. It has been observed that adjectives in Bengali generally occur in four different forms based on the suffixes attached. The first type of adjectives can form comparative and superlative degree by attaching the suffixes -tara and -tamo to the adjective word. These adjective stems are stored in the lexicon with adjective POS. The second set of suffixes (e.g., -gato, -karo etc.) identifies the POS of the wordform as adjective if only there is a noun entry of the desuffixed word in the lexicon. The third group of suffixes (e.g., -janok, -sulav etc.) identifies the POS of the wordform as adjective and the desuffixed word is included in the lexicon with noun POS. The last set of suffixes identifies the POS of the wordform as adjective. Hidden Markov Model Based POS Tagging A POS tagger based on the modified Hidden Markov Model (HMM) has been developed using a portion of the tagged Bengali news corpus. We have used a tagset having 27 dif- <s> BIB004 </s> Part of Speech Taggers for Morphologically Rich Indian Languages: A Survey <s> Support Vector Machine based tagging <s> Part of speech (POS) tagging is the task of labeling each word in a sentence with its appropriate syntactic category called part of speech. POS tagging is a very important preprocessing task for language processing activities. This paper reports about task of POS tagging for Bengali using support vector machine (SVM). The POS tagger has been developed using a tagset of 26 POS tags, defined for the Indian languages. The system makes use of the different contextual information of the words along with the variety of features that are helpful in predicting the various POS classes. The POS tagger has been trained, and tested with the 72,341, and 20 K wordforms, respectively. Experimental results show the effectiveness of the proposed SVM based POS tagger with an accuracy of 86.84%. Results show that the lexicon, named entity recognizer and different word suffixes are effective in handling the unknown word problems and improve the accuracy of the POS tagger significantly. Comparative evaluation results have demonstrated that this SVM based system outperforms the three existing systems based on the hidden markov model (HMM), maximum entropy (ME) and conditional random field (CRF). <s> BIB005 </s> Part of Speech Taggers for Morphologically Rich Indian Languages: A Survey <s> Support Vector Machine based tagging <s> This paper reports about the development of a Named Entity Recognition (NER) system for Bengali using the statistical Conditional Random Fields (CRFs). The system makes use of the different contextual information of the words along with the variety of features that are helpful in predicting the various named entity (NE) classes. A portion of the partially NE tagged Bengali news corpus, developed from the archive of a leading Bengali newspaper available in the web, has been used to develop the system. The training set consists of 150K words and has been manually annotated with a NE tagset of seventeen tags. Experimental results of the 10-fold cross validation test show the effectiveness of the proposed CRF based NER system with an overall average Recall, Precision and F-Score values of 93.8%, 87.8% and 90.7%, respectively. <s> BIB006
Support vector machine is a new generation learning system based on recent advances in statistical learning theory. It gives excellent performance in the applications like text categorization, handwritten character recognition, natural language processing, etc. It has many advantages over conventional statistical learning algorithms. Simple HMMs do not work well when small amount of labeled data are used to estimate the model parameters. Incorporating diverse features in an HMM based tagger is difficult and complicates the smoothing typically used in such taggers. In contrast, a ME  or a CRF BIB001 or a SVM BIB002 can deal with the diverse and overlapping features more efficiently. A POS tagger has been proposed in BIB003 that has shown an accuracy of 93.45% for Hindi with a tagset of 23 POS tags. SVMs have advantages over conventional statistical learning algorithms, such as Decision Tree, HMMs, ME from the following two aspects BIB005 : 1. SVMs have high generalization performance independent of dimension of feature vectors. Other algorithms require careful feature selection, which is usually optimized heuristically, to avoid over fitting. 2. SVMs can carry out their learning with all combinations of given features without increasing computational complexity by introducing the Kernel function. Conventional algorithms cannot handle these combinations efficiently. In this work, SVM based approach was used for the task of POS tagging. To improve the accuracy of the POS tagger, a lexicon BIB004 and a CRF-based NER system BIB006 have been used, along with the variety of contextual and word level features. The SVM based POS tagger has been developed using a corpus 72,341 word forms tagged with the 26 POS tags, defined for the Indian languages. Out of 72,341 word forms, around 15K word forms have been selected as the development set and the rest, i.e., 57,341 word forms have been used as the training set of the SVM based tagger in order to find out the best set of features for POS tagging in Bengali. The baseline model has been defined as the one where the POS tag probabilities depend only on the current word: In this model, each word in the test data will be assigned the POS tag, which occurred most frequently for that word in the training data. Features for part of speech (POS) tagging in Bengali have been identified based on the different possible combination of available word and tag context. The features also include prefix and suffix for all words. The term prefix/suffix is a sequence of first/last few characters of a word, which may not be a linguistically meaningful prefix/suffix. The use of prefix/suffix information works well for highly inflected languages like the Indian languages BIB005 . Numbers of experiments were conducted taking the different combinations from the set "F" to identify the best-suited set of features for the POS tagging task. From the analysis, the following combinations were found to give the best result: F={ w i-2 w i-1 w i w i+1 w i+2 , |prefix|<=3, |suffix|<=3, Dynamic POS tags of the previous two words, NE tags of the current and the previous words, Lexicon feature, Symbol feature, Digit feature, Length feature, Inflection lists}.