Multi-Layer Perceptron (MLP), Naive Bayes Classifier (NBC), Support Vector Machine (SVM), Gradient Boosting Classifier (GBC), Stochastic Gradient Descent (SGD), K Nearest Neighbour (K-NN) and Random Forest (RF)
EXTRACTIVE
Multi-Layer Perceptron, Naive Bayes Classifier, Support Vector Machine, Gradient Boosting Classifier, Stochastic Gradient Descent, K Nearest Neighbour, Random Forest
EXTRACTIVE
In this research, we briefly discuss the steps of QA system and compare the performance of seven machine learning based classifiers (Multi-Layer Perceptron (MLP), Naive Bayes Classifier (NBC), Support Vector Machine (SVM), Gradient Boosting Classifier (GBC), Stochastic Gradient Descent (SGD), K Nearest Neighbour (K-NN) and Random Forest (RF)) in classifying Bengali questions to classes based on their anticipated answers. Bengali questions have flexible inquiring ways, so there are many difficulties associated with Bengali QC BIBREF0. As there is no rich corpus of questions in Bengali Language available, collecting questions is an additional challenge. Different difficulties in building a QA System are mentioned in the literature BIBREF2 BIBREF3. The first work on a machine learning based approach towards Bengali question classification is presented in BIBREF0 that employ the Stochastic Gradient Descent (SGD).
In this research, we briefly discuss the steps of QA system and compare the performance of seven machine learning based classifiers (Multi-Layer Perceptron (MLP), Naive Bayes Classifier (NBC), Support Vector Machine (SVM), Gradient Boosting Classifier (GBC), Stochastic Gradient Descent (SGD), K Nearest Neighbour (K-NN) and Random Forest (RF)) in classifying Bengali questions to classes based on their anticipated answers. Bengali questions have flexible inquiring ways, so there are many difficulties associated with Bengali QC BIBREF0. As there is no rich corpus of questions in Bengali Language available, collecting questions is an additional challenge. Different difficulties in building a QA System are mentioned in the literature BIBREF2 BIBREF3. The first work on a machine learning based approach towards Bengali question classification is presented in BIBREF0 that employ the Stochastic Gradient Descent (SGD).
1911.03059
Introduction </s> Question Ġclassification Ġ( Q C ) Ġdeals Ġwith Ġquestion Ġanalysis Ġand Ġquestion Ġlabeling Ġbased Ġon Ġthe Ġexpected Ġanswer Ġtype . ĠThe Ġgoal Ġof ĠQC Ġis Ġto Ġassign Ġclasses Ġaccurately Ġto Ġthe Ġquestions Ġbased Ġon Ġexpected Ġanswer . ĠIn Ġmodern Ġsystem , Ġthere Ġare Ġtwo Ġtypes Ġof Ġquestions ĠB IB REF 0 . ĠOne Ġis ĠFact oid Ġquestion Ġwhich Ġis Ġabout Ġproviding Ġconcise Ġfacts Ġand Ġanother Ġone Ġis ĠComplex Ġquestion Ġthat Ġhas Ġa Ġpresupp osition Ġwhich Ġis Ġcomplex . ĠQuestion ĠAn sw ering Ġ( Q A ) ĠSystem Ġis Ġan Ġintegral Ġpart Ġof Ġour Ġdaily Ġlife Ġbecause Ġof Ġthe Ġhigh Ġamount Ġof Ġusage Ġof ĠInternet Ġfor Ġinformation Ġacquisition . ĠIn Ġrecent Ġyears , Ġmost Ġof Ġthe Ġresearch Ġworks Ġrelated Ġto ĠQ A Ġare Ġbased Ġon ĠEnglish Ġlanguage Ġsuch Ġas ĠIBM ĠWatson , ĠWolf ram ĠAlpha . ĠBeng ali Ġspeakers Ġoften Ġfall Ġin Ġdifficulty Ġwhile Ġcommunicating Ġin ĠEnglish ĠB IB REF 1 . </s> In Ġthis Ġresearch , Ġwe Ġbriefly Ġdiscuss Ġthe Ġsteps Ġof ĠQ A Ġsystem Ġand Ġcompare Ġthe Ġperformance Ġof Ġseven Ġmachine Ġlearning Ġbased Ġclass ifiers Ġ( Multi - Layer ĠPer cept ron Ġ( ML P ), ĠNa ive ĠBay es ĠClass ifier Ġ( NBC ), ĠSupport ĠVector ĠMachine Ġ( S VM ), ĠGrad ient ĠBoost ing ĠClass ifier Ġ( G BC ), ĠSt och astic ĠGrad ient ĠDes cent Ġ( SG D ), ĠK ĠNe arest ĠNeigh bour Ġ( K - NN ) Ġand ĠRandom ĠForest Ġ( RF )) Ġin Ġclass ifying ĠBeng ali Ġquestions Ġto Ġclasses Ġbased Ġon Ġtheir Ġanticipated Ġanswers . ĠBeng ali Ġquestions Ġhave Ġflexible Ġinqu iring Ġways , Ġso Ġthere Ġare Ġmany Ġdifficulties Ġassociated Ġwith ĠBeng ali ĠQC ĠB IB REF 0 . ĠAs Ġthere Ġis Ġno Ġrich Ġcorpus Ġof Ġquestions Ġin ĠBeng ali ĠLanguage Ġavailable , Ġcollecting Ġquestions Ġis Ġan Ġadditional Ġchallenge . ĠDifferent Ġdifficulties Ġin Ġbuilding Ġa ĠQ A ĠSystem Ġare Ġmentioned Ġin Ġthe Ġliterature ĠB IB REF 2 ĠB IB REF 3 . ĠThe Ġfirst Ġwork Ġon Ġa Ġmachine Ġlearning Ġbased Ġapproach Ġtowards ĠBeng ali Ġquestion Ġclassification Ġis Ġpresented Ġin ĠB IB REF 0 Ġthat Ġemploy Ġthe ĠSt och astic ĠGrad ient ĠDes cent Ġ( SG D ). </s> Related ĠWorks Ġ: :: ĠPopular ĠQuestion - An sw ering ĠSystems </s> Over Ġthe Ġyears , Ġa Ġhandful Ġof ĠQ A Ġsystems Ġhave Ġgained Ġpopularity Ġaround Ġthe Ġworld . ĠOne Ġof Ġthe Ġoldest ĠQ A Ġsystem Ġis ĠBASE BALL Ġ( created Ġon Ġ1961 ) ĠB IB REF 4 Ġwhich Ġanswers Ġquestion Ġrelated Ġto Ġbaseball Ġleague Ġin ĠAmerica Ġfor Ġa Ġparticular Ġseason . ĠL UN AR ĠB IB REF 5 Ġsystem Ġanswers Ġquestions Ġabout Ġsoil Ġsamples Ġtaken Ġfrom ĠApollo Ġlunar Ġexploration . ĠSome Ġof Ġthe Ġmost Ġpopular ĠQ A ĠSystems Ġare ĠIBM ĠWatson , ĠApple ĠSiri Ġand ĠWolf ram ĠAlpha . ĠExamples Ġof Ġsome ĠQ A Ġsystems Ġbased Ġon Ġdifferent Ġlanguages Ġare : ĠZhang ĠYu ĠChinese Ġquestion Ġclassification ĠB IB REF 6 Ġbased Ġon ĠIncre mental ĠModified ĠBay es , ĠArabic ĠQ A Ġsystem Ġ( A Q AS ) ĠB IB REF 7 Ġby ĠF . ĠA . ĠMohammed , ĠK . ĠN asser , Ġ& ĠH . ĠM . ĠHar b Ġand ĠSynt actic Ġopen Ġdomain ĠArabic ĠQ A Ġsystem Ġfor Ġfact oid Ġquestions ĠB IB REF 8 Ġby ĠFare ed Ġet Ġal . ĠQ A Ġsystems Ġhave Ġbeen Ġbuilt Ġon Ġdifferent Ġanalysis Ġmethods Ġsuch Ġas Ġmorph ological Ġanalysis ĠB IB REF 9 , Ġsynt act ical Ġanalysis ĠB IB REF 10 , Ġsemantic Ġanalysis ĠB IB REF 11 Ġand Ġexpected Ġanswer ĠType Ġanalysis ĠB IB REF 12 . </s> Related ĠWorks Ġ: :: ĠResearch ĠWorks ĠRelated Ġto ĠQuestion ĠClass ifications </s> Res ear ches Ġon Ġquestion Ġclassification , Ġquestion Ġtax onom ies Ġand ĠQ A Ġsystem Ġhave Ġbeen Ġundertaken Ġin Ġrecent Ġyears . ĠThere Ġare Ġtwo Ġtypes Ġof Ġapproaches Ġfor Ġquestion Ġclassification Ġaccording Ġto ĠBan er jee Ġet Ġal Ġin ĠB IB REF 13 Ġ- Ġby Ġrules Ġand Ġby Ġmachine Ġlearning Ġapproach . ĠRule Ġbased Ġapproaches Ġuse Ġsome Ġhard Ġcoded Ġgrammar Ġrules Ġto Ġmap Ġthe Ġquestion Ġto Ġan Ġappropriate Ġanswer Ġtype ĠB IB REF 14 ĠB IB REF 15 . ĠMachine ĠLearning Ġbased Ġapproaches Ġhave Ġbeen Ġused Ġby ĠZhang Ġet Ġal Ġand ĠMd . ĠAmin ul ĠIslam Ġet Ġal Ġin ĠB IB REF 16 Ġand ĠB IB REF 0 . ĠMany Ġclass ifiers Ġhave Ġbeen Ġused Ġin Ġmachine Ġlearning Ġfor ĠQC Ġsuch Ġas ĠSupport ĠVector ĠMachine Ġ( S VM ) ĠB IB REF 16 ĠB IB REF 17 , ĠSupport ĠVector ĠMachines Ġand ĠMaximum ĠEnt ropy ĠModel ĠB IB REF 18 , ĠNa ive ĠBay es Ġ( NB ), ĠKernel ĠNa ive ĠBay es Ġ( KN B ), ĠDecision ĠTree Ġ( DT ) Ġand ĠRule ĠInd uction Ġ( RI ) ĠB IB REF 13 . ĠIn ĠB IB REF 0 , Ġthey Ġclaimed Ġto Ġachieve Ġaverage Ġprecision Ġof Ġ0 . 9 55 62 Ġfor Ġcoarse Ġclass Ġand Ġ0 . 87 646 Ġfor Ġfiner Ġclass Ġusing ĠSt och astic ĠGrad ient ĠDes cent Ġ( SG D ). </s> Related ĠWorks Ġ: :: ĠResearch ĠWorks Ġin ĠBeng ali ĠLanguage </s> A ĠBeng ali ĠQC ĠSystem Ġwas Ġbuilt Ġby ĠSom n ath ĠBan er jee Ġand ĠS iv aji ĠBand y op ad hy ay ĠB IB REF 13 ĠB IB REF 19 ĠB IB REF 20 . ĠThey Ġproposed Ġa Ġtwo - layer Ġtax onomy Ġclassification Ġwith Ġ9 Ġcoarse - gr ained Ġclasses Ġand Ġ69 Ġfine - gr ained Ġclasses . ĠThere Ġare Ġother Ġresearch Ġworks ĠB IB REF 0 ĠB IB REF 21 Ġin ĠBeng ali ĠLanguage . ĠA Ġsurvey Ġwas Ġperformed Ġon Ġtext ĠQ A Ġtechniques ĠB IB REF 22 Ġwhere Ġthere Ġwas Ġan Ġanalysis Ġconducted Ġin ĠBeng ali ĠLanguage . ĠSy ed ĠMe hed i ĠHasan ĠN iro b Ġet Ġal Ġachieved Ġ88 . 62 % Ġaccuracy Ġby Ġusing Ġ380 Ġtop Ġfrequent Ġwords Ġas Ġthe Ġfeature Ġin Ġtheir Ġwork ĠB IB REF 17 . </s> Question ĠAn sw ering Ġ( Q A ) ĠSystem </s> Q A Ġsystem Ġresides Ġwithin Ġthe Ġscope Ġof ĠComputer ĠScience . ĠIt Ġdeals Ġwith Ġinformation Ġretrieval Ġand Ġnatural Ġlanguage Ġprocessing . ĠIts Ġgoal Ġis Ġto Ġautomatically Ġanswer Ġquestions Ġasked Ġby Ġhumans Ġin Ġnatural Ġlanguage . ĠIR - based ĠQ A , ĠKnowledge Ġbased Ġapproaches Ġand ĠHybrid Ġapproaches Ġare Ġthe ĠQ A Ġsystem Ġtypes . ĠT REC , ĠIBM - W atson , ĠGoogle Ġare Ġexamples Ġof ĠIR - based ĠQ A Ġsystems . ĠKnowledge Ġbased ĠQ A Ġsystems Ġare ĠApple ĠSiri , ĠWolf ram ĠAlpha . ĠExamples Ġof ĠHybrid Ġapproach Ġsystems Ġare ĠIBM ĠWatson Ġand ĠTrue ĠKnowledge ĠEv i . </s> Figure ĠFIG REF 4 Ġprovides Ġan Ġoverview Ġof ĠQ A ĠSystem . ĠThe Ġfirst Ġstep Ġof ĠQ A ĠSystem Ġis ĠQuestion ĠAnalysis . ĠQuestion Ġanalysis Ġhas Ġtwo Ġparts Ġ- Ġquestion Ġclassification Ġand Ġanother Ġquestion Ġformulation . ĠIn Ġquestion Ġclassification Ġstep , Ġthe Ġquestion Ġis Ġclassified Ġusing Ġdifferent Ġclass ifier Ġalgorithms . ĠIn Ġquestion Ġformulation , Ġthe Ġquestion Ġis Ġanalyzed Ġand Ġthe Ġsystem Ġcreates Ġa Ġproper ĠIR Ġquestion Ġby Ġdetecting Ġthe Ġentity Ġtype Ġof Ġthe Ġquestion Ġto Ġprovide Ġa Ġsimple Ġanswer . </s> The Ġnext Ġstep Ġis Ġdocuments Ġretrieval Ġand Ġanalysis . ĠIn Ġthis Ġstep , Ġthe Ġsystem Ġmatches Ġthe Ġquery Ġagainst Ġthe Ġsources Ġof Ġanswers Ġwhere Ġthe Ġsource Ġcan Ġbe Ġdocuments Ġor ĠWeb . ĠIn Ġthe Ġanswer Ġextraction Ġstep , Ġthe Ġsystem Ġextracts Ġthe Ġanswers Ġfrom Ġthe Ġdocuments Ġof Ġthe Ġsources Ġcollected Ġin Ġdocuments Ġretrieval Ġand Ġanalysis Ġphase . ĠThe Ġextracted Ġanswers Ġare Ġfiltered Ġand Ġevaluated Ġin Ġanswer Ġevaluation Ġphase Ġas Ġthere Ġcan Ġbe Ġmultiple Ġpossible Ġanswers Ġfor Ġa Ġquery . ĠIn Ġthe Ġfinal Ġstep , Ġan Ġanswer Ġof Ġthe Ġquestion Ġis Ġreturned . </s> Prop osed ĠMethod ology </s> We Ġuse Ġdifferent Ġtypes Ġof Ġclass ifiers Ġfor ĠQ A Ġtype Ġclassification . ĠWe Ġseparate Ġour Ġmethodology Ġinto Ġtwo Ġsections Ġsimilar Ġto ĠB IB REF 0 Ġ- Ġone Ġis Ġtraining Ġsection Ġand Ġanother Ġis Ġvalidation Ġsection Ġshown Ġin ĠFigure ĠFIG REF 5 . </s> We Ġuse Ġ10 Ġfold Ġcross Ġvalidation Ġwhere Ġwe Ġhave Ġ31 50 Ġand Ġ350 Ġquestions Ġin Ġour Ġtraining Ġset Ġand Ġvalidation Ġset Ġrespectively . ĠDuring Ġtraining , Ġafter Ġselecting Ġthe Ġpossible Ġclass Ġlabels , Ġthe Ġsystem Ġextracts Ġthe Ġfeatures Ġof Ġthe Ġquestions Ġand Ġcreates Ġa Ġmodel Ġby Ġpassing Ġthrough Ġa Ġclass ifier Ġalgorithm Ġwith Ġthe Ġextracted Ġfeatures Ġand Ġclass Ġlabels . ĠDuring Ġvalidation , Ġthe Ġsystem Ġextracts Ġthe Ġfeatures Ġof Ġthe Ġquestion Ġand Ġpasses Ġit Ġinto Ġthe Ġmodel Ġcreated Ġduring Ġtraining Ġand Ġpredicts Ġthe Ġanswer Ġtype . </s> Question ĠCollection Ġand ĠCategories </s> Though ĠBeng ali Ġis Ġthe Ġseventh Ġmost Ġspoken Ġlanguage Ġin Ġterms Ġof Ġnumber Ġof Ġnative Ġspeakers ĠB IB REF 23 , Ġthere Ġis Ġno Ġstandard Ġcorpus Ġof Ġquestions Ġavailable ĠB IB REF 0 . ĠWe Ġhave Ġcollected Ġtotal Ġ35 00 Ġquestions Ġfrom Ġthe ĠInternet Ġand Ġother Ġsources Ġsuch Ġas Ġbooks Ġof Ġgeneral Ġknowledge Ġquestions , Ġhistory Ġetc . ĠThe Ġcorpus Ġcontains Ġthe Ġquestions Ġand Ġthe Ġclasses Ġeach Ġquestion Ġbelongs Ġto . </s> The Ġset Ġof Ġquestion Ġcategories Ġis Ġknown Ġas Ġquestion Ġtax onomy ĠB IB REF 0 . ĠWe Ġhave Ġused Ġtwo Ġlayer Ġtax onomy Ġwhich Ġwas Ġproposed Ġby ĠXin ĠLi , ĠDan ĠRoth ĠB IB REF 24 . ĠThis Ġtwo Ġlayer Ġtax onomy Ġis Ġmade Ġup Ġof Ġtwo Ġclasses Ġwhich Ġare ĠCo arse ĠClass Ġand ĠFin er ĠClass . ĠThere Ġare Ġsix Ġcoarse Ġclasses Ġsuch Ġas ĠN umeric , ĠLocation , ĠEntity , ĠDescription , ĠHuman Ġand ĠAb bre viation Ġand Ġfifty Ġfiner Ġclasses Ġsuch Ġas Ġcity , Ġstate , Ġmountain , Ġdistance , Ġcount , Ġdefinition , Ġgroup , Ġexpression , Ġsubstance , Ġcreative , Ġvehicle Ġetc Ġas Ġshown Ġin Ġthe ĠTable ĠI ĠB IB REF 0 . ĠA Ġcoarse - gr ained Ġdescription Ġof Ġa Ġsystem Ġdenotes Ġlarge Ġcomponents Ġwhile Ġa Ġfine - gr ained Ġdescription Ġdenotes Ġsmaller Ġsub - comp onents Ġof Ġwhich Ġthe Ġlarger Ġones Ġare Ġcomposed Ġof . </s> Im plementation Ġof Ġthe ĠSystem Ġ: :: ĠFeature ĠExt raction </s> Question Ġword Ġanswer Ġphrases , Ġparts Ġof Ġspeech Ġtags , Ġparse Ġfeature , Ġnamed Ġentity Ġand Ġsem antically Ġrelated Ġwords Ġare Ġdifferent Ġfeatures Ġfrom Ġanswer Ġtype Ġdetection ĠB IB REF 18 . ĠWe Ġuse Ġquestion Ġword Ġand Ġphrases Ġas Ġfeatures Ġfor Ġanswer Ġtype Ġdetection . ĠWe Ġconsider Ġthe Ġfollowing Ġfeatures : </s> Im plementation Ġof Ġthe ĠSystem Ġ: :: ĠFeature ĠExt raction Ġ: :: ĠTF - ID F </s> Term ĠFrequency Ġ- ĠIn verse ĠDocument ĠFrequency Ġ( TF - ID F ) Ġis Ġa Ġpopular Ġmethod Ġused Ġto Ġidentify Ġthe Ġimportance Ġof Ġa Ġword Ġin Ġa Ġparticular Ġdocument . ĠTF - ID F Ġtransforms Ġtext Ġinto Ġmeaningful Ġnumeric Ġrepresentation . ĠThis Ġtechnique Ġis Ġwidely Ġused Ġto Ġextract Ġfeatures Ġfor ĠNatural ĠLanguage ĠProcessing Ġ( N LP ) Ġapplications ĠB IB REF 25 ĠB IB REF 26 . </s> Im plementation Ġof Ġthe ĠSystem Ġ: :: ĠFeature ĠExt raction Ġ: :: ĠWord Ġlevel ĠN - G rams </s> N - gram s Ġis Ġn - back Ġto Ġback Ġwords Ġin Ġa Ġtext . ĠQu eries Ġof Ġa Ġsame Ġclass Ġusually Ġshare Ġword Ġn - gram s ĠB IB REF 0 . ĠIn Ġthis Ġsystem , Ġwe Ġchoose Ġbi - gram Ġfor Ġextracting Ġfeatures . </s> Im plementation Ġof Ġthe ĠSystem Ġ: :: ĠFeature ĠExt raction Ġ: :: ĠStop ĠWords </s> We Ġuse Ġtwo Ġsetups Ġ( as Ġdone Ġin ĠB IB REF 0 ) Ġfor Ġour Ġsystem . ĠIn Ġthe Ġfirst Ġsetup , Ġwe Ġeliminate Ġthe Ġstop Ġwords Ġfrom Ġthe Ġtext Ġusing Ġanother Ġdataset Ġcontaining Ġonly Ġstop Ġwords . ĠAt Ġsecond Ġstep , Ġwe Ġwork Ġwithout Ġeliminating Ġthe Ġstop Ġwords Ġfrom Ġthe Ġtext Ġwhich Ġgives Ġbetter Ġresult Ġthan Ġthe Ġfirst Ġsetup . </s> Im plementation Ġof Ġthe ĠSystem Ġ: :: ĠClassification ĠAl gorith ms Ġ: :: ĠMulti ĠLayer ĠPer cept ron Ġ( ML P ) </s> ML P Ġcontains Ġthree Ġlayers Ġ- Ġan Ġinput Ġlayer , Ġan Ġoutput Ġlayer Ġand Ġsome Ġhidden Ġlayers . ĠInput Ġlayer Ġreceives Ġthe Ġsignal , Ġthe Ġoutput Ġlayer Ġgives Ġa Ġdecision Ġor Ġprediction Ġabout Ġthe Ġinput Ġand Ġthe Ġcomputation Ġof Ġthe ĠML P Ġis Ġconducted Ġin Ġthe Ġhidden Ġlayers . ĠIn Ġour Ġsystem , Ġwe Ġuse Ġ100 Ġlayers . ĠFor Ġweight Ġoptimization , Ġwe Ġuse ĠLimited - memory ĠBro yden âĢĵ F letcher âĢĵ Gold far b âĢĵ Sh ann o Ġ( L BF GS ) Ġoptimization Ġalgorithm . </s> Im plementation Ġof Ġthe ĠSystem Ġ: :: ĠClassification ĠAl gorith ms Ġ: :: ĠSupport ĠVector ĠMachine Ġ( S VM ) </s> S VM Ġgives Ġan Ġoptimal Ġhyper - plane Ġand Ġit Ġmaxim izes Ġthe Ġmargin Ġbetween Ġclasses . ĠWe Ġuse ĠRad ial ĠBas is ĠFunction Ġ( RB F ) Ġkernel Ġin Ġour Ġsystem Ġto Ġmake Ġdecision Ġboundary Ġcurve - shaped . ĠFor Ġdecision Ġfunction Ġshape , Ġwe Ġuse Ġthe Ġoriginal Ġone - vs - one Ġ( ovo ) Ġdecision Ġfunction . </s> Im plementation Ġof Ġthe ĠSystem Ġ: :: ĠClassification ĠAl gorith ms Ġ: :: ĠNa ive ĠBay esian ĠClass ifier Ġ( NBC ) </s> NBC Ġis Ġbased Ġon ĠBay es ' ĠThe orem Ġwhich Ġgives Ġprobability Ġof Ġan Ġevent Ġoccurrence Ġbased Ġon Ġsome Ġconditions Ġrelated Ġto Ġthat Ġevent . ĠWe Ġuse ĠMult in omial ĠNa ive ĠBay es ĠClass ifier Ġwith Ġsmoot hing Ġparameter Ġequals Ġto Ġ0 . 1 . ĠA Ġzero Ġprobability Ġcancel s Ġthe Ġeffects Ġof Ġall Ġthe Ġother Ġprobabilities . </s> Im plementation Ġof Ġthe ĠSystem Ġ: :: ĠClassification ĠAl gorith ms Ġ: :: ĠSt och astic ĠGrad ient ĠDes cent Ġ( SG D ) </s> St och astic Ġgradient Ġdescent Ġoptim izes Ġan Ġobjective Ġfunction Ġwith Ġsuitable Ġsmooth ness Ġproperties ĠB IB REF 27 . ĠIt Ġselects Ġfew Ġexamples Ġrandomly Ġinstead Ġof Ġwhole Ġdata Ġfor Ġeach Ġiteration . ĠWe Ġuse Ġ' L 2 ' Ġregular ization Ġfor Ġreduction Ġof Ġover fitting . </s> Im plementation Ġof Ġthe ĠSystem Ġ: :: ĠClassification ĠAl gorith ms Ġ: :: ĠGrad ient ĠBoost ing ĠClass ifier Ġ( G BC ) </s> Grad ient ĠBoost ing ĠClass ifier Ġproduces Ġa Ġprediction Ġmodel Ġconsisting Ġof Ġweak Ġprediction Ġmodels . ĠGrad ient Ġboosting Ġuses Ġdecision Ġtrees . ĠWe Ġuse Ġ100 Ġboosting Ġstages Ġin Ġthis Ġwork . </s> Im plementation Ġof Ġthe ĠSystem Ġ: :: ĠClassification ĠAl gorith ms Ġ: :: ĠK ĠNe arest ĠNeigh bour Ġ( K - NN ) </s> K - NN Ġis Ġa Ġsupervised Ġclassification Ġand Ġregression Ġalgorithm . ĠIt Ġuses Ġthe Ġneighbours Ġof Ġthe Ġgiven Ġsample Ġto Ġidentify Ġits Ġclass . ĠK Ġdetermines Ġthe Ġnumber Ġof Ġneighbours Ġneeded Ġto Ġbe Ġconsidered . ĠWe Ġset Ġthe Ġvalue Ġof ĠK Ġequals Ġto Ġ13 Ġin Ġthis Ġwork . </s> Im plementation Ġof Ġthe ĠSystem Ġ: :: ĠClassification ĠAl gorith ms Ġ: :: ĠRandom ĠForest Ġ( RF ) </s> RF Ġis Ġan Ġensemble Ġlearning Ġtechnique . ĠIt Ġconstructs Ġlarge Ġnumber Ġof Ġdecision Ġtrees Ġduring Ġtraining Ġand Ġthen Ġpredicts Ġthe Ġmajority Ġclass . ĠWe Ġuse Ġ500 Ġdecision Ġtrees Ġin Ġthe Ġforest Ġand Ġ" ent ropy " Ġfunction Ġto Ġmeasure Ġthe Ġquality Ġof Ġa Ġsplit . </s> Im plementation Ġof Ġthe ĠSystem Ġ: :: ĠResults Ġand ĠDiscussion </s> Table ĠII Ġshows Ġthe Ġaccuracy Ġand ĠF 1 Ġscore Ġfor Ġdifferent Ġclass ifiers Ġwith Ġand Ġwithout Ġeliminating Ġstop Ġwords Ġwhile Ġextracting Ġfeatures . ĠFigure ĠFIG REF 21 Ġshows Ġthe Ġaverage Ġresults Ġof Ġdifferent Ġclass ifiers Ġin Ġa Ġbar Ġchart Ġwith Ġand Ġwithout Ġeliminating Ġstop Ġwords Ġfrom Ġthe Ġquestions . </s> Overall , ĠSG D Ġhas Ġshown Ġthe Ġbest Ġperformance Ġon Ġour Ġdataset Ġas Ġit Ġintroduces Ġnon - linear ity Ġand Ġuses Ġback - prop ag ation Ġfor Ġupdating Ġparameter Ġweights Ġusing Ġloss Ġfunction Ġcalculated Ġon Ġtraining Ġset Ġinto Ġclassification . ĠK - NN Ġhas Ġshown Ġthe Ġweakest Ġperformance Ġoverall , Ġas Ġthis Ġalgorithm Ġhas Ġa Ġbad Ġreputation Ġof Ġnot Ġworking Ġwell Ġin Ġhigh Ġdimensional Ġdata ĠB IB REF 28 . ĠML P Ġand ĠS VM Ġhave Ġshown Ġsimilar Ġperformance . ĠML P Ġtakes Ġadvantage Ġof Ġmultiple Ġhidden Ġlayers Ġin Ġorder Ġto Ġtake Ġnon - lin early Ġsepar able Ġsamples Ġin Ġa Ġlin early Ġsepar able Ġcondition . ĠS VM Ġaccompl ishes Ġthis Ġsame Ġfeat Ġby Ġtaking Ġthe Ġsamples Ġto Ġa Ġhigher Ġdimensional Ġhyper plane Ġwhere Ġthe Ġsamples Ġare Ġlin early Ġsepar able . ĠGrad ient ĠBoost ing ĠClass ifier Ġ( G BC ) Ġand ĠRandom ĠForest Ġ( RF ) Ġboth Ġutilize Ġa Ġset Ġof Ġdecision Ġtrees Ġand Ġachieve Ġsimilar Ġresults Ġ( RF Ġperforms Ġslightly Ġwithout Ġeliminating Ġstop Ġwords ). ĠNa ive ĠBay esian ĠClass ifier Ġ( NBC ) Ġshows Ġperformance Ġon Ġper Ġwith ĠG BC Ġand ĠRF Ġalgorithms . ĠThe Ġoverall Ġbetter Ġperformance Ġof Ġall Ġthe Ġalgorithms Ġwhen Ġprovided Ġwith Ġstop Ġwords Ġshow Ġthe Ġimportance Ġof Ġstop Ġwords Ġin ĠBeng ali ĠQ A Ġclassification . </s> Figure ĠFIG REF 22 Ġshows Ġthe Ġpredictions Ġof Ġsome Ġparticular Ġquestions Ġby Ġeach Ġof Ġthe Ġclass ifiers . ĠThe Ġinput Ġis Ġa Ġfull Ġquestion Ġand Ġthe Ġoutput Ġis Ġthe Ġclass Ġof Ġthe Ġquestion . </s> Im plementation Ġof Ġthe ĠSystem Ġ: :: ĠComput ational ĠComplex ity </s> In ĠTable ĠT AB REF 24 , Ġn Ġ= ĠNo . Ġof Ġtraining Ġsample , Ġp Ġ= ĠNo . Ġof Ġfeatures , Ġn t rees Ġ= ĠNo . Ġof Ġtrees Ġ( for Ġmethods Ġbased Ġon Ġvarious Ġtrees ), Ġn sv Ġ= ĠNo . Ġof Ġsupport Ġvectors , Ġi Ġ= ĠNo . Ġof Ġiterations , Ġh Ġ= ĠNo . Ġof Ġnodes Ġin Ġeach Ġhidden Ġlayer , Ġk Ġ= ĠNo . Ġof Ġhidden Ġlayers Ġand Ġ$\ over line { m } $ Ġ= Ġthe Ġaverage Ġno . Ġof Ġnon - zero Ġattributes Ġper Ġsample . </s> Conclusion </s> By Ġimplementing Ġdifferent Ġmachine Ġlearning Ġbased Ġclass ifiers Ġon Ġour ĠBeng ali Ġquestion Ġcorpus , Ġwe Ġperform Ġa Ġcomparative Ġanalysis Ġamong Ġthem . ĠThe Ġquestion Ġclassification Ġimpacts Ġthe ĠQ A Ġsystem . ĠSo , Ġit Ġis Ġimportant Ġto Ġclassify Ġthe Ġquestion Ġmore Ġprecisely . ĠThis Ġwork Ġwill Ġhelp Ġthe Ġresearch Ġcommunity Ġto Ġchoose Ġa Ġproper Ġclassification Ġmodel Ġfor Ġsmart ĠBeng ali ĠQ A Ġsystem Ġdevelopment . ĠFuture Ġwork Ġshould Ġaim Ġat Ġdeveloping Ġa Ġricher Ġcorpus Ġof ĠBeng ali Ġquestions Ġwhich Ġwill Ġhelp Ġin Ġgetting Ġbetter Ġvector Ġrepresentation Ġof Ġwords Ġand Ġthus Ġwill Ġfacilitate Ġdeep Ġlearning Ġbased Ġautomatic Ġfeature Ġextraction .
Introduction Question classification (QC) deals with question analysis and question labeling based on the expected answer type. The goal of QC is to assign classes accurately to the questions based on expected answer. In modern system, there are two types of questions BIBREF0. One is Factoid question which is about providing concise facts and another one is Complex question that has a presupposition which is complex. Question Answering (QA) System is an integral part of our daily life because of the high amount of usage of Internet for information acquisition. In recent years, most of the research works related to QA are based on English language such as IBM Watson, Wolfram Alpha. Bengali speakers often fall in difficulty while communicating in English BIBREF1. In this research, we briefly discuss the steps of QA system and compare the performance of seven machine learning based classifiers (Multi-Layer Perceptron (MLP), Naive Bayes Classifier (NBC), Support Vector Machine (SVM), Gradient Boosting Classifier (GBC), Stochastic Gradient Descent (SGD), K Nearest Neighbour (K-NN) and Random Forest (RF)) in classifying Bengali questions to classes based on their anticipated answers. Bengali questions have flexible inquiring ways, so there are many difficulties associated with Bengali QC BIBREF0. As there is no rich corpus of questions in Bengali Language available, collecting questions is an additional challenge. Different difficulties in building a QA System are mentioned in the literature BIBREF2 BIBREF3. The first work on a machine learning based approach towards Bengali question classification is presented in BIBREF0 that employ the Stochastic Gradient Descent (SGD). Related Works ::: Popular Question-Answering Systems Over the years, a handful of QA systems have gained popularity around the world. One of the oldest QA system is BASEBALL (created on 1961) BIBREF4 which answers question related to baseball league in America for a particular season. LUNAR BIBREF5 system answers questions about soil samples taken from Apollo lunar exploration. Some of the most popular QA Systems are IBM Watson, Apple Siri and Wolfram Alpha. Examples of some QA systems based on different languages are: Zhang Yu Chinese question classification BIBREF6 based on Incremental Modified Bayes, Arabic QA system (AQAS) BIBREF7 by F. A. Mohammed, K. Nasser, & H. M. Harb and Syntactic open domain Arabic QA system for factoid questions BIBREF8 by Fareed et al. QA systems have been built on different analysis methods such as morphological analysis BIBREF9, syntactical analysis BIBREF10, semantic analysis BIBREF11 and expected answer Type analysis BIBREF12. Related Works ::: Research Works Related to Question Classifications Researches on question classification, question taxonomies and QA system have been undertaken in recent years. There are two types of approaches for question classification according to Banerjee et al in BIBREF13 - by rules and by machine learning approach. Rule based approaches use some hard coded grammar rules to map the question to an appropriate answer type BIBREF14 BIBREF15. Machine Learning based approaches have been used by Zhang et al and Md. Aminul Islam et al in BIBREF16 and BIBREF0. Many classifiers have been used in machine learning for QC such as Support Vector Machine (SVM) BIBREF16 BIBREF17, Support Vector Machines and Maximum Entropy Model BIBREF18, Naive Bayes (NB), Kernel Naive Bayes (KNB), Decision Tree (DT) and Rule Induction (RI) BIBREF13. In BIBREF0, they claimed to achieve average precision of 0.95562 for coarse class and 0.87646 for finer class using Stochastic Gradient Descent (SGD). Related Works ::: Research Works in Bengali Language A Bengali QC System was built by Somnath Banerjee and Sivaji Bandyopadhyay BIBREF13 BIBREF19 BIBREF20. They proposed a two-layer taxonomy classification with 9 coarse-grained classes and 69 fine-grained classes. There are other research works BIBREF0 BIBREF21 in Bengali Language. A survey was performed on text QA techniques BIBREF22 where there was an analysis conducted in Bengali Language. Syed Mehedi Hasan Nirob et al achieved 88.62% accuracy by using 380 top frequent words as the feature in their work BIBREF17. Question Answering (QA) System QA system resides within the scope of Computer Science. It deals with information retrieval and natural language processing. Its goal is to automatically answer questions asked by humans in natural language. IR-based QA, Knowledge based approaches and Hybrid approaches are the QA system types. TREC, IBM-Watson, Google are examples of IR-based QA systems. Knowledge based QA systems are Apple Siri, Wolfram Alpha. Examples of Hybrid approach systems are IBM Watson and True Knowledge Evi. Figure FIGREF4 provides an overview of QA System. The first step of QA System is Question Analysis. Question analysis has two parts - question classification and another question formulation. In question classification step, the question is classified using different classifier algorithms. In question formulation, the question is analyzed and the system creates a proper IR question by detecting the entity type of the question to provide a simple answer. The next step is documents retrieval and analysis. In this step, the system matches the query against the sources of answers where the source can be documents or Web. In the answer extraction step, the system extracts the answers from the documents of the sources collected in documents retrieval and analysis phase. The extracted answers are filtered and evaluated in answer evaluation phase as there can be multiple possible answers for a query. In the final step, an answer of the question is returned. Proposed Methodology We use different types of classifiers for QA type classification. We separate our methodology into two sections similar to BIBREF0 - one is training section and another is validation section shown in Figure FIGREF5. We use 10 fold cross validation where we have 3150 and 350 questions in our training set and validation set respectively. During training, after selecting the possible class labels, the system extracts the features of the questions and creates a model by passing through a classifier algorithm with the extracted features and class labels. During validation, the system extracts the features of the question and passes it into the model created during training and predicts the answer type. Question Collection and Categories Though Bengali is the seventh most spoken language in terms of number of native speakers BIBREF23, there is no standard corpus of questions available BIBREF0. We have collected total 3500 questions from the Internet and other sources such as books of general knowledge questions, history etc. The corpus contains the questions and the classes each question belongs to. The set of question categories is known as question taxonomy BIBREF0. We have used two layer taxonomy which was proposed by Xin Li, Dan Roth BIBREF24. This two layer taxonomy is made up of two classes which are Coarse Class and Finer Class. There are six coarse classes such as Numeric, Location, Entity, Description, Human and Abbreviation and fifty finer classes such as city, state, mountain, distance, count, definition, group, expression, substance, creative, vehicle etc as shown in the Table I BIBREF0. A coarse-grained description of a system denotes large components while a fine-grained description denotes smaller sub-components of which the larger ones are composed of. Implementation of the System ::: Feature Extraction Question word answer phrases, parts of speech tags, parse feature, named entity and semantically related words are different features from answer type detection BIBREF18. We use question word and phrases as features for answer type detection. We consider the following features: Implementation of the System ::: Feature Extraction ::: TF-IDF Term Frequency - Inverse Document Frequency (TF-IDF) is a popular method used to identify the importance of a word in a particular document. TF-IDF transforms text into meaningful numeric representation. This technique is widely used to extract features for Natural Language Processing (NLP) applications BIBREF25 BIBREF26. Implementation of the System ::: Feature Extraction ::: Word level N-Grams N-grams is n-back to back words in a text. Queries of a same class usually share word n-grams BIBREF0. In this system, we choose bi-gram for extracting features. Implementation of the System ::: Feature Extraction ::: Stop Words We use two setups (as done in BIBREF0) for our system. In the first setup, we eliminate the stop words from the text using another dataset containing only stop words. At second step, we work without eliminating the stop words from the text which gives better result than the first setup. Implementation of the System ::: Classification Algorithms ::: Multi Layer Perceptron (MLP) MLP contains three layers - an input layer, an output layer and some hidden layers. Input layer receives the signal, the output layer gives a decision or prediction about the input and the computation of the MLP is conducted in the hidden layers. In our system, we use 100 layers. For weight optimization, we use Limited-memory Broyden–Fletcher–Goldfarb–Shanno (LBFGS) optimization algorithm. Implementation of the System ::: Classification Algorithms ::: Support Vector Machine (SVM) SVM gives an optimal hyper-plane and it maximizes the margin between classes. We use Radial Basis Function (RBF) kernel in our system to make decision boundary curve-shaped. For decision function shape, we use the original one-vs-one (ovo) decision function. Implementation of the System ::: Classification Algorithms ::: Naive Bayesian Classifier (NBC) NBC is based on Bayes' Theorem which gives probability of an event occurrence based on some conditions related to that event. We use Multinomial Naive Bayes Classifier with smoothing parameter equals to 0.1. A zero probability cancels the effects of all the other probabilities. Implementation of the System ::: Classification Algorithms ::: Stochastic Gradient Descent (SGD) Stochastic gradient descent optimizes an objective function with suitable smoothness properties BIBREF27. It selects few examples randomly instead of whole data for each iteration. We use 'L2' regularization for reduction of overfitting. Implementation of the System ::: Classification Algorithms ::: Gradient Boosting Classifier (GBC) Gradient Boosting Classifier produces a prediction model consisting of weak prediction models. Gradient boosting uses decision trees. We use 100 boosting stages in this work. Implementation of the System ::: Classification Algorithms ::: K Nearest Neighbour (K-NN) K-NN is a supervised classification and regression algorithm. It uses the neighbours of the given sample to identify its class. K determines the number of neighbours needed to be considered. We set the value of K equals to 13 in this work. Implementation of the System ::: Classification Algorithms ::: Random Forest (RF) RF is an ensemble learning technique. It constructs large number of decision trees during training and then predicts the majority class. We use 500 decision trees in the forest and "entropy" function to measure the quality of a split. Implementation of the System ::: Results and Discussion Table II shows the accuracy and F1 score for different classifiers with and without eliminating stop words while extracting features. Figure FIGREF21 shows the average results of different classifiers in a bar chart with and without eliminating stop words from the questions. Overall, SGD has shown the best performance on our dataset as it introduces non-linearity and uses back-propagation for updating parameter weights using loss function calculated on training set into classification. K-NN has shown the weakest performance overall, as this algorithm has a bad reputation of not working well in high dimensional data BIBREF28. MLP and SVM have shown similar performance. MLP takes advantage of multiple hidden layers in order to take non-linearly separable samples in a linearly separable condition. SVM accomplishes this same feat by taking the samples to a higher dimensional hyperplane where the samples are linearly separable. Gradient Boosting Classifier (GBC) and Random Forest (RF) both utilize a set of decision trees and achieve similar results (RF performs slightly without eliminating stop words). Naive Bayesian Classifier (NBC) shows performance on per with GBC and RF algorithms. The overall better performance of all the algorithms when provided with stop words show the importance of stop words in Bengali QA classification. Figure FIGREF22 shows the predictions of some particular questions by each of the classifiers. The input is a full question and the output is the class of the question. Implementation of the System ::: Computational Complexity In Table TABREF24, n = No. of training sample, p = No. of features, ntrees = No. of trees (for methods based on various trees), nsv = No. of support vectors, i = No. of iterations, h = No. of nodes in each hidden layer, k = No. of hidden layers and $\overline{m}$ = the average no. of non-zero attributes per sample. Conclusion By implementing different machine learning based classifiers on our Bengali question corpus, we perform a comparative analysis among them. The question classification impacts the QA system. So, it is important to classify the question more precisely. This work will help the research community to choose a proper classification model for smart Bengali QA system development. Future work should aim at developing a richer corpus of Bengali questions which will help in getting better vector representation of words and thus will facilitate deep learning based automatic feature extraction.
what ml based approaches were compared?
f428618ca9c017e0c9c2a23515dab30a7660f65f
what Ġml Ġbased Ġapproaches Ġwere Ġcompared ?
Multi - Layer ĠPer cept ron Ġ( ML P ), ĠNa ive ĠBay es ĠClass ifier Ġ( NBC ), ĠSupport ĠVector ĠMachine Ġ( S VM ), ĠGrad ient ĠBoost ing ĠClass ifier Ġ( G BC ), ĠSt och astic ĠGrad ient ĠDes cent Ġ( SG D ), ĠK ĠNe arest ĠNeigh bour Ġ( K - NN ) Ġand ĠRandom ĠForest Ġ( RF )
<s> what Ġml Ġbased Ġapproaches Ġwere Ġcompared ? </s> Introduction </s> Question Ġclassification Ġ( Q C ) Ġdeals Ġwith Ġquestion Ġanalysis Ġand Ġquestion Ġlabeling Ġbased Ġon Ġthe Ġexpected Ġanswer Ġtype . ĠThe Ġgoal Ġof ĠQC Ġis Ġto Ġassign Ġclasses Ġaccurately Ġto Ġthe Ġquestions Ġbased Ġon Ġexpected Ġanswer . ĠIn Ġmodern Ġsystem , Ġthere Ġare Ġtwo Ġtypes Ġof Ġquestions ĠB IB REF 0 . ĠOne Ġis ĠFact oid Ġquestion Ġwhich Ġis Ġabout Ġproviding Ġconcise Ġfacts Ġand Ġanother Ġone Ġis ĠComplex Ġquestion Ġthat Ġhas Ġa Ġpresupp osition Ġwhich Ġis Ġcomplex . ĠQuestion ĠAn sw ering Ġ( Q A ) ĠSystem Ġis Ġan Ġintegral Ġpart Ġof Ġour Ġdaily Ġlife Ġbecause Ġof Ġthe Ġhigh Ġamount Ġof Ġusage Ġof ĠInternet Ġfor Ġinformation Ġacquisition . ĠIn Ġrecent Ġyears , Ġmost Ġof Ġthe Ġresearch Ġworks Ġrelated Ġto ĠQ A Ġare Ġbased Ġon ĠEnglish Ġlanguage Ġsuch Ġas ĠIBM ĠWatson , ĠWolf ram ĠAlpha . ĠBeng ali Ġspeakers Ġoften Ġfall Ġin Ġdifficulty Ġwhile Ġcommunicating Ġin ĠEnglish ĠB IB REF 1 . </s> In Ġthis Ġresearch , Ġwe Ġbriefly Ġdiscuss Ġthe Ġsteps Ġof ĠQ A Ġsystem Ġand Ġcompare Ġthe Ġperformance Ġof Ġseven Ġmachine Ġlearning Ġbased Ġclass ifiers Ġ( Multi - Layer ĠPer cept ron Ġ( ML P ), ĠNa ive ĠBay es ĠClass ifier Ġ( NBC ), ĠSupport ĠVector ĠMachine Ġ( S VM ), ĠGrad ient ĠBoost ing ĠClass ifier Ġ( G BC ), ĠSt och astic ĠGrad ient ĠDes cent Ġ( SG D ), ĠK ĠNe arest ĠNeigh bour Ġ( K - NN ) Ġand ĠRandom ĠForest Ġ( RF )) Ġin Ġclass ifying ĠBeng ali Ġquestions Ġto Ġclasses Ġbased Ġon Ġtheir Ġanticipated Ġanswers . ĠBeng ali Ġquestions Ġhave Ġflexible Ġinqu iring Ġways , Ġso Ġthere Ġare Ġmany Ġdifficulties Ġassociated Ġwith ĠBeng ali ĠQC ĠB IB REF 0 . ĠAs Ġthere Ġis Ġno Ġrich Ġcorpus Ġof Ġquestions Ġin ĠBeng ali ĠLanguage Ġavailable , Ġcollecting Ġquestions Ġis Ġan Ġadditional Ġchallenge . ĠDifferent Ġdifficulties Ġin Ġbuilding Ġa ĠQ A ĠSystem Ġare Ġmentioned Ġin Ġthe Ġliterature ĠB IB REF 2 ĠB IB REF 3 . ĠThe Ġfirst Ġwork Ġon Ġa Ġmachine Ġlearning Ġbased Ġapproach Ġtowards ĠBeng ali Ġquestion Ġclassification Ġis Ġpresented Ġin ĠB IB REF 0 Ġthat Ġemploy Ġthe ĠSt och astic ĠGrad ient ĠDes cent Ġ( SG D ). </s> Related ĠWorks Ġ: :: ĠPopular ĠQuestion - An sw ering ĠSystems </s> Over Ġthe Ġyears , Ġa Ġhandful Ġof ĠQ A Ġsystems Ġhave Ġgained Ġpopularity Ġaround Ġthe Ġworld . ĠOne Ġof Ġthe Ġoldest ĠQ A Ġsystem Ġis ĠBASE BALL Ġ( created Ġon Ġ1961 ) ĠB IB REF 4 Ġwhich Ġanswers Ġquestion Ġrelated Ġto Ġbaseball Ġleague Ġin ĠAmerica Ġfor Ġa Ġparticular Ġseason . ĠL UN AR ĠB IB REF 5 Ġsystem Ġanswers Ġquestions Ġabout Ġsoil Ġsamples Ġtaken Ġfrom ĠApollo Ġlunar Ġexploration . ĠSome Ġof Ġthe Ġmost Ġpopular ĠQ A ĠSystems Ġare ĠIBM ĠWatson , ĠApple ĠSiri Ġand ĠWolf ram ĠAlpha . ĠExamples Ġof Ġsome ĠQ A Ġsystems Ġbased Ġon Ġdifferent Ġlanguages Ġare : ĠZhang ĠYu ĠChinese Ġquestion Ġclassification ĠB IB REF 6 Ġbased Ġon ĠIncre mental ĠModified ĠBay es , ĠArabic ĠQ A Ġsystem Ġ( A Q AS ) ĠB IB REF 7 Ġby ĠF . ĠA . ĠMohammed , ĠK . ĠN asser , Ġ& ĠH . ĠM . ĠHar b Ġand ĠSynt actic Ġopen Ġdomain ĠArabic ĠQ A Ġsystem Ġfor Ġfact oid Ġquestions ĠB IB REF 8 Ġby ĠFare ed Ġet Ġal . ĠQ A Ġsystems Ġhave Ġbeen Ġbuilt Ġon Ġdifferent Ġanalysis Ġmethods Ġsuch Ġas Ġmorph ological Ġanalysis ĠB IB REF 9 , Ġsynt act ical Ġanalysis ĠB IB REF 10 , Ġsemantic Ġanalysis ĠB IB REF 11 Ġand Ġexpected Ġanswer ĠType Ġanalysis ĠB IB REF 12 . </s> Related ĠWorks Ġ: :: ĠResearch ĠWorks ĠRelated Ġto ĠQuestion ĠClass ifications </s> Res ear ches Ġon Ġquestion Ġclassification , Ġquestion Ġtax onom ies Ġand ĠQ A Ġsystem Ġhave Ġbeen Ġundertaken Ġin Ġrecent Ġyears . ĠThere Ġare Ġtwo Ġtypes Ġof Ġapproaches Ġfor Ġquestion Ġclassification Ġaccording Ġto ĠBan er jee Ġet Ġal Ġin ĠB IB REF 13 Ġ- Ġby Ġrules Ġand Ġby Ġmachine Ġlearning Ġapproach . ĠRule Ġbased Ġapproaches Ġuse Ġsome Ġhard Ġcoded Ġgrammar Ġrules Ġto Ġmap Ġthe Ġquestion Ġto Ġan Ġappropriate Ġanswer Ġtype ĠB IB REF 14 ĠB IB REF 15 . ĠMachine ĠLearning Ġbased Ġapproaches Ġhave Ġbeen Ġused Ġby ĠZhang Ġet Ġal Ġand ĠMd . ĠAmin ul ĠIslam Ġet Ġal Ġin ĠB IB REF 16 Ġand ĠB IB REF 0 . ĠMany Ġclass ifiers Ġhave Ġbeen Ġused Ġin Ġmachine Ġlearning Ġfor ĠQC Ġsuch Ġas ĠSupport ĠVector ĠMachine Ġ( S VM ) ĠB IB REF 16 ĠB IB REF 17 , ĠSupport ĠVector ĠMachines Ġand ĠMaximum ĠEnt ropy ĠModel ĠB IB REF 18 , ĠNa ive ĠBay es Ġ( NB ), ĠKernel ĠNa ive ĠBay es Ġ( KN B ), ĠDecision ĠTree Ġ( DT ) Ġand ĠRule ĠInd uction Ġ( RI ) ĠB IB REF 13 . ĠIn ĠB IB REF 0 , Ġthey Ġclaimed Ġto Ġachieve Ġaverage Ġprecision Ġof Ġ0 . 9 55 62 Ġfor Ġcoarse Ġclass Ġand Ġ0 . 87 646 Ġfor Ġfiner Ġclass Ġusing ĠSt och astic ĠGrad ient ĠDes cent Ġ( SG D ). </s> Related ĠWorks Ġ: :: ĠResearch ĠWorks Ġin ĠBeng ali ĠLanguage </s> A ĠBeng ali ĠQC ĠSystem Ġwas Ġbuilt Ġby ĠSom n ath ĠBan er jee Ġand ĠS iv aji ĠBand y op ad hy ay ĠB IB REF 13 ĠB IB REF 19 ĠB IB REF 20 . ĠThey Ġproposed Ġa Ġtwo - layer Ġtax onomy Ġclassification Ġwith Ġ9 Ġcoarse - gr ained Ġclasses Ġand Ġ69 Ġfine - gr ained Ġclasses . ĠThere Ġare Ġother Ġresearch Ġworks ĠB IB REF 0 ĠB IB REF 21 Ġin ĠBeng ali ĠLanguage . ĠA Ġsurvey Ġwas Ġperformed Ġon Ġtext ĠQ A Ġtechniques ĠB IB REF 22 Ġwhere Ġthere Ġwas Ġan Ġanalysis Ġconducted Ġin ĠBeng ali ĠLanguage . ĠSy ed ĠMe hed i ĠHasan ĠN iro b Ġet Ġal Ġachieved Ġ88 . 62 % Ġaccuracy Ġby Ġusing Ġ380 Ġtop Ġfrequent Ġwords Ġas Ġthe Ġfeature Ġin Ġtheir Ġwork ĠB IB REF 17 . </s> Question ĠAn sw ering Ġ( Q A ) ĠSystem </s> Q A Ġsystem Ġresides Ġwithin Ġthe Ġscope Ġof ĠComputer ĠScience . ĠIt Ġdeals Ġwith Ġinformation Ġretrieval Ġand Ġnatural Ġlanguage Ġprocessing . ĠIts Ġgoal Ġis Ġto Ġautomatically Ġanswer Ġquestions Ġasked Ġby Ġhumans Ġin Ġnatural Ġlanguage . ĠIR - based ĠQ A , ĠKnowledge Ġbased Ġapproaches Ġand ĠHybrid Ġapproaches Ġare Ġthe ĠQ A Ġsystem Ġtypes . ĠT REC , ĠIBM - W atson , ĠGoogle Ġare Ġexamples Ġof ĠIR - based ĠQ A Ġsystems . ĠKnowledge Ġbased ĠQ A Ġsystems Ġare ĠApple ĠSiri , ĠWolf ram ĠAlpha . ĠExamples Ġof ĠHybrid Ġapproach Ġsystems Ġare ĠIBM ĠWatson Ġand ĠTrue ĠKnowledge ĠEv i . </s> Figure ĠFIG REF 4 Ġprovides Ġan Ġoverview Ġof ĠQ A ĠSystem . ĠThe Ġfirst Ġstep Ġof ĠQ A ĠSystem Ġis ĠQuestion ĠAnalysis . ĠQuestion Ġanalysis Ġhas Ġtwo Ġparts Ġ- Ġquestion Ġclassification Ġand Ġanother Ġquestion Ġformulation . ĠIn Ġquestion Ġclassification Ġstep , Ġthe Ġquestion Ġis Ġclassified Ġusing Ġdifferent Ġclass ifier Ġalgorithms . ĠIn Ġquestion Ġformulation , Ġthe Ġquestion Ġis Ġanalyzed Ġand Ġthe Ġsystem Ġcreates Ġa Ġproper ĠIR Ġquestion Ġby Ġdetecting Ġthe Ġentity Ġtype Ġof Ġthe Ġquestion Ġto Ġprovide Ġa Ġsimple Ġanswer . </s> The Ġnext Ġstep Ġis Ġdocuments Ġretrieval Ġand Ġanalysis . ĠIn Ġthis Ġstep , Ġthe Ġsystem Ġmatches Ġthe Ġquery Ġagainst Ġthe Ġsources Ġof Ġanswers Ġwhere Ġthe Ġsource Ġcan Ġbe Ġdocuments Ġor ĠWeb . ĠIn Ġthe Ġanswer Ġextraction Ġstep , Ġthe Ġsystem Ġextracts Ġthe Ġanswers Ġfrom Ġthe Ġdocuments Ġof Ġthe Ġsources Ġcollected Ġin Ġdocuments Ġretrieval Ġand Ġanalysis Ġphase . ĠThe Ġextracted Ġanswers Ġare Ġfiltered Ġand Ġevaluated Ġin Ġanswer Ġevaluation Ġphase Ġas Ġthere Ġcan Ġbe Ġmultiple Ġpossible Ġanswers Ġfor Ġa Ġquery . ĠIn Ġthe Ġfinal Ġstep , Ġan Ġanswer Ġof Ġthe Ġquestion Ġis Ġreturned . </s> Prop osed ĠMethod ology </s> We Ġuse Ġdifferent Ġtypes Ġof Ġclass ifiers Ġfor ĠQ A Ġtype Ġclassification . ĠWe Ġseparate Ġour Ġmethodology Ġinto Ġtwo Ġsections Ġsimilar Ġto ĠB IB REF 0 Ġ- Ġone Ġis Ġtraining Ġsection Ġand Ġanother Ġis Ġvalidation Ġsection Ġshown Ġin ĠFigure ĠFIG REF 5 . </s> We Ġuse Ġ10 Ġfold Ġcross Ġvalidation Ġwhere Ġwe Ġhave Ġ31 50 Ġand Ġ350 Ġquestions Ġin Ġour Ġtraining Ġset Ġand Ġvalidation Ġset Ġrespectively . ĠDuring Ġtraining , Ġafter Ġselecting Ġthe Ġpossible Ġclass Ġlabels , Ġthe Ġsystem Ġextracts Ġthe Ġfeatures Ġof Ġthe Ġquestions Ġand Ġcreates Ġa Ġmodel Ġby Ġpassing Ġthrough Ġa Ġclass ifier Ġalgorithm Ġwith Ġthe Ġextracted Ġfeatures Ġand Ġclass Ġlabels . ĠDuring Ġvalidation , Ġthe Ġsystem Ġextracts Ġthe Ġfeatures Ġof Ġthe Ġquestion Ġand Ġpasses Ġit Ġinto Ġthe Ġmodel Ġcreated Ġduring Ġtraining Ġand Ġpredicts Ġthe Ġanswer Ġtype . </s> Question ĠCollection Ġand ĠCategories </s> Though ĠBeng ali Ġis Ġthe Ġseventh Ġmost Ġspoken Ġlanguage Ġin Ġterms Ġof Ġnumber Ġof Ġnative Ġspeakers ĠB IB REF 23 , Ġthere Ġis Ġno Ġstandard Ġcorpus Ġof Ġquestions Ġavailable ĠB IB REF 0 . ĠWe Ġhave Ġcollected Ġtotal Ġ35 00 Ġquestions Ġfrom Ġthe ĠInternet Ġand Ġother Ġsources Ġsuch Ġas Ġbooks Ġof Ġgeneral Ġknowledge Ġquestions , Ġhistory Ġetc . ĠThe Ġcorpus Ġcontains Ġthe Ġquestions Ġand Ġthe Ġclasses Ġeach Ġquestion Ġbelongs Ġto . </s> The Ġset Ġof Ġquestion Ġcategories Ġis Ġknown Ġas Ġquestion Ġtax onomy ĠB IB REF 0 . ĠWe Ġhave Ġused Ġtwo Ġlayer Ġtax onomy Ġwhich Ġwas Ġproposed Ġby ĠXin ĠLi , ĠDan ĠRoth ĠB IB REF 24 . ĠThis Ġtwo Ġlayer Ġtax onomy Ġis Ġmade Ġup Ġof Ġtwo Ġclasses Ġwhich Ġare ĠCo arse ĠClass Ġand ĠFin er ĠClass . ĠThere Ġare Ġsix Ġcoarse Ġclasses Ġsuch Ġas ĠN umeric , ĠLocation , ĠEntity , ĠDescription , ĠHuman Ġand ĠAb bre viation Ġand Ġfifty Ġfiner Ġclasses Ġsuch Ġas Ġcity , Ġstate , Ġmountain , Ġdistance , Ġcount , Ġdefinition , Ġgroup , Ġexpression , Ġsubstance , Ġcreative , Ġvehicle Ġetc Ġas Ġshown Ġin Ġthe ĠTable ĠI ĠB IB REF 0 . ĠA Ġcoarse - gr ained Ġdescription Ġof Ġa Ġsystem Ġdenotes Ġlarge Ġcomponents Ġwhile Ġa Ġfine - gr ained Ġdescription Ġdenotes Ġsmaller Ġsub - comp onents Ġof Ġwhich Ġthe Ġlarger Ġones Ġare Ġcomposed Ġof . </s> Im plementation Ġof Ġthe ĠSystem Ġ: :: ĠFeature ĠExt raction </s> Question Ġword Ġanswer Ġphrases , Ġparts Ġof Ġspeech Ġtags , Ġparse Ġfeature , Ġnamed Ġentity Ġand Ġsem antically Ġrelated Ġwords Ġare Ġdifferent Ġfeatures Ġfrom Ġanswer Ġtype Ġdetection ĠB IB REF 18 . ĠWe Ġuse Ġquestion Ġword Ġand Ġphrases Ġas Ġfeatures Ġfor Ġanswer Ġtype Ġdetection . ĠWe Ġconsider Ġthe Ġfollowing Ġfeatures : </s> Im plementation Ġof Ġthe ĠSystem Ġ: :: ĠFeature ĠExt raction Ġ: :: ĠTF - ID F </s> Term ĠFrequency Ġ- ĠIn verse ĠDocument ĠFrequency Ġ( TF - ID F ) Ġis Ġa Ġpopular Ġmethod Ġused Ġto Ġidentify Ġthe Ġimportance Ġof Ġa Ġword Ġin Ġa Ġparticular Ġdocument . ĠTF - ID F Ġtransforms Ġtext Ġinto Ġmeaningful Ġnumeric Ġrepresentation . ĠThis Ġtechnique Ġis Ġwidely Ġused Ġto Ġextract Ġfeatures Ġfor ĠNatural ĠLanguage ĠProcessing Ġ( N LP ) Ġapplications ĠB IB REF 25 ĠB IB REF 26 . </s> Im plementation Ġof Ġthe ĠSystem Ġ: :: ĠFeature ĠExt raction Ġ: :: ĠWord Ġlevel ĠN - G rams </s> N - gram s Ġis Ġn - back Ġto Ġback Ġwords Ġin Ġa Ġtext . ĠQu eries Ġof Ġa Ġsame Ġclass Ġusually Ġshare Ġword Ġn - gram s ĠB IB REF 0 . ĠIn Ġthis Ġsystem , Ġwe Ġchoose Ġbi - gram Ġfor Ġextracting Ġfeatures . </s> Im plementation Ġof Ġthe ĠSystem Ġ: :: ĠFeature ĠExt raction Ġ: :: ĠStop ĠWords </s> We Ġuse Ġtwo Ġsetups Ġ( as Ġdone Ġin ĠB IB REF 0 ) Ġfor Ġour Ġsystem . ĠIn Ġthe Ġfirst Ġsetup , Ġwe Ġeliminate Ġthe Ġstop Ġwords Ġfrom Ġthe Ġtext Ġusing Ġanother Ġdataset Ġcontaining Ġonly Ġstop Ġwords . ĠAt Ġsecond Ġstep , Ġwe Ġwork Ġwithout Ġeliminating Ġthe Ġstop Ġwords Ġfrom Ġthe Ġtext Ġwhich Ġgives Ġbetter Ġresult Ġthan Ġthe Ġfirst Ġsetup . </s> Im plementation Ġof Ġthe ĠSystem Ġ: :: ĠClassification ĠAl gorith ms Ġ: :: ĠMulti ĠLayer ĠPer cept ron Ġ( ML P ) </s> ML P Ġcontains Ġthree Ġlayers Ġ- Ġan Ġinput Ġlayer , Ġan Ġoutput Ġlayer Ġand Ġsome Ġhidden Ġlayers . ĠInput Ġlayer Ġreceives Ġthe Ġsignal , Ġthe Ġoutput Ġlayer Ġgives Ġa Ġdecision Ġor Ġprediction Ġabout Ġthe Ġinput Ġand Ġthe Ġcomputation Ġof Ġthe ĠML P Ġis Ġconducted Ġin Ġthe Ġhidden Ġlayers . ĠIn Ġour Ġsystem , Ġwe Ġuse Ġ100 Ġlayers . ĠFor Ġweight Ġoptimization , Ġwe Ġuse ĠLimited - memory ĠBro yden âĢĵ F letcher âĢĵ Gold far b âĢĵ Sh ann o Ġ( L BF GS ) Ġoptimization Ġalgorithm . </s> Im plementation Ġof Ġthe ĠSystem Ġ: :: ĠClassification ĠAl gorith ms Ġ: :: ĠSupport ĠVector ĠMachine Ġ( S VM ) </s> S VM Ġgives Ġan Ġoptimal Ġhyper - plane Ġand Ġit Ġmaxim izes Ġthe Ġmargin Ġbetween Ġclasses . ĠWe Ġuse ĠRad ial ĠBas is ĠFunction Ġ( RB F ) Ġkernel Ġin Ġour Ġsystem Ġto Ġmake Ġdecision Ġboundary Ġcurve - shaped . ĠFor Ġdecision Ġfunction Ġshape , Ġwe Ġuse Ġthe Ġoriginal Ġone - vs - one Ġ( ovo ) Ġdecision Ġfunction . </s> Im plementation Ġof Ġthe ĠSystem Ġ: :: ĠClassification ĠAl gorith ms Ġ: :: ĠNa ive ĠBay esian ĠClass ifier Ġ( NBC ) </s> NBC Ġis Ġbased Ġon ĠBay es ' ĠThe orem Ġwhich Ġgives Ġprobability Ġof Ġan Ġevent Ġoccurrence Ġbased Ġon Ġsome Ġconditions Ġrelated Ġto Ġthat Ġevent . ĠWe Ġuse ĠMult in omial ĠNa ive ĠBay es ĠClass ifier Ġwith Ġsmoot hing Ġparameter Ġequals Ġto Ġ0 . 1 . ĠA Ġzero Ġprobability Ġcancel s Ġthe Ġeffects Ġof Ġall Ġthe Ġother Ġprobabilities . </s> Im plementation Ġof Ġthe ĠSystem Ġ: :: ĠClassification ĠAl gorith ms Ġ: :: ĠSt och astic ĠGrad ient ĠDes cent Ġ( SG D ) </s> St och astic Ġgradient Ġdescent Ġoptim izes Ġan Ġobjective Ġfunction Ġwith Ġsuitable Ġsmooth ness Ġproperties ĠB IB REF 27 . ĠIt Ġselects Ġfew Ġexamples Ġrandomly Ġinstead Ġof Ġwhole Ġdata Ġfor Ġeach Ġiteration . ĠWe Ġuse Ġ' L 2 ' Ġregular ization Ġfor Ġreduction Ġof Ġover fitting . </s> Im plementation Ġof Ġthe ĠSystem Ġ: :: ĠClassification ĠAl gorith ms Ġ: :: ĠGrad ient ĠBoost ing ĠClass ifier Ġ( G BC ) </s> Grad ient ĠBoost ing ĠClass ifier Ġproduces Ġa Ġprediction Ġmodel Ġconsisting Ġof Ġweak Ġprediction Ġmodels . ĠGrad ient Ġboosting Ġuses Ġdecision Ġtrees . ĠWe Ġuse Ġ100 Ġboosting Ġstages Ġin Ġthis Ġwork . </s> Im plementation Ġof Ġthe ĠSystem Ġ: :: ĠClassification ĠAl gorith ms Ġ: :: ĠK ĠNe arest ĠNeigh bour Ġ( K - NN ) </s> K - NN Ġis Ġa Ġsupervised Ġclassification Ġand Ġregression Ġalgorithm . ĠIt Ġuses Ġthe Ġneighbours Ġof Ġthe Ġgiven Ġsample Ġto Ġidentify Ġits Ġclass . ĠK Ġdetermines Ġthe Ġnumber Ġof Ġneighbours Ġneeded Ġto Ġbe Ġconsidered . ĠWe Ġset Ġthe Ġvalue Ġof ĠK Ġequals Ġto Ġ13 Ġin Ġthis Ġwork . </s> Im plementation Ġof Ġthe ĠSystem Ġ: :: ĠClassification ĠAl gorith ms Ġ: :: ĠRandom ĠForest Ġ( RF ) </s> RF Ġis Ġan Ġensemble Ġlearning Ġtechnique . ĠIt Ġconstructs Ġlarge Ġnumber Ġof Ġdecision Ġtrees Ġduring Ġtraining Ġand Ġthen Ġpredicts Ġthe Ġmajority Ġclass . ĠWe Ġuse Ġ500 Ġdecision Ġtrees Ġin Ġthe Ġforest Ġand Ġ" ent ropy " Ġfunction Ġto Ġmeasure Ġthe Ġquality Ġof Ġa Ġsplit . </s> Im plementation Ġof Ġthe ĠSystem Ġ: :: ĠResults Ġand ĠDiscussion </s> Table ĠII Ġshows Ġthe Ġaccuracy Ġand ĠF 1 Ġscore Ġfor Ġdifferent Ġclass ifiers Ġwith Ġand Ġwithout Ġeliminating Ġstop Ġwords Ġwhile Ġextracting Ġfeatures . ĠFigure ĠFIG REF 21 Ġshows Ġthe Ġaverage Ġresults Ġof Ġdifferent Ġclass ifiers Ġin Ġa Ġbar Ġchart Ġwith Ġand Ġwithout Ġeliminating Ġstop Ġwords Ġfrom Ġthe Ġquestions . </s> Overall , ĠSG D Ġhas Ġshown Ġthe Ġbest Ġperformance Ġon Ġour Ġdataset Ġas Ġit Ġintroduces Ġnon - linear ity Ġand Ġuses Ġback - prop ag ation Ġfor Ġupdating Ġparameter Ġweights Ġusing Ġloss Ġfunction Ġcalculated Ġon Ġtraining Ġset Ġinto Ġclassification . ĠK - NN Ġhas Ġshown Ġthe Ġweakest Ġperformance Ġoverall , Ġas Ġthis Ġalgorithm Ġhas Ġa Ġbad Ġreputation Ġof Ġnot Ġworking Ġwell Ġin Ġhigh Ġdimensional Ġdata ĠB IB REF 28 . ĠML P Ġand ĠS VM Ġhave Ġshown Ġsimilar Ġperformance . ĠML P Ġtakes Ġadvantage Ġof Ġmultiple Ġhidden Ġlayers Ġin Ġorder Ġto Ġtake Ġnon - lin early Ġsepar able Ġsamples Ġin Ġa Ġlin early Ġsepar able Ġcondition . ĠS VM Ġaccompl ishes Ġthis Ġsame Ġfeat Ġby Ġtaking Ġthe Ġsamples Ġto Ġa Ġhigher Ġdimensional Ġhyper plane Ġwhere Ġthe Ġsamples Ġare Ġlin early Ġsepar able . ĠGrad ient ĠBoost ing ĠClass ifier Ġ( G BC ) Ġand ĠRandom ĠForest Ġ( RF ) Ġboth Ġutilize Ġa Ġset Ġof Ġdecision Ġtrees Ġand Ġachieve Ġsimilar Ġresults Ġ( RF Ġperforms Ġslightly Ġwithout Ġeliminating Ġstop Ġwords ). ĠNa ive ĠBay esian ĠClass ifier Ġ( NBC ) Ġshows Ġperformance Ġon Ġper Ġwith ĠG BC Ġand ĠRF Ġalgorithms . ĠThe Ġoverall Ġbetter Ġperformance Ġof Ġall Ġthe Ġalgorithms Ġwhen Ġprovided Ġwith Ġstop Ġwords Ġshow Ġthe Ġimportance Ġof Ġstop Ġwords Ġin ĠBeng ali ĠQ A Ġclassification . </s> Figure ĠFIG REF 22 Ġshows Ġthe Ġpredictions Ġof Ġsome Ġparticular Ġquestions Ġby Ġeach Ġof Ġthe Ġclass ifiers . ĠThe Ġinput Ġis Ġa Ġfull Ġquestion Ġand Ġthe Ġoutput Ġis Ġthe Ġclass Ġof Ġthe Ġquestion . </s> Im plementation Ġof Ġthe ĠSystem Ġ: :: ĠComput ational ĠComplex ity </s> In ĠTable ĠT AB REF 24 , Ġn Ġ= ĠNo . Ġof Ġtraining Ġsample , Ġp Ġ= ĠNo . Ġof Ġfeatures , Ġn t rees Ġ= ĠNo . Ġof Ġtrees Ġ( for Ġmethods Ġbased Ġon Ġvarious Ġtrees ), Ġn sv Ġ= ĠNo . Ġof Ġsupport Ġvectors , Ġi Ġ= ĠNo . Ġof Ġiterations , Ġh Ġ= ĠNo . Ġof Ġnodes Ġin Ġeach Ġhidden Ġlayer , Ġk Ġ= ĠNo . Ġof Ġhidden Ġlayers Ġand Ġ$\ over line { m } $ Ġ= Ġthe Ġaverage Ġno . Ġof Ġnon - zero Ġattributes Ġper Ġsample . </s> Conclusion </s> By Ġimplementing Ġdifferent Ġmachine Ġlearning Ġbased Ġclass ifiers Ġon Ġour ĠBeng ali Ġquestion Ġcorpus , Ġwe Ġperform Ġa Ġcomparative Ġanalysis Ġamong Ġthem . ĠThe Ġquestion Ġclassification Ġimpacts Ġthe ĠQ A Ġsystem . ĠSo , Ġit Ġis Ġimportant Ġto Ġclassify Ġthe Ġquestion Ġmore Ġprecisely . ĠThis Ġwork Ġwill Ġhelp Ġthe Ġresearch Ġcommunity Ġto Ġchoose Ġa Ġproper Ġclassification Ġmodel Ġfor Ġsmart ĠBeng ali ĠQ A Ġsystem Ġdevelopment . ĠFuture Ġwork Ġshould Ġaim Ġat Ġdeveloping Ġa Ġricher Ġcorpus Ġof ĠBeng ali Ġquestions Ġwhich Ġwill Ġhelp Ġin Ġgetting Ġbetter Ġvector Ġrepresentation Ġof Ġwords Ġand Ġthus Ġwill Ġfacilitate Ġdeep Ġlearning Ġbased Ġautomatic Ġfeature Ġextraction .