LSTM Language Model Based Korean Sentence Generation
The recurrent neural network (RNN) is a deep learning model which is suitable to sequential or length variable data. The Long Short Term Memory (LSTM) mitigates the vanishing gradient problem of RNNs so that LSTM can maintain the long term dependency among the constituents of the given input sequence. In this paper, we propose a LSTM based language model which can predict following words of a given incomplete sentence to generate a complete sentence. To evaluate our method, we trained our model using multiple Korean corpora then generated the incomplete part of Korean sentences. The result shows that our language model was able to generate the fluent Korean sentences. We also show that the word based model generated better sentences compared to the other settings. bon yeonguneun 2016nyeondo dunoehangug21peulreoseusaeobe yihayeo jiweoneul bada suhaengdoeeossseubnida. First Author Automation and Systems Research Institute (ASRI) Department of Electrical and Computer Engineering, Seoul National University, ad26kr@snu.ac.kr, hagsaenghoeweon deg Corresponding Author Automation and Systems Research Institute (ASRI) Department of Electrical and Computer Engineering, Seoul National University, kjung@snu.ac.kr, jeonghoeweon Department of Electrical and Computer Engineering, Seoul National University, wangcho2k@snu.ac.kr Department of Electrical and Computer Engineering, Seoul National University, zd9370@snu.ac.kr nonmunbeonho:KICS2015 12 388, Received December 10, 2015; Revised April 6, 2016; Accepted May 23, 2016 I. seo ron coegeun gigyehagseub(Machine Learning) yeongu mic eungyongbun yaga daepog hwagdaedoeneun gaunde [1,2] dib reoning(Deep Learning)i saerobge jumogbadgo issda. dib reoningeun yeoreo eunniggyeceung(Hidden Layer)euro guseongdoen ingongsingyeongmang (Artificial Neural Network)eul hagseubhago iyonghaneun bang beobeuroseo, GPU(Graphics Processing Unit)wa gateun go seongneung gyesan hadeuweeoyi baldal mic pungbuhan deiteoyi cugjeog gwa dongsie yeoreo bunyaeseo gijon gigyehagseub gibeobeul ddwieo nonmun LSTM eoneomodel giban hangugeo munjang saengseong 593 neomneun gangryeoghan daeaneuro busanghago issda. dib reoningeun yeongsang insiggwa gati gijon gigyehagseub bunyaeseo hwalbalhi yeongudoe go issneun deiteo insig bunyaneun mulronigo, helseukeeo, mu seon senseo neteuweokeu(Wireless Sensor Network) deungyi bunyaedo coegeun pog neolbge jeogyongdoego issda. dib reoning gibeob jung hanaro coegeun yeonguga hwalbalhage i rueojigo issneun sunhwansingyeongmang(Recurrent Neural Network; RNN)eun gwageo sigan dangye(Time Step)yi eun nigsangtae(Hidden State)wa hyeon sigan dangyeyi siseutem ibryeog gabseul saeroun eunnig sangtaereul gyesanhaneun de sayonghaneun gujo jeog teugjingi issda. hajiman RNNyi gibonjeogin sunhwangujoneun hagseubgwajeongeseo yeonsogdoen ibryeoggabsyi giliga neuleoname ddara cugjeogdoeneun jeongboreul ilhge doeneun giulgi somyeolmunje (Vanishing Gradient Problem [5] ro inhae eungyongjeogin ceug myeoneseo ddwieonan seonggwareul boiji moshaesseuna, hagseub seongneungeul hyangsangsikineun dayanghan gibeobdeuli gaebaldoemyeonseo dayanghan bunya e neolri jeogyongdoegi sijaghaessda. teughi LSTM(Long Short Term Memory) RNN [6] yi gyeongu, RNNyi gujojeog gaeseon eul tonghae hagseub neungryeogeul keuge hyangsangsikyeo yeonsogjeogigo gili ga gin hagseub deiteoga gajigo issneun suncajeog yijonseongeul hyo gwajeogeuro hagseub hal bbunman anira gwageo sigan dangye jeongboyi jungyodoreul panbyeolhayeo eolmana manheun gwageo jeongboreul hyeonjaegga ji ggeuleool geosinji, hogeun hyeonjae sigan dangyeyi jeongboreul eol mana sayonghal geosinji gyeoljeong hal su issgi ddaemune gin sigan dongan natananeun ibryeog deiteoyi yijonseongeul gijonyi RNN boda deoug hyogwajeogeuro hagseubhaneun geosi ganeunghada. geureoh gi ddaemune LSTMyi gyeongu giliga gin ibryeog deiteoreul jeog yonghaneun jageobeseo gijon RNNboda joheun seongneungeul boinda. bon yeongueseoneun LSTMe giban hangugeo eoneo model (Language Model)eul tonghae bulwanjeonhan hangugeo munjang hog eun daneoga jueojyeosseul ddae ieojineun munjangeul jadongeuro saeng seonghal su issneun munjang saengseong siseutemeul guhyeonhanda. eoneo mo deleun yeoreo gaeyi munseona munjanggwa gateun haedang eoneoyi malmung ci(Corpus) deiteoreul hagseubhayeo, jueojin daneoyi baeyeoli haedang eoneoe isseoseo eolmana inganyi eoneowa yusahanji reul pyohyeonhal su issneun hwagryul mohyeongida. ireultemyeon, jeogjeolha ge guhyeon mic hagseubdoen hangugeo eoneo modeleseoneun 'babeul meog da' ga 'meogda babeul' yi gyeonguboda nopeun hwagryul gabseul boil geosida. LSTMe gibanhan eoneomodeleun jubyeon daneoyi dong siculhyeonhwagryule gibanhan Unigram eoneomodel mic N gram eoneomodelbodaneun munjangeseo natananeun daneo ganyi suncajeog yijongwangyereul boda hyogwajeogeuro hagseub hal su issda. ireohge hagseubhan eoneo modeleul batangeuro munjangyi ilbuga jueojyeoss eul ddae munjangyi nameoji daneodeuleul suncajeogeuro curon mic saeng seonghaneun munjang saengseong algorijeumeul guhyeonhamyeo, bon nonmuneseo neun silje hangugeo malmungcireul iyonghan eoneomodel hagseub mic mun jang saengseong silheomeul tonghae guhyeondoen siseutemi malmungcirobuteo hagseubhan hangugeoyi teugjinge majneun munjangeul saengseonghaneun jireul salpyeobol geosida. bon nonmunyi guseongeun daeumgwa gatda. je2jangeseoneun sunhwan singyeongmang mic sunhwansingyeongmangeul jeogyonghan eoneo model gwanryeon yeongu deuleul salpyeobol geosigo, 3jangeseoneun sunhwansingyeongmangyi suhagjeog jeongyi mic hunryeon bangbeobe daehae ribyureul hal geosida. 4jangeseo neun bon yeongueseo guhyeonhan sunhwansingyeongmang giban eoneo modelyi gujoreul salpyeobol geosigo, 5jangeseoneun silje hangugeo deiteo reul hwalyonghayeo sunhwansingyeongmang giban eoneomodeleul sayonghan mun jang saengseong silheomeul jinhaenghan gyeolgwareul seolmyeonghan dwi majimageuro 6jangeseo gyeolroneul doculharyeo handa.