We examine a methodology using neural language models ( LMs ) for analyzing the word order of language . This LM-based method has the potential to overcome the difficulties existing methods face , such as the propagation of preprocessor errors in count-based methods . In this study , we explore whether the LMbased method is valid for analyzing the word order . As a case study , this study focuses on Japanese due to its complex and flexible word order . To validate the LM-based method , we test ( i ) parallels between LMs and human word order preference , and ( ii ) consistency of the results obtained using the LM-based method with previous linguistic studies . Through our experiments , we tentatively conclude that LMs display sufficient word order knowledge for usage as an analysis tool . Finally , using the LMbased method , we demonstrate the relationship between the canonical word order and topicalization , which had yet to be analyzed by largescale experiments . Model Verbs whose type-A examples prefer the ACC-DAT order Verbs whose type-B examples prefer the ACC-DAT order CLM " 預ける " ( deposit ) , " 置く " ( place ) , " 持つ " ( have ) , " 入 れる " ( put in ) , " 納める " ( pay ) , " 郵送 " ( mail ) , " 供 給 " ( supply ) , " 出す " ( put out ) , " 運ぶ " ( transport ) , " 流 す " ( shed ) , " 掛ける " ( hang ) , " 飾る " ( decorate ) , " 広 げる " ( spread ) , " 移す " ( transfer ) , " 残す " ( leave ) , " 配 送 " ( deliver ) , " 送る " ( send ) , " 投げる " ( throw ) , " 送 付 " ( send ) , " 返却 " ( return ) , " 届ける " ( send ) , " 戻す " ( return ) , " 着ける " ( wear ) , " 上げる " ( increase ) , " 落と す " ( drop ) , " 載せる " ( publish ) , " 変更 " ( change ) , " 納 入 " ( deliver ) , " 卸す " ( unload ) , " 掲載 " ( publish ) , " 通 す " ( get X through ) " 配 布 " ( distribute ) , " 渡 す " ( pass ) , " プ レ ゼ ン ト " ( present ) , " 合わせる " ( match ) , " 見せる " ( show ) , " 提 供 " ( offer ) , " 与える " ( give ) , " 当てる " ( hit ) , " 回す " ( turn ) , " 追加 " ( add ) , " 貸す " ( lend ) , " 展示 " ( exhibit ) , " 据える " ( lay ) , " 依頼 " ( request ) , " 挿入 " ( insert ) , " 纏 める " ( collect ) , " 請求 " ( claim ) SLM " 預ける " ( deposit ) , " 置く " ( place ) , " 頼む " ( ask ) , " 入 れる " ( put in ) , " 納める " ( pay ) , " 郵送 " ( mail ) , " 出 す " ( put out ) , " 運ぶ " ( transport ) , " 流す " ( shed ) , " 掛け る " ( hang ) , " 広げる " ( spread ) , " 移す " ( transfer ) , " 残 す " ( leave ) , " リクエスト " ( request ) , " 配送 " ( deliver ) , " 送る " ( send ) , " 投げる " ( throw ) , " 送付 " ( send ) , " 求 める " ( ask ) , " 提出 " ( submit ) , " 届ける " ( deliver ) , " 要 求 " ( request ) , " 戻す " ( return ) , " 寄付 " ( donate ) , " 寄贈 " ( donation ) , " 着ける " ( wear ) , " 乗せる " ( place ) , " 上げ る " ( increase ) , " 落とす " ( drop ) , " 貼る " ( stick ) , " 分け る " ( divide ) , " ばらまく " ( scatter ) , " はめる " ( fit ) , " 支 払う " ( pay ) , " 配達 " ( deliver ) , " 卸す " ( unload ) , " 纏め る " ( collect ) , " 通す " ( get X through ) " プレゼント " ( present ) , " 持つ " ( have ) , " 合わせる " ( match ) , " 見せる " ( show ) , " 向ける " ( point ) , " 提供 " ( offer ) , " 装備 " ( equip ) , " 追加 " ( add ) , " 展示 " ( exhibit ) , " 据える " ( lay ) , " 採用 " ( adopt ) Speakers sometimes have a range of options for word order in conveying a similar meaning . A typical case in English is dative alternation : ( 1 ) a. A teacher gave a student a book . b. A teacher gave a book to a student . Even for such a particular alternation , several studies ( Bresnan et al . , 2007 ; Hovav and Levin , 2008 ; Colleman , 2009 ) investigated the factors determining this word order and found that the choice is not random . For analyzing such linguistic phenomena , linguists repeat the cycle of constructing hypotheses and testing their validity , usually through psychological experiments or count-based methods . However , these approaches sometimes face difficulties , such as scalability issues in psychological experiments and the propagation of preprocessor errors in count-based methods . Compared to the typical approaches for evaluating linguistic hypotheses , approaches using LMs have potential advantages ( Section 3.2 ) . In this study , we examine the methodology of using LMs for analyzing word order ( Figure 1 ) . To validate the LM-based method , we first examine if there is a parallel between canonical word order and generation probability of LMs for each word order . Futrell and Levy ( 2019 ) reported that English LMs have human-like word order preferences , which can be one piece of evidence for validating the LM-based method . However , it is not clear whether the above assumption is valid even in languages with more flexible word order . In this study , we specifically focus on the Japanese language due to its complex and flexible word order . There are many claims on the canonical word order of Japanese , and it has attracted considerable attention from linguists and natural language processing ( NLP ) researchers for decades ( Hoji , 1985 ; Saeki , 1998 ; Miyamoto , 2002 ; Matsuoka , 2003 ; Koizumi and Tamaoka , 2004 ; Nakamoto et al . , 2006 ; Shigenaga , 2014 ; Sasano and Okumura , 2016 ; Orita , 2017 ; Asahara et al . , 2018 ) . We investigated the validity of using Japanese LMs for canonical word order analysis by conducting two sets of experiments : ( i ) comparing word order preference in LMs to that in Japanese speakers ( Section 4 ) , and ( ii ) checking the consistency Topic Time Location Subject ( Adverb ) Indirect object Direct object Verb Notation TOP TIM LOC NOM -DAT ACC -Typical particle " は " ( wa ) " に " ( ni ) " で " ( de ) " が " ( ga ) -"に " ( ni ) " を " ( o ) -Related section 6 5.2 5.2 5.2 5.3 5.1 5.1 5.1 Table 1 : Overview of the typical cases in Japanese , their typical particles , and the sections where the corresponding case is analyzed . The well-known canonical word order of Japanese is listed from left to right . between the preference of LMs with previous linguistic studies ( Section 5 ) . From our experiments , we tentatively conclude that LMs display sufficient word order knowledge for usage as an analysis tool , and further explore potential applications . Finally , we analyzed the relationship between topicalization and word order of Japanese by taking advantage of the LM-based method ( Section 6 ) . In summary , we : • Discuss and validate the use of LMs as a tool for word order analysis as well as investigate the sensitivity of LMs against different word orders in non-European language ( Section 3 ) ; • Find encouraging parallels between the results obtained with the LM-based method and those with the previously established method on various hypotheses of canonical word order of Japanese ( Sections 4 and 5 ) ; and • Showcase the advantages of an LM-based method through analyzing linguistic phenomena that is difficult to explore with the previous data-driven methods ( Section 6 ) . We have proposed to use LMs as a tool for analyzing word order in Japanese . Our experimental results support the validity of using Japanese LMs for canonical word order analysis , which has the potential to broaden the possibilities of linguistic research . From an engineering view , this study supports the use of LMs for scoring Japanese word order automatically . From the viewpoint of the linguistic field , we provide additional empirical evidence to various word order hypotheses as well as demonstrate the validity of the LM-based method . We plan to further explore the capability of LMs on other linguistic phenomena related to word order , such as " given new ordering " ( Nakagawa , 2016 ; Asahara et al . , 2018 ) . Since LMs are language-agnostic , analyzing word order in another language with the LM-based method would also be an interesting direction to investigate . Furthermore , we would like to extend a comparison between machine and human language processing beyond the perspective of word order .
Linguistical approaches to analyze word order phenomena have scalability and preprocessor error propagation problems, and the use of language models is limited in English.
They validate language models as a tool to study word order in Japanese by examining the relationship between canonical word order and generation probability.
They show that language models have sufficient word order knowledge in Japanese to be used as a tool for linguists.