This paper presents an effective method for generating natural language sentences from their underlying meaning representations . The method is built on top of a hybrid tree representation that jointly encodes both the meaning representation as well as the natural language in a tree structure . By using a tree conditional random field on top of the hybrid tree representation , we are able to explicitly model phrase-level dependencies amongst neighboring natural language phrases and meaning representation components in a simple and natural way . We show that the additional dependencies captured by the tree conditional random field allows it to perform better than directly inverting a previously developed hybrid tree semantic parser . Furthermore , we demonstrate that the model performs better than a previous state-of-the-art natural language generation model . Experiments are performed on two benchmark corpora with standard automatic evaluation metrics . One of the ultimate goals in the field of natural language processing ( NLP ) is to enable computers to converse with humans through human languages . To achieve this goal , two important issues need to be studied . First , it is important for computers to capture the meaning of a natural language sentence in a meaning representation . Second , computers should be able to produce a humanunderstandable natural language sentence from its meaning representation . These two tasks are referred to as semantic parsing and natural language generation ( NLG ) , respectively . In this paper , we use corpus-based statistical methods for constructing a natural language generation system . Given a set of pairs , where each pair consists of a natural language ( NL ) sentence and its formal meaning representation ( MR ) , a learning method induces an algorithm that can be used for performing language generation from other previously unseen meaning representations . A crucial question in any natural language processing system is the representation used . Meaning representations can be in the form of a tree structure . In Lu et al . ( 2008 ) , we introduced a hybrid tree framework together with a probabilistic generative model to tackle semantic parsing , where tree structured meaning representations are used . The hybrid tree gives a natural joint tree representation of a natural language sentence and its meaning representation . A joint generative model for natural language and its meaning representation , such as that used in Lu et al . ( 2008 ) has several advantages over various previous approaches designed for semantic parsing . First , unlike most previous approaches , the generative approach models a simultaneous generation process for both NL and MR . One elegant property of such a joint generative model is that it allows the modeling of both semantic parsing and natural language generation within the same process . Second , the generative process proceeds as a recursive top-down Markov process in a way that takes advantage of the tree structure of the MR . The hybrid tree generative model proposed in Lu et al . ( 2008 ) was shown to give stateof-the-art accuracy in semantic parsing on benchmark corpora . While semantic parsing with hybrid trees has been studied in Lu et al . ( 2008 ) , its inverse task -NLG with hybrid trees -has not yet been explored . We believe that the properties that make the hybrid trees effective for semantic parsing also make them effective for NLG . In this paper , we develop systems for the generation task by building on top of the generative model introduced in Lu et al . ( 2008 ) ( referred to as the LNLZ08 system ) . We first present a baseline model by directly " inverting " the LNLZ08 system , where an NL sentence is generated word by word . We call this model the direct inversion model . This model is unable to model some long range global dependencies over the entire NL sentence to be generated . To tackle several weaknesses exhibited by the baseline model , we next introduce an alternative , novel model that performs generation at the phrase level . Motivated by conditional random fields ( CRF ) ( Lafferty et al . , 2001 ) , a different parameterization of the conditional probability of the hybrid tree that enables the model to encode some longer range dependencies amongst phrases and MRs is used . This novel model is referred to as the tree CRF-based model . Evaluation results for both models are presented , through which we demonstrate that the tree CRF-based model performs better than the direct inversion model . We also compare the tree CRFbased model against the previous state-of-the-art model of Wong and Mooney ( 2007 ) . Furthermore , we evaluate our model on a dataset annotated with several natural languages other than English ( Japanese , Spanish , and Turkish ) . Evaluation results show that our proposed tree CRF-based model outperforms the previous model . In this paper , we presented two novel models for the task of generating natural language sentences from given meaning representations , under a hybrid tree framework . We first built a simple direct inversion model as a baseline . Next , to address the limitations associated with the direct inversion model , a tree CRF-based model was introduced . We evaluated both models on standard benchmark corpora . Evaluation results show that the tree CRF-based model performs better than the direct inversion model , and that the tree CRF-based model also outperforms WASP -1 + + , which was a previous state-of-the-art system reported in the literature .
While hybrid trees are shown to be effective for semantic parsing, their application for text generation is under explored.
They propose a phrase-level tree conditional random field that uses a hybrid tree of a meaning representation for the text generation model.
Experiments in four languages with automatic evaluation metrics show that the proposed conditional random field-based model outperforms the previous state-of-the-art system.