Semantic parsing which maps a natural language sentence into a formal machine-readable representation of its meaning, is highly constrained by the limited annotated training data. Inspired by the idea of coarse-to-fine, we propose a general-to-detailed neural network(GDNN) by incorporating cross-domain sketch(CDS) among utterances and their logic forms. For utterances in different domains, the General Network will extract CDS using an encoder-decoder model in a multi-task learning setup. Then for some utterances in a specific domain, the Detailed Network will generate the detailed target parts using sequence-to-sequence architecture with advanced attention to both utterance and generated CDS. Our experiments show that compared to direct multi-task learning, CDS has improved the performance in semantic parsing task which converts users' requests into meaning representation language(MRL). We also use experiments to illustrate that CDS works by adding some constraints to the target decoding process, which further proves the effectiveness and rationality of CDS. Recently many natural language processing (NLP) tasks based on the neural network have shown promising results and gained much attention because these studies are purely data-driven without linguistic prior knowledge. Semantic parsing task which maps a natural language sentence into a machine-readable representation BID6 ), as a particular translation task, can be treated as a sequence-to-sequence problem BID3 ). Lately, a compositional graph-based semantic meaning representation language (MRL) has been introduced BID14 ), which converts utterance into logic form (action-object-attribute), increasing the ability to represent complex requests. This work is based on MRL format for semantic parsing task.Semantic parsing highly depends on the amount of annotated data and it is hard to annotate the data in logic forms such as Alexa MRL. Several researchers have focused on the area of multi-task learning and transfer learning BID10 , BID6 , BID15 ) with the observation that while these tasks differ in their domains and forms, the structure of language composition repeats across domains BID12 ). Compared to the model trained on a single domain only, a multi-task model that shares information across domains can improve both performance and generalization. However, there is still a lack of interpretations why the multi-task learning setting works BID26 ) and what the tasks have shared. Some NLP studies around language modeling BID18 , BID29 , BID2 ) indicate that implicit commonalities of the sentences including syntax and morphology exist and can share among domains, but these commonalities have not been fully discussed and quantified.To address this problem, in this work, compared to multi-task learning mentioned above which directly use neural networks to learn shared features in an implicit way, we try to define these cross-domain commonalities explicitly as cross-domain sketch (CDS). E.g., Search weather in 10 days in domain Weather and Find schedule for films at night in domain ScreeningEvent both have action SearchAction and Attribute time, so that they share a same MRL structure like SearchAction(Type(time@?)), where Type indicates domain and ? indicates attribute value which is copying from the original utterance. We extract this domain general MRL structure as CDS. Inspired by the research of coarse-to-fine BID4 ), we construct a two-level encoder-decoder by using CDS as a middle coarse layer. We firstly use General Network to get the CDS for every utterance in all domains. Then for a single specific domain, based on both utterance and extracted CDS, we decode the final target with advanced attention while CDS can be seen as adding some constraints to this process. The first utterance-CDS process can be regarded as a multi-task learning setup since it is suitable for all utterances across the domains. This work mainly introducing CDS using multi-task learning has some contributions listed below: 1) We make an assumption that there exist cross-domain commonalities including syntactic and phrasal similarity for utterances and extract these commonalities as cross-domain sketch (CDS) which for our knowledge is the first time. We then define CDS on two different levels (action-level and attribute-level) trying to seek the most appropriate definition of CDS.2) We propose a general-to-detailed neural network by incorporating CDS as a middle coarse layer. CDS is not only a high-level extraction of commonalities across all the domains, but also a prior information for fine process helping the final decoding.3) Since CDS is cross-domain, our first-level network General Network which encodes the utterance and decodes CDS can be seen as a multi-task learning setup, capturing the commonalities among utterances expressions from different domains which is exactly the goal of multi-task learning. In this paper, we propose the concept of cross-domain sketch (CDS) which extracts some shared information across domains, trying to fully utilize the cross-domain commonalities such as syntactic and phrasal similarity in human expressions. We try to define CDS on two levels and give some examples to illustrate our idea. We also present a general-to-detailed neural network (GDNN) for converting an utterance into a logic form based on meaning representation language (MRL) form. The general network, which is meant to extract cross-domain commonalities, uses an encoderdecoder model to obtain CDS in a multi-task setup. Then the detailed network generates the final domain-specific target by exploiting utterance and CDS simultaneously via attention mechanism. Our experiments demonstrate the effectiveness of CDS and multi-task learning.CDS is able to generalize over a wide range of tasks since it is an extraction to language expressions. Therefore, in the future, we would like to perfect the CDS definition and extend its' ontology to other domains and tasks. Besides, in this paper, we use attention mechanism to make use of CDS which is still a indirect way. We would like to explore more effective ways such as constraint decoding to further enhance the role of CDS.
General-to-detailed neural network(GDNN)  with Multi-Task Learning by incorporating cross-domain sketch(CDS) for semantic parsing
{"src": "scitldr-AIC_train", "paper_id": "r1fO8oC9Y7"}