We propose a new framework for entity and event extraction based on generative adversarial imitation learning -- an inverse reinforcement learning method using generative adversarial network (GAN). We assume that instances and labels yield to various extents of difficulty and the gains and penalties (rewards) are expected to be diverse. We utilize discriminators to estimate proper rewards according to the difference between the labels committed by ground-truth (expert) and the extractor (agent).   Experiments also demonstrate that the proposed framework outperforms state-of-the-art methods. Event extraction (EE) is a crucial information extraction (IE) task that focuses on extracting structured information (i.e., a structure of event trigger and arguments, "what is happening", and "who or what is involved ") from unstructured texts. In most recent five years, many event extraction approaches have brought forth encouraging results by retrieving additional related text documents BID18 , introducing rich features of multiple categories [Li et al., 2013 BID26 , incorporating relevant information within or beyond context BID23 , Judea and Strube, 2016 BID24 BID7 and adopting neural network frameworks BID4 , Nguyen and Grishman, 2015 BID8 , Nguyen et al., 2016 BID8 , Nguyen and Grishman, 2018 BID17 , Huang et al., 2018 BID13 BID27 .There are still challenging cases: for example, in the following sentences: "Masih's alleged comments of blasphemy are punishable by death under Pakistan Penal Code" and "Scott is charged with first-degree homicide for the death of an infant.", the word death can trigger an Execute event in the former sentence and a Die event in the latter one. With similar local information (word embeddings) or contextual features (both sentences include legal events), supervised models pursue the probability distribution which resembles that in the training set (in ACE2005 data, we have overwhelmingly more Die annotation on death than Execute), and will label both as Die event, causing error in the former instance.Such mistake is due to the lack of a mechanism that explicitly deals with wrong and confusing labels. Many multi-classification approaches utilize cross-entropy loss, which aims at boosting the probability of the correct labels. Many approaches -including AdaBoost which focuses weights on difficult cases -usually treat wrong labels equally and merely inhibits them indirectly. Models are trained to capture features and weights to pursue correct labels, but will become vulnerable and unable to avoid mistakes when facing ambiguous instances, where the probabilities of the confusing and wrong labels are not sufficiently "suppressed". Therefore , exploring information from wrong labels is a key to make the models robust.In this paper, we propose a dynamic mechanism -inverse reinforcement learning -to directly assess correct and wrong labels on instances in entity and event extraction. We assign explicit scores on cases -or rewards in terms of Reinforcement Learning (RL). We adopt discriminators from generative adversarial networks (GAN) to estimate the reward values. Discriminators ensures the highest reward for ground-truth (expert) and the extractor attempts to imitate the expert by pursuing highest rewards. For challenging cases, if the extractor continues selecting wrong labels, the GAN keeps expanding the margins between rewards for ground-truth labels and (wrong) extractor labels and eventually deviates the extractor from wrong labels.The main contributions of this paper can be summarized as follows: • We apply reinforcement learning framework to event extraction tasks, and the proposed framework is an end-to-end and pipelined approach that extracts entities and event triggers and determines the argument roles for detected entities.• With inverse reinforcement learning propelled by GAN, we demonstrate that a dynamic reward function ensures more optimal performance in a complicated RL task. In this paper, we propose an end-to-end entity and event extraction framework based on inverse reinforcement learning. Experiments have demonstrated that the performance benefits from dynamic reward values estimated from discriminators in GAN, and we also demonstrate the performance of recent embedding work in the experiments. In the future, besides releasing the source code, we also plan to further visualize the reward values and attempt to interpret these rewards so that researchers and event extraction system developers are able to better understand and explore the algorithm and remaining challenges. Our future work also includes using cutting edge approaches such as BERT BID6 , and exploring joint model in order to alleviate impact from upstream errors in current pipelined framework.
We use dynamic rewards to train event extractors.
{"src": "scitldr-AIC_train", "paper_id": "BJlsVZ966m"}