We address the problem of open-set authorship verification, a classification task that consists of attributing texts of unknown authorship to a given author when the unknown documents in the test set are excluded from the training set. We present an end-to-end model-building process that is universally applicable to a wide variety of corpora with little to no modification or fine-tuning. It relies on transfer learning of a deep language model and uses a generative adversarial network and a number of text augmentation techniques to improve the model's generalization ability. The language model encodes documents of known and unknown authorship into a domain-invariant space, aligning document pairs as input to the classifier, while keeping them separate. The resulting embeddings are used to train to an ensemble of recurrent and quasi-recurrent neural networks. The entire pipeline is bidirectional; forward and backward pass results are averaged. We perform experiments on four traditional authorship verification datasets, a collection of machine learning papers mined from the web, and a large Amazon-Reviews dataset. Experimental results surpass baseline and current state-of-the-art techniques, validating the proposed approach. We investigate the applicability of transfer learning techniques to Authorship Verification (AV) problems, and propose a a method that uses some of the most recent advances in deep learning to achieve state of the art results on a variety of datasets. AV seeks to determine whether two or more text documents have been written by the same author. Some applications of AV include plagiarism analysis, sock-puppet detection, blackmailing, and email spoofing prevention BID7 . Traditionally, studies on AV consider a closed and limited set of authors, and a closed set of documents written by such authors. During the training step, some of these documents (sometimes as long as a novel) are used. The goal can be formulated as to successfully identify whether the authors of a pair of documents are identical BID14 BID19 BID11 . This type of AV tasks assumes access to the writing samples of all possible authors during the training step, which is not realistic. Recently, the AV problem has changed to reflect realistic -and more challenging-scenarios. The goal is no longer to individually learn the writing style of the authors (like in traditional AV methods), but to learn what differentiates two different authors within a corpus. This task involves predicting authorship of documents that may not have been previously encountered within the training set; in fact, the presence of the authors in the training data is not guaranteed either. That is, the test set may contain out of training sample data; given a set of authors of unknown papers contained within the training data, A unknown train , and a set of authors of unknown papers in the test data, A unknown test , it is neither unreasonable nor unexpected to find that A unknown train ∩A unknown test = ∅. Some other challenges arise in modern AV tasks, making authorship verification of a given pair of documents hard to infer. One is the lack of training data, which can manifest itself in any one or more of the following: the training set may be small, samples of available writings may be limited, or the length of the given documents may be insufficient. Another is the test and train documents belonging to different genre and/or topics, both within their respective sets as well as between the train and the test set -implying they were likely drawn from different distributions. The challenge is to ensure robustness in a multitude of possible scenarios. Regardless of the AV problem specifics, generally we assume a training dataset made of sets of triples: DISPLAYFORM0 with x i X known , x j X unknown a realization from random variables X known and X unknown , and the label y i,j Y is drawn from a random variable Y , producing a total of P sets of realizations, each potentially by a different author, thus forming up to P source domains, because it can be argued that a collection of literary works by one author forms a latent domain of it's own. The goal is to learn a prediction function f : X → Y that can generalize well and make accurate predictions regarding documents written by authors both inside and outside of the training set, even if those documents were not seen in training. Less formally, in AV the task is composed of multiple sub-problems: for each given sub-set of texts, we are provided one or more documents that need to be verified and one or more that are known to be of identical authorship. We approach the AV problem by designing a straightforward deep document classification model that relies on transfer learning a deep language model, ensembles, an adversary, differential learning rates, and data augmentation. In order to ensure the design's versatility and robustness, we perform authorship verification on a collection of datasets that have little in common in terms of size, distribution, origins, and manner they were designed. For evaluation, we consider standard AV corpora with minimal amount of training data, PAN-2013 BID12 , PAN-2014E and PAN-2014N BID27 , PAN-2015 BID28 , a collection of scientific papers mined from the web BID2 , and Amazon Reviews dataset BID8 . The proposed approach performs well in all scenarios with no specific modifications and minimal fine-tuning, defeating all baselines, PAN competition winners, as well as the recent Transformation Encoder and PRNN models that were recently shown to perform well on AV tasks. BID8 . Authorship verification has always been a challenging problem. It can be even more difficult when no writing samples of questioned author/authors is given. In this paper, we explore the possibility of a more general approach to the problem, one that does not rely on having most of the authors within the training set. To this end, we use transfer and adversarial learning learning, data augmentation, ensemble methods, and cutting edge developments in training deep models to produce an architecture that is to the best of our knowledge novel at least to problem setting. Our design exhibits a high degree of robustness and stability when dealing with out-of-sample (previously unseen) authors and lack of training data and delivers state-of-the-art performance.
We propose and end-to-end model-building process that is universally applicable to a wide variety of authorship verification corpora and outperforms state-of-the-art with little to no modification or fine-tuning.
{"src": "scitldr-AIC_train", "paper_id": "BkgdPnjQ84"}