In many domains, especially enterprise text analysis, there is an abundance of data which can be used for the development of new AI-powered intelligent experiences to improve people's productivity. However, there are strong-guarantees of privacy which prevent broad sampling and  labeling of personal text data to learn or evaluate models of interest. Fortunately, in some cases like enterprise email, manual annotation is possible on certain public datasets. The hope is that models trained on these public datasets would perform well on the target private datasets of interest. In this paper, we study the challenges of transferring information from one email dataset to another, for predicting user intent. In particular, we present approaches to characterizing the transfer gap in text corpora from both an intrinsic and extrinsic point-of-view, and evaluate several proposed methods in the literature for bridging this gap. We conclude with raising issues for further discussion in this arena. Using publicly available text data to train predictive models for use in privacy-aware enterprise settings is a very fruitful direction in the area of document understanding. However, when the labeled training dataset (source domain) is different from the unlabeled test (target domain), the two datasets likely follow different distributions. This application setting violates the i.i.d. assumption made by classic supervised learning methods and calls for domain adaptation techniques to properly account for this difference. State of the art domain adaptation techniques are generally developed and evaluated using a limited number of benchmark datasets and under constrained settings. The extent to which these methods are applicable for predictive settings over enterprise text data has neither been explored nor characterized in detail. To explore the effectiveness of state of the art domain adaptation methodology in enterprise text data, we focus on communication intent prediction in enterprise email. In particular, we use two public enterprise email datasets (Avocado, an IT company, and Enron, an oil company) to systematically analyze the transfer problem in enterprise email. The two intent prediction tasks that we focus on in this study are Meeting Intent (the email expresses an intent to meet with the recipient) and Commitment Intent (the email expresses an action the sender intends to take in the future) -both of which are binary classification tasks.
Insights on the domain adaptation challenge, when predicting user intent in enterprise email.
{"src": "scitldr-AIC_train", "paper_id": "BJgAGp5qLH"}