Entailment Relation Aware Paraphrase Generation

We introduce a new task of entailment relation aware paraphrase generation which aims at generating a paraphrase conforming to a given entailment relation (e.g. equivalent, forward entailing, or reverse entailing) with respect to a given input. We propose a reinforcement learning-based weakly-supervised paraphrasing system, ERAP, that can be trained using existing paraphrase and natural language inference (NLI) corpora without an explicit task-specific corpus. A combination of automated and human evaluations show that ERAP generates paraphrases conforming to the specified entailment relation and are of good quality as compared to the baselines and uncontrolled paraphrasing systems. Using ERAP for augmenting training data for downstream textual entailment task improves performance over an uncontrolled paraphrasing system, and introduces fewer training artifacts, indicating the benefit of explicit control during paraphrasing.


Introduction
Paraphrase is "an alternative surface form in the same language expressing the same semantic content as the original form" (Madnani and Dorr 2010). Although the logical definition of paraphrase requires strict semantic equivalence (or bi-directional entailment (Androutsopoulos and Malakasiotis 2010)) between a sequence and its paraphrase, data-driven paraphrasing accepts a broader definition of approximate semantic equivalence (Bhagat and Hovy 2013). Moreover, existing automatically curated paraphrase resources do not align with this logical definition. For instance, pivot-based paraphrasing rules extracted by Ganitkevitch, Van Durme, and Callison-Burch (2013) contain hypernym or hyponym pairs, e.g. due to variation in the discourse structure of translations, and unrelated pairs, e.g. due to misalignments or polysemy in the foreign language.
While this flexibility of approximate semantic equivalence allows for greater diversity in expressing a sequence, it comes at the cost of the ability to precisely control the semantic entailment relationship (henceforth "entailment relation") between a sequence and its paraphrase. This trade-off severely limits the applicability of paraphrasing systems or resources to a variety of downstream natural language understanding (NLU) tasks (e.g. machine translation, question Copyright © 2022, Association for the Advancement of Artificial Intelligence (www.aaai.org). All rights reserved. Figure 1: Entailment-unaware system might output approximately equivalent paraphrases. Label preserving augmentations generated using such system for textual entailment task can result in incorrect labels (red). Explicit entailment relation control in entailment-aware system helps in reducing such incorrectly labeled augmentations (green).
Existing works identify directionality (forward, reverse, bi-directional, or no implication) of paraphrase and inference rules (Bhagat, Pantel, and Hovy 2007), and add semantics (natural logic entailment relationships such as equivalence, forward or reverse entailment, etc.) to data-driven paraphrasing resources (Pavlick et al. 2015) leading to improvements in lexical expansion and proof-based RTE systems, respectively. However, entailment relation control in paraphrase generation is, to our knowledge, a relatively unexplored topic, despite its potential benefit to downstream applications (Madnani and Dorr 2010) such as Multi-Document Summarization (MDS) (or Information Retrieval (IR)) wherein having such a control could allow the MDS (or IR) system to choose either the more specific (reverse entailing) or general (forward entailing) sentence (or query) depending on the purpose of the summary (or user needs).
To address the lack of entailment relation control in para-phrasing systems, we introduce a new task of entailment relation aware paraphrase generation: given a sequence and an entailment relation, generate a paraphrase which conforms to the given entailment relation. We consider three entailment relations (controls) in the spirit of monotonicity calculus (Valencia 1991): (1) Equivalence (≡) refers to semantically equivalent paraphrases (e.g. synonyms) where input sequence entails its paraphrase and vice-versa; (2) Forward Entailment (⊏) refers to paraphrases that loose information from the input or generalizes it (e.g. hypernyms) i.e. input sequence entails its paraphrase; (3) Reverse Entailment (⊐) refers to paraphrases that add information to the input or makes it specific (e.g. hyponyms) i.e. input sequence is entailed by its paraphrase. The unavailability of paraphrase pairs annotated with such a relation makes it infeasible to directly train a sequence-to-sequence model for this task. Collecting such annotations for existing large paraphrase corpora such as ParaBank (Hu et al. 2019b) or ParaNMT  is expensive due to scale. We address this challenge in 3 ways: (1) by building a novel entailment relation oracle based on natural language inference task (NLI) (Bowman et al. 2015a;Williams, Nangia, and Bowman 2018) to obtain weak-supervision for entailment relation for existing paraphrase corpora; (2) by recasting an existing NLI dataset, SICK (Marelli et al. 2014), into a small supervised dataset for this task, and (3) by proposing Entailment Relation Aware Paraphraser (ERAP) which is a reinforcement learning based (RL-based) weakly-supervised system that can be trained only using existing paraphrase and NLI corpora, with or without weak-supervision for entailment relation.
Intrinsic and extrinsic evaluations show advantage of entailment relation aware (henceforth "entailment-aware") paraphrasing systems over entailment-unaware (standard uncontrolled paraphrase generation) counterparts. Intrinsic evaluation of ERAP (via a combination of automatic and human measures) on recasted SICK ( §3) dataset shows that generated paraphrases conform to the given entailment relation with high accuracy while maintaining good or improved paraphrase quality when compared against entailment-unaware baselines. Extrinsic data-augmentation experiments ( §5) on textual entailment task show that augmenting training sets using entailment-aware paraphrasing system leads to improved performance over entailmentunaware paraphrasing system, and makes it less susceptible to making incorrect predictions on adversarial examples.

Entailment Relation Aware Paraphraser
Task Definition Given a sequence of tokens X = [x 1 , . . . , x n ], and an entailment relation R ∈{Equivalence (≡), Forward Entailment (⊏), Reverse Entailment (⊐)}, we generate a paraphrase Y = [y 1 , . . . , y m ] such that the entailment relationship between X and Y is R.Ŷ is the generated paraphrase and Y is the reference paraphrase.
Neural paraphrasing systems (Prakash et al. 2016;Li et al. 2018) employ a supervised sequence-to-sequence model to generate paraphrases. However, building a supervised model for this task requires paraphrase pairs with entailment rela- Figure 2: ERAP: Generator takes in a sequence X, an entailment relation R, and outputs a paraphraseŶ .Ŷ is scored by various scorers in the evaluator and a combined score (known as reward) is sent back to train the generator. Hypothesis-only adversary is adversarially trained onŶ and predictions from the entailment relation consistency scorer. tion annotations. To address this, we propose an RL-based paraphrasing system ERAP which can be trained with existing paraphrase and NLI corpora without any additional annotations. ERAP (Figure 2) consists of a paraphrase generator ( §2.1) and an evaluator ( §2.2) comprising of various scorers to assess the quality of generated paraphrases for different aspects. Scores from the evaluator are combined ( §2.3) to provide feedback to the generator in the form of rewards. Employing RL allows us to explicitly optimize the generator over measures accounting for the quality of generated paraphrases, including the non-differentiable ones.

Paraphrase Generator
The generator is a transformer-based (Vaswani et al. 2017) sequence-to-sequence model which takes (X, R) and gener-atesŶ . We denote the generator as G(Ŷ |X,R; θ g ), where θ g refers to parameters of the generator. We incorporate the entailment relation as a special token prepended to the input sequence. This way, entailment relation receives special treatment (Kobus, Crego, and Senellart 2017) and the generator learns to generate paraphrases for a given X, and R.

Paraphrase Evaluator
The evaluator comprises of several scorers to asses the quality of the generated paraphrase for three aspects: semantic similarity with the input, expression diversity from the input, and entailment relation consistency. It provides rewards to the paraphrases generated by the generator as feedback which is used to update the parameters of the generator. We describe the various scorers below.
Semantic Similarity Scorer provides reward which encourages the generated paraphraseŶ to have similar meaning as the input sequence X. We use MoverScore (Zhao et al. 2019) to measure the semantic similarity between the generated paraphrase and the input, denoted as r s (X,Ŷ ). MoverScore combines contextualized representations with word mover's distance (Kusner et al. 2015) and has shown high correlation with human judgment of text quality.
Expression Diversity Scorer rewards the generated paraphrase to ensure that it uses different tokens or surface form to express the input. We measure this aspect by computing n-grams dissimilarity (inverse BLUE (Papineni et al. 2002)), Following Hu et al. (2019b), we use modified BLEU without length penalty to avoid generating short paraphrases which can result in high inverse BLEU scores.
Entailment Relation Consistency Scorer is a novel scorer designed to reward the generated paraphrase in such a way that encourages it to adhere to the given entailment relation R. To compute the reward, we build an oracle O(X, Y ) (details in §3) based on natural language inferencing (NLI) and use likelihood of the given entailment relation from the Oracle as the score. r l (X,Ŷ , R) = O(l = R|X,Ŷ ). As will be discussed further in §4.3, we found that entailment relation consistency scorer can result in generator learning simple heuristics (e.g. adding same adjective such as 'desert', or trailing tokens like 'and says' or 'with mexico' for ⊐ or short outputs for ⊏) leading to degenerate paraphrases having high consistency score. Inspired by the idea of hypothesis-only baselines (Poliak et al. 2018) for NLI task, we build a novel RoBERTa-based Hypothesis-only Adversary, A(l|Ŷ ), to penalize the generated paraphrases resorting to such heuristics. The adversary is a 3-class classifier trained on the paraphrases generated during the training phase with the oracle prediction for (X,Ŷ ) pair as the ground-truth. The Adversary loss L(A) is where |C| = 3 is the number of entailment relations. Training the adversary in this way helps in adapting to the heuristics taken by the generator during the course of training. The generator and the adversary are trained alternatively, similar to a GAN (Goodfellow et al. 2014) setup. The penalty is computed as the likelihood of entailment relation being R using the adversary, p l (Ŷ , R) = A(l = R|Ŷ ). We only penalize those generated paraphrases for which predicted relation is same as the input relation because incorrect prediction denotes no heuristic is taken by the generator.

Reinforcement Learning Setup
The output paraphrases from the generator are sent to the scorers for evaluation. The various scores from the scorers are combined to give feedback (in the form of reward) to the generator to update its parameters and to improve the quality of the generated paraphrases conforming to the given relation. We emphasize that although the scores from our scorers are not differentiable with respect to θ g , we can still use them by employing RL (the REINFORCE algorithm) to update the parameters of the generator (Williams 1992).
In RL paradigm, state at time t is defined as s t = (X, R,Ŷ 1∶t−1 ) whereŶ 1∶t−1 refers to the first t − 1 tokens that are already generated in the paraphrase. The action at time t is the t th token to be generated. Let V be the vocabulary, and T be the maximum output length. The total expected reward of the current generator is then given by where P(y t |s t ) is the likelihood of token y t given the current state s t , and Q(y t , s t ) is the cumulative discounted reward for a paraphrase extended fromŶ 1∶t−1 . The total reward, Q, is defined as the sum of the token level rewards.
where r(s τ , y τ ) is the reward of token y τ at state s τ , and γ ∈ (0, 1) is a discounting factor so that the future rewards have decreasing weights, since their estimates are less accurate. If we consider thatŶ 1∶t−1 has been given then for every y t , the total expected reward becomes Sequence Sampling To obtain r(s t , y t ) at each time step t, we need scores for each token. However, by design these scorers only evaluate complete sequences instead of single token or partial sequences. We therefore use the technique of rolling out (Yu et al. 2017), where the generator "rolls out" a given sub-sequenceŶ 1∶t to generate complete sequence by sampling the remaining part of the sequenceŶ t+1∶T . Following Gong et al. (2019), we use a combination of beam search and multinomial sampling to balance reward estimation accuracy at each time step and diversity of the generated sequence. We first generate a reference paraphraseŶ The combined score f (s t , y t ) for an action y t at state s t is computed by averaging the score of the complete sequences rolled out fromŶ where α, β, δ, and n are hyperparameters empirically set to 0.4, 0.4, 0.2, and 2, respectively. These parameters control the trade-off between different aspects for this multiobjective task. Following Siddique, Oymak, and Hristidis (2020), we threshold 1 the scorers' scores so that the final reward maintains a good balance across various scores. For example, generating diverse tokens at the expense of losing too much on the semantic similarity is not desirable. Similarly, copying the input sequence as-is to the generation is clearly not a paraphrase (i.e., r s (X,Ŷ ) = 1). We define reward r(s t , y t ) for action y t at state s t as The discounted cumulative reward Q(s t , y t ) is then computed from the rewards r(s τ , y τ ) at each time step using Eq. 3 and the total expected reward is derived using Eq. 4 The generator loss L(G) is defined as −J(G).

Training Details
Pre-training Pre-training has been shown to be critical for RL to work in unsupervised settings (Siddique, Oymak, and Hristidis 2020;Gong et al. 2019) therefore, we pre-train the generator on existing large paraphrase corpora e.g. ParaBank (Hu et al. 2019b) or ParaNMT (Wieting and Gimpel 2018) in two ways; (1) Entailment-aware uses Oracle ( §3) to obtain entailment relation for paraphrase pairs in the train set of paraphrase corpora, filter the semantically-divergent ( §3) pairs, upsample or downsample to have balanced data across relations, and train the generator with weak-supervision for entailment relation and goldparaphrases, and (2) Entailment-unaware trains the generator on paraphrase pairs as-is without any entailment relation. Pre-training is done in a supervised manner with the cross-entropy loss and offers immediate benefits for generator to learn paraphrasing transformations and have warmstart leading to faster model training.
RL-based Fine-tuning We fine-tune the generator using feedback from the evaluator on recasted SICK dataset (details in §3). For any practical purposes, our RL-finetuning approach only requires input sequences without any annotations for entailment relation or ground-truth paraphrases. However, for a fair comparison against supervised or weakly-supervised baselines ( §4.1), we use the goldentailment relation for recasted SICK during RL fine-tuning.

Collecting Labeled Paraphrase Data
Entailment-aware paraphrasing requires paraphrase pairs annotated with entailment relation. However, collecting entailment relation annotations for large size of paraphrase corpora such as ParaBank 2 (Hu et al. 2019b) is too costly. To obtain entailment relations automatically for available paraphrase pairs, we train a NLI classifier and use it to derive the entailment relations as described below. Entailment Relation Oracle NLI is a standard natural language understanding task of determining whether a hypothesis h is true (entailment 3 E), false (contradiction C), or undetermined (neutral N) given a premise p (MacCartney 2009). To build an entailment relation oracle, O(X, Y ), we first train a RoBERTa-based  3-class classifier, o(l|⟨p, h⟩), to predict the uni-directional (E, N, C) labels given a ⟨p, h⟩ pair. This classifier is then run forwards (⟨X, Y ⟩) and backwards (⟨Y, X⟩) on the paraphrase pairs to get the uni-directional predictions which are further used to derive the entailment relations as follows.
The Oracle serves two purposes: (1) generates weaksupervision for entailment relations for existing paraphrase corpora, and (2)

Intrinsic Evaluation
Here we provide details on the entailment-aware and unaware comparison models, and the evaluation measures.

Comparison Models
To contextualize ERAP's performance, we train several related models including supervised and weakly-supervised,  Train  1344  684  684  420  1274  2524  641  Dev  196  63  63  43  143  281  71  Test  1386  814  814  494  1404  2790  712   Table 1: E, N, C denote entailment, neutral, and contradiction, respectively. Others refers to neutral or invalid relation. entailment-aware and unaware models to obtain lower and upper bound performance on recasted SICK as follows: (1) the generator is trained on recasted SICK in an entailment-aware (Seq2seq-A) and unaware (Seq2seq-U) supervised setting; (2) the generator is pre-trained on Para-Bank dataset in entailment-aware (Pre-trained-A) and unaware (Pre-trained-U) setting to directly test on the test set of recasted SICK; (3) the pre-trained generators are fine-tuned on recasted SICK in entailment-aware (Finetuned-A) and unaware (Fine-tuned-U) supervised setting; (4) multiple outputs (k∈{1, 5, 10, 20}) are sampled using nucleus sampling (Holtzman et al. 2019) from Seq2seq-U (Re-rank-s2s-U) or Fine-tuned-U (Re-rank-FT-U) and reranked based on the combined score f (s t , y t ). The highest scoring output is considered as the final output for Re-rank-s2s-U and Re-rank-FT-U.

Evaluation Measures
Automatic evaluation to evaluate the quality of paraphrases is primarily done using iBLEU (

Results and Analysis
To use paraphrasing models for downstream tasks, we need to ensure that the generated paraphrases conform to the specified entailment relation and are of good quality.   Automatic evaluation We first evaluate the pre-trained generators on a held-out set from ParaBank containing 500 examples for each relation. Table 2 shows that the entailment-aware generator outperforms its unaware counterpart across all the measures. This boost is observed with weak-supervision for entailment relation demonstrating the good quality of weak-supervision. Next, We evaluate ERAP variants against the comparison models ( §4.1) on the recasted SICK test samples belonging to ≡, ⊏, ⊐ relation and report the results in Table 3 4 . Entailment-aware (-A) variants outperform corresponding unaware (-U) variants on iBLEU score, while outperforming the majority-class (i.e., ≡) copy-input baseline (except for Seq2seq-A). Weakly-supervised pre-training helps in boosting the performance in terms of iBLEU and R-Consistency score as evident from higher scores for Finetuned-A(U) model over Seq2seq-A(U). Poor performance of Seq2seq variants is because of the small dataset size and much harder multi-objective task. Re-ranking outputs from Fine-tuned-U achieve higher iBLEU and consistency score than Pre-trained-A which is explicitly trained with weaksupervision for relation. However, this comes at the computational cost of sampling multiple 5 (k=20) outputs. Improved performance of Fine-tuned models over Seq2seq indicates the importance of pre-training. Both the ERAP variants achieve higher iBLEU and consistency than its lowerbounding (Pre-trained) models but the outputs show less diversity in expression and make conservative lexical or  Table 4: Ablation of scorers in ERAP. Con, Sim, Div refers to relation consistency, semantic similarity, and expression diversity scorers. Underline denote more copying of input for Diversity score and presence of heuristics in outputs for R-Consistency score than gold-references.
syntactic changes. These results look encouraging until we notice Copy-input (last row) which achieves high BLEU and iBLEU, indicating that these metrics fail to punish against copying through the input (an observation consistent with Niu et al. (2020)).

Ablation analysis of each scorer
We demonstrate the effectiveness of each scorer in ERAP via an ablation study in Table 4. Using only consistency scorer for rewarding the generated paraphrases, a significant improvement in consistency score is observed as compared to Pre-trained-A and Gold-references. However, this high score may occur at the cost of semantic similarity (e.g. 1 in Table 5) wherein output conforms to ⊏ relation at the cost of losing much of the content. Adding similarity scorer, helps in retaining some of the content (higher BLEU and iBLEU) but results in copying (low diversity) from the input. Addition of diversity scorer helps in introducing diversity in expression. However, model is still prone to heuristics (e.g. losing most of the content from input (1 in Table 5), or adding irrelevant tokens 'with mexico' or 'desert' (2 in Table 5)) for ensuring high consistency score. Introducing Adversary reduces the heuristics learned by the generator. Together all the scorers help maintain a good balance for this multi-objective task.
Human evaluation We report the human evaluation for 25 test outputs each from 8 models for 4 measures in Table 6. ERAP-A achieves the highest consistency while maintaining a good balance between similarity, diversity and grammaticality. Re-rank-s2s-U has the highest diversity which comes at the cost of semantic similarity and grammaticality (e.g. 3 in Table 5). A strikingly different observation is high similarity and low diversity of Pre-trained variants, reinforcing the issues with existing automatic measures.

Extrinsic Evaluation
The intrinsic evaluations show that ERAP produces quality paraphrases while adhering to the specified entailment relation. Next, we examine the utility of entailment-aware paraphrasing models over unaware models for a downstream application, namely paraphrastic data augmentation for textual entailment task. Given two sentences, a premise p and a hypothesis h, the task of textual entailment is to determine if a human would infer h is true from p. Prior work has shown that paraphrastic augmentation of textual entailment datasets improve performance (Hu et al. 2019a); however, these ap-1. Qualitative outputs for ablation study Input ⊏ a shirtless man is escorting a horse that is pulling a carriage along a road Reference a shirtless man is leading a horse that is pulling a carriage Only Con a shirtless man escorts it. Con+Sim a shirtless man is escorting a horse. Con+Sim+Div a shirtless man escorts a horse.

ERAP-A
a shirtless person is escorting a horse who is dragging a carriage.

Example of heuristic learned w/o Hypothesis-only Adversary Input ⊐
a man and a woman are walking through a wooded area Reference a man and a woman are walking together through the woods -Adversary a desert man and a woman walk through a wooded area ERAP-A a man and a woman are walking down a path through a wooded area Input ⊐ four girls are doing backbends and playing outdoors Reference four kids are doing backbends in the park -Adversary four girls do backbends and play outside with mexico.

ERAP-A
four girls do backbends and play games outside.
3. Output from comparison models and ERAP Input ≡ a ribbon is being twirled by a girl in pink Reference a girl in pink is twirling a ribbon Seq2seq-U a girl is talking to a girl in pink Seq2seq-A a girl in blue is surfing on a pink Fine-tuned-U a girl in pink is turning a tape Fine-tuned-A a girl in pink is circling the ribbon. Re-rank-U a girl is talking to a girl in a pink built Re-rank-A a girl in a pink jacket is swinging a ribbon Pre-trained-U a girl in pink is circling the ribbon. Pre-trained-A the ribbon is being twisted by a girl in pink ERAP-U the ribbon is being twisted by the girl in the pink ERAP-A a girl in pink is twirling a ribbon.  proaches make the simplifying assumption that entailment relations are preserved under paraphrase, which is not always the case (see Figure 1 and 30% of ParaBank pairs were found to be semantically-divergent using Oracle). We use SICK NLI dataset for this task because we have a paraphrasing system trained on similar data distribution 6 . We hypothesize that entailment-aware augmentations will result in fewer label violations, and thus overall improved performance on the textual entailment task. Moreover, explicit control over the entailment relation allows for greater variety of augmentations that can be generated (an exhaustive list of label preserving augmentations based on entailment relation between a p (or h) and its paraphrase is pre- sented in Table 7) with entailment-aware models. Paraphrastic Data Augmentation We generate paraphrases for all premises p ∈ P and hypotheses h ∈ H present in the train set of SICK NLI using entailmentaware and unaware models. We obtain augmentation data by combining all the paraphrases (generated using entailmentaware models) with original data and label them as per   Table 8. As per our hypothesis, models trained with augmentations generated using entailmentaware models result in improved accuracy on both original as well as adversarial test samples over those trained with entailment-unaware augmentations. Textual entailment model trained only on SICK NLI data performs the best on adversarial test set as expected and proves that although augmentation helps in boosting the performance of a model, it introduces augmentation artifacts during training.

Related Work
Paraphrase generation is a common NLP task with widespread applications. Earlier approaches are rulebased (Barzilay, McKeown, and Elhadad 1999;Ellsworth and Janin 2007) or data-driven (Madnani and Dorr 2010). Recent, supervised deep learning approaches use LSTMs (Prakash et al. 2016), VAEs (Gupta et al. 2018), pointer-generator networks (See, Liu, and Manning 2017), and transformer-based  sequence-to-sequence models. Li et al. (2018) use RL for supervised paraphrasing. Unsupervised paraphrasing is a challenging and emerging NLP task with limited efforts. Bowman et al. (2015b) train VAE to sample less controllable paraphrases. Others use metropolis-hastings (Miao et al. 2019), simulated annealing  or dynamic-blocking (Niu et al. 2020) to add constraints to the decoder at test time. Siddique, Oymak, and Hristidis (2020) use RL to maximize expected reward based on adequacy, fluency and diversity. Our RLbased approach draws inspiration from this work by introducing oracle and hypothesis-only adversary.
Controllable text generation is a closely related field with efforts been made to add lexical (Hu et al. 2019a;Garg et al. 2021) or syntactic control (Iyyer et al. 2018;Chen et al. 2019;Goyal and Durrett 2020) to improve diversity of paraphrases. However, ours is the first work which introduces a semantic control for paraphrase generation. Style transfer is a related field that aims at transforming an input to adhere to a specified target attribute (e.g. sentiment, formality). RL has been used to explicitly reward the output to adhere to a target attribute (Gong et al. 2019;Sancheti et al. 2020;Luo et al. 2019;Liu, Neubig, and Wieting 2020;Goyal et al. 2021). The target attributes are only a function of the output and defined at a lexical level. However, we consider a relation control which is a function of both the input and the output, and is defined at a semantic level.

Conclusion
We introduce a new task of entailment-relation-aware paraphrase generation and propose a RL-based weaklysupervised model (ERAP) that can be trained without a taskspecific corpus. Additionally, an existing NLI corpora is recasted to curate a small annotated dataset for this task, and provide performance bounds for it. A novel Oracle is proposed to obtain weak-supervision for relation control for existing paraphrase corpora. ERAP is shown to generate paraphrases conforming to the specified relation while maintaining quality of the paraphrase. Intrinsic and Extrinsic experiments demonstrate the utility of entailment-relation control, indicating a fruitful direction for future research.

A.5 Inference
We use beam search with beam width of 5, minimum and maximum output length set at 5 and 40, respectively. All the models were trained and tested on 8 NVIDIA Tesla V100 SXM2 32GB GPU machine.

B Recasting SICK
We use examples ∈{S1aS2a, S1aS2b, S1bS2a, S1bS2b, S1aS1b} meaning preserving transformations. In case of bidirectional entailing examples, we double the data by using both original as well as reverse pair and label it as ≡. In case of entailment from only premise to hypothesis, we add this sample with ⊏ relation and reverse sample with ⊐ relation.

C.1 Re-ranking Baseline Results
Re-rank baselines are built over Seq2seq-U (Fine-tuned-U) resulting in Re-rank-s2s-U (Re-rank-FT-U). Seq2seq-U (or Fine-tuned-U) is used to generate multiple outputs (k∈{1, 5, 10, 20}) for an input using nucleus sampling (Holtzman et al. 2019) and the outputs are scored with the scorers for a desired entailment relation to obtain a combined score f (s t , y t ) for each output. Highest scoring output is considered as the final output. Re-ranking results for k ∈ {1, 5, 10} are reported in Table 9.  Table 10: Automatic evaluation of paraphrases from ERAP against entailment-aware (A) and unaware (U) models. R-Consistency is measured only for models conditioned (R-Test) on R at test time. Shaded rows denote upper-and lower-bound models. ⋆ denotes that only pre-training is done in entailment-unaware setting. Bold-face denote best in each block and * denote best overall.