KEML: A Knowledge-Enriched Meta-Learning Framework for Lexical Relation Classification

Lexical relations describe how concepts are semantically related, in the form of relation triples. The accurate prediction of lexical relations between concepts is challenging, due to the sparsity of patterns indicating the existence of such relations. We propose the Knowledge-Enriched Meta-Learning (KEML) framework to address the task of lexical relation classification. In KEML, the LKB-BERT (Lexical Knowledge Base-BERT) model is presented to learn concept representations from massive text corpora, with rich lexical knowledge injected by distant supervision. A probabilistic distribution of auxiliary tasks is defined to increase the model's ability to recognize different types of lexical relations. We further combine a meta-learning process over the auxiliary task distribution and supervised learning to train the neural lexical relation classifier. Experiments over multiple datasets show that KEML outperforms state-of-the-art methods.


Introduction
As an important type of linguistic resources, lexical relations describe semantic associations between concepts. Such resources are organized as backbones in lexicons (Miller, 1995), semantic networks (Speer et al., 2017), etc. The explicit usage of such resources has benefited a variety of NLP tasks, including relation extraction (Shen et al., 2018), question answering (Yang et al., 2017) and machine translation (Thompson et al., 2019).
To accumulate such knowledge, Lexical Relation Classification (LRC) is a basic NLP task to classify concepts into a finite set of lexical relations. In the literature, pattern-based and distributional methods are two major types of LRC models Wang et al., 2017a). However, compared to the classification of factual relations for knowledge graph population (Liu et al., 2017;Wu and He, 2019), the accurate classification of lexical relations has more challenges. i) Most lexical relations represent the common sense of human knowledge, not frequently expressed in texts explicitly 1 . Apart from Hearst patterns (Hearst, 1992) for hypernymy ("is-a") extraction, textual patterns that indicate the existence of other types of lexical relations remain few, leading to the "pattern sparsity" problem Washio and Kato, 2018a). ii) Distributional models assume concepts with similar contexts have similar embeddings (Mikolov et al., 2013;Bojanowski et al., 2017). Representations of a concept pair learned by traditional word embedding models are not sufficient to distinguish different types of lexical relations (Glavas and Vulic, 2018;Ponti et al., 2019). iii) Many LRC datasets are highly imbalanced w.r.t. training instances of different lexical relations, and may contain randomly paired concepts. It is difficult for models to distinguish whether a concept pair has a particular type of lexical relation, or has very weak or no semantic relatedness.
In this work, the Knowledge-Enriched Meta-Learning (KEML) framework is presented to address these challenges for LRC, consisting of three modules: Knowledge Encoder, Auxiliary Task Generator and Relation Leaner. In Knowledge Encoder, we propose the LKB-BERT (Lexical Knowledge Base-BERT) model to learn relation-sensitive concept representations. LKB-BERT is built upon BERT (Devlin et al., 2019) and trained via new distant supervised learning tasks over lexical knowledge bases, which encodes both language patterns and relational lexical knowledge into the model. In Auxiliary Task Generator, we treat recognizing single type of lexical relations as auxiliary tasks.
Based on meta-learning (Finn et al., 2017), a probabilistic task distribution is properly defined for the model to optimize, which addresses the imbalanced property and the existence of random relations in LRC datasets. In Relation Leaner, we combine a gradient-based meta-learning process over the auxiliary task distribution and supervised learning to train the final neural relation classifier. Especially, a relation recognition cell is designed and integrated into the neural network for the purpose.
This paper makes the following contributions: • We propose LKB-BERT to learn concept representations for LRC, considering unstructured texts and relational lexical knowledge. • A meta-learning process with auxiliary tasks for single relation recognition is proposed to improve the performance of LRC.

Related Work
In this section, we overview related work on LRC, pre-trained language models and meta-learning. As summarized in , Lexical Relation Classification (LRC) models are categorized into two major types: pattern-based and distributional. Pattern-based approaches extract patterns w.r.t. a concept pair from texts as features to predict its lexical relation. For hypernymy relations, Hearst patterns (Hearst, 1992) are most influential, often used for the construction of largescale taxonomies (Wu et al., 2012). To learn patterns representations,  exploit LSTM-based RNNs to encode dependency paths of patterns. Roller et al. (2018);  calculate Hearst pattern-based statistics from texts and design hypernymy measures to predict the degrees of hypernymy between concepts. For other types of relations, LexNET  extends the network architecture  for multi-way classification of lexical relations. Nguyen et al. (2016Nguyen et al. ( , 2017 design path-based neural networks to distinguish antonymy and synonymy. However, these methods may suffer from the lack of patterns and the occurrence of concept pairs in texts (Washio and Kato, 2018a).
With the rapid development of deep neural language models, distributional models attract more interest. While traditional methods directly leverage the two concepts' embeddings as features for classifier training (Weeds et al., 2014;Vylomova et al., 2016;Roller and Erk, 2016), they may suffer from the "lexical memorization" problem (Levy et al., 2015). Recently, more complicated neural networks are proposed to encode the semantics of lexical relations. Attia et al. (2016) formulate LRC as a multi-task learning task and propose a convolutional neural network for LRC. Mrksic et al. (2017) propose the Attract-Repe model to learn the semantic specialization of word embeddings. Glavas and Vulic (2018) introduce the Specialization Tensor Model, which learns multiple relationsensitive specializations of concept embeddings. SphereRE (Wang et al., 2019) encodes concept pairs in the hyperspherical embedding space and achieves state-of-the-art results. There exist some models to learn word-pair representations for other NLP tasks (Washio and Kato, 2018b;Joshi et al., 2019;Camacho-Collados et al., 2019). KEML is also distributional, further improving LRC by training meta-learners over neural language models.
Pre-trained language models have gained attention from the NLP community. ELMo (Peters et al., 2018) learns context-sensitive embeddings for each token form both left-to-right and rightto-left. BERT (Devlin et al., 2019) is a notable work, employing layers of transformer encoders to learn language representations. Follow-up works include Transformer-XL , XL-Net , ALBERT (Lan et al., 2019) and many more Yet another direction is to fuse additional knowledge sources into BERT-like models. ERNIE  incorporates the rich semantics of entities in the model. KG-BERT (Yao et al., 2019) and K-BERT (Liu et al., 2019) employ relation prediction objectives in knowledge graphs as additional learning tasks. In our work, we leverage the conceptual facts in lexical knowledge bases to improve the representation learning for LRC.
Meta-learning is a learning paradigm to train models that can adapt to a variety of different tasks with little training data (Vanschoren, 2018), mostly applied to few-shot learning. In NLP, metalearning algorithms have not been extensively employed, mostly due to the large numbers of training examples required to train the model for different NLP tasks. Existing models mostly focus on training meta-learners for single applications, such as link prediction , dialog systems (Madotto et al., 2019) and semantic parsing (Guo et al., 2019). Dou et al. (2019) leverage meta-learning for various low-resource natural language understanding tasks. KEML is one of the early attempts to improve LRC via gradient-based meta-learning (Finn et al., 2017). Since the mechanism of meta-learning is not our major research focus, we do not further elaborate.

The KEML Framework
In this section, we formally describe the KEML framework for LRC. A brief technical flow is presented, followed by its algorithmic details.

A Brief Overview of KEML
We first overview the LRC task briefly. Denote (x i , y i ) as an arbitrary concept pair. The goal of LRC is to learn a classifier f to predict the lexical relation r i ∈ R between x i and y i , based on a labeled, training set D = {(x i , y i , r i )}. Here, R is the collection of all pre-defined lexical relation types (e.g., hypernymy, synonymy), (possibly) including a special relation type RAN ("random"), depending on different task settings. It means that x i and y i are randomly paired, without clear association with any lexical relations.
The framework of KEML is illustrated in Figure 1, with three modules introduced below: Knowledge Encoder. Representation learning for LRC is significantly different from learning traditional word embeddings. This is because some lexical concepts are naturally Multiword Expressions (e.g., card game, orange juice), in which the entire sequences of tokens should be encoded in the embedding space (Cordeiro et al., 2016). Additionally, these models are insufficient to capture the lexical relations between concepts, due to the pattern sparsity issue (Washio and Kato, 2018a). Hence, the semantics of concepts should be encoded from a larger corpus and rich language resources. Inspired by BERT (Devlin et al., 2019), and its extensions, we propose the LKB-BERT (Lexical Knowledge Base-BERT) model to encode the semantics of concepts from massive text corpora and lexical knowledge bases. LKB-BERT employs the neural architecture and pre-trained parameters of BERT (Devlin et al., 2019) for token encoding, and imposes two new distant supervised learning objectives over lexical knowledge bases (such as WordNet (Miller, 1995)) as fine-tuning tasks. After model training, each concept x i receives the em-beddings of the last transformer encoding layer of LKB-BERT as the its representation. Denote the embeddings of x i as x i , with the dimension as d.
Auxiliary Task Generator. As discussed, training concept embedding based classifiers directly may produce sub-optimal results due to the highly imbalanced nature of LRC training sets and the existence of RAN relations. Inspired by the design philosophy of meta-learning (Finn et al., 2017(Finn et al., , 2018 and its NLP applications Madotto et al., 2019), we regard the relation classifier f as the meta-learner, and design a series of auxiliary tasks to update model parameters. Each task aims at distinguishing between concept pairs that have a particular relation r ∈ R and randomly paired concepts. The training sets for auxiliary tasks are sampled from subsets of D. Denote the collection of all auxiliary tasks as T . The metalearner f is optimized over a probabilistic distributions of tasks p(T ). By designing p(T ) properly, the relation classifier f is capable of alleviating the imbalanced classification and the RAN relation problems of LRC at the same time.
Relation Leaner. Finally, we design a two-stage algorithm to train the neural relation classifier f : i) meta-learning and ii) supervised learning. In the meta-learning stage, the adapted model parameters of neural networks are iteratively learned over the distribution p(T ). Therefore, the neural network learns how to recognize specific lexical relations, with the guidance of the underlying lexical knowledge base. Here, a special cell, i.e., Single Relation Recognition Cell is designed and integrated into the neural network. In the supervised learning stage, we fine-tune meta-learned parameters to obtain the multi-way classification model for LRC over D.

Knowledge Encoder
We consider the training of LKB-BERT as a variant of the fine-tune process of BERT (Devlin et al., 2019). In original BERT, the inputs are arbitrary spans of token sequences. To encode the semantics of concept pairs, we combine a concept pair (x i , y i ) to form a sequence of tokens, separated by a special token "[SEP]" as the input for LKB-BERT (see Figure 1). We first initialize all the model parameters of transformer encoders to be the same as BERT's pre-training results. Different from any standard fine-tuning tasks in BERT (Devlin et al., 2019) or KG-BERT (Yao et al., 2019), to address the RAN problem, LKB-BERT learns to classify a concept LKB-BERT Lexical KB [CLS] Tok 1  Figure 1: The high-level framework of KEML (best viewed in color).
pair (x i , y i ) into the lexical relation collection R (including the special relation type RAN).
Let KB be the collection of labeled concept pairs in lexical knowledge bases. For each concept pair with its label (x i , y i , r i ) ∈ KB, we compute τ r (x i , y i ) as the predicted score w.r.t. the lexical relation r by LKB-BERT's transformer encoders (we have ∀r ∈ R, τ r (x i , y i ) ∈ [0, 1] and r∈R τ r (x i , y i ) = 1). The first loss, i.e., the multiway relation classification loss L (1) KB is defined as: 3 where 1(·) is the indicator function that returns 1 if the input expression is true; and 0 otherwise.
To improve LKB-BERT's ability to recognize concept pairs without any lexical relations, we add a binary cross-entropy loss for LKB-BERT to optimize. We only need LKB-BERT to learn whether a concept pair (x i , y i ) are randomly paired. Let ¬ RAN be any non-random lexical relation types in R. The complete objective of LKB-BERT is: In KEML, we regard lexical relations sampled from WordNet (Miller, 1995) and in training sets as sources of KB. As for the neural network structure, LKB-BERT has two sets of classification outputs.
Refer to the C 1 and C 2 units of LKB-BERT.

Auxiliary Task Generator
Although LKB-BERT is capable of learning deep concept representations, using such features for classifier training is insufficient. The reasons are twofold. i) Direct classification can suffer from "lexical memorization" (Levy et al., 2015), meaning that the relation classifier f only learns the individual characteristics of two concepts alone. ii) The LRC datasets are highly imbalanced. For example, in the widely used dataset EVALution (Santus et al., 2015), the numbers of training instances w.r.t. several lexical relation types are very few. Hence, the learning bias of the classifier trained by naive approaches is almost unavoidable. Finn et al. (2017) observe that meta-learning achieves better parameter initialization for few-shot learning, compared to multi-task learning across all the tasks. In KEML, we propose a series of auxiliary tasks T , where each task (named Single Relation Recognition) T r ∈ T corresponds to a specific type of lexical relation r ∈ R (excluding RAN). The goal is to distinguish concept pairs with the lexical relation type r and randomly paired concepts. Let S r and S RAN be the collection of concept pairs with lexical relations as r and RAN, respectively, randomly sampled from the training set D. The goal of learning auxiliary task T r is to minimize the following loss function L(T r ): where q r (x i , y i ) is the predicted probability of the concept pair (x i , y i ) having the lexical relation r.
A remaining problem is the design of the probabilistic distribution of auxiliary tasks p(T ). We need to consider two issues. i) The semantics of all types of lexical relations should be fully learned. ii) Assume the batch sizes for all tasks are the same, i.e., ∀r p , r q ∈ R \ {RAN}, |S rp | = |S rq |. Tasks related to lexical relations with more training instances should be learned more frequently by the meta-learner. Let D r be the subset of the training set D with the lexical relation as r. ∀r p , r q ∈ R \ {RAN}, if |D rp | > |D rq |, we have the sampling probability p(T rp ) > p(T rq ). Hence, we define p(T rp ) empirically as follows: where γ > 0 is the smoothing factor. The expectation of all the losses of auxiliary tasks (represented as L(T )) is: E(L(T )) = r * ∈R\{RAN} p(T r ) · L(T r ), which is the real learning objective that these auxiliary tasks aim to optimize.

Relation Learner
In this part, we introduce the meta-learning algorithm for LRC. Assume the relation classifier f is parameterized by θ, with learning and metalearning rates as α and . Relation Learner has two stages: i) meta-learning and ii) supervised learning. For each iteration in meta-learning, we sample N auxiliary tasks from p(T ). For each auxiliary task T r , we learn adapted parameters based on two sampled subsets: S r and S RAN , to make the model to recognize one specific type of lexical relations. After that, the adapted parameters on each task T r are averaged and updated to θ. We simplify the meta-update step by only taking firstorder derivatives (Nichol et al., 2018) to avoid the time-consuming second-order derivative computation. For supervised learning, we fine-tune the parameters θ of the classifier f to obtain the multiway LRC model over the entire training set D. The algorithmic description is shown in Algorithm 1.
Algorithm 1 Meta-Learning Algorithm for LRC 1: Initialize model parameters θ; 2: while not converge do 3: Sample N auxiliary tasks Tr 1 , Tr 2 , · · · , Tr N from the task distribution p(T ); 4: for each auxiliary task Tr do 5: Sample a batch (positive samples Sr and negative samples S RAN ) from the training set D; 6: Update adapted parameters: θr ← θ − α∇L(Tr) based on Sr and S RAN ; 7: end for 8: Update meta-parameters: θ ← θ − ∇ Tr L(Tr); 9: end while 10: Fine-tune θ over D by standard supervised learning LRC; Concept 1 (x i ) Concept 2 (y i )  Finally, we describe the neural network structure for LRC. In this network, the Single Relation Recognition Cell (SRR Cell) is deigned for learning auxiliary tasks and enabling knowledge injection, with the structure illustrated in Figure 2. For each lexical relation r ∈ R \ {RAN}, we extract the relation prototype r proto from the lexical knowledge base KB by averaging all the embedding offsets of concept pairs (x i , y i ) with relation r:

Dense Layers
We use x i − y i as features because the Diff model is effective for representing semantic relations (Fu et al., 2014;Vylomova et al., 2016;Wang et al., 2019). Consider the SRR Cell structure in Figure 2. Given the inputs x i , y i and r proto , we compute the d-dimensional hidden states U 1 and U 2 by: where W 1 ∈ R 2d×d , W 2 ∈ R 2d×d , b 1 ∈ R 1×d and b 2 ∈ R 1×d are the weights and biases of these states. This can be interpreted as inferring the embeddings of relation objects or subjects, given the relation prototype and subjects/objects as inputs, similar to knowledge graph completion (Wang et al., 2017b). Next, we compute the offsets U 1 − y i and U 2 − x i and two new d-dimensional hidden states U 3 and U 4 , with W 3 ∈ R d×d , W 4 ∈ R d×d , b 3 ∈ R 1×d and b 4 ∈ R 1×d as learnable parameters: We can see that if x i and y i actually have the lexical relation r, U 3 and U 4 should be good indicators of the existence of such relations. For example, one way to interpret U 1 and U 3 is that U 1 tries to infer the relation object given x i and r proto as inputs, and U 3 makes the judgment by comparing U 1 and the true relation object embedding y i . Hence, the network learns whether r proto is a good fit for the concept pair (x i , y i ). The functionalities of U 2 and U 4 are similar, only with directions reversed. After that, we concatenate U 3 and U 4 as part of the inputs for the next layer.
Re-consider the entire network structure in Figure 1. For each concept pair (x i , y i ), we compare x i and y i with all relation prototypes (treated as constants in the network). The results are represented by 2(|R|−1) vectors of hidden states. After that, a dense layer and multiple output layers are connected. During the meta-learning stage, we train |R| − 1 binary meta-classifiers to minimize L(T r ) (T r ∈ T ), with meta-parameters updated. In the supervised learning stage, we discard all the output layers of meta-classifiers, and train the final LRC model. This is because the numbers of output units of meta-classifiers and the final classifier are different. The parameters of the last layer can not be re-used. We also need to note that additional skip connections between x i , y i and the dense layer are added, in order to improve the effect of back propagation during training (He et al., 2016).
Discussion. KEML employs a "divide-andconquer" strategy for LRC. In mate-learning, each SRR Cell learns the semantics of single lexical relation type, and also handles the RAN problem. Hence, the supervised learning process of the relation classifier can be improved with better parameter initializations. Unlike traditional metalearning (Finn et al., 2017(Finn et al., , 2018, KEML does not contain meta-testing steps (since LRC is not a fewshot learning problem), but takes advantages of both meta-learning and supervised learning.

Experiments
We conduct extensive experiments to evaluate KEML over multiple benchmark datasets, and compare it with state-of-the-art methods.

Datasets and Experimental Settings
We employ Google's pre-trained BERT model 4 to initialize the parameters of LKB-BERT. The lexical knowledge base KB contains 16.7K relation triples 5 . Following Wang et al. (2019) (which produced state-of-the-art results for LRC previously), we use the five public benchmark datasets for multiway classification of lexical relations to evaluate KEML, namely, K&H+N (Necsulescu et al., 2015), BLESS Lenci, 2011), ROOT09 (Santus et al., 2016b), EVALution (Santus et al., 2015) and CogALex-V Subtask 2 (Santus et al., 2016a). Due to space limitation, we do not introduce all the datasets here. Refer to Wang et al. (2019) for the statistical summarization of all the datasets. K&H+N, BLESS, ROOT09 and EVALution are partitioned into training, validation and testing sets, following the extract same settings as in . The CogALex-V dataset has training and testing sets only, with no validation sets provided (Santus et al., 2016a). Hence, we randomly sample 80% of the training set to learn the parameters, and use the rest for parameter tuning.
The default hyper-parameter settings of KEML are as follows: N = |R| − 1 6 , γ = 1 and α = = 10 −3 . We use tanh as the activation function, and Adam as the optimizer to train the neural network. All the model parameters are l 2 -regularized, with the hyper-parameter λ = 10 −3 . The batch size is set as 256. The dimension of hidden layers is set as the same of d (768 for the base BERT model). The number of parameters of the final neural classifier is around 7M to 24M, depending on the number of classes. The algorithms are implemented with TensorFlow and trained with NVIDIA Tesla P100 GPU. For evaluation, we use Precision, Recall and F1 as metrics, reported as the average of all the classes, weighted by the support.

General Experimental Results
We follow the experimental steps (Wang et al., 2019) to evaluate KEML over K&H+N, BLESS, ROOT09 and EVALution. Since EVALution does not contain RAN relations, during the meta-learning process of auxiliary task T r , we randomly sample relation triples from D that do not have the relation r, and take them as S RAN . We manually tune the regularization hyperparameter λ from 10 −2 to 10 −4 using the validation set (based on F1) and report the performance over the testing set. As for baselines, we consider traditional distributional models Concat (Baroni et al., 2012) and Diff (Weeds et al., 2014), pattern-based models NPB , LexNET , NPB+Aug and LexNET+Aug (Washio and Kato, 2018a), and the state-of-the-art model SphereRE (Wang et al., 2019). We refer readers to the following papers Washio and Kato, 2018a;Wang et al., 2019) for the detailed descriptions of these baselines. Additionally, we implement two variants of our approach: i) LKB-BERT (using trained concept representations to predict lexical relations) and ii) KEML-S (KEML without the meta-learning stage). The results of KEML and all the baselines are summarized in Table 1. As shown, KEML outperforms all baselines, especially over BLESS, ROOT09 and EVALution. As for K&H+N, KEML produces a slightly better F1 score (0.3%) than the strongest baseline SphereRE (Wang et al., 2019). A probable cause is that K&H+N is an "easy" dataset (99% F1 by SphereRE), leaving little room for improvement. Comparing KEML against LKB-BERT and KEML-S, we can conclude that, the knowledge enrichment technique for concept representation learning and the meta-learning algorithm are highly beneficial for accurate prediction of lexical relations.

Results of CogALex-V Shared Task
We evaluate KEML over the CogALex-V Shared Task (Subtask 2) (Santus et al., 2016a). This dataset is most challenging as it contains a large number of random word pairs and disables "lexical memorization". The organizer requires participants to  discard the results of the random class from average, and report the F1 scores for each type of lexical relations. We consider two top systems reported in this task (i.e., GHHH (Attia et al., 2016) and LexNET ), as well as two recent models that have been evaluated over the shared task (i.e., STM (Glavas and Vulic, 2018) and SphereRE (Wang et al., 2019)) as strong competitors. Because the training set contains an overwhelming number of random word pairs, during the training process of KEML-S and KEML, we randomly discard 70% (manually tuned) of the random pairs in each epoch 7 . Results are reported in Table 2, showing that KEML achieves the highest F1 score of 50.0%. It also has highest scores on three types of lexical relations: synonymy (SYN), hypernymy (HYP) and meronymy (MER).

Detailed Analysis of KEML
To facilitate deeper understanding, we conduct additional experiments to analyze KEML's components. We first study how knowledge-enriched concept representation learning benefits LRC. We implement three models:    described previously. We take the two concepts' representations as features ( x i and y i ) to train relation classifiers for LRC and report the results over the validation sets. For fair comparison, we use neural networks with one hidden layer (with the dimension d, and the activation function tanh) as classifiers in all the experiments and present the results in Table 4. We can see that the improvement of using L KB = L (2) KB as the objective function is consistent across all the datasets (which ranges from 0.8% to 1.9% in terms of F1). Particularly, LKB-BERT outperforms the strongest baseline SphereRE in a few cases (for example, the dataset ROOT09, as shown in Table 1 and Table 4). Hence, LKB-BERT is capable of encoding knowledge in patterns and lexical knowledge bases to represent the semantics of lexical relations.
Next, we look into the meta-learning process in KEML. We test whether SRR Cells can distinguish a specific type of lexical relations from random concept pairs. In each iteration of meta-learning, we sample another batch of positive and negative samples from D and compute the accuracy of single relation recognition. Figure 3 illustrates how accuracies changes through time in K&H+N, BLESS and ROOT09. Within 100 iterations, our models can achieve desirable performance efficiently, achieving good parameter initializations for LRC. This experiment also explains why KEML produces better results than KEML-S.

Error Analysis
We analyze prediction errors produced by KEML. Because the inputs of our task are very simple and the interpretation of deep neural language models is still challenging, the error analysis process is rather difficult. Here, we analyze the causes of errors from a linguistic point of view, with some cases presented in Table 3. As seen, some types of lexical relations are very similar in semantics. For instance, concept pairs with the synonymy relation and the co-hyponymy relation are usually mapped similar positions in the embedding space. Hence, it is difficult for models to distinguish the differences between the two relations without rich contextual information available. Another problem is that some of the relations are "blurry" in semantics, making KEML hard to discriminate between these relations and random relations.

Conclusion and Future Work
In this paper, we present the Knowledge-Enriched Meta-Learning (KEML) framework to distinguish different lexical relations. Experimental results confirm that KEML outperforms state-of-the-art approaches. Future work includes: i) improving concept representation learning with deep neural language models; ii) integrating richer linguistic and commonsense knowledge into KEML; and iii) extending KEML to other similar semantics-intensive NLP tasks, such as natural language inference.