The clustering-based unsupervised relation discovery method has gradually become one of the important methods of open relation extraction ( OpenRE ) . However , high-dimensional vectors can encode complex linguistic information which leads to the problem that the derived clusters can not explicitly align with the relational semantic classes . In this work , we propose a relationoriented clustering model and use it to identify the novel relations in the unlabeled data . Specifically , to enable the model to learn to cluster relational data , our method leverages the readily available labeled data of pre-defined relations to learn a relationoriented representation . We minimize distance between the instance with same relation by gathering the instances towards their corresponding relation centroids to form a cluster structure , so that the learned representation is cluster-friendly . To reduce the clustering bias on predefined classes , we optimize the model by minimizing a joint objective on both labeled and unlabeled data . Experimental results show that our method reduces the error rate by 29.2 % and 15.7 % , on two datasets respectively , compared with current SOTA methods . Relation extraction ( RE ) , a crucial basic task in the field of information extraction , is of the utmost practical interest to various fields including web search ( Xiong et al . , 2017 ) , knowledge base completion ( Bordes et al . , 2013 ) , and question answering ( Yu et al . , 2017 ) . However , conventional RE paradigms such as supervision and distant supervision are generally designed for pre-defined relations , which can not deal with new emerging relations in the real world . Under this background , open relation extraction ( OpenRE ) has been widely studied for its use Figure 1 : Although both instances S 2 and S 3 express founded relation while S 1 expresses CEO relation , the distance between S 1 and S 2 is still smaller than that between S 2 and S 3 . This is because there may be more similar surface information ( e.g. word overlapping ) or syntactic structure between S 1 and S 2 , thus the derived clusters can not explicitly align with relations . in extracting new emerging relational types from open-domain corpora . The approaches used to handle open relations roughly fall into one of two groups . The first group is open information extraction ( OpenIE ) ( Etzioni et al . , 2008 ; Yates et al . , 2007 ; Fader et al . , 2011 ) , which directly extracts related phrases as representations of different relational types . However , if not properly canonicalized , the extracted relational facts can be redundant and ambiguous . The second group is unsupervised relation discovery ( Yao et al . , 2011 ; Shinyama and Sekine , 2006 ; Simon et al . , 2019 ) . In this type of research , much attention has been focused on unsupervised clustering-based RE methods , which cluster and recognize relations from high-dimensional representations ( Elsahar et al . , 2017 ) . Recently , the self-supervised signals in pretrained language model are further exploited for clustering optimization ( Hu et al . , 2020 ) . However , many studies show that highdimensional embeddings can encode complex linguistic information such as morphological ( Peters et al . , 2018 ) , local syntactic ( Hewitt and Manning , 2019 ) , and longer range semantic information ( Jawahar et al . , 2019 ) . Consequently , the distance of representation is not completely consistent with relational semantic similarity . Although Hu et al . ( 2020 ) use self-supervised signals to optimize clustering , there is still no guarantee that the learned clusters will explicitly align with the desired relational semantic classes ( Xing et al . , 2002 ) . As shown in Figure 1 , we use the method proposed by Hu et al . ( 2020 ) to get the instance representations . Although both instances S 2 and S 3 express the founded relation , the euclidean distance between them is larger than that between S 1 and S 2 , which express different relation . Obviously , the clustering algorithm tends to group instances S 1 and S 2 together , rather than S 2 and S 3 which express the same relation . In this work , we propose a relation-oriented clustering method . To enable the model to learn to cluster relational data , pre-defined relations and their existing labeled instances are leveraged to optimize a non-linear mapping , which transforms high-dimensional entity pair representations into relation-oriented representations . Specifically , we minimize distance between the instances with same relation by gathering the instances representation towards their corresponding relation centroids to form the cluster structure , so that the learned representation is cluster-friendly . In order to reduce the clustering bias on the predefined classes , we iteratively train the entity pair representations by optimizing a joint objective function on the labeled and unlabeled subsets of the data , improving both the supervised classification of the labeled data , and the clustering of the unlabeled data . In addition , the proposed method can be easily extended to incremental learning by classifying the pre-defined and novel relations with a unified classifier , which is often desirable in real-world applications . Our experimental results show that our method outperforms current state-of-the-art methods for OpenRE . Our codes are publicly available at Github * . To summarize , the main contributions of our work are as follows : ( 1 ) we propose a novel relation-oriented clustering method RoCORE to enable model to learn to cluster relational data ; ( 2 ) the proposed method achieves the incremental learning of unlabeled novel relations , which is often desirable in real-world applications ; ( 3 ) experimental results show that our method reduces * https://github.com / Ac-Zyx / RoCORE . the error rate by 29.2 % and 15.7 % , on two realworld datasets respectively , compared with current state-of-the-art OpenRE methods . In this work , we introduce a relation-oriented clustering method that extends the current unsupervised clustering-based OpenRE method . The proposed method leverages the labeled data of pre-defined relations to learn a relation-oriented representation from which the derived clusters explicitly align with relational classes . Iterative joint training method effectively reduces the unwanted bias on labeled data . In addition , the proposed method can be easily extended to incremental learning of novel relations . Experimental results show that our method outperforms SOTA methods for OpenRE .
Even though high-dimensional vectors that can encode complex information used for relation extraction are not guaranteed to be consistent with relational semantic similarity.
They propose to use available relation labeled data to obtain relation-oriented representation by minimizing the distance between the same relation instances.
The proposed approach can reduce error rates significantly from the best models for open relation extraction.