Traditional noun phrase coreference resolution systems represent features only of pairs of noun phrases . In this paper , we propose a machine learning method that enables features over sets of noun phrases , resulting in a first-order probabilistic model for coreference . We outline a set of approximations that make this approach practical , and apply our method to the ACE coreference dataset , achieving a 45 % error reduction over a comparable method that only considers features of pairs of noun phrases . This result demonstrates an example of how a firstorder logic representation can be incorporated into a probabilistic model and scaled efficiently . Noun phrase coreference resolution is the problem of clustering noun phrases into anaphoric sets . A standard machine learning approach is to perform a set of independent binary classifications of the form " Is mention a coreferent with mention b ? " This approach of decomposing the problem into pairwise decisions presents at least two related difficulties . First , it is not clear how best to convert the set of pairwise classifications into a disjoint clustering of noun phrases . The problem stems from the transitivity constraints of coreference : If a and b are coreferent , and b and c are coreferent , then a and c must be coreferent . This problem has recently been addressed by a number of researchers . A simple approach is to perform the transitive closure of the pairwise decisions . However , as shown in recent work ( McCallum and Wellner , 2003 ; Singla and Domingos , 2005 ) , better performance can be obtained by performing relational inference to directly consider the dependence among a set of predictions . For example , McCallum and Wellner ( 2005 ) apply a graph partitioning algorithm on a weighted , undirected graph in which vertices are noun phrases and edges are weighted by the pairwise score between noun phrases . A second and less studied difficulty is that the pairwise decomposition restricts the feature set to evidence about pairs of noun phrases only . This restriction can be detrimental if there exist features of sets of noun phrases that can not be captured by a combination of pairwise features . As a simple example , consider prohibiting coreferent sets that consist only of pronouns . That is , we would like to require that there be at least one antecedent for a set of pronouns . The pairwise decomposition does not make it possible to capture this constraint . In general , we would like to construct arbitrary features over a cluster of noun phrases using the full expressivity of first-order logic . Enabling this sort of flexible representation within a statistical model has been the subject of a long line of research on first-order probabilistic models ( Gaifman , 1964 ; Halpern , 1990 ; Paskin , 2002 ; Poole , 2003 ; Richardson and Domingos , 2006 ) . Conceptually , a first-order probabilistic model can be described quite compactly . A configuration of the world is represented by a set of predi- cates , each of which has an associated real-valued parameter . The likelihood of each configuration of the world is proportional to a combination of these weighted predicates . In practice , however , enumerating all possible configurations , or even all the predicates of one configuration , can result in intractable combinatorial growth ( de Salvo Braz et al . , 2005 ; Culotta and McCallum , 2006 ) . In this paper , we present a practical method to perform training and inference in first-order models of coreference . We empirically validate our approach on the ACE coreference dataset , showing that the first-order features can lead to an 45 % error reduction . We have presented learning and inference procedures for coreference models using first-order features . By relying on sampling methods at training time and approximate inference methods at testing time , this approach can be made scalable . This results in a coreference model that can capture features over sets of noun phrases , rather than simply pairs of noun phrases . This is an example of a model with extremely flexible representational power , but for which exact inference is intractable . The simple approximations we have described here have enabled this more flexible model to outperform a model that is simplified for tractability . A short-term extension would be to consider features over entire clusterings , such as the number of clusters . This could be incorporated in a ranking scheme , as in Ng ( 2005 ) . Future work will extend our approach to a wider variety of tasks . The model we have described here is specific to clustering tasks ; however a similar formulation could be used to approach a number of language processing tasks , such as parsing and relation extraction . These tasks could benefit from first-order features , and the present work can guide the approximations required in those domains . Additionally , we are investigating more sophisticated inference algorithms that will reduce the greediness of the search procedures described here .
Existing approaches treat noun phrase coreference resolution as a set of independent binary classifications limiting the features to be only pairs of noun phrases.
They propose a machine learning method that uses sets of noun phrases as features that are coupled with a sampling method to enable scalability.
Evaluation on the ACE coreference dataset, the proposed method achieves a 45% error reduction over a previous method.