Several studies on sentence processing suggest that the mental lexicon keeps track of the mutual expectations between words . Current DSMs , however , represent context words as separate features , thereby loosing important information for word expectations , such as word interrelations . In this paper , we present a DSM that addresses this issue by defining verb contexts as joint syntactic dependencies . We test our representation in a verb similarity task on two datasets , showing that joint contexts achieve performances comparable to single dependencies or even better . Moreover , they are able to overcome the data sparsity problem of joint feature spaces , in spite of the limited size of our training corpus . Distributional Semantic Models ( DSMs ) rely on the Distributional Hypothesis ( Harris , 1954 ; Sahlgren , 2008 ) , stating that words occurring in similar contexts have similar meanings . On such theoretical grounds , word co-occurrences extracted from corpora are used to build semantic representations in the form of vectors , which have become very popular in the NLP community . Proximity between word vectors is taken as an index of meaning similarity , and vector cosine is generally adopted to measure such proximity , even though other measures have been proposed ( Weeds et al . , 2004 ; Santus et al . , 2016 ) . Most of DSMs adopt a bag-of-words approach , that is they turn a text span ( i.e. , a word window or a parsed sentence ) into a set of words and they register separately the co-occurrence of each word with a given target . The problem with this approach is that valuable information concerning word interrelations in a context gets lost , because words co-occurring with a target are treated as independent features . This is why works like Ruiz-Casado et al . ( 2005 ) , Agirre et al . ( 2009 ) and Melamud et al . ( 2014 ) proposed to introduce richer contexts in distributional spaces , by using entire word windows as features . These richer contexts proved to be helpful to semantically represent verbs , which are characterized by highly context-sensitive meanings , and complex argument structures . In fact , two verbs may share independent words as features despite being very dissimilar from the semantic point of view . For instance kill and heal share the same object nouns in The doctor healed the patient and the The poison killed the patient , but are highly different if we consider their joint dependencies as a single context . Nonetheless , richer contexts like these suffer from data sparsity , therefore requiring either larger corpora or complex smoothing processes . In this paper , we propose a syntactically savvy notion of joint contexts . To test our representation , we implement several DSMs and we evaluate them in a verb similarity task on two datasets . The results show that , even using a relatively small corpus , our syntactic joint contexts are robust with respect to data sparseness and perform similarly or better than single dependencies in a wider range of parameter settings . The paper is organized as follows . In Section 2 , we provide psycholinguistic and computational background for this research , describing recent models based on word windows . In Section 3 , we describe our reinterpretation of joint contexts with syntactic dependencies . Evaluation settings and results are presented in Section 4 . In this paper , we have presented our proposal for a new type of vector representation based on joint features , which should emulate more closely the general knowledge about event participants that seems to be the organizing principle of our mental lexicon . A core issue of previous studies was the data sparseness challenge , and we coped with it by means of a more abstract , syntactic notion of joint context . The models using joint dependencies were able at least to perform comparably to traditional , dependency-based DSMs . In our experiments , they even achieved the best correlation scores across several parameter settings , especially after the application of SVD . We want to emphasize that previous works such as Agirre et al . ( 2009 ) already showed that large word windows can have a higher discriminative power than indipendent features , but they did it by using a huge training corpus . In our study , joint context-based representations derived from a small corpus such as RCV1 are already showing competitive performances . This result strengthens our belief that dependencies are a possible solution for the data sparsity problem of joint feature spaces . We also believe that verb similarity might not be the best task to show the usefulness of joint contexts for semantic representation . The main goal of the present paper was to show that joint contexts are a viable option to exploit the full potential of distributional information . Our successful tests on verb similarity prove that syntactic joint contexts do not suffer of data sparsity and are also able to beat other types of representations based on independent word features . Moreover , syntactic joint contexts are much simpler and more competitive with respect to window-based ones . The good performance in the verb similarity task motivates us to further test syntactic joint contexts on a larger range of tasks , such as word sense disambiguation , textual entailment and classification of semantic relations , so that they can unleash their full potential . Moreover , our proposal opens interesting perspectives for computational psycholinguistics , especially for modeling those semantic phenomena that are inherently related to the activation of event knowledge ( e.g. thematic fit ) .
Providing richer contexts to Distributional Semantic Models improves by taking word interrelations into account but it would suffer from data sparsity.
They propose a Distributional Semantic Model that incorporates verb contexts as joint syntactic dependencies so that it emulates knowledge about event participants.
They show that representations obtained by the proposed model outperform more complex models on two verb similarity datasets with a limited training corpus.