Word embeddings are already well studied in the general domain, usually trained on large text corpora, and have been evaluated for example on word similarity and analogy tasks, but also as an input to downstream NLP processes. In contrast, in this work we explore the suitability of word embedding technologies in the specialized digital humanities domain. After training embedding models of various types on two popular fantasy novel book series, we evaluate their performance on two task types: term analogies, and word intrusion. To this end, we manually construct test datasets with domain experts. Among the contributions are the evaluation of various word embedding techniques on the different task types, with the findings that even embeddings trained on small corpora perform well for example on the word intrusion task. Furthermore, we provide extensive and high-quality datasets in digital humanities for further investigation, as well as the implementation to easily reproduce or extend the experiments.
Linzen @cite_13 discusses potential pitfalls of the vector offset method of analogy, and presents baselines to improve the utility of vector space evaluations. We apply the suggested baselines in our evaluations.
@cite_13
2963176474 2141599568 1541459201 2921341714
The offset method for solving word analo- gies has become a standard evaluation tool for vector-space semantic models: it is considered desirable for a space to repre- sent semantic relations as consistent vec- tor offsets. We show that the method's re- liance on cosine similarity conflates offset consistency with largely irrelevant neigh- borhood structure, and propose simple baselines that should be used to improve the utility of the method in vector space evaluation. Continuous space language models have recently demonstrated outstanding results across a variety of tasks. In this paper, we examine the vector-space word representations that are implicitly learned by the input-layer weights. We find that these representations are surprisingly good at capturing syntactic and semantic regularities in language, and that each relationship is characterized by a relation-specific vector offset. This allows vector-oriented reasoning based on the offsets between words. For example, the male female relationship is automatically learned, and with the induced vector representations, “King Man + Woman” results in a vector very close to “Queen.” We demonstrate that the word vectors capture syntactic regularities by means of syntactic analogy questions (provided with this paper), and are able to correctly answer almost 40 of the questions. We demonstrate that the word vectors capture semantic regularities by using the vector offset method to answer SemEval-2012 Task 2 questions. Remarkably, this method outperforms the best previous systems. For similarity search in high-dimensional vector spaces (or ‘HDVSs’), researchers have proposed a number of new methods (or adaptations of existing methods) based, in the main, on data-space partitioning. However, the performance of these methods generally degrades as dimensionality increases. Although this phenomenon-known as the ‘dimensional curse’-is well known, little or no quantitative a.nalysis of the phenomenon is available. In this paper, we provide a detailed analysis of partitioning and clustering techniques for similarity search in HDVSs. We show formally that these methods exhibit linear complexity at high dimensionality, and that existing methods are outperformed on average by a simple sequential scan if the number of dimensions exceeds around 10. Consequently, we come up with an alternative organization based on approximations to make the unavoidable sequential scan as fast as possible. We describe a simple vector approximation scheme, called VA-file, and report on an experimental evaluation of this and of two tree-based index methods (an R*-tree and an X-tree). Correlation-based techniques used for frame synchronization can suffer significant performance degradation over multi-path frequency-selective channels. In this paper, we propose a joint frame synchronization and channel estimation (JFSCE) framework as a remedy to this problem. This framework, however, increases the size of the resulting combined channel vector which should capture both the channel impulse response vector and the frame boundary offset and, therefore, its estimation becomes more challenging. On the other hand, because the combined channel vector is sparse, sparse channel estimation methods can be applied. We propose several JFSCE methods using popular sparse signal recovery algorithms which exploit the sparsity of the combined channel vector. Subsequently, the sparse channel vector estimate is used to design a sparse equalizer. Our simulation results and experimental measurements using software defined radios show that in some scenarios our proposed method improves the overall system performance significantly, in terms of the mean square error between the transmitted and the equalized symbols compared to the conventional method.