Previous work has modeled the compositionality of words by creating character-level models of meaning, reducing problems of sparsity for rare words. However, in many writing systems compositionality has an effect even on the character-level: the meaning of a character is derived by the sum of its parts. In this paper, we model this effect by creating embeddings for characters based on their visual characteristics, creating an image for the character and running it through a convolutional neural network to produce a visual character embedding. Experiments on a text classification task demonstrate that such model allows for better processing of instances with rare characters in languages such as Chinese, Japanese, and Korean. Additionally, qualitative analyses demonstrate that our proposed model learns to focus on the parts of characters that carry semantic content, resulting in embeddings that are coherent in visual space.
Several techniques for reducing the rare words effects have been introduced in the literature, including spelling expansion @cite_7 , dictionary term expansion @cite_7 , proper name transliteration @cite_25 , treating words as a sequence of characters @cite_19 , subword units @cite_35 , and reading text as bytes @cite_38 . However, most of these techniques still have no mechanism for handling low frequency characters, which are the target of this work.
@cite_38 @cite_35 @cite_7 @cite_19 @cite_25
2153903004 1973550659 2036723249 2333611780
We present four techniques for online handling of Out-of-Vocabulary words in Phrase-based Statistical Machine Translation. The techniques use spelling expansion, morphological expansion, dictionary term expansion and proper name transliteration to reuse or extend a phrase table. We compare the performance of these techniques and combine them. Our results show a consistent improvement over a state-of-the-art baseline in terms of BLEU and a manual error analysis. Statistical methods to extract translational equivalents from non-parallel corpora hold the promise of ensuring the required coverage and domain customisation of lexicons as well as accelerating their compilation and maintenance. A challenge for these methods are rare, less common words and expressions, which often have low corpus frequencies. However, it is rare words such as newly introduced terminology and named entities that present the main interest for practical lexical acquisition. In this article, we study possibilities of improving the extraction of low-frequency equivalents from bilingual comparable corpora. Our work is carried out in the general framework which discovers equivalences between words of different languages using similarities between their occurrence patterns found in respective monolingual corpora. We develop a method that aims to compensate for insufficient amounts of corpus evidence on rare words: prior to measuring cross-language similarities, the method uses same-language corpus data to model co-occurrence vectors of rare words by predicting their unseen co-occurrences and smoothing rare, unreliable ones. Our experimental evaluation demonstrates that the proposed method delivers a consistent and significant improvement on the conventional approach to this task. This paper presents a novel preprocessing method based on mathematical morphology techniques to improve the subsequent thresholding quality of raw degraded word images. The raw degraded word images contain undesirable shapes called critical shadows on the background that cause noise in binary images. This noise constitutes obstacles to posterior segmentation of characters. Direct application of a thresholding method produces inadequate binary versions of these degraded word images. Our preprocessing method called Shadow Location and Lightening (SL*L) adaptively, accurately and without manual fine-tuning of parameters locates these critical shadows on grayscale degraded images using morphological operations, and lightens them before applying eventual thresholding process. In this way, enhanced binary images without unpredictable and inappropriate noise can be provided to subsequent segmentation of characters. Then, adequate binary characters can be segmented and extracted as input data to optical character recognition (OCR) applications saving computational effort and increasing recognition rate. The proposed method is experimentally tested with a set of several raw degraded images extracted from real photos acquired by unsophisticated imaging systems. A qualitative analysis of experimental results led to conclusions that the thresholding result quality was significantly improved with the proposed preprocessing method. Also, a quantitative evaluation using a testing data of 1194 degraded word images showed the essentiality and effectiveness of the proposed preprocessing method to increase segmentation and recognition rates of their characters. Furthermore, an advantage of the proposed method is that Otsu's method as a simple and easily implementable global thresholding technique can be sufficient to reducing computational load. The problem of rare and unknown words is an important issue that can potentially influence the performance of many NLP systems, including both the traditional count-based and the deep learning models. We propose a novel way to deal with the rare and unseen words for the neural network models using attention. Our model uses two softmax layers in order to predict the next word in conditional language models: one predicts the location of a word in the source sentence, and the other predicts a word in the shortlist vocabulary. At each time-step, the decision of which softmax layer to use choose adaptively made by an MLP which is conditioned on the context. We motivate our work from a psychological evidence that humans naturally have a tendency to point towards objects in the context or the environment when the name of an object is not known. We observe improvements on two tasks, neural machine translation on the Europarl English to French parallel corpora and text summarization on the Gigaword dataset using our proposed model.