This paper describes Artex, another algorithm for Automatic Text Summarization. In order to rank sentences, a simple inner product is calculated between each sentence, a document vector (text topic) and a lexical vector (vocabulary used by a sentence). Summaries are then generated by assembling the highest ranked sentences. No ruled-based linguistic post-processing is necessary in order to obtain summaries. Tests over several datasets (coming from Document Understanding Conferences (DUC), Text Analysis Conferences (TAC), evaluation campaigns, etc.) in French, English and Spanish have shown that summarizer achieves interesting results.
. Simple rule-based method is used for sentence splitting. Documents are chunked at the period, exclamation and question mark. . Words lowercased and cleared up from sloppy punctuation. Words with less than 2 occurrences ( @math ) are eliminated ( Hapax legomenon presents once in a document). Words that do not carry meaning such as functional or very common words are removed. Small stop-lists (depending of language) are used in this step. . Remaining words are replaced by their canonical form using lemmatization, stemming, ultra-stemming or none of them (raw text). Four methods of normalization were applied after filtering: Lemmatization by simples dictionaries of morphological families. These dictionaries have 1.32M, 208K and 316K words-entries in Spanish, English and French, respectively. Porter's Stemming, available at Snowball (web site http: snowball.tartarus.org texts stemmersoverview.html ) for English, Spanish, French among other languages. Ultra-stemming. This normalization seems be very efficient and it produces a compact matrix representation @cite_17 . Ultra-stemming consider only the @math first letters of each word. For example, in the case of ultra-stemming (first letter, Fix @math ), inflected verbs like sing'', song'', sings'', singing''... or proper names smith'', snowboard'', sex'',... are replaced by the letter s ''.
@cite_17
1579835159
In Automatic Text Summarization, preprocessing is an important phase to reduce the space of textual representation. Classically, stemming and lemmatization have been widely used for normalizing words. However, even using normalization on large texts, the curse of dimensionality can disturb the performance of summarizers. This paper describes a new method for normalization of words to further reduce the space of representation. We propose to reduce each word to its initial letters, as a form of Ultra-stemming. The results show that Ultra-stemming not only preserve the content of summaries produced by this representation, but often the performances of the systems can be dramatically improved. Summaries on trilingual corpora were evaluated automatically with Fresa. Results confirm an increase in the performance, regardless of summarizer system used.