Visualizing Natural Language Descriptions: A Survey <s> Story Picturing Engine <s> We present here SIMPLIcity (semantics-sensitive integrated matching for picture libraries), an image retrieval system, which uses semantics classification methods, a wavelet-based approach for feature extraction, and integrated region matching based upon image segmentation. An image is represented by a set of regions, roughly corresponding to objects, which are characterized by color, texture, shape, and location. The system classifies images into semantic categories. Potentially, the categorization enhances retrieval by permitting semantically-adaptive searching methods and narrowing down the searching range in a database. A measure for the overall similarity between images is developed using a region-matching scheme that integrates properties of all the regions in the images. The application of SIMPLIcity to several databases has demonstrated that our system performs significantly better and faster than existing ones. The system is fairly robust to image alterations. <s> BIB001 </s> Visualizing Natural Language Descriptions: A Survey <s> Story Picturing Engine <s> In this paper, we present an approach towards automated story picturing based on mutual reinforcement principle. Story picturing refers to the process of illustrating a story with suitable pictures. In our approach, semantic keywords are extracted from the story text and an annotated image database is searched to form an initial picture pool. Thereafter, a novel image ranking scheme automatically determines the importance of each image. Both lexical annotations and visual content of an image play a role in determining its rank. Annotations are processed using the Wordnet to derive a lexical signature for each image. An integrated region based similarity is also calculated between each pair of images. An overall similarity measure is formed using lexical and visual features. In the end, a mutual reinforcement based rank is calculated for each image using the image similarity matrix. We also present a human behavior model based on a discrete state Markov process which captures the intuition for our technique. Experimental results have demonstrated the effectiveness of our scheme <s> BIB002 </s> Visualizing Natural Language Descriptions: A Survey <s> Story Picturing Engine <s> We present an unsupervised approach to automated story picturing. Semantic keywords are extracted from the story, an annotated image database is searched. Thereafter, a novel image ranking scheme automatically determines the importance of each image. Both lexical annotations and visual content play a role in determining the ranks. Annotations are processed using the Wordnet. A mutual reinforcement-based rank is calculated for each image. We have implemented the methods in our Story Picturing Engine (SPE) system. Experiments on large-scale image databases are reported. A user study has been performed and statistical analysis of the results has been presented. <s> BIB003
The story picturing engine BIB002 BIB003 addresses the mapping process of a given textual story to a set of representative pictures by focusing on " quantifying image importance in a pool of images." This system receives input stories such as "Vermont is mostly a rural state. The countryside has the cozy feeling of a place which . . . " BIB003 and ranks the related and available images accordingly as the output. This system is a pipeline of three processes as follows. First, the descriptor keywords are extracted from the story. For this purpose, the stop-words are eliminated using a manually crafted dictionary and then a subset of the remaining words is selected based on a combination of bag-of-words model and named-entity recognition. The utilized bag-ofwords model uses WordNet to determine the polysemy count the number of senses 242 of a given word of the words. Among them, nouns, adjectives, adverbs, and verbs with a low polysemy count (i.e., less ambiguity) are selected as descriptor keywords. A naive named-entity recognizer is used to extract the proper nouns such as names of places and people based on the beginning letter of the words. Those images that contain at least one keyword and one named entity are retrieved from a local annotated image database. The next step in the pipeline is to estimate the similarity between pairs of images based on their visual and lexical features, which is calculated based on a linear combination of Integrated Region Matching (IRM) distance BIB001 and WordNet hierarchy. Finally, the images are ranked based on a mutual reinforcement method and top k ranked images are retrieved. This system is basically an image search engine that gets a given description as a query and retrieves and ranks the related images. Despite the good accuracy and performance of the story picturing engine, it only retrieves one picture for a given story and ignores many aspects such as the temporal or spatial relations.