Visualizing Natural Language Descriptions: A Survey <s> Other Systems <s> An animation system has been developed which generates animations from stories written in natural language. The system consists of three modules: story understanding module, stage directing module and action generating module. The story understanding module extracts actions that are not explicitly described in the story and makes a scenario. The stage directing module adapts the scenario by determining the actors' positions on the stage and setting the stage. Actors are defined as 3-Dimensional articulated figures. Each component of an actor has its primitive motion method. To achieve complicated actions, primitive motions are combined. Referring to these complicated actions, the action generating module produces animated sequences from the adapted scenario. These three modules are tightly coupled with their knowledge bases. As an example, the story of the “Hare and Tortoise” from Aesop's Fables, written for elementary schoolchildren is used. This example proves that it is possible to produce computer animation directly from the story written in natural language, now in Japanese. <s> BIB001 </s> Visualizing Natural Language Descriptions: A Survey <s> Other Systems <s> When speech understanding systems are used in real applications, they will have to deal with phenomena peculiar to spontaneous speech. People use language differently when they speak than when they write. Spoken language contains many interjections, filled pauses, etc. Speakers often don't use well-formed sentences. They speak in phrases, have restarts, etc. Systems designed for written or read text will encounter serious difficulties processing such input. This paper outlines our strategy for dealing with spontaneous spoken input in a speech recognition system. <s> BIB002 </s> Visualizing Natural Language Descriptions: A Survey <s> Other Systems <s> This paper presents a statistical model which trains from a corpus annotated with Part Of Speech tags and assigns them to previously unseen text with state of the art accuracy The model can be classi ed as a Maximum Entropy model and simultaneously uses many contextual features to predict the POS tag Furthermore this paper demonstrates the use of specialized fea tures to model di cult tagging decisions discusses the corpus consistency problems discovered during the implementation of these features and proposes a training strategy that mitigates these problems <s> BIB003 </s> Visualizing Natural Language Descriptions: A Survey <s> Other Systems <s> INGENIAS is both a methodology and a set of tools for development of multi-agent systems (MAS). As a methodology, it tries to integrate results from other proposals and considers the MAS from five complementary viewpoints: organization, agent, tasks/goals, interactions, and environment. It is supported by a set of tools for modelling (graphical editor), documentation and code generation (for different agent platforms). INGENIAS is the result of the experience developing MAS in different areas, such as workflow management systems, recommender systems, Robocode teams, and PC assistants <s> BIB004 </s> Visualizing Natural Language Descriptions: A Survey <s> Other Systems <s> Visualizing natural language description is a difficult and complex task. When dealing with the process of generating images from natural language descriptions, we firstly should consider the real world and find out what key visual information can be extracted from the sentences which represents the most fundament concepts in both virtual and real environments. In this paper, we present the result of a prototype system called 3DSV (3D Story Visualiser) that generates a virtual scene by using simplified story-based descriptions. In particular, we describe the methodology used to parameterize the visual and describable words into XML formatted data structure. Then we discuss how to interpret the parameterized data and create an interactive real-time 3D virtual environment. <s> BIB005 </s> Visualizing Natural Language Descriptions: A Survey <s> Other Systems <s> This paper describes a virtual storytelling medium, called Interactive e-Hon, for helping children understand content from the Web or other electronic devices. It works by transforming electronic content into an easily understandable “storybook world.”In this world, easy-to-understand content is generated by creating 3D animations that include content and metaphors, and by using a child-parent model with dialogue expression and a question-answering style comprehensible to children. <s> BIB006 </s> Visualizing Natural Language Descriptions: A Survey <s> Other Systems <s> This paper describes a system, called Interactive e-Hon, for helping children understand difficult content. It works by transforming text into an easily understandable storybook style with animation and dialogue. In this system, easy-to-understand content is created by a semantic tag generator through natural language processing, an animation generator using an animation achieve and animation tables, a dialogue generator using semantic tag information, and a story generator using the Soar AI engine. Through the results of an experiment, this paper describes the advantages of attracting interest, supporting and facilitating understanding, and improving parent-child communication by using Interactive e-Hon. <s> BIB007 </s> Visualizing Natural Language Descriptions: A Survey <s> Other Systems <s> Visualization and interaction give virtual environments the capacity to represent and communicate information with more realistic images and more interaction. However, there is a lack of low cost and easy to use tools for non-professionals to create their own interactive 3D virtual environments. In this paper, we present a prototype system called 3DSV (3D Story Visualiser) that generates a virtual scene by using simplified language descriptions. This prototype system is based on the integration of advances in text-to-visualization and computer graphics technology to generate 3D virtual environment. This makes it relative easier and faster for non-professionals to create 3D environments. <s> BIB008 </s> Visualizing Natural Language Descriptions: A Survey <s> Other Systems <s> Using the Storytelling Alice programming environment to create computer-animated movies inspires middle school girls' interest in learning to program computers. <s> BIB009 </s> Visualizing Natural Language Descriptions: A Survey <s> Other Systems <s> This paper describes a strategy for automatically converting fiction text into 3D animations. It assumes the existence of fiction text annotated with avatar, object, setting, transition and relation annotations, and presents a transformation process that converts annotated text into quantified constraint systems, the solutions to which are used in the population of 3D environments. Constraint solutions are valid over temporal intervals, ensuring that consistent dynamic behaviour is produced. A substantial level of automation is achieved, while providing opportunities for creative manual intervention in animation process. The process is demonstrated using annotated examples drawn from popular fiction text that are converted into animation sequences, confirming that the desired results can be achieved with only high-level human direction. <s> BIB010 </s> Visualizing Natural Language Descriptions: A Survey <s> Other Systems <s> In this paper, we propose a machine learning-based NLP system for automatically creating animated storyboards using the action descriptions of movie scripts. We focus particularly on the importance of verb semantics when generating graphics commands, and find that semantic role labelling boosts performance and is relatively robust to the effects of unseen verbs. <s> BIB011 </s> Visualizing Natural Language Descriptions: A Survey <s> Other Systems <s> The last years have been very prolific in the study of methods, methodologies and even development process in the area of Agent Oriented Software Engineering. However, there are two aspects that are not developed sufficiently, the first is the acquisition and establishment of requirements and goals for Multiagent Systems from the descriptions of customers / users. The second is a successful implementation of reverse engineering to implementations undertaken in order to retrieve a detailed design of the application obtained. The concern of this work focuses on the first of these aspects. The proposal applies techniques of natural language processing, throughout the entire lifecycle of multi-system, to obtain a model and improve product required. One noteworthy point of this proposal is that the client is actively involved in the development process in a manner non-attendance. Lastly, the proposal allows maximize obtaining initial models for different projects, minimizing their involvement in this process. <s> BIB012 </s> Visualizing Natural Language Descriptions: A Survey <s> Other Systems <s> In this paper, we introduce an animation production system, which generates a story from the web text in a specific domain and presents it in 3D animation. With our approach, 3D animation is generated in real-time from a web recipe in natural language. As a user starts the animation, only domain related texts are extracted using their associated tags and their semantics are analyzed by a semantic parser. The results of this semantic analysis are mapped to the computer animation commands through the action generation algorithms. To infer the hidden context within the text, we designed and leveraged the recipe ontology. A set of XML schemas has been designed to store different representations for web recipes; from the natural language representation to the animation representation. <s> BIB013 </s> Visualizing Natural Language Descriptions: A Survey <s> Other Systems <s> This paper builds on previous research into the development of the TMap3D system which allows for the creation of historical 3D spatiotemporal visualisations from natural language narratives [1]. This paper addresses the use of deductive 3D modelling techniques to generate the realism of the object behaviours, the environmental conditions and their interactions within the visualisation. The authors define deductive 3D modelling techniques as follows: (i) Given an object and its specification the Vist3D system (formally TMap3D) can deduce the object's visualised behaviour in response to a given "real world" instruction from the natural language narrative. (ii) The Vist3D system can deduce visualised environmental conditions from the natural language narrative. (iii) The Vist3D system can deduce the interactions between the objects (e.g. ship) and the environmental conditions (e.g. rough sea) to generate, for example, a ship rolling in rough seas. <s> BIB014 </s> Visualizing Natural Language Descriptions: A Survey <s> Other Systems <s> Usually people make a visual representation of what they read. This is a fact inherent to their nature. Many efforts have been done in the last years to try to find a direct method to translate a text wrote on a natural language to a graphical representation. Some solutions try to represent mainly a static scene with the objects appearing in the text, also there are few approaches that try to build an animation from the action described in text. But in any case all the approaches found on literature are not reciprocal approaches; that is, if the visual result is not satisfactory another text must be wrote in order to get a better one. Software Engineering and in particular Agent Oriented Software Engineering treat this kind of problematic using model driven techniques over different intermediates models. In this paper such kind of solution is proposed to get not only a visual model of the text wrote, but also to get an intermediate model that could be used to modify the visual model without modify the text and also to propose an alternative narration if necessary. In order to validate the visual model obtained, a previous multi-agent system has been adapted to provide a representation to be run on the Alice System. <s> BIB015 </s> Visualizing Natural Language Descriptions: A Survey <s> Other Systems <s> Creating 3D animation is a labor-intensive and time-consuming process requiring designers to learn and utilize a complex combination of menus, dialog boxes, buttons and manipulation interfaces for a given stand-alone animation design software. On the other hand, conceptual simplicity and naturalness of visualizing imaginations from lingual descriptions motivates researchers for developing automatic animation generation systems using natural language interfaces. In this research, we introduce an interactive and adaptive animation generation system that utilizes data-driven techniques to extract the required common-sense and domain-specific knowledge from web. This system is capable of creating 3D animation based on user's lingual commands. It uses the user interactions as a relevance feedback to learn the implicit design knowledge, correct the extracted knowledge, and manipulate the dynamics of the virtual world in an active and incremental manner. Moreover, system is designed based on a multi-agent methodology which provides it with distributed processing capabilities and cross-platform characteristics. In this paper, we will focus on information retrieval agent which is responsible for extracting numeric data utilized in object attributes, spatiotemporal relations, and environment dynamics using web mining techniques. <s> BIB016
One of the early text-to-animation synthesis systems is the Story Driven Animation System (SDAS) introduced in BIB001 . This Japanese system consists of three modules, including story understanding, stage directing, and action generating modules implemented in the Prolog and Lisp programming languages. The system input is restricted to unambiguous text that can only contain sentences describing actions in a time sequence. The story understanding module performs syntactic and semantic analyses. However, the original article does not explain the applied techniques. It also uses an assumption-based reasoning that adds very simple implicit assertions about the story. The stage direction module exploits a few simple heuristics to position the actors and set the background based on the extracted information and generated assertions. The action generating module uses a set of model descriptions and motion descriptions. A very limited set of simple articulated figures and primitive joint motions are defined and combined to create a simple animation. 3DSV  BIB005 BIB008 attempts to create an interactive interface for animating 3D stages of simple stories described in restricted sentences. The stage includes objects, their attributes, and simple spatial relations. The spatial relations are captured using a set of regular expressions and represented in XML format. 3DSV utilizes an XML-based knowledgebase to parametrize the extracted properties of the stage. The knowledgebase contains visual descriptions of the objects, attributes, and spatial relations. The information extracted from the input text and the knowledgebase are then integrated into an XML representation that is converted to the VRML format. The VRML is animated within a Java applet and lets the user manipulate the stage using mouse commands. Despite the simplicity and restricted nature of 3DSV, it provides the users with cross-platform functionalities. Interactive e-Hon BIB007 BIB006 is a Japanese multi-modal storytelling system designed for facilitating the interactions between parents and children by animating and explaining the difficult concepts in a simpler form using Web content. This system uses a Japanese morphological analyzer and a lexicon to extract time, space, weather, objects, and actions from the story. The extracted information is matched against two lookup tables, including a background table and an action table. The time, space, and weather are matched against the background table to provide the animation with the appropriate static background. The objects and the actions are matched against an action table that is used to retrieve a corresponding recorded animation from a database. This system mostly relies on lookup tables and binary matching algorithms, which severely limits its capability of semantic analyses. A semi-automatic system developed at Rhodes University  BIB010 generates animations from given annotated fiction texts. The basic assumption in this system is that the characters, objects, environment configuration, spatial relations, and the character transitions in the text are annotated in a well-formed structure in advance. It uses the annotations of the characters and the objects to query a 3D model database. The system exploits a query expansion mechanism using WordNet to enhance the possibility of finding proper models. It also uses annotated spatial information to construct a spatio-temporal constraint network. It provides the users with an interface to alter the constraint network to increase the artistic aspects of the generated animation. The layout constraints are satisfied using an incremental greedy algorithm. The system is developed in Python, and the extracted models and the environment are rendered using Blender3D. This system lets the user modify the animation by manipulating the constraints and provides a robust model matching scheme using the query expansion mechanism. On the other hand, it requires annotated fiction texts as its input, which is a labor-intensive and tedious task. A data-driven system developed at the University of Melbourne BIB011 attempts to train a classifier to ground high-level verbs into a set of low-level graphical tasks. To carry out this task, it extracts verb features, collocation features, and semantic role features from the scripts. It also extracts binary spatial features from the virtual stage. These linguistic and visual features are then used to train a maximum entropy classifier BIB003 to decide the next graphical action. Despite its interesting approach for co-training the semantic and stage features, it fails to provide a proper means of interaction. Web2Animation BIB013 is a multi-modal pedagogical system that uses Web content related to recipes to create an online animation to teach the users how to cook. Converting the Web content to an animation is done within three steps, including extracting relevant text, capturing semantics, and animating actions. The relevant recipe information is located by traversing the HTML tags and analyzed using the Phoenix parser BIB002 . The extracted instructions are mapped to a few actions and the captured ingredients are associated with a set of objects. A domain-specific ontology is utilized to match the actions with their graphical representation. The ingredients are also matched against their graphical models. Finally, a user-created screenplay is used to synchronize the animation with a monologue explaining the recipe. In this system, the user has to craft the screenplay, which interferes with its pedagogical purpose. Also, considering the noise-prone nature of Web content, it is not clear how well the system will behave in mining useful content. Vist3D BIB014  ] is a domain-specific system for creating 3D animation of historical naval battles from narratives provided by the users. The system uses a manually populated ship specification database and a temporal database. The temporal database is populated by a narrative analyzer that extracts the time and date and [Subject][V erb][Object] structures using a set of regular expressions. The retrieved information from these two databases is converted to VRML format. Vist3D is designed in a very restricted way. Similar to NALIG, it can only detect very simple syntactic structures and utilizes a very small dictionary. A different approach that relies on service-oriented and multi-agent design methodology is proposed in BIB015 . It models the agents using NLP4INGENIAS BIB012 , which is a multi-agent system based on the INGENIAS framework BIB004 . NLP4INGENIAS exploits natural language descriptions to model the agents and supports user-in-the-loop disambiguation of the descriptions. The acquired agent models are fed to Alice BIB009 -a rapid prototyping environment for generating virtual environments to render the world and agents. This system approaches the text-to-animation conversion problem from a software engineering point of view. It exploits agile software development using existing platforms rather than struggling with theoretical sophistications. On the other hand, it is restricted to the limitations of its building blocks and cannot tailor them to meet its specific requirements. An adaptive animation generation system is introduced in Hassani and Lee BIB016 . This system is a multi-agent and data-driven system that utilizes statistical Web content mining techniques for extracting the attribute values of objects such as relative sizes and velocities. The system consists of three interacting agents, including an information retrieval agent, a cognitive agent, and a language processing agent. The cognitive agent contains a knowledgebase and a planner to decide the actions. It also interacts with visualization interface in terms of high-level visual operations and perceptions. The authors mostly focus on the information retrieval agent and do not elaborate on the language processing agent. The reported accuracy of the retrieved results is promising. However, the results are only provided for simulating the solar system and it is not clear whether it can generalize to other scenarios as well. The language processing agent employs a set of regular expressions for extracting the embedded information and query generation.