Visualizing Natural Language Descriptions: A Survey <s> IVELL <s> In this paper, a general architecture is proposed for developing embodied conversational agents with fuzzy ontology knowledge base. The proposed architecture enables agents to interact with the user via multimodal channels in a virtual reality environment for the purpose of language learning. The agents play the role of emotional, rational, and friendly partners who provide a specific domain of knowledge based on user's queries in natural language. These queries are performed by an optimized fuzzy search engine. Two scenarios including two virtual airports and a virtual electronic gadget shop are implemented in this architecture to improve users' oral skills. The results show the users' average oral skills improved 11%. Moreover, 80% of the users ranked agents' logical sequence of actions and the total speed of responses as very good, and 90% of them evaluated agents' appropriateness of responses as very good based on Likert scale. <s> BIB001 </s> Visualizing Natural Language Descriptions: A Survey <s> IVELL <s> In this paper, we present an intelligent architecture, called intelligent virtual environment for language learning, with embedded pedagogical agents for improving listening and speaking skills of non-native English language learners. The proposed architecture integrates virtual environments into the Intelligent Computer-Assisted Language Learning. This architecture supports visual, auditory, and haptic channels of interaction. It allows pedagogical ideas about language skills to be implemented and validated with a minimum design time. Moreover, we design a computational model to evaluate learner's proficiency level, and an automatic adaptation mechanism which adjusts to the learner's learning curve. We have implemented two scenarios based on the proposed architecture to teach learners how to communicate in public places such as airports and TV stores. Inputs to this system include learner's speech and hand motion, and outputs include graphical scenes, force feedback, and speech by a few embodied agents. ... <s> BIB002
Intelligent Virtual Environment for Language Learning (IVELL) BIB001 BIB002 is a domainspecific multi-modal virtual reality system that consists of a few Embodied Conversational Agents (ECAs). It is designed to improve the speaking and listening skills of non-native users in English. IVELL implements a few scenarios, such as an airport and shopping mall, in which learners speak to domain-specific agents such as an immigration agent while manipulating the virtual world using a haptic robot. The agents can alter the difficulty level of the conversation by automatically evaluating the user's linguistic proficiency. Each agent consists of an abstract layer and an embodied layer. The abstract layer consists of a language interpreter, user evaluator, fuzzy knowledgebase, haptic interpreter, language generator, and action coordinator. The language interpreter lemmatizes and parses the inputs using the OpenNLP tool and matches the results against deterministic finite automata to capture the user's intentions. The user evaluator uses a weighted model to score the user's proficiency. The knowledgebase is a light domain-specific fuzzy ontology that keeps knowledge about the predefined tasks. The haptic interpreter maps the low-level force and position vectors acquired from a haptic robot to high-level perceptions. The language generator generates a set of answers with different difficulty levels based on the knowledge extracted from the knowledgebase. The action coordinator synchronizes the graphical actions, haptic actions, and output speech whose score is the closest to the user's proficiency level. The embodied layer contains a speech recognizer, a text-to-speech engine, a haptic interface, and an avatar controller. The system is developed in C#.Net and uses Autodesk 3DMax and 3DVIA Virtools to model and render the environment. Different modules within this system communicate synchronously through TCP/IP protocol which provides the system with distributed processing capabilities. IVELL is an interactive system that can adapt its interactions based on the user's proficiency level. It also utilizes a natural language generator to augment the interactions. Moreover, it asks user's help when it is not able to understand her utterance. Nevertheless, it uses a very limited approach to capture the semantics. Also similar to previous systems, it uses a set of limited and hardwired actions.