Visualizing Natural Language Descriptions: A Survey <s> PAR <s> We describe a workbench (XTAG) for the development of tree-adjoining grammars and their parsers, and discuss some issues that arise in the design of the graphical interface.Contrary to string rewriting grammars generating trees, the elementary objects manipulated by a tree-adjoining grammar are extended trees (i.e. trees of depth one or more) which capture syntactic information of lexical items. The unique characteristics of tree-adjoining grammars, its elementary objects found in the lexicon (extended trees) and the derivational history of derived trees (also a tree), require a specially crafted interface in which the perspective has shifted from a string-based to a tree-based system. XTAG provides such a graphical interface in which the elementary objects are trees (or tree sets) and not symbols (or strings).The kernel of XTAG is a predictive left to right parser for unification-based tree-adjoining grammar [Schabes, 1991]. XTAG includes a graphical editor for trees, a graphical tree printer, utilities for manipulating and displaying feature structures for unification-based tree-adjoining grammar, facilities for keeping track of the derivational history of TAG trees combined with adjoining and substitution, a parser for unification based tree-adjoining grammars, utilities for defining grammars and lexicons for tree-adjoining grammars, a morphological recognizer for English (75 000 stems deriving 280 000 inflected forms) and a tree-adjoining grammar for English that covers a large range of linguistic phenomena.Considerations of portability, efficiency, homogeneity and ease of maintenance, lead us to the use of Common Lisp without its object language addition and to the use of the X Window interface to Common Lisp (CLX) for the implementation of XTAG.XTAG without the large morphological and syntactic lexicons is public domain software. The large morphological and syntactic lexicons can be obtained through an agreement with ACL's Data Collection Initiative.XTAG runs under Common Lisp and X Window (CLX). <s> BIB001 </s> Visualizing Natural Language Descriptions: A Survey <s> PAR <s> This book describes a continuing research effort at the University of Pennsylvania to model and control simulated humanoid figures. Profs. Badler and Webber have directed this work for many years; Dr. Phillips did his Ph.D. work under Prof. Badler. The Badler-Webber tandem was one of the first examples of what is now something of a trend in academe---collaboration between researchers in computer graphics and AI, especially natural-language processing. At Penn, this collaboration has focused on the long-term goal of:b creating a biomechanically realistic human model;b facilitating behavioral control of the model; andb using natural language to guide and instruct the model. <s> BIB002 </s> Visualizing Natural Language Descriptions: A Survey <s> PAR <s> The computation speed and control methods needed to portray 3D virtual humans suitable for interactive applications have improved dramatically in recent years. Real-time virtual humans show increasingly complex features along the dimensions of appearance, function, time, autonomy, and individuality. The virtual human architecture we’ve been developing at the University of Pennsylvania is representative of an emerging generation of such architectures and includes low-level motor skills, a mid-level parallel automata controller, and a high-level conceptual representation for driving virtual humans through complex tasks. The architecture—called Jack— provides a level of abstraction generic enough to encompass natural-language instruction representation as well as direct links from those instructions to animation control. <s> BIB003 </s> Visualizing Natural Language Descriptions: A Survey <s> PAR <s> Smart avatars are virtual human representations controlled by real people. Given instructions interactively, smart avatars can act as autonomous or reactive agents. During a real-time simulation, a user should be able to dynamically refine his or her avatar’s behavior in reaction to simulated stimuli without having to undertake a lengthy off-line programming session. In this paper, we introduce an architecture, which allows users to input immediate or persistent instructions using natural language and see the agents’ resulting behavioral changes in the graphical output of the simulation. <s> BIB004
Parameterized Action Representation (PAR) developed at the University of Pennsylvania, is a framework for controlling virtual humans using natural language commands in a context-sensitive fashion  BIB003 BIB004 . The main focus of PAR is to develop a comprehensive knowledge representation scheme to reflect the input commands on agents' behaviors. The structure of PAR contains applicability condition, start, results, participants, semantics, path, purpose, termination, duration, manner, sub-actions, parent action, previous action, concurrent action, and next action  . In this representation, the applicability condition is a Boolean expression that indicates the feasibility of an action for a given agent. The start and result indicate the states and time stamps of a given action performed by the agent and the beginning and termination of that action. The participants refer to the agent that is executing the current PAR and the passive objects that are related to that action. The semantics include a set of Boolean pre-conditions and post-conditions of an action that must be satisfied to let the agent perform that action. They also embed the motion and the force of the action that should be applied. The path denotes the start and end points, direction, and distance of a motion. The purpose determines whether the current action should satisfy a set of conditions or trigger another action and the termination determines the condition for terminating the current action. PAR structure also contains a set of pointers to other PARs, including parent, next, previous, and concurrent actions. The execution architecture of PAR implemented in C++ and Python is a reactive framework designed to handle the PAR representation. The architecture consists of five components, including language converter, database manager, execution engine, agent process, and visualizer. The language converter parses the input commands using an XTAG parser BIB001 and uses a naive string matching algorithm to find the corresponding 3D objects and agents from the database through the database manager. It also captures the verbs and the adjectives in order to construct the corresponding PAR representation of the given verbal command. The execution engine synchronizes the actions using its universal clock and passes the received PAR structures to the corresponding agent processes. It also controls the visualizer to update the environment. Each active agent within the virtual world is assigned with an agent process that handles a queue of PARs (i.e., PatNet data structure) to be executed BIB002 . The visualizer uses OpenGL to render the virtual world and its inhabitants are based on the received commands from the execution engine. The PAR architecture relies on shallow parsing rather than attempting to capture the deep semantics. It also does not support the deliberative planning that is essential for generating plans for complicated goals. The lack of interactivity is another drawback of this architecture.