We present the design and prototype implementation of a scientific visualization language called Zifazah for composing 3D visualizations of diffusion tensor magnetic resonance imaging (DT-MRI or DTI) data. Unlike existing tools allowing flexible customization of data visualizations that are programmer-oriented, we focus on domain scientists as end users in order to enable them to freely compose visualizations of their scientific data set. We analyzed end-user descriptions extracted from interviews with neurologists and physicians conducting clinical practices using DTI about how they would build and use DTI visualizations to collect syntax and semantics for the language design, and have discovered the elements and structure of the proposed language. Zifazah makes use of the initial set of lexical terms and semantics to provide a declarative language in the spirit of intuitive syntax and usage. This work contributes three, among others, main design principles for scientific visualization language design as well as a practice of such language for DTI visualization with Zifazah. First, Zifazah incorporated visual symbolic mapping based on color, size and shape, which is a sub-set of Bertin's taxonomy migrated to scientific visualizations. Second, Zifazah is defined as a spatial language whereby lexical representation of spatial relationship for 3D object visualization and manipulations, which is characteristic of scientific data, can be programmed. Third, built on top of Bertin's semiology, flexible data encoding specifically for scientific visualizations is integrated in our language in order to allow end users to achieve optimal visual composition at their best. Along with sample scripts representative of our language design features, some new DTI visualizations as the running results created by end users using the novel visualization language have also been presented.
Recently Metoyer et. al. @cite_29 report from an exploratory study a set of design implications for the design of visualization languages and toolkits. More specifically, their findings inform visualization language design through the way end users describe visualizations and their inclination to using ambiguous and relative, instead of definite and absolute, terms that can be refined later via a feedback loop provided by the language. Emphatically, their findings also disclose that end users tend to express in generally high-level semantics. During the design of our visualization language, we have benefited from these findings and actually have reflected them in the development of .
@cite_29
1519642193 2486648592 2273038706 2963890019
While information visualization tools support the representation of abstract data, their ability to enhance one’s understanding of complex relationships can be hindered by a limited set of predefined charts. To enable novel visualization over multiple variables, we propose a declarative language for specifying informational graphics from first principles. The language maps properties of generic objects to graphical representations based on scaled interpretations of data values. An iterative approach to constraint solving that involves user advice enables the optimization of graphic layouts. The flexibility and expressiveness of a powerful but relatively easy to use grammar supports the expression of visualizations ranging from the simple to the complex. Despite the recent popularity of visual analytics focusing on big data, little is known about how to support users that use visualization techniques to explore multi-dimensional datasets and accomplish specific tasks. Our lack of models that can assist end-users during the data exploration process has made it challenging to learn from the user's interactive and analytical process. The ability to model how a user interacts with a specific visualization technique and what difficulties they face are paramount in supporting individuals with discovering new patterns within their complex datasets. This paper introduces the notion of visualization systems understanding and modeling user interactions with the intent of guiding a user through a task thereby enhancing visual data exploration. The challenges faced and the necessary future steps to take are discussed; and to provide a working example, a grammar-based model is presented that can learn from user interactions, determine the common patterns among a number of subjects using a K-Reversible algorithm, build a set of rules, and apply those rules in the form of suggestions to new users with the goal of guiding them along their visual analytic process. A formal evaluation study with 300 subjects was performed showing that our grammar-based model is effective at capturing the interactive process followed by users and that further research in this area has the potential to positively impact how users interact with a visualization system. The complex compositional structure of language makes problems at the intersection of vision and language challenging. But language also provides a strong prior that can result in good superficial performance, without the underlying models truly understanding the visual content. This can hinder progress in pushing state of art in the computer vision aspects of multi-modal AI. In this paper, we address binary Visual Question Answering (VQA) on abstract scenes. We formulate this problem as visual verification of concepts inquired in the questions. Specifically, we convert the question to a tuple that concisely summarizes the visual concept to be detected in the image. If the concept can be found in the image, the answer to the question is "yes", and otherwise "no". Abstract scenes play two roles (1) They allow us to focus on the high-level semantics of the VQA task as opposed to the low-level recognition problems, and perhaps more importantly, (2) They provide us the modality to balance the dataset such that language priors are controlled, and the role of vision is essential. In particular, we collect fine-grained pairs of scenes for every question, such that the answer to the question is "yes" for one scene, and "no" for the other for the exact same question. Indeed, language priors alone do not perform better than chance on our balanced dataset. Moreover, our proposed approach matches the performance of a state-of-the-art VQA approach on the unbalanced dataset, and outperforms it on the balanced dataset. The complex compositional structure of language makes problems at the intersection of vision and language challenging. But language also provides a strong prior that can result in good superficial performance, without the underlying models truly understanding the visual content. This can hinder progress in pushing state of art in the computer vision aspects of multi-modal AI. In this paper, we address binary Visual Question Answering (VQA) on abstract scenes. We formulate this problem as visual verification of concepts inquired in the questions. Specifically, we convert the question to a tuple that concisely summarizes the visual concept to be detected in the image. If the concept can be found in the image, the answer to the question is "yes", and otherwise "no". Abstract scenes play two roles (1) They allow us to focus on the highlevel semantics of the VQA task as opposed to the low-level recognition problems, and perhaps more importantly, (2) They provide us the modality to balance the dataset such that language priors are controlled, and the role of vision is essential. In particular, we collect fine-grained pairs of scenes for every question, such that the answer to the question is "yes" for one scene, and "no" for the other for the exact same question. Indeed, language priors alone do not perform better than chance on our balanced dataset. Moreover, our proposed approach matches the performance of a state-of-the-art VQA approach on the unbalanced dataset, and outperforms it on the balanced dataset.