Multimodality can make (especially mobile) device interaction more efficient. Sensors and communication capabilities of modern smartphones and tablets lay the technical basis for its implementation. Still, mobile platforms do not make multimodal interaction support trivial. Building multimodal applications requires various APIs with different paradigms, high-level interpretation of contextual data, and a method for fusing individual inputs and outputs. To reduce this effort, we created a framework that simplifies and accelerates the creation of multimodal applications for prototyping and research. It provides an abstraction of information representations in different modalities, unifies access to implicit and explicit information, and wires together the logic behind context-sensitive modality switches. In the paper, we present the structure and features of our framework, and validate it by four implemented demonstrations of different complexity.
For rapid development of context-sensitive applications in the research context, several toolkits and frameworks have been presented, starting with the by @cite_1 . The toolkit by Fogarty and Hudson @cite_5 supports the development of applications that use sensor-based statistical models. Thereby, contextual information can be used to adapt certain settings automatically. With our framework, we focus on wiring of functionality, as often the impacting factors for a desired action are quite clear. There exist alternative approaches (e.g., support vector machines, Bayesian or neural networks), but machine learning often makes it hard to understand a certain action has been performed. Some toolkits address particular use cases, e.g. physical mobile interaction @cite_19 or proxemic interaction @cite_10 . Du and Wang @cite_16 developed a model and implementation framework to simulate context events with Symbian phones. used JCOP (Context-Oriented Programming) extensions to the Java language that support context-dependent execution for Android programming @cite_14 , which, however, complicates a quick integration in any existing Android project and working environment.
@cite_14 @cite_1 @cite_19 @cite_5 @cite_16 @cite_10
2006015625 2163419627 656949909 2079397285 2090764929 2132168989
The behavior of mobile applications is particularly affected by their execution context, such as location and state a the mobile device. Among other approaches, context-oriented programming can help to achieve context-dependent behavior without sacrificing modularity or adhering to a certain framework or library by enabling fine-grained adaptation of default behavior per control-flow. However, context information relevant for mobile applications is mostly defined by external events and sensor data rather than by code and control flow. To accommodate this, the JCop language provides a more declarative approach by pointcut-like adaptation rules. In this paper, we explain how we applied JCop to the development of Android applications for which we extended the language semantics for static contexts and modified the compiler. Additionally, we outline the successful implementation of a simple, proof-of-concept mobile application using our approach and report on promising early evaluation results. Computing devices and applications are now used beyond the desktop, in diverse environments, and this trend toward ubiquitous computing is accelerating. One challenge that remains in this emerging research field is the ability to enhance the behavior of any application by informing it of the context of its use. By context, we refer to any information that characterizes a situation related to the interaction between humans, applications, and the surrounding environment. Context-aware applications promise richer and easier interaction, but the current state of research in this field is still far removed from that vision. This is due to 3 main problems: (a) the notion of context is still ill defined, (b) there is a lack of conceptual models and methods to help drive the design of context-aware applications, and (c) no tools are available to jump-start the development of context-aware applications. In this anchor article, we address these 3 problems in turn. We first define context, identify categories of contextual information, and characterize context-aware application behavior. Though the full impact of context-aware computing requires understanding very subtle and high-level notions of context, we are focusing our efforts on the pieces of context that can be inferred automatically from sensors in a physical environment. We then present a conceptual framework that separates the acquisition and representation of context from the delivery and reaction to context by a context-aware application. We have built a toolkit, the Context Toolkit, that instantiates this conceptual framework and supports the rapid development of a rich space of context-aware applications. We illustrate the usefulness of the conceptual framework by describing a number of context-aware applications that have been prototyped using the Context Toolkit. We also demonstrate how such a framework can support the investigation of important research challenges in the area of context-aware computing. Mobile interactions with the real world, meaning a person using her mobile device as a mediator for the interaction with smart objects, is becoming more and more popular in industry and academia. Typical technologies supporting this kind of interactions are Radio Frequency Identification (RFID), Near Field Communication (NFC), visual marker recognition, or Bluetooth. Currently, there is only very little tool support for developing systems based on this kind of mobile interactions. In this paper we motivate such tool support, discuss its requirements and present the architecture and implementation of the Physical Mobile Interaction Framework (PMIF). This framework comprises several components that support different implementations of the interaction techniques touching, pointing, scanning and user-mediated object selection. In addition to that PMIF also abstracts from specific techniques and technologies and provides a generic framework for the uniform integration and simple use of the supported interaction techniques Furthermore we discuss seven prototypes that were implemented using PMIF which show the maturity of the framework. Sensor based statistical models promise to support a variety of advances in human computer interaction, but building applications that use them is currently difficult and potential advances go unexplored. We present Subtle, a toolkit that removes some of the obstacles to developing and deploying applications using sensor based statistical models of human situations. Subtle provides an appropriate and extensible sensing library, continuous learning of personalized models, fully automated high level feature generation, and support for using learned models in deployed applications. By removing obstacles to developing and deploying sensor based statistical models, Subtle makes it easier to explore the design space surrounding sensor based statistical models of human situations. Subtle thus helps to move the focus of human computer interaction research onto applications and datasets, instead of the difficulties of developing and deploying sensor based statistical models. This research aims at facilitating the development of context-aware application software for mobile devices by providing a programming model, an implementation framework and a development environment. The programming model provides a multi-layered software architecture for context-aware application programming. The model supports developers to define contexts, behaviors and context-behavior binding rules through specifications and automates generation of context-aware application code based on the specifications. The implementation framework is a backbone program that implements the programming model. It facilitates the development in reducing the effort on the common tasks of context-awareness and help developers focus on the application-specific components. The development environment provides a series of tools to support the development of context-aware applications. These tools simplify the development process and provide the developed applications with robustness and testability. People naturally understand and use proxemic relationships (e.g., their distance and orientation towards others) in everyday situations. However, only few ubiquitous computing (ubicomp) systems interpret such proxemic relationships to mediate interaction (proxemic interaction). A technical problem is that developers find it challenging and tedious to access proxemic information from sensors. Our Proximity Toolkit solves this problem. It simplifies the exploration of interaction techniques by supplying fine-grained proxemic information between people, portable devices, large interactive surfaces, and other non-digital objects in a room-sized environment. The toolkit offers three key features. 1) It facilitates rapid prototyping of proxemic-aware systems by supplying developers with the orientation, distance, motion, identity, and location information between entities. 2) It includes various tools, such as a visual monitoring tool, that allows developers to visually observe, record and explore proxemic relationships in 3D space. (3) Its flexible architecture separates sensing hardware from the proxemic data model derived from these sensors, which means that a variety of sensing technologies can be substituted or combined to derive proxemic information. We illustrate the versatility of the toolkit with proxemic-aware systems built by students.