The software of robotic assistants needs to be verified, to ensure its safety and functional correctness. Testing in simulation allows a high degree of realism in the verification. However, generating tests that cover both interesting foreseen and unforeseen scenarios in human-robot interaction (HRI) tasks, while executing most of the code, remains a challenge. We propose the use of belief-desire-intention (BDI) agents in the test environment, to increase the level of realism and human-like stimulation of simulated robots. Artificial intelligence, such as agent theory, can be exploited for more intelligent test generation. An automated testbench was implemented for a simulation in Robot Operating System (ROS) and Gazebo, of a cooperative table assembly task between a humanoid robot and a person. Requirements were verified for this task, and some unexpected design issues were discovered, leading to possible code improvements. Our results highlight the practicality of BDI agents to automatically generate valid and human-like tests to get high code coverage, compared to hand-written directed tests, pseudorandom generation, and other variants of model-based test generation. Also, BDI agents allow the coverage of combined behaviours of the HRI system with more ease than writing temporal logic properties for model checking.
To the best of our knowledge, BDI agents have not been used as the modelling formalism for model-based test generation before. A multi-agent framework has been proposed in @cite_31 for model-based test generation in software testing. Agent programs are in charge of exploring a UML model of the code, generating all the scenarios of the if-then-else conditions and branches. BDI agents have been tested, with respect to the interaction behaviours of multi-agent systems @cite_30 , or single agents (units) in terms of the correctness of their beliefs (e.g., value combinations), plans (e.g., triggering the correct plan according to the context), and events or messages (e.g., sending them at the right time) @cite_12 @cite_3 @cite_26 . In this paper we turn the table and introduce BDI agents into the test environment, for intuitive and effective test generation.
@cite_30 @cite_26 @cite_3 @cite_31 @cite_12
2949647627 1983537749 1497120980 2049199149
The challenges of robotic software testing extend beyond conventional software testing. Valid, realistic and interesting tests need to be generated for multiple programs and hardware running concurrently, deployed into dynamic environments with people. We investigate the use of BeliefDesire-Intention (BDI) agents as models for test generation, in the domain of human-robot interaction (HRI) in simulations. These models provide rational agency, causality, and a reasoning mechanism for planning, which emulate both intelligent and adaptive robots, as well as smart testing environments directed by humans. We introduce reinforcement learning (RL) to automate the exploration of the BDI models using a reward function based on coverage feedback. Our approach is evaluated using a collaborative manufacture example, where the robotic software under test is stimulated indirectly via a simulated human co-worker. We conclude that BDI agents provide intuitive models for test generation in the HRI domain. Our results demonstrate that RL can fully automate BDI model exploration, leading to very effective coverage-directed test generation. Software testing remains the most widely used approach to verification in industry today, consuming between 30-50 percent of the entire development cost. Test input selection for intelligent agents presents a problem due to the very fact that the agents are intended to operate robustly under conditions which developers did not consider and would therefore be unlikely to test. Using methods to automatically generate and execute tests is one way to provide coverage of many conditions without significantly increasing cost. However, one problem using automatic generation and execution of tests is the oracle problem: How can we automatically decide if observed program behavior is correct with respect to its specification? In this paper, we present a model-based oracle generation method for unit testing belief-desire-intention agents. We develop a fault model based on the features of the core units to capture the types of faults that may be encountered and define how to automatically generate a partial, passive oracle from the agent design models. We evaluate both the fault model and the oracle generation by testing 14 agent systems. Over 400 issues were raised, and these were analyzed to ascertain whether they represented genuine faults or were false positives. We found that over 70 percent of issues raised were indicative of problems in either the design or the code. Of the 19 checks performed by our oracle, faults were found by all but 5 of these checks. We also found that 8 out the 11 fault types identified in our fault model exhibited at least one fault. The evaluation indicates that the fault model is a productive conceptualization of the problems to be expected in agent unit testing and that the oracle is able to find a substantial number of such faults with relatively small overhead in terms of false positives. Preface. 1 Introduction. 1.1 Autonomous Agents. 1.2 Characteristics of Agents. 1.3 Multi-Agent Systems. 1.4 Hello World! 2 The BDI Agent Model. 2.1 Agent-Oriented Programming. 2.2 Practical Reasoning. 2.3 A Computational Model of BDI Practical Reasoning. 2.4 The Procedural Reasoning System. 2.5 Agent Communication. 3 The Jason Agent Programming Language. 3.1 Beliefs. 3.2 Goals. 3.3 Plans. 3.4 Example: A Complete Agent Program. 3.5 Exercises. 4 Jason Interpreter. 4.1 The Reasoning Cycle. 4.2 Plan Failure. 4.3 Interpreter Configuration and Execution Modes. 4.4 Pre-Defined Plan Annotations. 4.5 Exercises. 5 Environments. 5.1 Support for Defining Simulated Environments. 5.2 Example: Running a System of Multiple Situated Agents. 5.3 Exercises. 6 Communication and Interaction. 6.1 Available Performatives. 6.2 Informal Semantics of Receiving Messages. 6.3 Example: Contract Net Protocol. 6.4 Exercises. 7 User-Defined Components. 7.1 Defining New Internal Actions. 7.2 Customising the Agent Class. 7.3 Customising the Overall Architecture. 7.4 Customising the Belief Base. 7.5 Pre-Processing Directives. 7.6 Exercises. 8 Advanced Goal-Based Programming. 8.1 BDI Programming. 8.2 Declarative (Achievement) Goal Patterns. 8.3 Commitment Strategy Patterns. 8.4 Other Useful Patterns. 8.5 Pre-Processing Directives for Plan Patterns. 9 Case Studies. 9.1 Case Study I: Gold Miners. 9.2 Case Study II: Electronic Bookstore. 10 Formal Semantics. 10.1 Semantic Rules. 10.2 Semantics of Message Exchange in a Multi-Agent System. 10.3 Semantic Rules for Receiving Messages. 10.4 Semantics of the BDI Modalities for AgentSpeak. 11 Conclusions. 11.1 Jason and Agent-Oriented Programming. 11.2 Ongoing Work and Related Research. 11.3 General Advice on Programming Style and Practice. A Reference Guide. A.1 EBNF for the Agent Language. A.2 EBNF for the Multi-Agent Systems Language. A.3 Standard Internal Actions. A.4 Pre-Defined Annotations. A.5 Pre-Processing Directives. A.6 Interpreter Configuration. Bibliography. This article discusses a formal belief, desire, intention (BDI)-based agent model for theory of mind (ToM). The model uses BDI concepts to describe the reasoning process of an agent that reasons about the reasoning process of another agent, which is also based on BDI concepts. We discuss three different application areas and illustrate how the model can be applied to each of them. We explore a case study for each of the application areas and apply our model to it. For each case study, a number of simulation experiments are described, and their results are discussed.