The software of robotic assistants needs to be verified, to ensure its safety and functional correctness. Testing in simulation allows a high degree of realism in the verification. However, generating tests that cover both interesting foreseen and unforeseen scenarios in human-robot interaction (HRI) tasks, while executing most of the code, remains a challenge. We propose the use of belief-desire-intention (BDI) agents in the test environment, to increase the level of realism and human-like stimulation of simulated robots. Artificial intelligence, such as agent theory, can be exploited for more intelligent test generation. An automated testbench was implemented for a simulation in Robot Operating System (ROS) and Gazebo, of a cooperative table assembly task between a humanoid robot and a person. Requirements were verified for this task, and some unexpected design issues were discovered, leading to possible code improvements. Our results highlight the practicality of BDI agents to automatically generate valid and human-like tests to get high code coverage, compared to hand-written directed tests, pseudorandom generation, and other variants of model-based test generation. Also, BDI agents allow the coverage of combined behaviours of the HRI system with more ease than writing temporal logic properties for model checking.
To the best of our knowledge, BDI agents have not been used as the modelling formalism for model-based test generation before. A multi-agent framework has been proposed in @cite_31 for model-based test generation in software testing. Agent programs are in charge of exploring a UML model of the code, generating all the scenarios of the if-then-else conditions and branches. BDI agents have been tested, with respect to the interaction behaviours of multi-agent systems @cite_30 , or single agents (units) in terms of the correctness of their beliefs (e.g., value combinations), plans (e.g., triggering the correct plan according to the context), and events or messages (e.g., sending them at the right time) @cite_12 @cite_3 @cite_26 . In this paper we turn the table and introduce BDI agents into the test environment, for intuitive and effective test generation.
@cite_30 @cite_26 @cite_3 @cite_31 @cite_12
2048949155 1983537749 2388593457 2005083693 2174334857
Autonomous agents perform on behalf of the user to achieve defined goals or objectives. They are situated in dynamic environment and are able to operate autonomously to achieve their goals. In a multiagent system, agents cooperate with each other to achieve a common goal. Testing of multiagent systems is a challenging task due to the autonomous and proactive behavior of agents. However, testing is required to build confidence into the working of a multiagent system. Prometheus methodology is a commonly used approach to design multiagents systems. Systematic and thorough testing of each interaction is necessary. This paper proposes a novel approach to testing of multiagent systems based on Prometheus design artifacts. In the proposed approach, different interactions between the agent and actors are considered to test the multiagent system. These interactions include percepts and actions along with messages between the agents which can be modeled in a protocol diagram. The protocol diagram is converted into a protocol graph, on which different coverage criteria are applied to generate test paths that cover interactions between the agents. A prototype tool has been developed to generate test paths from protocol graph according to the specified coverage criterion. Software testing remains the most widely used approach to verification in industry today, consuming between 30-50 percent of the entire development cost. Test input selection for intelligent agents presents a problem due to the very fact that the agents are intended to operate robustly under conditions which developers did not consider and would therefore be unlikely to test. Using methods to automatically generate and execute tests is one way to provide coverage of many conditions without significantly increasing cost. However, one problem using automatic generation and execution of tests is the oracle problem: How can we automatically decide if observed program behavior is correct with respect to its specification? In this paper, we present a model-based oracle generation method for unit testing belief-desire-intention agents. We develop a fault model based on the features of the core units to capture the types of faults that may be encountered and define how to automatically generate a partial, passive oracle from the agent design models. We evaluate both the fault model and the oracle generation by testing 14 agent systems. Over 400 issues were raised, and these were analyzed to ascertain whether they represented genuine faults or were false positives. We found that over 70 percent of issues raised were indicative of problems in either the design or the code. Of the 19 checks performed by our oracle, faults were found by all but 5 of these checks. We also found that 8 out the 11 fault types identified in our fault model exhibited at least one fault. The evaluation indicates that the fault model is a productive conceptualization of the problems to be expected in agent unit testing and that the oracle is able to find a substantial number of such faults with relatively small overhead in terms of false positives. Abstract Software testing is done to find errors and fix them to improve quality. Software testing is a laborious and time–consuming work which spends almost 50 of the software system development resources. The number of test cases required to develop error-free software is high and when done manually the process of test case generation is time consuming and error prone. As systems are increasing in complexity, more systems perform mission-critical functions and the dependability requirements such as safety, reliability and security are vital to the users of the system. The competitive market place forces companies to define new approaches to reduce time-to-market as well as development cost of the systems. To increase the effectiveness and efficiency of the testing process and to reduce the overall development cost for software system, new approaches are required for test automation. The automation of test case generation is the most important aspect of test automation. No powerful test data generation tools combining structural and functional testing are commercially available today. The objective of the proposed system is to automate the process of test case generation for both structural and model based testing by deployment of an agent based framework. This framework automatically generates an optimized Test Suite for system under test that can assist the testers and developers and enable them to correct the errors in earlier stages of software development. The deployment of agent based framework reduces considerable amount of execution time when compared with the system without agents. Although agent technology is gaining world wide popularity, a hindrance to its uptake is the lack of proper testing mechanisms for agent based systems. While many traditional software testing methods can be generalized to agent systems, there are many aspects that are different and which require an understanding of the underlying agent paradigm. In this paper we present certain aspects of a testing framework that we have developed for agent based systems. The testing framework is a model based approach using the design models of the Prometheus agent development methodology. In this paper we focus on unit testing and identify the appropriate units, present mechanisms for generating suitable test cases and for determining the order in which the units are to be tested, present a brief overview of the unit testing process and an example. Although we use the design artefacts from Prometheus the approach is suitable for any plan and event based agent system.