The software of robotic assistants needs to be verified, to ensure its safety and functional correctness. Testing in simulation allows a high degree of realism in the verification. However, generating tests that cover both interesting foreseen and unforeseen scenarios in human-robot interaction (HRI) tasks, while executing most of the code, remains a challenge. We propose the use of belief-desire-intention (BDI) agents in the test environment, to increase the level of realism and human-like stimulation of simulated robots. Artificial intelligence, such as agent theory, can be exploited for more intelligent test generation. An automated testbench was implemented for a simulation in Robot Operating System (ROS) and Gazebo, of a cooperative table assembly task between a humanoid robot and a person. Requirements were verified for this task, and some unexpected design issues were discovered, leading to possible code improvements. Our results highlight the practicality of BDI agents to automatically generate valid and human-like tests to get high code coverage, compared to hand-written directed tests, pseudorandom generation, and other variants of model-based test generation. Also, BDI agents allow the coverage of combined behaviours of the HRI system with more ease than writing temporal logic properties for model checking.
Testing of robotic systems can be performed in a real-life setting @cite_1 , completely in simulation @cite_9 , or in combinations of simulation and real components @cite_4 , i.e., hardware-in-the-loop. Our BDI approach offers a novel solution for the two latter cases. In our approach, we explore the code mainly for finding and eliminating functional bugs, i.e., for safety and functional soundness, although runtime bugs can be found by instrumenting the code with relevant assertion monitors.
@cite_9 @cite_1 @cite_4
308585474 2145257696 1774952121
In robotics, a reliable simulation tool is an important design and test resource because the performance of algorithms is evaluated before being implemented in real mobile robots. The virtual environment makes it possible to conduct extensive experiments in controlled scenarios, without the dependence of a physical platform, in a faster and inexpensive way. Although, simulators should be able to represent all the relevant characteristics that are present in the real environment, like dynamic (shape, mass, surface friction, etc.), impact simulation, realistic noise, among other factors, in order to guarantee the accuracy and reliability of the results. Abstract Context Testing complex industrial robots (CIRs) requires testing several interacting control systems. This is challenging, especially for robots performing process-intensive tasks such as painting or gluing, since their dedicated process control systems can be loosely coupled with the robot’s motion control. Objective Current practices for validating CIRs involve manual test case design and execution. To reduce testing costs and improve quality assurance, a trend is to automate the generation of test cases. Our work aims to define a cost-effective automated testing technique to validate CIR control systems in an industrial context. Method This paper reports on a methodology, developed at ABB Robotics in collaboration with SIMULA, for the fully automated testing of CIRs control systems. Our approach draws on continuous integration principles and well-established constraint-based testing techniques. It is based on a novel constraint-based model for automatically generating test sequences where test sequences are both generated and executed as part of a continuous integration process. Results By performing a detailed analysis of experimental results over a simplified version of our constraint model, we determine the most appropriate parameterization of the operational version of the constraint model. This version is now being deployed at ABB Robotics’s CIR testing facilities and used on a permanent basis. This paper presents the empirical results obtained when automatically generating test sequences for CIRs at ABB Robotics. In a real industrial setting, the results show that our methodology is not only able to detect reintroduced known faults, but also to spot completely new faults. Conclusion Our empirical evaluation shows that constraint-based testing is appropriate for automatically generating test sequences for CIRs and can be faithfully deployed in an industrial context. Developing control software for teams of autonomous mobile robots is a challenging task, which can be facilitated using frameworks with ready to use components. But testing and debugging the resulting system as teached in modern software engineering to be free of errors and tolerant to sensor noise in a real world scenario is to a large extend beyond the scope of current approaches. In this paper multilevel testing strategies using the developed frameworks RoboFrame and MuRoSimF are presented. Testing incorporating automated tests, online and offline analysis and software-in-the-loop (SIL) tests in combination with real robot hardware or an adequate simulation are highly facilitated by the two frameworks. Thus the efficiency of validation of complex real world applications is improved. In this way potential errors can be identified early in the development process and error situations in real world operations can be reduced significantly.