We quantify the accuracy of various simulators compared to a real world robotic reaching and interaction task. Simulators are used in robotics to design solutions for real world hardware without the need for physical access. The reality gap' prevents solutions developed or learnt in simulation from performing well, or at at all, when transferred to real-world hardware. Making use of a Kinova robotic manipulator and a motion capture system, we record a ground truth enabling comparisons with various simulators, and present quantitative data for various manipulation-oriented robotic tasks. We show the relative strengths and weaknesses of numerous contemporary simulators, highlighting areas of significant discrepancy, and assisting researchers in the field in their selection of appropriate simulators for their use cases. All code and parameter listings are publicly available from: this https URL .
 Contact behaviors in physics simulations are important for real-time interactive applications, especially in virtual reality applications where user's body parts are tracked and interact with the environment via contact. For these contact simulations, it is ideal to have small changes in initial condition yield predictable changes in the output. Predictable simulation is key for success in iterative learning processes as well, such as learning controllers for manipulations or locomotion tasks. Here, we present an extensive comparison of contact simulations using Bullet Physics, Dynamic Animation and Robotics Toolkit DART, MuJoCo, and Open Dynamics Engine, with a focus on predictability of behavior. We first tune each engine to match an analytical solution as closely as possible and then compare the results for a more complex simulation. We found that in the commonly available physics engines, small changes in initial condition can sometimes induce different sequences of contact events to occur and ultimately lead to a vastly different result. Our results confirmed that parameter settings do matter a great deal and suggest that there may be a trade-off between accuracy and predictability. Copyright © 2016 John Wiley & Sons, Ltd. Purpose – In this study, a new methodology to evaluate the performance of physics simulation engines (PSEs) when used in haptic virtual assembly applications is proposed. This methodology can be used to assess the performance of any physics engine. To prove the feasibility of the proposed methodology, two-third party PSEs – Bullet and PhysXtm – were evaluated. The paper aims to discuss these issues. Design methodology approach – Eight assembly tests comprising variable geometric and dynamic complexity were conducted. The strengths and weaknesses of each simulation engine for haptic virtual assembly were identified by measuring different parameters such as task completion time, influence of weight perception and force feedback. Findings – The proposed tests have led to the development of a standard methodology by which physics engines can be compared and evaluated. The results have shown that when the assembly comprises complex shapes, Bullet has better performance than PhysX. It was also observed that the...