Nowadays, web services play a major role in the development of enterprise applications. Many such applications are now developed using a service-oriented architecture (SOA), where microservices is one of its most popular kind. A RESTful web service will provide data via an API over the network using HTTP, possibly interacting with databases and other web services. Testing a RESTful API poses challenges, as inputs outputs are sequences of HTTP requests responses to a remote server. Many approaches in the literature do black-box testing, as the tested API is a remote service whose code is not available. In this paper, we consider testing from the point of view of the developers, which do have full access to the code that they are writing. Therefore, we propose a fully automated white-box testing approach, where test cases are automatically generated using an evolutionary algorithm. Tests are rewarded based on code coverage and fault finding metrics. We implemented our technique in a tool called EVOMASTER, which is open-source. Experiments on two open-source, yet non-trivial RESTful services and an industrial one, do show that our novel technique did automatically find 38 real bugs in those applications. However, obtained code coverage is lower than the one achieved by the manually written test suites already existing in those services. Research directions on how to further improve such approach are therefore discussed.
This paper provides users and system integrators with an overview of service-oriented architecture (SOA) testing's fundamental technical issues and solutions, focusing on Web services as a practical implementation of the SOA model. The paper discusses SOA testing across two dimensions: testing perspectives, wherein various stakeholders have different needs and raise different testing requirements; and testing level, wherein each SOA testing level poses unique challenges Service Oriented Architecture (SOA) is changing the way in which software applications are designed, deployed and maintained. A service-oriented application consists of the runtime composition of autonomous services that are typically owned and controlled by different organizations. This decentralization impacts on the dependability of applications that consist of dynamic services agglomerates, and challenges their validation. Diﬀerent techniques can be used or combined for the verification of dependability aspects, spanning over traditional oﬀ-line testing approaches up till monitoring and on-line testing. In this chapter we discuss issues and opportunities of SOA validation, identify three different stages for validation along the service life-cycle model, and overview some proposed research approaches and tools. The emphasis is on on-line testing, which to us is the most peculiar stage in the SOA validation process. Finally, we claim that on-line testing is only possible within an agreed governance framework. Testing of Service Oriented Architectures (SOA) plays a critical role in ensuring a successful deployment in any enterprise. SOA testing must span several levels, from individual services to inter-enterprise federations of systems, and must cover functional and non-functional aspects. SOA unique combination of features, such as run-time discovery of services, ultra-late binding, QoS aware composition, and SLA automated negotiation, challenge many existing testing techniques. As an example, run-time discovery and ultra-late binding entail that the actual configuration of a system is known only during the execution, and this makes many existing integration testing techniques inadequate. Similarly, QoS aware composition and SLA automated negotiation means that a service may deliver with different performances in different contexts, thus making most existing performance testing techniques to fail. Whilst SOA testing is a recent area of investigation, the literature presents a number of approaches and techniques that either extend traditional testing or develop novel ideas with the aim of addressing the specific problems of testing service-centric systems. This chapter reports a survey of recent research achievements related to SOA testing. Challenges are analyzed from the viewpoints of different stakeholders and solutions are presented for different levels of testing, including unit, integration, and regression testing. The chapter covers both functional and non-functional testing, and explores ways to improve the testability of SOA.