Cloud Service Providers deliver their products in form of ”as-a-Service”, which are typically categorized by the level of abstraction. This approach hides the implementation details and shows only functionality to the user. However, the problem is that it is hard to measure the performance of Cloud services, because they behave like black boxes. Especially with Function-as-a-Service it is even more difficult because it completely hides server and infrastructure management from users by design. Cloud Service Prodivers usually restrict the maximum size of code, memory and runtime of Cloud Functions. Nevertheless, users need clarification if more ressources are needed to deliver services in high quality. In this regard, we present the architectural design of a new Function-as-a-Service benchmarking tool, which allows users to evaluate the performance of Cloud Functions. Furthermore, the capabilities of the framework are tested on an isolated platform with a specific workload. The results show that users are able to get insights into Function-as-a-Service environments. This, in turn, allows users to identify factors which may slow down or speed up the performance of Cloud Functions.
We present the design of a novel performance-oriented serverless computing platform implemented in. NET, deployed in Microsoft Azure, and utilizing Windows containers as function execution environments. Implementation challenges such as function scaling and container discovery, lifecycle, and reuse are discussed in detail. We propose metrics to evaluate the execution performance of serverless platforms and conduct tests on our prototype as well as AWS Lambda, Azure Functions, Google Cloud Functions, and IBM's deployment of Apache OpenWhisk. Our measurements show the prototype achieving greater throughput than other platforms at most concurrency levels, and we examine the scaling and instance expiration trends in the implementations. Additionally, we discuss the gaps and limitations in our current design, propose possible solutions, and highlight future research.