Cloud Service Providers deliver their products in form of ”as-a-Service”, which are typically categorized by the level of abstraction. This approach hides the implementation details and shows only functionality to the user. However, the problem is that it is hard to measure the performance of Cloud services, because they behave like black boxes. Especially with Function-as-a-Service it is even more difficult because it completely hides server and infrastructure management from users by design. Cloud Service Prodivers usually restrict the maximum size of code, memory and runtime of Cloud Functions. Nevertheless, users need clarification if more ressources are needed to deliver services in high quality. In this regard, we present the architectural design of a new Function-as-a-Service benchmarking tool, which allows users to evaluate the performance of Cloud Functions. Furthermore, the capabilities of the framework are tested on an isolated platform with a specific workload. The results show that users are able to get insights into Function-as-a-Service environments. This, in turn, allows users to identify factors which may slow down or speed up the performance of Cloud Functions.
@cite_8 present an approach for performance evaluation of Cloud functions and take the heterogeneity aspects into account. For this purpose, the authors developed a framework with two suites of computing-intensive benchmarks for performance evaluation of Cloud functions. Their results show the heterogeneity of Cloud Function Providers, the relation between function size and performance and how providers interpret the resource allocation policies differently. The authors conclude that there is need of research that should analyse the impact of parallelism, delays, and warm-up on performance.
@cite_8
2028837511 1525444182 2786869547 1651603302
In this paper, we address the cloud service trustworthiness evaluation problem, which in essence is a multi-attribute decision-making problem, by proposing a novel evaluation model based on the fuzzy gap measurement and the evidential reasoning approach. There are many sources of uncertainties in the process of cloud service trustworthiness evaluation. In addition to the intrinsic uncertainties, cloud service providers face the problem of discrepant evaluation information given by different users from different perspectives. To address these problems, we develop a novel fuzzy gap evaluation approach to assess cloud service trustworthiness and to provide evaluation values from different perspectives. From the evaluation values, the perception-importance, delivery-importance, and perception-delivery gaps are generated. These three gaps reflect the discrepancy evaluation of cloud service trustworthiness in terms of perception utility, delivery utility, and importance utility, respectively. Finally, the gap measurement of each perspective is represented by a belief structure and aggregated using the evidential reasoning approach to generate final evaluation results for informative and robust decision making. From this hybrid two-stage evaluation process, cloud service providers can get improvement suggestions from intermediate information derived from the gap measurement, which is the main advantage of this evaluation process. There is an increasing need for cloud service performance that can be tailored to customer requirements. In the context of jobs submitted to cloud computing clusters, a crucial requirement is the specification of job completion-times. A natural way to model this specification, is through client job utility functions that are dependent on job completion-times. We present a method to allocate and schedule heterogeneous resources to jointly optimize the utilities of jobs in a cloud. Specifically: (i) we formulate a completion-time optimal resource allocation (CORA) problem to apportion cluster resources across the jobs that enforces max-min fairness among job utilities, and (ii) starting with an integer programming problem, we perform a series of steps to transform it into an equivalent linear programming problem, and (iii) we implement the proposed framework as a utility-aware resource scheduler in the widely used Hadoop data processing framework, and finally (iv) through extensive experiments with real-world datasets, we show that our prototype achieves significant performance improvement over existing resource-allocation policies. Cloud Functions, often called Function-as-a-Service (FaaS), pioneered by AWS Lambda, are an increasingly popular method of running distributed applications. As in other cloud offerings, cloud functions are heterogeneous, due to different underlying hardware, runtime systems, as well as resource management and billing models. In this paper, we focus on performance evaluation of cloud functions, taking into account heterogeneity aspects. We developed a cloud function benchmarking framework, consisting of one suite based on Serverless Framework, and one based on HyperFlow. We deployed the CPU-intensive benchmarks: Mersenne Twister and Linpack, and evaluated all the major cloud function providers: AWS Lambda, Azure Functions, Google Cloud Functions and IBM OpenWhisk. We make our results available online and continuously updated. We report on the initial results of the performance evaluation and we discuss the discovered insights on the resource allocation policies. Benchmarking the performance of public cloud providers is a common research topic. Previous work has already extensively evaluated the performance of different cloud platforms for different use cases, and under different constraints and experiment setups. In this article, we present a principled, large-scale literature review to collect and codify existing research regarding the predictability of performance in public Infrastructure-as-a-Service (IaaS) clouds. We formulate 15 hypotheses relating to the nature of performance variations in IaaS systems, to the factors of influence of performance variations, and how to compare different instance types. In a second step, we conduct extensive real-life experimentation on four cloud providers to empirically validate those hypotheses. We show that there are substantial differences between providers. Hardware heterogeneity is today less prevalent than reported in earlier research, while multitenancy has a dramatic impact on performance and predictability, but only for some cloud providers. We were unable to discover a clear impact of the time of the day or the day of the week on cloud performance.