For the architecture community, reasonable simulation time is a strong requirement in addition to performance data accuracy. However, emerging big data and AI workloads are too huge at binary size level and prohibitively expensive to run on cycle-accurate simulators. The concept of data motif, which is identified as a class of units of computation performed on initial or intermediate data, is the first step towards building proxy benchmark to mimic the real-world big data and AI workloads. However, there is no practical way to construct a proxy benchmark based on the data motifs to help simulation-based research. In this paper, we embark on a study to bridge the gap between data motif and a practical proxy benchmark. We propose a data motif-based proxy benchmark generating methodology by means of machine learning method, which combine data motifs with different weights to mimic the big data and AI workloads. Furthermore, we implement various data motifs using light-weight stacks and apply the methodology to five real-world workloads to construct a suite of proxy benchmarks, considering the data types, patterns, and distributions. The evaluation results show that our proxy benchmarks shorten the execution time by 100s times on real systems while maintaining the average system and micro-architecture performance data accuracy above 90 , even changing the input data sets or cluster configurations. Moreover, the generated proxy benchmarks reflect consistent performance trends across different architectures. To facilitate the community, we will release the proxy benchmarks on the project homepage this http URL
Using reduced data input is one way to reduce execution time. Previous work @cite_51 @cite_49 adopts reduced data set for the SPEC benchmark and maintains similar architecture behaviors with that of using the full reference data sets. However, it does not suit for big data and AI workloads, since they are data centric computing.
@cite_51 @cite_49
1838930658 2131667412 2788459922 2005232436
The large input datasets in the SPEC 2000 benchmark suite result in unreasonably long simulation times when using detailed execution-driven simulators for evaluating future computer architectures ideas. To address this problem, we have an ongoing project to reduce the execution times of the SPEC 2000 benchmarks in a quantitatively defensible way. Upon completion of this work 1, we will have smaller input datasets for several SPEC2000 benchmarks. The programs using our reduced input datasets will produce execution profiles that accurately reflect the program behavior of the full reference dataset, as measured using standard statistical tests. In the process of reducing and verifying the SPEC2000 Benchmark datasets, we also obtain instuction mix, memory behavior, and instructions per cycle characterization information about each benchmark program. Computer architects must determine how tomost effectively use finite computational resources whenrunning simulations to evaluate new architectural ideas.To facilitate efficient simulations with a range of benchmarkprograms, rn have developed the MinneSPEC inputset for the SPEC CPU 2000 benchmark suite. Thisnew workload allows computer architects to obtain simulationresults in a reasonable time using existing sirnulators.While the MinneSPEC workload is derived from thestandard SPEC CPU 2000 warklcad, it is a valid benchmarksuite in and of itself for simulation-based research.MinneSPEC also may be used to run Iarge numbers ofsimulations to find "sweet spots" in the evaluation parameterspace. This small number of promising designpoints subsequently may be investigated in more detailwith the full SPEC reference workload. In the processof developing the MinneSPEC datasets, we quantify itsdifferences in terms of function-level execution patterns,instruction mixes, and memory behaviors compared tothe SPEC programs when executed with the reference inputs.We find that for some programs, the MinneSPECprofiles match the SPEC reference dataset program behaviorvery closely. For other programs, however, theMinneSPEC inputs produce significantly different programbehavior. The MinneSPEC workload has been recognizedby SPEC and is distributed with Version 1.2 andhigher of the SPEC CPU 2000 benchmark suite. This paper presents the interesting observation that by performing fewer of the optimizations available in a standard compiler optimization level such as -02, while preserving their original ordering, significant savings can be achieved in both execution time and energy consumption. This observation has been validated on two embedded processors, namely the ARM Cortex-M0 and the ARM Cortex-M3, using two different versions of the LLVM compilation framework; v3.8 and v5.0. Experimental evaluation with 71 embedded benchmarks demonstrated performance gains for at least half of the benchmarks for both processors. An average execution time reduction of 2.4 and 5.3 was achieved across all the benchmarks for the Cortex-M0 and Cortex-M3 processors, respectively, with execution time improvements ranging from 1 up to 90 over the -02. The savings that can be achieved are in the same range as what can be achieved by the state-of-the-art compilation approaches that use iterative compilation or machine learning to select flags or to determine phase orderings that result in more efficient code. In contrast to these time consuming and expensive to apply techniques, our approach only needs to test a limited number of optimization configurations, less than 64, to obtain similar or even better savings. Furthermore, our approach can support multi-criteria optimization as it targets execution time, energy consumption and code size at the same time. The difficulties posed by GPGPU programming and the need to increase productivity have guided research towards directive-based high-level programs for accelerators. This effort has led to the definition of the OpenACC industry standard. It significantly simplifies writing code for graphics engines leaving the programmer the opportunity to tune the application for the target hardware and input. In this paper we address the problem of choosing the best mapping of sequential OpenACC loops to the parallel thread-space for a given program and input size. We show that auto-tuning on mapping parameters can improve performance by up to 4.8x over the default chosen by a state-of-the-art compiler. To reduce the overhead of auto-tuning we introduce a search technique that exploits similarities in behaviour across inputs using a nearest neighbour approach. This dramatically reduces the search for a good mapping (by 97 compared to random search). Finally we propose a heuristic for stopping the focused search which, averaged across 12 benchmarks and 30 input sizes each, achieves a speedup over the default of 1.26x with only 8 sampling runs.