Basic block reordering is an important step for profile-guided binary optimization. The state-of-the-art for basic block reordering is to maximize the number of fall-through branches. However, we demonstrate that such orderings may impose suboptimal performance on instruction and I-TLB caches. We propose a new algorithm that relies on a model combining the effects of fall-through and caching behavior. As details of modern processor caching is quite complex and often unknown, we show how to use machine learning in selecting parameters that best trade off different caching effects to maximize binary performance. An extensive evaluation on a variety of applications, including Facebook production workloads, the open-source compiler Clang, and SPEC CPU 2006 benchmarks, indicate that the new method outperforms existing block reordering techniques, improving the resulting performance of large-scale data-center applications. We have open sourced the code of the new algorithm as a part of a post-link binary optimization tool, BOLT.
The work by Pettis and Hansen @cite_15 is the basis for the majority of modern code reordering techniques. The goal is to create chains of basic blocks that are frequently executed together in the order. As discussed in , many variants of the technique have been suggested in the literature and implemented in various tools @cite_6 @cite_1 @cite_7 @cite_4 @cite_13 @cite_33 @cite_19 @cite_26 @cite_31 . Similar to our work, the techniques are operating with a control flow graph and try to lay out basic blocks tackling a variant of the Traveling Salesman Problem . Alternative models have been studied by @cite_20 , @cite_30 , and Lavaee and Ding @cite_29 , where a temporal-relation graph is taken into account. Temporal affinities between code instructions can be utilized for reducing conflict cache misses @cite_17 and improving the performance of multiple applications using a shared cache @cite_23 . We emphasize that according to our experiments, the performance of a front-end bound large-scale binary can be largely predicted by its control flow graph without considering more expensive models.
@cite_30 @cite_31 @cite_26 @cite_4 @cite_33 @cite_7 @cite_29 @cite_1 @cite_6 @cite_19 @cite_23 @cite_15 @cite_13 @cite_20 @cite_17
2143043833 2153073666 2140987889 2298866224 2034350206 2079816878 2100045639 2058030415 1997649574
We explore the use of compiler optimizations, which optimize the layout of instructions in memory. The target is to enable the code to make better use of the underlying hardware resources regardless of the specific details of the processor architecture in order to increase fetch performance. The Software Trace Cache (STC) is a code layout algorithm with a broader target than previous layout optimizations. We target not only an improvement in the instruction cache hit rate, but also an increase in the effective fetch width of the fetch engine. The STC algorithm organizes basic blocks into chains trying to make sequentially executed basic blocks reside in consecutive memory positions, then maps the basic block chains in memory to minimize conflict misses in the important sections of the program. We evaluate and analyze in detail the impact of the STC, and code layout optimizations in general, on the three main aspects of fetch performance; the instruction cache hit rate, the effective fetch width, and the branch prediction accuracy. Our results show that layout optimized, codes have some special characteristics that make them more amenable for high-performance instruction fetch. They have a very high rate of not-taken branches and execute long chains of sequential instructions; also, they make very effective use of instruction cache lines, mapping only useful instructions which will execute close in time, increasing both spatial and temporal locality. High instruction cache hit rates are key to high performance. One known technique to improve the hit rate of caches is to minimize cache interference by improving the layout of the basic blocks of the code. However, the performance impact of this technique has been reported for application code only, even though there is evidence that the operating system often uses the cache heavily and with less uniform patterns than applications. It is unknown how well existing optimizations perform for systems code acid whether better optimizations can be found. We address this problem in this paper. This paper characterizes, in detail, the locality patterns of the operating system code and shows that there is substantial locality Unfortunately, caches are not able to extract much of it: Rarely-executed special-case code disrupts spatial locality loops with few iterations that call routines make loop locality hard to exploit, and plenty of loop-less code hampers temporal locality. Based on our observations, we propose an algorithm to expose these localities and reduce interference in the cache. For a range of cache sizes, associativities, lines sizes, and organizations, we show that we reduce total instruction miss rates by 31-86 percent, or up to 2.9 absolute points. Using a simple model, this corresponds to execution time reductions of the order of 10-25 percent. In addition, our optimized operating system combines well with optimized and unoptimized applications. Branch alignment reorders the basic blocks of a program to minimize pipeline penalties due to control-transfer instructions. Prior work in branch alignment has produced useful heuristic methods. We present a branch alignment algorithm that usually achieves the minimum possible pipeline penalty and on our benchmarks averages within 0.3 of a provable optimum. We compare the control penalties and running times of our algorithm to an older, greedy approach and observe that both the greedy method and our method are close to the lower bound on control penalties, suggesting that greedy is good enough. Surprisingly, in actual execution our method produces programs that run noticeably faster than the greedy method. We also report results from training and testing on different data sets, validating that our results can be achieved in real-world usage. Training and testing on different data sets slightly reduced the benefits from both branch alignment algorithms, but the ranking of the algorithms does not change, and the bulk of the benefits remain. Many optimizing compilers use basic block reordering to reduce conditional branch misprediction penalties, decrease the number of unconditional branches and to improve the instruction cache performance. To perform this optimization one has to solve an NP-hard problem of finding a covering of a weighted control flow graph with a node-disjoint set of simple paths with maximal total weight; the problem is usually solved heuristically. We present the precise method which decomposes the initial graph into the set of subgraphs of certain kind — hammocks — and then solves a subproblem for each subgraph by an exponential branch-and-bound algorithm. The algorithm was tested on several real-world C programs, and its running time was as small as the rest of the compilation time for 91 of the tested graphs. Unfortunately it still failed to finish in a reasonable time for the remaining graphs since they didn’t have enough hammocks. We consider fixing up this drawback as a future work. A point-disjoint path cover of a directed graph is a collection of point-disjoint paths (some paths possibly having zero length) which covers all the points. A path cover which minimizes the number of paths corresponds to an optimal sequence of the steps of a computer program for efficient coding and documentation. The minimization problem for the general directed graph is hard in the sense of being NP-complete. In the case of cycle-free digraphs, however, the problem is polynomial, for it is shown that it can be reduced to the maximum-matching problem. A heuristic given here for finding a near optimal path cover for the general case is based upon applying the maximum-matching algorithm to the subgraphs of an interval decomposition. Code layout optimization seeks to reorganize the instructions of a program to better utilize the cache. On multicore, parallel executions improve the throughput but may significantly increase the cache contention, because the co-run programs share the cache and in the case of hyper-threading, the instruction cache. In this paper, we extend the reference affinity model for use in whole-program code layout optimization. We also implement the temporal relation graph (TRG) model used in prior work for comparison. For code reorganization, we have developed both function reordering and inter-procedural basic-block reordering. We implement the two models and the two transformations in the LLVM compiler. Experimental results on a set of benchmarks show frequently 20 to 50 reduction in instruction cache misses. By better utilizing the shared cache, the new techniques magnify the throughput improvement of hyper-threading by 8 . Several researchers have proposed algorithms for basic block reordering. We call these branch alignment algorithms. The primary emphasis of these algorithms has been on improving instruction cache locality, and the few studies concerned with branch prediction reported small or minimal improvements. As wide-issue architectures become increasingly popular the importance of reducing branch costs will increase, and branch alignment is one mechanism which can effectively reduce these costs. In this paper, we propose an improved branch alignment algorithm that takes into consideration the architectural cost model and the branch prediction architecture when performing the basic block reordering. We show that branch alignment algorithms can improve a broad range of static and dynamic branch prediction architectures. We also show that a program performance can be improved by approximately 5 even when using recently proposed, highly accurate branch prediction architectures. The programs are compiled by any existing compiler and then transformed via binary transformations. When implementing these algorithms on a Alpha AXP 21604 up to a 16 reduction in total execution time is achieved. Instruction cache performance is critical to instruction fetch efficiency and overall processor performance. The layout of an executable has a substantial effect on the cache miss rate during execution. This means that the performance of an executable can be improved significantly by applying a codeplacement algorithm that minimizes instruction cache conflicts. Alternatively, the hardware configuration of the instruction cache itself may greatly influence cache performance. For instance, increasing associativity or selective placement of data in the cache may significantly reduce conflict misses as As the gap between memory and processor performance continues to widen, it becomes increasingly important to exploit cache memory eflectively. Both hardware and aoftware approaches can be explored to optimize cache performance. Hardware designers focus on cache organization issues, including replacement policy, associativity, line size and the resulting cache access time. Software writers use various optimization techniques, including software prefetching, data scheduling and code reordering. Our focus is on improving memory usage through code reordering compiler techniques.In this paper we present a link-time procedure mapping algorithm which can significantly improve the eflectiveness of the instruction cache. Our algorithm produces an improved program layout by performing a color mapping of procedures to cache lines, taking into consideration the procedure size, cache size, cache line size, and call graph. We use cache line coloring to guide the procedure mapping, indicating which cache lines to avoid when placing a procedure in the program layout. Our algorithm reduces on average the instruction cache miss rate by 40 over the original mapping and by 17 over the mapping algorithm of Pettis and Hansen [12].