Computational fluid dynamics (CFD) requires a vast amount of compute cycles on contemporary large-scale parallel computers. Hence, performance optimization is a pivotal activity in this field of computational science. Not only does it reduce the time to solution, but it also allows to minimize the energy consumption. In this work we study performance optimizations for an MPI-parallel lattice Boltzmann-based flow solver that uses a sparse lattice representation with indirect addressing. First we describe how this indirect addressing can be minimized in order to increase the single-core and chip-level performance. Second, the communication overhead is reduced via appropriate partitioning, but maintaining the single core performance improvements. Both optimizations allow to run the solver at an operating point with minimal energy consumption.
In this work we present a modified version of the two-grid one-step algorithm with non-temporal stores (OS-NT) and the AA-pattern algorithm. Both are augmented by a technique called RIA (reduced indirect addressing), which can avoid the indirect access under certain conditions. This optimization is based on the idea of run length encoding and enables a reduction of the loop balance @math . It also allows for partial vectorization, which is usually incompatible with indirect addressing unless the hardware supports efficient gather operations. RIA and partial vectorization were already implemented in the code we employed for our earlier analysis @cite_14 ; here we describe them in due detail.
@cite_14
2341508215 2952901036 1522470213 2020461703
In this paper, we propose a new decomposition approach named the proximal primal dual algorithm (Prox-PDA) for smooth nonconvex linearly constrained optimization problems. The proposed approach is primal-dual based, where the primal step minimizes certain approximation of the augmented Lagrangian of the problem, and the dual step performs an approximate dual ascent. The approximation used in the primal step is able to decompose the variable blocks, making it possible to obtain simple subproblems by leveraging the problem structures. Theoretically, we show that whenever the penalty parameter in the augmented Lagrangian is larger than a given threshold, the Prox-PDA converges to the set of stationary solutions, globally and in a sublinear manner (i.e., certain measure of stationarity decreases in the rate of @math , where @math is the iteration counter). Interestingly, when applying a variant of the Prox-PDA to the problem of distributed nonconvex optimization (over a connected undirected graph), the resulting algorithm coincides with the popular EXTRA algorithm [ 2014], which is only known to work in convex cases. Our analysis implies that EXTRA and its variants converge globally sublinearly to stationary solutions of certain nonconvex distributed optimization problem. There are many possible extensions of the Prox-PDA, and we present one particular extension to certain nonconvex distributed matrix factorization problem. We quantify a source of ineffectual computations when processing the multiplications of the convolutional layers in Deep Neural Networks (DNNs) and propose Pragmatic (PRA), an architecture that exploits it improving performance and energy efficiency. The source of these ineffectual computations is best understood in the context of conventional multipliers which generate internally multiple terms, that is, products of the multiplicand and powers of two, which added together produce the final product [1]. At runtime, many of these terms are zero as they are generated when the multiplicand is combined with the zero-bits of the multiplicator. While conventional bit-parallel multipliers calculate all terms in parallel to reduce individual product latency, PRA calculates only the non-zero terms using a) on-the-fly conversion of the multiplicator representation into an explicit list of powers of two, and b) hybrid bit-parallel multplicand bit-serial multiplicator processing units. PRA exploits two sources of ineffectual computations: 1) the aforementioned zero product terms which are the result of the lack of explicitness in the multiplicator representation, and 2) the excess in the representation precision used for both multiplicants and multiplicators, e.g., [2]. Measurements demonstrate that for the convolutional layers, a straightforward variant of PRA improves performance by 2.6x over the DaDiaNao (DaDN) accelerator [3] and by 1.4x over STR [4]. Similarly, PRA improves energy efficiency by 28 and 10 on average compared to DaDN and STR. An improved cross lane synchronication scheme boosts performance improvements to 3.1x over DaDN. Finally, Pragmatic benefits persist even with an 8-bit quantized representation [5]. We describe new data structures for full and packed storage of dense symmetric triangular arrays that generalize both. Using the new data structures one is led to several new algorithms that save "half" the storage for symmetric matrices and outperform the current blocked based level 3 algorithms in LAPACK. We concentrate on the simplest forms of the new algorithms and show they are a direct generalization of LINPACK. This means that level 3 BLAS's are not required to obtain level 3 performance. The replacement for Level 3 BLAS are so-called kernel routines, see [1], and on IBM platforms they are producible from simple textbook type codes, by the XLF Fortran compiler. In the sequel I will label these "vanilla" codes. On Power3 with a peak performance of 800 MFlops, the results for Cholesky factorization at order n ? 200 is over 720 MFlops and then reaches 735 MFlops at n = 400. Using conventional full format LAPACK DPOTRF with ESSL BLAS's one first gets to 600 MFlops at n ? 600 and only reaches a peak of 620 MFlops. For this result we used simple square blocked full matrix data formats where the blocks themselves are stored in column major (Fortran) order or row major (C) order. The simple algorithms of LU factorization with partial pivoting for this new data format is a direct generalization of LINPACK algorithm DGEFA. Again, no conventional level 3 BLAS's are required; the replacements are again so-called kernel routines. Programming for squared blocked full matrix format can be accomplished in standard Fortran through the use of three and four dimensional arrays. Thus, no new compiler support is necessary. Also we mention that other more complicated algorithms are possible; e.g., recursive ones. The recursive algorithms are also easily programmed via the use of tables that address where the blocks are stored in the two dimensional recursive block array. Finally, we describe block hybrid formats. Doing so allows one to use no additional storage over conventional (full and packed) matrix storage. This means the new algorithms are completely portable. In image processing, the Rudin-Osher-Fatemi (ROF) model [L. Rudin, S. Osher, and E. Fatemi, Phys. D, 60 (1992), pp. 259-268] based on total variation (TV) minimization has proven to be very useful. So far many researchers have contributed to designing fast numerical schemes and overcoming the nondifferentiability of the model. Methods considered to be particularly efficient for the ROF model include the Chan-Golub-Mulet (CGM) primal-dual method [T.F. Chan, G.H. Golub, and P. Mulet, SIAM J. Sci. Comput., 20 (1999), pp. 1964-1977], Chambolle's dual method [A. Chambolle, J. Math. Imaging Vis., 20 (2004), pp. 89-97], the splitting and quadratic penalty-based method [Y. Wang, J. Yang, W. Yin, and Y. Zhang, SIAM J. Imaging Sci., 1 (2008), pp. 248-272], and the split Bregman iteration [T. Goldstein and S. Osher, SIAM J. Imaging Sci., 2 (2009), pp. 323-343], as well as the augmented Lagrangian method [X.C. Tai and C. Wu, Lecture Notes in Comput. Sci. 5567, Springer-Verlag, Berlin, 2009, pp. 502-513]. In this paper, we first review the augmented Lagrangian method for the ROF model and then provide some convergence analysis and extensions to vectorial TV and high order models. All the algorithms and analysis will be presented in the discrete setting, which is much clearer for practical implementation than the continuous setting as in Tai and Wu, above. We also present, in the discrete setting, the connections between the augmented Lagrangian method, the dual methods, and the split Bregman iteration. Using our extensions and observations, we can easily figure out CGM and the split Bregman iteration for vectorial TV and high order models, which, to the best of our knowledge, have not been presented in the literature. Numerical examples demonstrate the efficiency and accuracy of our method, especially in the image deblurring case.