Code optimization is an intricate task that is getting more complex as computing systems evolve. Managing the program optimization process, including the implementation and evaluation of code variants, is tedious, inefficient, and errors are likely to be introduced in the process. Moreover, because each platform typically requires a different sequence of transformations to fully harness its computing power, the optimization process complexity grows as new platforms are adopted. To address these issues, systems and frameworks have been proposed to automate the code optimization process. They, however, have not been widely adopted and are primarily used by experts with deep knowledge about underlying architecture and compiler intricacies. This article describes the requirements that we believe necessary for making automatic performance tuning more broadly used, especially in complex, long-lived high-performance computing applications. Besides discussing limitations of current systems and strategies to overcome these, we describe the design of a system that is able to semi-automatically generate efficient platform-specific code. In the proposed system, the code optimization is programmer-guided, separately from application code, on an external file in what we call optimization programming. The language to program the optimization process is able to represent complex collections of transformations and, as a result, generate efficient platform-specific code. A database manages different optimized versions of code regions, providing a pragmatic approach to performance portability, and the framework itself has separate components, allowing the optimized code to be used on systems without installing all of the modules required for the code generation. We present experiments on two different platforms to illustrate the generation of efficient platform-specific code that performs comparable to hand-optimized, vendor-provided code.
AnselJChanCWongYL, et al. (2009) PetaBricks: a language and compiler for algorithmic choice. In: ACM SIGPLAN Conference on Programming Language Design and Implementation, Dublin, Ireland, 15–21 June 2009, pp. 38–49. ACM. DOI: 10.1145/1543135.1542481.
2.
AnselJKamilSVeeramachaneniK, et al. (2014) OpenTuner: an extensible framework for program autotuning. In: Proceedings of the 23rd International Conference on Parallel Architectures and Compilation, PACT ‘14, Edmonton, Alberta, Canada, 24–27 August 2014, pp. 303–316. New York, NY, USA: ACM. ISBN 978-1-4503-2809-8, DOI: 10.1145/2628071.2628092.
3.
BalaySAbhyankarSAdamsMF, et al. (2018) PETSc web page. Available at: http://www.mcs.anl.gov/petsc (accessed 24 July 2019).
4.
BalaySGroppWDMcInnesLC, et al. (1997) Efficient management of parallelism in object oriented numerical software libraries. In: ArgeEBruasetAMLangtangenHP (eds) Modern Software Tools in Scientific Computing. Basel: Birkhäuser Press, pp. 163–202.
5.
BasiliVRCarverJCCruzesD, et al. (2008) Understanding the high-performance-computing community: a software engineer’s perspective. IEEE Software25(4): 29–36.
6.
BergstraJYaminsDCoxDD (2013) Making a science of model search: hyperparameter optimization in hundreds of dimensions for vision architectures. In: Proceedings of the 30th International Conference on International Conference on Machine Learning - Volume 28, ICML’13. JMLR.org, Atlanta, GA, USA, 16–21 June 2013, pp. I–115–I–123. Available at: http://dl.acm.org/citation.cfm?id=3042817.3042832
7.
BilmesJAsanovicKChinCW, et al. (1997) Optimizing matrix multiply using PHiPAC: a portable, high-performance, ANSI C coding methodology. In: Proceedings of the 11th International Conference on Supercomputing, ICS ‘97, Vienna, Austria, 7–11 July 1997, pp. 340–347. New York, NY, USA: ACM. ISBN 0-89791-902-5, DOI:10.1145/263580.263662. Available at: http://doi.acm.org/10.1145/263580.263662
8.
BondhugulaUHartonoARamanujamJ, et al. (2008) A practical automatic polyhedral parallelizer and locality optimizer. In: Proceedings of the 29th ACM SIGPLAN Conference on Programming Language Design and Implementation, PLDI ‘08, Tucson, AZ, USA, 7–13 June 2008, pp. 101–113. New York, NY, USA: ACM. ISBN 978-1-59593-860-2, DOI:10.1145/1375581.1375595. Available at: http://doi.acm.org/10.1145/1375581.1375595
9.
ChenCChameJHallM (2008) CHiLL: A framework for composing high-level loop transformations. Technical report, University of Utah.
10.
ChungIHHollingsworthJK (2004) Using information from prior runs to improve automated tuning systems. In: Proceedings of the 2004 ACM/IEEE Conference on Supercomputing, SC ‘04, Pittsburgh, PA, USA, 6–12 November 2004, p. 30. Washington, DC, USA: IEEE Computer Society. ISBN 0-7695-2153-3, DOI: 10.1109/SC.2004.65.
11.
DagumLMenonR (1998) OpenMP: an industry standard API for shared-memory programming. IEEE Computational Science and Engineering5(1): 46–55.
12.
DattaKKamilSWilliamsS, et al. (2009) Optimization and performance modeling of stencil computations on modern microprocessors. SIAM Review51(1): 129–159.
13.
DonadioSBrodmanJRoederT, et al. (2006) A language for the compact representation of multiple program versions. In: Proceedings of the 18th International Conference on Languages and Compilers for Parallel Computing, LCPC’05, Hawthorne, NY, USA, 20–22 October 2005, pp. 136–151. Berlin, Heidelberg: Springer-Verlag. ISBN 3-540-69329-7, 978-3-540-69329 -1, DOI: 10.1007/978-3-540-69330-7_10.
14.
EdwardsHCTrottCRSunderlandD (2014) Kokkos: enabling manycore performance portability through polymorphic memory access patterns. Journal of Parallel and Distributed Computing74(12): 3202–3216. Available at: http://www.sciencedirect.com/science/article/pii/S0743731514001257 (accessed 24 July 2019). Domain-Specific Languages and High-Level Frameworks for High-Performance Computing.
15.
FatahalianKKnightTJHoustonM, et al. (2006) Sequoia: programming the memory hierarchy. In: Proceedings of the ACM/IEEE of SC 2006 Conference, Tampa, FL, USA, 11–17 November 2006, pp. 4–4. DOI: 10.1109/SC.2006.55.
16.
FrigoM (1999) A fast Fourier transform compiler. In: Proceedings of the ACM SIGPLAN 1999 Conference on Programming Language Design and Implementation, PLDI ‘99, Atlanta, Georgia, USA, pp. 169–180. New York, NY, USA: ACM. ISBN 1-58113-094-5, DOI: 10.1145/301618.301661.
17.
HartonoANorrisBSadayappanP (2009) Annotation-based empirical performance tuning using Orio. In: 2009 IEEE International Symposium on Parallel Distributed Processing, May 2009, pp. 1–11. DOI: 10.1109/IPDPS.2009.5161004.
18.
HornungRDKeaslerJA (2014) The RAJA portability layer: Overview and status. Technical Report LLNL-TR-661403, Lawrence Livermore National Lab. DOI: 10.2172/1169830.
19.
KeryellRAncourtCCoelhoF, et al. (1996) Pips: A workbench for building interprocedural parallelizers, compilers and optimizers. Technical Report A/289, Centre de Recherche en Informatique, Ecole des Mines de Paris.
20.
LiXGarzaránMJPaduaD (2004) A dynamically tuned sorting library. In: Proceedings of the International Symposium on Code Generation and Optimization: Feedback-Directed and Runtime Optimization, CGO ‘04, pp. 111. Washington, DC, USA: IEEE Computer Society. ISBN 0-7695-2102-9, Available at: http://dl.acm.org/citation.cfm?id=977395.977663 (accessed 24 July 2019).
21.
LidmanJQuinlanDJLiaoC, et al. (2012) ROSE: FTTransform - a source-to-source translation framework for exascale fault-tolerance research. In: IEEE/IFIP International Conference on Dependable Systems and Networks Workshops (DSN 2012), Boston, MA, USA, 25–28 June 2012, pp. 1–6. IEEE. DOI:10.1109/DSNW.2012.6264672.
22.
MuralidharanSShantharamMHallM, et al. (2014) Nitro: a framework for adaptive code variant tuning. In: 2014 IEEE 28th International Parallel and Distributed Processing Symposium, Phoenix, AZ, USA, 19–23 May 2014, pp. 501–512. IEEE. DOI: 10.1109/IPDPS.2014.59.
23.
PennycookSJSewallJDLeeVW (2016) A metric for performance portability. arXiv e-prints. Available at: https://arxiv.org/abs/1611.07409 (accessed 24 July 2019).
24.
PüschelMMouraJMFJohnsonJ, et al. (2005) SPIRAL: Code generation for DSP transforms. Proceedings of the IEEE, special issue on “Program Generation, Optimization, and Adaptation93(2): 232–275.
25.
Ragan-KelleyJAdamsASharletD, et al. (2017) Halide: decoupling algorithms from schedules for high-performance image processing. ACM Commun61(1): 106–115.
26.
SteuwerMFenschCLindleyS, et al. (2015) Generating performance portable code using rewrite rules: from high-level functional expressions to high-performance OpenCL code. ACM SIGPLAN Notices50(9): 205–217.
27.
TangYChowdhuryRAKuszmaulBC, et al. (2011) The Pochoir stencil compiler. In: Proceedings of the Twenty-third Annual ACM Symposium on Parallelism in Algorithms and Architectures SPAA ‘11, San Jose, California, USA, pp. 117–128. New York, NY, USA: ACM. ISBN 978-1-4503-0743-7, DOI: 10.1145/1989493.1989508.
28.
TangeO (2011) GNU parallel - the command-line power tool. login: The USENIX Magazine36(1): 42–47. Available at: http://www.gnu.org/s/parallel (accessed 24 July 2019).
29.
TeixeiraTSFXAncourtCPaduaD, et al. (2019) Locus: a system and a language for program optimization. In: Proceedings of the 2019 IEEE/ACM International Symposium on Code Generation and Optimization CGO 2019, Washington, DC, USA, pp. 217–228. Piscataway, NJ, USA: IEEE Press.
30.
TiwariAHollingsworthJKChenC, et al. (2011) Auto-tuning full applications: a case study. International Journal of High Performance Computing Applications25(3): 286–294.
31.
VuducRDemmelJWYelickKA (2005) OSKI: a library of automatically tuned sparse matrix kernels. Journal of Physics: Conference Series16(1): 521. Availble at: http://stacks.iop.org/1742-6596/16/i=1/a=071
32.
WhaleyRCDongarraJJ (1998) Automatically tuned linear algebra software. In: Proceedings of the 1998 ACM/IEEE Conference on Supercomputing, SC ‘98, pp. 1–27. Washington, DC, USA: IEEE Computer Society. ISBN 0-89791-984-X. Available at: http://dl.acm.org/citation.cfm?id=509058.509096 (accessed 24 July 2019).
YiQSeymourKYouH, et al. (2007) POET: parameterized optimizations for empirical tuning. In: 2007 IEEE International Parallel and Distributed Processing Symposium, Rome, Italy, 26–30 March 2007, pp. 1–8. IEEE. DOI: 10.1109/IPDPS.2007.370637.
