The implementation and performance of a hybrid OpenMP/ MPI parallel communication strategy for an unstructured mesh computational fluid dynamics code is described. The solver is cache efficient and fully vectorizable, and is parallelized using a two-level hybrid MPI-OpenMP implementation suitable for shared and/or distributed memory architectures, as well as clusters of shared memory machines. Parallelism is obtained through domain decomposition for both communication models. Single processor computational rates as well as scalability curves are given on various architectures. For the architectures studied in this work, the OpenMP or hybrid OpenMP/MPI communication strategies achieved no appreciable performance benefit over an exclusive MPI communication strategy.
Get full access to this article
View all access options for this article.
References
1.
CuthillE., and McKeeJ.1969. Reducing the bandwidth of sparse symmetric matrices. In Proc. ACM Nat. Conference, pp. 157–172.
2.
GroppW., LuskE., and SkjellumA.1994. Using MPI: Portable Parallel Programming with the Message Passing Interface.MIT Press, Cambridge, MA.
3.
HendricksonB., and LelandR.1995. The Chaco user's guide: Version 2.0. Tech. Rep. SAND94-2692, Sandia National Laboratories, Albuquerque, NM.
4.
KarypisG., and KumarV.1995. A fast and high quality multilevel scheme for partitioning irregular graphs. Technical Report 95-035, University of Minnesota. A short version appears in Intl. Conf. on Parallel Processing, 1995.
5.
LallemandM., SteveH., and DervieuxA.1992. Unstructured multigridding by volume agglomeration: Current status. Computers and Fluids21(3): 397–433.
6.
MavriplisD.J.1998a. Multigrid strategies for viscous flow solvers on anisotropic unstructured meshes. Journal of Computational Physics145(1):141–165.
7.
MavriplisD.J.1998b. Three dimensional high-lift analysis using a parallel unstructured multigrid solver. In Proceedings of the 16th AIAA Applied Aerodynamics Conference, Albuquerque, NM, pp. 376–390. AIAA Paper 98-2619-CP.
8.
MavriplisD.J.1999. Directional agglomeration multigrid techniques for high-Reynolds number viscous flows. AIAA Journal37(10):1222–1230.
9.
MavriplisD.J., and PirzadehS.1999. Large-scale parallel unstructured mesh computations for 3D high-lift analysis. AIAA Journal of Aircraft36(6):987–998.
10.
MavriplisD.J., and VenkatakrishnanV.1997. A unified multigrid solver for the Navier-Stokes equations on mixed element meshes. International Journal for Computational Fluid Dynamics8:247–263.
11.
MoranoE., and DervieuxA.1993. Looking for O(N) Navier-Stokes solutions on non-structured meshes. In 6th Copper Mountain Conf. on Multigrid Methods, pp. 449–464. NASA Conference Publication 3224.
Ollivier-GoochC.1995. Towards problem-independent multigrid convergence rates for unstructured mesh methods I: Inviscid and laminar flows. In Proceedings of the 6th International Symposium on CFD, Lake Tahoe, NV.
PierceN., and GilesM.1996. Preconditioning on stretched meshes. AIAA Paper 96-0889.
16.
RiemslaghK., and DickE.1993. A multigrid method for steady Euler equations on unstructured adaptive grids. In 6th Copper Mountain Conf. on Multigrid Methods, NASA conference publication 3224' pp. 527–542.
17.
RoeP.L.1981. Approximate Riemann solvers, parameter vectors and difference schemes. J. Comp. Phys.43(2): 357–372.
18.
SmithW.A.1990. Multigrid solution of transonic flow on unstructured grids. In Recent Advances and Applications in Computational Fluid Dynamics. Proceedings of the ASME Winter Annual Meeting, Ed. O. Baysal.
19.
SpalartP.R., and AllmarasS.R., 1994. A one-equation turbulence model for aerodynamic flows. La Recherche Aérospatiale1:5–21.
20.
van LeerB., TaiC.H., and PowellK.G.1989. Design of optimally-smoothing multi-stage schemes for the Euler equations. AIAA Paper 89-1933, June.
