Minimizing the memory usage with parallel out-of-core multi-frontal direct solver

Maciej Paszyński

doi:10.24423/cames.78

Maciej Paszyński Department of Computer Science, Faculty of Computer Science, Electronics and Telecommunications, AGH University of Science and Technology, Kraków

DOI: http://dx.doi.org/10.24423/cames.78

Abstract

This paper presents the out-of-core solver for three-dimensional multiphysics problems. In particular, our study focuses on the three-dimensional simulations of the linear elasticity coupled with acoustics. The out-of-core solver is designed with three principles in mind. First, to store the dense matrices associated with the nodes of the elimination tree with blocks related to nodes of the mesh, where many degrees of freedom may be located in the case of multiphysics computations with high order polynomials. The second principle is to minimize the memory usage. This is obtained by dumping out all local systems from the entire elimination tree to the disk during the elimination stage. The local systems are reutilized later during the backward substitution stage. The third principle is that the communication in the parallel version of the out-of-core solver occurs through the parallel file system. The memory usage of the solver is compared against the state-of-the-art MUMPS solver.

Keywords

multi-frontal direct solver, finite element method, out-of-core, parallel simulations, multiphysics,

References

[1] M. Paszyński, D. Pardo, C. Torres-Verdin, L. Demkowicz, V. Calo. A parallel direct solver for the self-adaptive hp finite element method. Journal of Parallel and Distributed Computing, 70(3): 270–281, 2010.
[2] M. Paszyński, R. Schaefer. Graph grammar driven partial differential equations solver. Concurrency and Computations: Practise and Experience, 22(9): 1063–1097, 2010.
[3] A. Szymczak, M. Paszyński. Graph grammar based Petri net controlled direct sovler algorithm. Computer Science, 11: 65–79, 2010.
[4] M. Paszyński, R. Schaefer. Reutilization of partial LU factorizations for self-adaptive hp Finite Element Method solver. Lecture Notes in Computer Science, 5101: 965–974, 2008.
[5] M. Paszyński, L. Demkowicz. Parallel, fully automatic, hp-adaptive 3D finite element package. Engineering with Computers, 22: 255–276, 2006.
[6] L. Demkowicz, J. Kurtz, D. Pardo, M. Paszyński, W. Rachowicz, A. Zdunek. Computing with hp-Adaptive Finite Elements, Vol. II. Frontiers: Three Dimensional Elliptic and Maxwell Problems with Applications. Chapman & Hall/CRC Applied Mathematics & Nonlinear Science, 2007.
[7] M. Paszyński, D. Pardo, A. Paszyńska. Parallel multi-frontal solver for p-adaptive finite element modeling of multi-physics computational problems. Journal of Computational Science, 1: 48–54, 2010.
[8] L. Demkowicz, P. Gatto, J. Kurtz, M. Paszynski, W. Rachowicz, E. Bleszyński, M. Bleszyński, M. Hamilton, C. Champlin, D. Pardo. Modeling of bone conduction of sound in human head using hp finite elements. Part I. Code design and modification. Computer Methods in Applied Mechanics and Engineering, 200: 1757–1773, 2011.
[9] D. Pardo. Integration of hp-Adaptivity with a Two-Grid Solver. PhD. Dissertation. University of Texas at Austin, 2004.
[10] D. Pardo, L. Demkowicz, C. Torres-Verdin, M. Paszyński. A self-adaptive goal-oriented hp finite element method with electromagnetic applications. Pt. 2, Electrodynamics. Computer Methods in Applied Mechanics and Engineering, 196: 3585–3597, 2007.
[11] P.R. Amestoy, I.S. Duff, J. Koster, J.-Y. L’Excellent. A fully asynchronous multifrontal solver using distributed dynamic scheduling. SIAM Journal of Matrix Analysis and Applications, 23(1): 15–41, 2001.
[12] P.R. Amestoy, A. Guermouche, J.-Y. L’Excellent, S. Pralet. Hybrid scheduling for the parallel solution of linear systems. Parallel Computing, 32(2): 136–156, 2006.
[13] Multi-frontal Massively Parallel Sparse direct solver (MUMPS), http://graal.ens-lyon.fr/MUMPS.
[14] D. Pardo, M.J. Nam, C. Torres-Verdin, M. Hoversten, I. Garay. Simulation of Marine Controlled Source Electro-magnetic Measurements Using a Parallel Fourier hp-Finite Element Method. Computational Geosciences, 15(1): 53–67, 2011.
[15] N.I.M. Gould, J.A. Scott, Hu Yifan. Numerical Evaluation of Sparse Direct Solvers for the Solution of Large Sparse Symmetric Linear Systems of Equations. ACM Transactions on Mathematical Software (TOMS), 33(2)10: 300–331, 2007.
[16] S. Fialko. A block sparse shared-memory multifrontal finite element solver for problems of structural mechanics. Computer Assisted Mechanics and Engineering Sciences, 16: 117–131, 2009.
[17] S. Fialko. A Sparse Shared-Memory Multifrontal Solver in SCAD Software, Proceedings of the International Multiconference on Computer Science and Information Technology. October 22–28, 2009,Wisła, Poland, 3: 277–283, 2008.
[18] S. Fialko. The block substructure multifrontal method for solution of large finite element equation SETS. Technical Transactions, 1-NP/2009 (8): 175–188, 2009.
[19] A. George, J.W.H. Liu. Computer solution of sparse positive definite systems. New Jersey: Prentice-Hall, Inc. Englewood Cliffs, 1981.
[20] S. Fialko. The Nested Substructures Method for Solving Large Finite-Element Systems as Applied to Thin-Walled Shells with High Ribs. International Applied Mechanics, 39(3): 324–332, 2003.
[21] P. Gend, J.T. Oden, R. van de Geijn. A parallel multifrontal algorithm and its implementation. Computer Methods in Applied Mechanics and Engineering, 149: 289–301, 1997.
[22] C. Ashcraft, J.W.H. Liu. Robust Ordering of Sparse Matrices Using Multisection, Technical Report CS 96-01. Department of Computer Science, York University, Ontario, Canada, 1996.
[23] I. Babuška, B. Guo. The hp-version of the finite element method, Part I: The basic approximation results. Computational Mechanics, 1: 21–41, 1986.
[24] I. Babuška, B. Guo. The hp-version of the finite element method, Part II: General results and applications. Computational. Mechanics, 1: 203–220, 1986.
[25] M. Paszyński, D. Pardo, A. Paszyńska. Parallel multi-frontal solver for p adaptive finite element modeling of multi-physics computational problems. Journal of Computational Science, 1: 48–54, 2010.