How to decompose a mesh of huge size effectively?

General Discussion
1

Hi FreeFem Community,

I am working on a code to decompose a 3D mesh by calling FFDDM. The mesh was generated with Gmsh, and has about 46 million elements. The code is running on the HPC of our Lab, and each node has 1T memeory and 56 cores. The key codes read:

include “ffddm.idp”
include “macro_ddm.idp”
load “gmsh”
load “msh3”
mesh3 ThGlobal = gmshload3(“xxx.msh”);
ffddmbuildDmesh(M3d, ThGlobal, mpiCommWorld);

The related options in the command line are: -ffddm_overlap 10 -ffddm_partitioner 1.

I tried to run the code on 1 or 2 nodes to partition the mesh into 56 or 112 subdomains, it terminates with the error: Failed to allocate memory for adjncy.

I also launched the task by allocating 4 nodes, but the log stays at the building stage for over 4 hours: Building decomposition from mesh of 45826143 elements.

Is there more effective method to decompose such a big mesh?

2

Use macro_ddm.idp with ParMETIS instead of METIS (-Dpartitioner=parmetis).

3

Thanks to your quick reply.

Is there a complete example to illustrate the usage of ParMETIS?

4

There is nothing to illustrate beside setting the macro partitioner to parmetis as shown above.

1 Like