Error from the code with PETSc when the mesh is refined

General Discussion
1

Dear all,

I am facing a problem. I compiled a code with PETSc to solve the elastic problem of the 3D model. The code with PETSc can normally run when the mesh of the model is rough, while the code reports error when the mesh is refined. The size of the refined mesh file is about 854 MB. I know that the size of the refined mesh file is so big, but it is needed for my problem. The contents of the reported error are shown in the below figures:

微信图片_20221023201520
微信图片_20221023201520718×672 126 KB

微信图片_20221023201540
Another problem is that the code cannot run when the number of processors is more than 8, there are 32 processors in my computer.
Please give me some advice.
The files of code and two mesh (rough and fine, respectively) are attached to this post.
Parallel_11.edp (5.4 KB)
Quartz_water_calcite6.edp (2.2 MB)
The second file is the rough mesh file with the format of .msh.
The size of the refined mesh file is about 854 MB, and hence cannot be uploaded. If you need this file, I can sent it to you by email.
Thank you in advance.

Best,
Liu

2

There is a nan in the either the linear or the bilinear form, so the program cannot continue.

3

Dear Professor,
Thank you for your reply. Why the code can normally run when the mesh of the model is rough?

4

Maybe the LU factorization cannot run properly because the matrix is too big?

5

Can I solve this problem if I increase the number of processors?

6

You can try, or you could use another solver.

7

Professor,
The problem is that the code cannot run when the number of processors is more than 8 even using the rough mesh, there are 32 processors in my computer. Could you give me some advice?
And which solver do you think can better solve this problem？

8

You can look at the PETSc manual for a list of available solvers: KSP: Linear System Solvers — PETSc 3.18.0 documentation.

9

Ok, Thank you very much professor.

10

Dear Professor,

According to your previous advice, I have tried all the solvers from the site you provided. The code can run with 25 processors. However, the code would report the error when the iterations was about 44, as shown in the below figures,

1
1959×413 190 KB

2
2967×414 27.8 KB

Is this because the mesh (854 MB) is too large or the performance of the computer is still insufficient？
Please give me some advice, thank you very much.
Best,
Liu

11

That’s a plausible cause. You can try to use the additional command line option -Dpartitioner=parmetis to see if this fixes things.

12

Many thanks for your reply.
Is your mean that “set(B, sparams = “-ksp_view -ksp_max_it 100 -Dpartitioner=parmetis -pc_type ksp”, bs=3, );”?

13

No, I mean on the command line that you use to launch your code.

14

Ok, I see. I will try it.
Thank you very much.

15

Professor, the additional command line option -Dpartitioner=parmetis can not solve the problem. Do you have another advice? Otherwise, I can only use a coarser grid, even if the result will be not ideal.

16

I cannot give more advice without a proper back trace.

17

Ok. Now, I am trying the other solvers. I would provide the feedback as soon as I get the new results. Thank you very much.

18

Dear professor,

Sorry for my late reply. The additional command line option -Dpartitioner=parmetis can not solve the problem. The error is same to the reported error when the option -Dpartitioner=parmetis was not used, as shown

1
1959×413 190 KB

2
2967×414 27.8 KB

Could you give me more advice?

Best,
Liu

19

I cannot give more advice without a proper back trace.

20

Sorry. Professor, what exactly does a proper back trace mean?