Dear Prj,
Sorry for late reply.
I did what you told me and here is the output.
make -j4 check
make check-TESTS
make[1]: Entering directory /home/yim/FreeFem-sources/examples/hpddm' make[2]: Entering directory /home/yim/FreeFem-sources/examples/hpddm’
XFAIL: withPartitioning.edp
XFAIL: buildRecursive.edp
XFAIL: createPartition.edp
XFAIL: reconstructDmesh.edp
XFAIL: diffusion-2d.edp
XFAIL: diffusion-mg-2d.edp
XFAIL: diffusion-substructuring-2d.edp
XFAIL: diffusion-substructuring-withPartitioning-2d.edp
XFAIL: diffusion-3d.edp
XFAIL: diffusion-simple-3d.edp
XFAIL: diffusion-periodic-2d.edp
XFAIL: elasticity-2d.edp
XFAIL: elasticity-substructuring-2d.edp
XFAIL: elasticity-3d.edp
XFAIL: elasticity-simple-3d.edp
PASS: elasticity-block.edp
XFAIL: heat-2d.edp
XFAIL: heat-io-2d.edp
XFAIL: heat-3d.edp
XFAIL: helmholtz-2d.edp
XFAIL: helmholtz-mg-2d.edp
XFAIL: iterative.edp
XFAIL: maxwell-3d.edp
XFAIL: stokes-2d.edp
XFAIL: stokes-3d.edp
XFAIL: stokes-io-3d.edp
XFAIL: heat-torus-3d-surf.edp
XFAIL: bratu-2d-PETSc.edp
XFAIL: diffusion-2d-PETSc.edp
XFAIL: diffusion-3d-PETSc.edp
XFAIL: diffusion-periodic-2d-PETSc.edp
XFAIL: diffusion-periodic-balanced-2d-PETSc.edp
XFAIL: elasticity-2d-PETSc.edp
XFAIL: elasticity-3d-PETSc.edp
XFAIL: elasticity-SNES-3d-PETSc.edp
XFAIL: heat-2d-PETSc.edp
XFAIL: laplace-lagrange-PETSc.edp
XFAIL: natural-convection-fieldsplit-2d-PETSc.edp
XFAIL: neo-Hookean-2d-PETSc.edp
XFAIL: newton-2d-PETSc.edp
XFAIL: newton-adaptmesh-2d-PETSc.edp
XFAIL: newton-vi-2d-PETSc.edp
XFAIL: newton-vi-adaptmesh-2d-PETSc.edp
XFAIL: block-PETSc.edp
XFAIL: laplace-RT-2d-PETSc.edp
XFAIL: stokes-2d-PETSc.edp
XFAIL: stokes-fieldsplit-2d-PETSc.edp
XFAIL: stokes-block-2d-PETSc.edp
XFAIL: stokes-3d-PETSc.edp
XFAIL: stokes-fieldsplit-3d-PETSc.edp
XFAIL: transpose-solve-PETSc.edp
XFAIL: bratu-hpddm-2d-PETSc.edp
XFAIL: vi-2d-PETSc.edp
PASS: orego-Tao-PETSc.edp
XFAIL: heat-TS-2d-PETSc.edp
XFAIL: heat-TS-RHS-2d-PETSc.edp
XFAIL: advection-TS-2d-PETSc.edp
PASS: toy-Tao-PETSc.edp
XFAIL: minimal-surface-Tao-2d-PETSc.edp
XFAIL: Schur-complement-PETSc.edp
XFAIL: maxwell-2d-PETSc.edp
XFAIL: maxwell-3d-PETSc.edp
XFAIL: laplace-adapt-3d-PETSc.edp
XFAIL: diffusion-mg-3d-PETSc.edp
XFAIL: save-load-Dmesh.edp
XFAIL: navier-stokes-2d-PETSc.edp
XFAIL: transfer.edp
SKIP: distributed-parmmg.edp
SKIP: laplace-adapt-dist-3d-PETSc.edp
XFAIL: laplace-2d-SLEPc.edp
XFAIL: laplace-spherical-harmonics-2d-SLEPc.edp
XFAIL: laplace-torus-2d-SLEPc.edp
XFAIL: schrodinger-harmonic-oscillator-1d-SLEPc.edp
XFAIL: schrodinger-square-well-1d-SLEPc.edp
XFAIL: schrodinger-axial-well-2d-SLEPc.edp
XFAIL: schrodinger-harmonic-oscillator-2d-SLEPc.edp
XFAIL: laplace-beltrami-3d-surf-SLEPc.edp
XFAIL: laplace-beltrami-3d-line-SLEPc.edp
XFAIL: diffusion-2d-PETSc-complex.edp
XFAIL: helmholtz-2d-PETSc-complex.edp
XFAIL: helmholtz-mg-2d-PETSc-complex.edp
XFAIL: maxwell-mg-3d-PETSc-complex.edp
XFAIL: laplace-2d-SLEPc-complex.edp
XFAIL: navier-stokes-2d-SLEPc-complex.edp
XFAIL: helmholtz-3d-surf-PETSc-complex.edp
XFAIL: helmholtz-3d-line-PETSc-complex.edp
XFAIL: helmholtz-coupled-2d-PETSc-complex.edp
Testsuite summary for FreeFEM 4.6
TOTAL: 87
PASS: 3
SKIP: 2
XFAIL: 82
FAIL: 0
XPASS: 0
ERROR: 0
============================================================================
make[2]: Leaving directory /home/yim/FreeFem-sources/examples/hpddm' make[1]: Leaving directory /home/yim/FreeFem-sources/examples/hpddm’
Most of them are XFAIL… What this means?
When I see for example, diffusion-3d.edp.log, it says this.
‘mpiexec’ -np 4 …/…/src/mpi/FreeFem+±mpi -nw ./diffusion-3d.edp
Fatal error in PMPI_Init_thread: Other MPI error, error stack:
MPIR_Init_thread(805)…: fail failed
MPID_Init(1859)…: channel initialization failed
MPIDI_CH3_Init(126)…: fail failed
MPID_nem_init_ckpt(857)…: fail failed
MPIDI_CH3I_Seg_commit(427): PMI_KVS_Get returned 4
In: PMI_Abort(69777679, Fatal error in PMPI_Init_thread: Other MPI error, error stack:
MPIR_Init_thread(805)…: fail failed
MPID_Init(1859)…: channel initialization failed
MPIDI_CH3_Init(126)…: fail failed
MPID_nem_init_ckpt(857)…: fail failed
MPIDI_CH3I_Seg_commit(427): PMI_KVS_Get returned 4)
INTERNAL ERROR: invalid error code ffffffff (Ring Index out of range) in MPID_nem_gen2_module_get_from_bc:262
INTERNAL ERROR: invalid error code ffffffff (Ring Index out of range) in MPID_nem_gen2_module_get_from_bc:262
INTERNAL ERROR: invalid error code ffffffff (Ring Index out of range) in MPID_nem_gen2_module_get_from_bc:262
INTERNAL ERROR: invalid error code ffffffff (Ring Index out of range) in MPID_nem_gen2_module_get_from_bc:262
[1] Abort: Error code in polled desc!
at line 2502 in file …/…/src/mpid/ch3/channels/nemesis/netmod/ofa/ofa_init.c
[2] Abort: Error code in polled desc!
at line 2502 in file …/…/src/mpid/ch3/channels/nemesis/netmod/ofa/ofa_init.c
===================================================================================
= BAD TERMINATION OF ONE OF YOUR APPLICATION PROCESSES
= PID 4159 RUNNING AT fidis
= EXIT CODE: 9
= CLEANING UP REMAINING PROCESSES
= YOU CAN IGNORE THE BELOW CLEANUP MESSAGES
===================================================================================
= BAD TERMINATION OF ONE OF YOUR APPLICATION PROCESSES
= PID 4162 RUNNING AT fidis
= EXIT CODE: 9
= CLEANING UP REMAINING PROCESSES
= YOU CAN IGNORE THE BELOW CLEANUP MESSAGES
Intel® MPI Library troubleshooting guide:
https://software.intel.com/node/561764
HOWEVER, when I run the code with srun with slurm, I have good results. Do you want me to do the test agian with srun?
— 0/4 - diffusion-3d.edp - input parameters: refinement factor = 1 - overlap = 1
– Cube nv=8 nt=6 nbe=12 kind= 6
– Cube nv=8 nt=6 nbe=12 kind= 6
– Cube nv=8 nt=6 nbe=12 kind= 6
– FESpace: Nb of Nodes 8 Nb of DoF 8
– FESpace: Nb of Nodes 8 Nb of DoF 8
– Cube nv=1331 nt=6000 nbe=1200 kind= 6
– Cube nv=1331 nt=6000 nbe=1200 kind= 6
– Build Nodes/DF on mesh : n.v. 1331, n. elmt. 6000, n b. elmt. 1200
– Build Nodes/DF on mesh : n.v. 1331, n. elmt. 6000, n b. elmt. 1200
nb of Nodes 6000 nb of DoF 6000 DFon=0001
nb of Nodes 6000 nb of DoF 6000 DFon=0001
– FESpace: Nb of Nodes 6000 Nb of DoF 6000
– FESpace: Nb of Nodes 1331 Nb of DoF 1331
– FESpace: Nb of Nodes 6000 Nb of DoF 6000
– FESpace: Nb of Nodes 1331 Nb of DoF 1331
— global mesh of 6000 elements (prior to refinement) partitioned with metis --metisESCOA: 4-way Edge-Cut: 4, Balance: 1.02 Nodal=0/Dual 1
(in 1.995611e-02)
– FESpace: Nb of Nodes 400 Nb of DoF 400
– FESpace: Nb of Nodes 409 Nb of DoF 409
– FESpace: Nb of Nodes 662 Nb of DoF 662
– FESpace: Nb of Nodes 716 Nb of DoF 716
– Cube nv=8 nt=6 nbe=12 kind= 6
– FESpace: Nb of Nodes 8 Nb of DoF 8
– FESpace: Nb of Nodes 8 Nb of DoF 8
– Cube nv=1331 nt=6000 nbe=1200 kind= 6
– Cube nv=1331 nt=6000 nbe=1200 kind= 6
– Build Nodes/DF on mesh : n.v. 1331, n. elmt. 6000, n b. elmt. 1200
– Build Nodes/DF on mesh : n.v. 1331, n. elmt. 6000, n b. elmt. 1200
nb of Nodes 6000 nb of DoF 6000 DFon=0001
– FESpace: Nb of Nodes 6000 Nb of DoF 6000
nb of Nodes 6000 nb of DoF 6000 DFon=0001
– FESpace: Nb of Nodes 6000 Nb of DoF 6000
– FESpace: Nb of Nodes 1331 Nb of DoF 1331
– FESpace: Nb of Nodes 1331 Nb of DoF 1331
– FESpace: Nb of Nodes 392 Nb of DoF 392
– FESpace: Nb of Nodes 408 Nb of DoF 408
– FESpace: Nb of Nodes 645 Nb of DoF 645
– FESpace: Nb of Nodes 746 Nb of DoF 746
– Build Nodes/DF on mesh : n.v. 645, n. elmt. 2675, n b. elmt. 702
nb of Nodes 2675 nb of DoF 2675 DFon=0001
– FESpace: Nb of Nodes 2675 Nb of DoF 2675
– Build Nodes/DF on mesh : n.v. 716, n. elmt. 3013, n b. elmt. 752
– Build Nodes/DF on mesh : n.v. 662, n. elmt. 2756, n b. elmt. 714
nb of Nodes 3013 nb of DoF 3013 DFon=0001
– FESpace: Nb of Nodes 3013 Nb of DoF 3013
nb of Nodes 2756 nb of DoF 2756 DFon=0001
– FESpace: Nb of Nodes 2756 Nb of DoF 2756
– Build Nodes/DF on mesh : n.v. 746, n. elmt. 3174, n b. elmt. 764
nb of Nodes 3174 nb of DoF 3174 DFon=0001
– FESpace: Nb of Nodes 3174 Nb of DoF 3174
– FESpace: Nb of Nodes 513 Nb of DoF 513
– FESpace: Nb of Nodes 530 Nb of DoF 530
– FESpace: Nb of Nodes 547 Nb of DoF 547
– FESpace: Nb of Nodes 569 Nb of DoF 569
– FESpace: Nb of Nodes 513 Nb of DoF 513
– FESpace: Nb of Nodes 197 Nb of DoF 197
– FESpace: Nb of Nodes 530 Nb of DoF 530
– FESpace: Nb of Nodes 206 Nb of DoF 206
– FESpace: Nb of Nodes 513 Nb of DoF 513
– FESpace: Nb of Nodes 220 Nb of DoF 220
– FESpace: Nb of Nodes 513 Nb of DoF 513
– FESpace: Nb of Nodes 530 Nb of DoF 530
– FESpace: Nb of Nodes 220 Nb of DoF 220
– FESpace: Nb of Nodes 530 Nb of DoF 530
– FESpace: Nb of Nodes 547 Nb of DoF 547
– FESpace: Nb of Nodes 206 Nb of DoF 206
– FESpace: Nb of Nodes 547 Nb of DoF 547
– FESpace: Nb of Nodes 163 Nb of DoF 163
– FESpace: Nb of Nodes 547 Nb of DoF 547
– FESpace: Nb of Nodes 197 Nb of DoF 197
– FESpace: Nb of Nodes 547 Nb of DoF 547
– FESpace: Nb of Nodes 569 Nb of DoF 569
– FESpace: Nb of Nodes 163 Nb of DoF 163
– FESpace: Nb of Nodes 569 Nb of DoF 569
– FESpace: Nb of Nodes 220 Nb of DoF 220
– FESpace: Nb of Nodes 569 Nb of DoF 569
– FESpace: Nb of Nodes 220 Nb of DoF 220
– FESpace: Nb of Nodes 569 Nb of DoF 569
— partition of unity built (in 5.068111e-02)
– FESpace: Nb of Nodes 645 Nb of DoF 645
– FESpace: Nb of Nodes 662 Nb of DoF 662
– FESpace: Nb of Nodes 716 Nb of DoF 716
– FESpace: Nb of Nodes 746 Nb of DoF 746
– FESpace: Nb of Nodes 513 Nb of DoF 513
– FESpace: Nb of Nodes 530 Nb of DoF 530
– Cube nv=8 nt=6 nbe=12 kind= 6
– FESpace: Nb of Nodes 547 Nb of DoF 547
– Cube nv=8 nt=6 nbe=12 kind= 6
– FESpace: Nb of Nodes 569 Nb of DoF 569
– Cube nv=8 nt=6 nbe=12 kind= 6
– Cube nv=8 nt=6 nbe=12 kind= 6
times: compile 0.06s, execution 0.29s, mpirank:1
times: compile 0.05s, execution 0.29s, mpirank:2
######## We forget of deleting 0 Nb pointer, 0Bytes , mpirank 1, memory leak =956464
CodeAlloc : nb ptr 6807, size :631600 mpirank: 1
times: compile 0.05s, execution 0.29s, mpirank:3
######## We forget of deleting 0 Nb pointer, 0Bytes , mpirank 2, memory leak =809072
CodeAlloc : nb ptr 6807, size :631600 mpirank: 2
######## We forget of deleting 0 Nb pointer, 0Bytes , mpirank 3, memory leak =956480
CodeAlloc : nb ptr 6807, size :631600 mpirank: 3
times: compile 1.600000e-01s, execution 1.800000e-01s, mpirank:0
######## We forget of deleting 0 Nb pointer, 0Bytes , mpirank 0, memory leak =809072
CodeAlloc : nb ptr 6807, size :631600 mpirank: 0
Ok: Normal End