MPI test for script in (examples/mpi)

General Discussion
1

Hello everyone.

I’m not sure why all of scripts which need load "MUMPS_mpi" should be run with using the sudo command

  1. Env : ubuntu22.04 with FreeFem++ compiled using mpich2
  2. Installation information :
    ./configure --enable-download --enable-optim --with-slepc-include=/usr/local/slepc/include --with-hdf5=/usr/bin/h5cc --with-mpi=/usr/local/bin/mpich/bin/mpic++ then using sudo make install after getting 3rd packages and sudo make -j 32 check
  • minimal example
(base) cyc@cyc-Z10PA-D8-Series:~/Desktop/Fem/FreeFem/FreeFemMODS/test_parallel$ FreeFem++ test_mumps.edp 
-- FreeFem++ v4.12 (Wed Jun  7 10:36:46 AM CST 2023 - git v4.12)
   file : test_mumps.edp
 Load: lg_fem lg_mesh lg_mesh3 eigenvalue 
    1 : load "MUMPS_mpi"
Load error: MUMPS_mpi
	 fail: 
 dlerror : /usr/local/lib/ff++/4.12/lib/MUMPS_mpi.so: cannot open shared object file: No such file or directory
list prefix: './' '/usr/local/lib/ff++/4.12/lib/' list suffix: '' , '.so' 
  current line = 1
Load error : MUMPS_mpi
	line number :1, MUMPS_mpi
error Load error : MUMPS_mpi
	line number :1, MUMPS_mpi
 code = 2 mpirank: 0
(base) cyc@cyc-Z10PA-D8-Series:~/Desktop/Fem/FreeFem/FreeFemMODS/test_parallel$ ff-mpirun -np 4 test_mumps.edp 
'/usr/local/ff-petsc/r/bin/mpiexec' -np 4 /usr/local/bin/FreeFem++-mpi -nw 'test_mumps.edp'
-- FreeFem++ v4.12 (Wed Jun  7 10:36:46 AM CST 2023 - git v4.12)
   file : test_mumps.edp
 Load: lg_fem lg_mesh lg_mesh3 eigenvalue parallelempi 
    1 : load "MUMPS_mpi"  current line = 1 mpirank 1 / 4
  current line = 1 mpirank 3 / 4
  current line = 1 mpirank 2 / 4

Load error: MUMPS_mpi
	 fail: 
 dlerror : /usr/local/bin/mpich/lib/libmpifort.so.12: undefined symbol: PMPIX_Start_progress_thread
list prefix: './' '/usr/local/lib/ff++/4.12/lib/mpi/' list suffix: '' , '.so' 
  current line = 1 mpirank 0 / 4
Load error : MUMPS_mpi
	line number :1, MUMPS_mpi
error Load error : MUMPS_mpi
	line number :1, MUMPS_mpi
 code = 2 mpirank: 0
(base) cyc@cyc-Z10PA-D8-Series:~/Desktop/Fem/FreeFem/FreeFemMODS/test_parallel$ sudo ff-mpirun -np 4 test_mumps.edp 
[sudo] password for cyc: 
'/usr/local/ff-petsc/r/bin/mpiexec' -np 4 /usr/local/bin/FreeFem++-mpi -nw 'test_mumps.edp'
-- FreeFem++ v4.12 (Wed Jun  7 10:36:46 AM CST 2023 - git v4.12)
   file : test_mumps.edp
 Load: lg_fem lg_mesh lg_mesh3 eigenvalue parallelempi 
    1 : load "MUMPS_mpi"
    2 : "" sizestack + 1024 =1072  ( 48 )

times: compile 0.007603s, execution times: compile 0.007594s, execution 5e-05s,  mpirank:2times: compile 0.007601s, execution 4.7e-05s,  mpirank:3
times: compile 0.00765s, execution 3.4e-05s,  mpirank:0
 CodeAlloc : nb ptr  39374.4e-05s,  mpirank:1
 CodeAlloc : nb ptr  3937,  size :536648 mpirank: 1

 CodeAlloc : nb ptr  3937,  size :536648 mpirank: 2
 CodeAlloc : nb ptr  3937,  size :536648 mpirank: 3
,  size :536648 mpirank: 0
Ok: Normal End

for Stokes-v1-matrix-mumps.edp

  • Error Code
'/usr/local/ff-petsc/r/bin/mpiexec' -np 4 /usr/local/bin/FreeFem++-mpi -nw 'Stokes-v1-matrix-mumps.edp'
/usr/local/bin/FreeFem++-mpi: /home/cyc/mambaforge/lib/libcurl.so.4: no version information available (required by /lib/x86_64-linux-gnu/libhdf5_serial.so.103)
/usr/local/bin/FreeFem++-mpi: /home/cyc/mambaforge/lib/libcurl.so.4: no version information available (required by /lib/x86_64-linux-gnu/libhdf5_serial.so.103)
/usr/local/bin/FreeFem++-mpi: /home/cyc/mambaforge/lib/libcurl.so.4: no version information available (required by /lib/x86_64-linux-gnu/libhdf5_serial.so.103)
/usr/local/bin/FreeFem++-mpi: /home/cyc/mambaforge/lib/libcurl.so.4: no version information available (required by /lib/x86_64-linux-gnu/libhdf5_serial.so.103)
-- FreeFem++ v4.12 (Wed Jun  7 10:36:46 AM CST 2023 - git v4.12)
   file : Stokes-v1-matrix-mumps.edp
 Load: lg_fem lg_mesh lg_mesh3 eigenvalue parallelempi 
    1 : //  run with MPI:  ff-mpirun -np 4 script.edp
    2 : // NBPROC 4
    3 : // PARAM -n 5
    4 : 
    5 : /*
    6 : to change locatation of daynamic lib. set   freefem++.pref   file like :
    7 : more freefem++.pref
    8 : loadpath = "../../plugin/seq/"
    9 : loadpath += "./"
   10 : */
   11 : 
   12 : load "msh3"
   13 : //load "medit"
   14 : load "MUMPS_mpi"
Load error: MUMPS_mpi
	 fail: 
 dlerror : ../../plugin/seq/MUMPS_mpi.so: cannot open shared object file: No such file or directory
list prefix: '../../plugin/mpi/' '' '../../plugin/seq/' list suffix: '' , '.so' 
  current line = 14 mpirank 0 / 4
Load error : MUMPS_mpi
	line number :14, MUMPS_mpi
error Load error : MUMPS_mpi
	line number :14, MUMPS_mpi
 code = 2 mpirank: 0
  current line = 14  current line = 14 mpirank 3 / 4
 mpirank 2 / 4
  current line = 14 mpirank 1 / 4
  • Output of locate MUMPS_mpi 
(base) cyc@cyc-Z10PA-D8-Series:~/dev/FreeFem-sources/examples/mpi$ locate MUMPS_mpi
/home/cyc/dev/FreeFem-sources/plugin/mpi/MUMPS_mpi.cpp
/home/cyc/dev/FreeFem-sources/plugin/mpi/MUMPS_mpi.o
/home/cyc/dev/FreeFem-sources/plugin/mpi/MUMPS_mpi.so
/usr/local/FreeFem/lib/ff++/4.12/lib/mpi/MUMPS_mpi.so
/usr/local/lib/ff++/4.12/lib/mpi/MUMPS_mpi.so
  • config of freefem++.pref
(base) cyc@cyc-Z10PA-D8-Series:~/dev/FreeFem-sources/examples/mpi$ which ff-mpirun
/usr/local/bin/ff-mpirun
(base) cyc@cyc-Z10PA-D8-Series:~/dev/FreeFem-sources/examples/mpi$ cat /usr/local/lib/ff++/4.12/etc/freefem++.pref
loadpath += "./"
loadpath += "/usr/local/lib/ff++/4.12/lib"
includepath += "/usr/local/lib/ff++/4.12/idp"
  • with sudo
(base) cyc@cyc-Z10PA-D8-Series:~/dev/FreeFem-sources/examples/mpi$ sudo ff-mpirun Stokes-v1-matrix-mumps.edp
[sudo] password for cyc: 
'/usr/local/ff-petsc/r/bin/mpiexec' /usr/local/bin/FreeFem++-mpi -nw 'Stokes-v1-matrix-mumps.edp'
-- FreeFem++ v4.12 (Wed Jun  7 10:36:46 AM CST 2023 - git v4.12)
   file : Stokes-v1-matrix-mumps.edp
 Load: lg_fem lg_mesh lg_mesh3 eigenvalue parallelempi 
    1 : //  run with MPI:  ff-mpirun -np 4 script.edp
    2 : // NBPROC 4
    3 : // PARAM -n 5
    4 : 
    5 : /*
    6 : to change locatation of daynamic lib. set   freefem++.pref   file like :
    7 : more freefem++.pref
    8 : loadpath = "../../plugin/seq/"
    9 : loadpath += "./"
   10 : */
   11 : 
   12 : load "msh3"
   13 : //load "medit"
   14 : load "MUMPS_mpi"
   15 : include "getARGV.idp" // for gestion of FreeFem++ argument and in version 3.10-1 FH
    2 :  // F. Hecht 
    3 :  // Usage:  getARGV(n,defaultvalue) // get the fist used default valeu
    4 :  //  or     getARGV(after,defaultvalue) // get the arg after after 
    5 :  // the type of delfaut value given the return type: int,double, string
    6 :  // Modif version 3.54-2  Jan 2018 (add ones include)
    7 :  IFMACRO(!getARGVidp)
    8 &  macro getARGVidp 1 //
    9 &  
   10 &  
   11 &  func int usedARGV(int n)
   12 & {
   13 &   int k=1,ii=1,kk=1,ret=-1;
   14 &   for(int i=1;i<ARGV.n;++i)
   15 &    {
   16 &    // cout <<i<< " "<< ARGV[i] << " " <<(ARGV[i]=="-v") << " " << kk << "==" 
   17 &    //      << n << " " << ARGV[i].rfind("dp") << " " <<ARGV[i].length-2  << endl;
   18 &         
   19 &        if(ARGV[i]=="-v") i++;
   20 &        else if(ARGV[i]=="-fglut") i++;
   21 &        else if(ARGV[i]=="-ffg") i++;
   22 &        else if(ARGV[i]=="-glut") i++;
   23 &        else if(ARGV[i]=="-f") i++;
   24 &        else if(ARGV[i]=="-nw") ii;
   25 &        else if(ARGV[i]=="-wait") ii;
   26 &        else if(ARGV[i]=="-ne") ii;
   27 &        else if(ARGV[i]=="-cd") ii;
   28 &        //else if(ARGV[i].rfind(".edp")==ARGV[i].length-4 ) ii;
   29 &        else if(i==1) ii;
   30 &        else if(kk++==n) {ret=i;}
   31 &       // else cout << " +++ \n";
   32 &      }
   33 &    //  cout << ret << endl;
   34 &     return ret;
   35 & }
   36 & 
   37 & func int usedARGV(string after)
   38 & {
   39 &   int ret=-1;
   40 &   for(int i=ARGV.n-1;i>=0;--i)        
   41 &        if(ARGV[i]==after) { ret=++i; break;}
   42 &   if(ARGV.n<ret) ret=-1;
   43 &   return ret;
   44 & }
   45 & 
   46 & func int getARGV(int n,int default)
   47 & {
   48 &    int d=default;
   49 &    int k=usedARGV(n);
   50 &    if(k>0) d=strtol(ARGV[k]);
   51 &     return d;
   52 & }
   53 & func real getARGV(int n,real default)
   54 & {
   55 &    real d=default;
   56 &    int k=usedARGV(n);
   57 &    if(k>0) d=strtod(ARGV[k]);
   58 &    return d;
   59 & }
   60 & func string getARGV(int n,string default)
   61 & {
   62 &    string d=default;
   63 &    int k=usedARGV(n);
   64 &    if(k>0) d=ARGV[k];
   65 &     return d;
   66 & }
   67 & 
   68 & func int getARGV(string after,int default)
   69 & {
   70 &    int d=default;
   71 &    int k=usedARGV(after);
   72 &    if(k>0) d=strtol(ARGV[k]);
   73 &     return d;
   74 & }
   75 & func real getARGV(string after,real default)
   76 & {
   77 &    real d=default;
   78 &    int k=usedARGV(after);
   79 &    if(k>0) d=strtod(ARGV[k]);
   80 &    return d;
   81 & }
   82 & func string getARGV(string after,string default)
   83 & {
   84 &    string d=default;
   85 &    int k=usedARGV(after);
   86 &    if(k>0) d=ARGV[k];
   87 &     return d;
   88 & }
   89 & 
   90 & /*
   91 & cout << getARGV(1,100) << endl;
   92 & cout << getARGV(2,200.) << endl;
   93 & cout << getARGV(3,"300.000") << endl;
   94 & cout << getARGV("-n"," xxx") << endl;
   95 & */
   96 & ENDIFMACRO
    8 @  macro getARGVidp 1 //
    9 @  
   10 @  
   11 @  func int usedARGV(int n)
   12 @ {
   13 @   int k=1,ii=1,kk=1,ret=-1;
   14 @   for(int i=1;i<ARGV.n;++i)
   15 @    {
   16 @    // cout <<i<< " "<< ARGV[i] << " " <<(ARGV[i]=="-v") << " " << kk << "==" 
   17 @    //      << n << " " << ARGV[i].rfind("dp") << " " <<ARGV[i].length-2  << endl;
   18 @         
   19 @        if(ARGV[i]=="-v") i++;
   20 @        else if(ARGV[i]=="-fglut") i++;
   21 @        else if(ARGV[i]=="-ffg") i++;
   22 @        else if(ARGV[i]=="-glut") i++;
   23 @        else if(ARGV[i]=="-f") i++;
   24 @        else if(ARGV[i]=="-nw") ii;
   25 @        else if(ARGV[i]=="-wait") ii;
   26 @        else if(ARGV[i]=="-ne") ii;
   27 @        else if(ARGV[i]=="-cd") ii;
   28 @        //else if(ARGV[i].rfind(".edp")==ARGV[i].length-4 ) ii;
   29 @        else if(i==1) ii;
   30 @        else if(kk++==n) {ret=i;}
   31 @       // else cout << " +++ \n";
   32 @      }
   33 @    //  cout << ret << endl;
   34 @     return ret;
   35 @ }
   36 @ 
   37 @ func int usedARGV(string after)
   38 @ {
   39 @   int ret=-1;
   40 @   for(int i=ARGV.n-1;i>=0;--i)        
   41 @        if(ARGV[i]==after) { ret=++i; break;}
   42 @   if(ARGV.n<ret) ret=-1;
   43 @   return ret;
   44 @ }
   45 @ 
   46 @ func int getARGV(int n,int default)
   47 @ {
   48 @    int d=default;
   49 @    int k=usedARGV(n);
   50 @    if(k>0) d=strtol(ARGV[k]);
   51 @     return d;
   52 @ }
   53 @ func real getARGV(int n,real default)
   54 @ {
   55 @    real d=default;
   56 @    int k=usedARGV(n);
   57 @    if(k>0) d=strtod(ARGV[k]);
   58 @    return d;
   59 @ }
   60 @ func string getARGV(int n,string default)
   61 @ {
   62 @    string d=default;
   63 @    int k=usedARGV(n);
   64 @    if(k>0) d=ARGV[k];
   65 @     return d;
   66 @ }
   67 @ 
   68 @ func int getARGV(string after,int default)
   69 @ {
   70 @    int d=default;
   71 @    int k=usedARGV(after);
   72 @    if(k>0) d=strtol(ARGV[k]);
   73 @     return d;
   74 @ }
   75 @ func real getARGV(string after,real default)
   76 @ {
   77 @    real d=default;
   78 @    int k=usedARGV(after);
   79 @    if(k>0) d=strtod(ARGV[k]);
   80 @    return d;
   81 @ }
   82 @ func string getARGV(string after,string default)
   83 @ {
   84 @    string d=default;
   85 @    int k=usedARGV(after);
   86 @    if(k>0) d=ARGV[k];
   87 @     return d;
   88 @ }
   89 @ 
   90 @ /*
   91 @ cout << getARGV(1,100) << endl;
   92 @ cout << getARGV(2,200.) << endl;
   93 @ cout << getARGV(3,"300.000") << endl;
   94 @ cout << getARGV("-n"," xxx") << endl;
   95 @ */
   96 @ 
   16 : 
   17 : verbosity=0;
   18 : real ttgv=1e10;
   19 : 
   20 : 
   21 : 
   22 : int nn=getARGV("-n",15);
   23 : int pplot=getARGV("-plot",0);
   24 : 
   25 : 
   26 : int[int] ll=[1,1,1,1,1,2];
   27 : mesh3 Th;
   28 : if(mpirank==0)
   29 : {
   30 : 	Th=cube(nn,nn,nn,label=ll);
   31 : 	Th=change(Th,fregion=nuTriangle%mpisize);
   32 : }
   33 : broadcast(processor(0),Th);
   34 : fespace VVh(Th,[P2,P2,P2,P1]);
   35 : fespace UUh(Th,[P2,P2,P2]);
   36 : fespace Uh(Th,P2);
   37 : fespace Ph(Th,P1);
   38 : 
   39 : macro Grad(u) [dx(u),dy(u),dz(u)] )  // EOM
   40 : macro div(u1,u2,u3) (dx(u1)+dy(u2)+dz(u3))  )  //EOM
   41 : 
   42 : func fup = (1-x)*(x)*y*(1-y)*16;
   43 : 
   44 : 
   45 : VVh [u1,u2,u3,p];
   46 : VVh [v1,v2,v3,q];
   47 : 
   48 : real timeI=mpiWtime();
   49 : real time1=mpiWtime();
   50 : varf vStokes([u1,u2,u3,p],[v1,v2,v3,q]) =
   51 :   int3d(Th,qforder=3,mpirank)( Grad(u1)  [dx(u1),dy(u1),dz(u1)]'*Grad(v1)  [dx(v1),dy(v1),dz(v1)] +  Grad(u2)  [dx(u2),dy(u2),dz(u2)]'*Grad(v2)  [dx(v2),dy(v2),dz(v2)] +  Grad(u3)  [dx(u3),dy(u3),dz(u3)]'*Grad(v3)  [dx(v3),dy(v3),dz(v3)]   //' for emacs
   52 :                   - div(u1,u2,u3)   (dx(u1)+dy(u2)+dz(u3)) *q - div(v1,v2,v3)   (dx(v1)+dy(v2)+dz(v3)) *p + 1e-10*q*p )
   53 :   + on(2,u1=fup,u2=0,u3=0) + on(1,u1=0,u2=0,u3=0) ;
   54 : 
   55 : 
   56 : matrix MStokes=vStokes(VVh,VVh,tgv=ttgv);
   57 : if(mpirank==0)
   58 : cout << "size of matrix " << MStokes.n << " x " << MStokes.m << " nn nzero coef  = " << MStokes.nbcoef << endl;
   59 : time1=mpiWtime()-time1;
   60 : 
   61 : real timeF=mpiWtime();
   62 : set(MStokes,solver=sparsesolver,master=-1);
   63 : timeF=mpiWtime()-timeF;
   64 : 
   65 : real time2=mpiWtime();
   66 : real[int] b=vStokes(0,VVh);
   67 : time2=mpiWtime()-time2;
   68 : 
   69 : real time3=mpiWtime();
   70 : u1[] = MStokes^-1*b;
   71 : time3=mpiWtime()-time3;
   72 : 
   73 : timeI=mpiWtime()-timeI;
   74 : 
   75 : if(mpirank==0)
   76 : {
   77 : cout << "============= CPU TIME ============" << endl;
   78 : cout << "size of matrix " << MStokes.n << " x " << MStokes.m << " nn nzero coef  = " << MStokes.nbcoef << endl;
   79 : cout << " matrix                  " <<  time1 << endl;
   80 : cout << " Fact                    " <<  timeF << endl;
   81 : cout << " second member           " <<  time2 << endl;
   82 : cout << " solve                   " <<  time3 << endl;
   83 : cout << "                          ------------" << endl;
   84 : cout << " all                     " <<  timeI << endl;
   85 : cout << "============= CPU TIME ============" << endl;
   86 : }
   87 : //if(mpirank==0 && pplot) medit("UV2 PV2",Th,[u1,u2,u3],p);
   88 :  sizestack + 1024 =2432  ( 1408 )

size of matrix 93469 x 93469 nn nzero coef  = 8718991
============= CPU TIME ============
size of matrix 93469 x 93469 nn nzero coef  = 8718991
 matrix                  2.39332
 Fact                    0.000302255
 second member           0.00842016
 solve                   13.2696
                          ------------
 all                     15.6716
============= CPU TIME ============
2

You should use a prefix when configuring. Then, you won’t need sudo.

3

Thanks a lot for your quick reply, but I tried adding --prefix= when recompiling, and it didn’t work :disappointed:

4

Please send the new output you get.

5

new out put

  • as user
(base) cyc@cyc-Z10PA-D8-Series:~/Desktop/Fem/FreeFem/FreeFemMODS/test_parallel$ ff-mpirun -np 4 script.edp 
'/usr/local/bin/mpich/bin/mpiexec' -np 4 /usr/local/FreeFem/bin/FreeFem++-mpi -nw 'script.edp'
-- FreeFem++ v4.12 (Thu Jun  8 12:28:00 AM CST 2023 - git v4.12)
   file : script.edp
 Load: lg_fem lg_mesh lg_mesh3 eigenvalue parallelempi 
    1 : //  run with MPI:  ff-mpirun -np 4 script.edp
    2 : // NBPROC 4
    3 : // PARAM -n 5
    4 : 
    5 : /*
    6 : to change locatation of daynamic lib. set   freefem++.pref   file like :
    7 : more freefem++.pref
    8 : loadpath = "../../plugin/seq/"
    9 : loadpath += "./"
   10 : */
   11 : 
   12 : load "msh3"
   13 : //load "medit"
   14 : load "MUMPS_mpi"
Load error: MUMPS_mpi
	 fail: 
 dlerror : /usr/local/bin/mpich/lib/libmpifort.so.12: undefined symbol: PMPIX_Start_progress_thread
list prefix: './' '/usr/local/FreeFem/lib/ff++/4.12/lib/mpi/' list suffix: '' , '.so' 
  current line = 14 mpirank 0 / 4
Load error : MUMPS_mpi
	line number :14, MUMPS_mpi
error Load error : MUMPS_mpi
	line number :14, MUMPS_mpi
 code = 2 mpirank: 0
  current line = 14 mpirank 3 / 4
  current line = 14 mpirank 1 / 4
  • as sudo
(base) cyc@cyc-Z10PA-D8-Series:~/Desktop/Fem/FreeFem/FreeFemMODS/test_parallel$ sudo ff-mpirun -np 4 script.edp 
[sudo] password for cyc: 
sudo: ff-mpirun: command not found

i am sorry that i have no access to upload the config.log file as a new user. The following is part of output of config.log

This file contains any messages produced by compilers while
running configure, to aid debugging if configure makes a mistake.

It was created by FreeFEM configure 4.12, which was
generated by GNU Autoconf 2.71.  Invocation command line was

  $ ./configure --enable-download --enable-optim --with-hdf5=/usr/bin/h5cc --with-mpi=/usr/local/bin/mpich/bin/mpic++ --prefix=/usr/local/FreeFem

## --------- ##
## Platform. ##
## --------- ##

hostname = cyc-Z10PA-D8-Series
uname -m = x86_64
uname -r = 5.19.0-43-generic
uname -s = Linux
uname -v = #44~22.04.1-Ubuntu SMP PREEMPT_DYNAMIC Mon May 22 13:39:36 UTC 2

/usr/bin/uname -p = x86_64
/bin/uname -X     = unknown

/bin/arch              = x86_64
/usr/bin/arch -k       = unknown
/usr/convex/getsysinfo = unknown
/usr/bin/hostinfo      = unknown
/bin/machine           = unknown
/usr/bin/oslevel       = unknown
/bin/universe          = unknown

PATH: /usr/local/bin/mpich/bin/
PATH: /home/cyc/mambaforge/bin/
PATH: /home/cyc/mambaforge/condabin/
PATH: /usr/local/texlive/2022/texmf-dist/scripts/latexindent/
PATH: /usr/local/texlive/2022/bin/x86_64-linux/
PATH: /usr/local/bin/
PATH: /usr/local/hdf5/bin/
PATH: /home/cyc/.local/bin/
PATH: /home/cyc/bin/
PATH: /usr/local/sbin/
PATH: /usr/local/bin/
PATH: /usr/sbin/
PATH: /usr/bin/
PATH: /sbin/
PATH: /bin/
PATH: /usr/games/
PATH: /usr/local/games/
PATH: /snap/bin/
PATH: /snap/bin/
PATH: /usr/local/go/bin/
PATH: /usr/local/go/bin/
PATH: /home/cyc/bin/
PATH: ./
PATH: /usr/lib/
PATH: /home/cyc/lib/
PATH: /usr/local/hdf5/lib/
PATH: /usr/local/petsc/lib/
PATH: /usr/local/cuda/lib64/
PATH: /usr/local/cuda/extras/CPUTI/lib64/
PATH: /usr/local/cuda/bin/
6

dlerror : /usr/local/bin/mpich/lib/libmpifort.so.12: undefined symbol: PMPIX_Start_progress_thread, you are probably mixing different MPI implementations. I would recommend that you start from scratch, by removing all previous installations and use a prefix with no sudo needed, e.g., ${HOME}/FreeFem-install.

7
  • FreeFem++
(base) cyc@cyc-Z10PA-D8-Series:~/Desktop/Fem/FreeFem/FreeFemMODS/test_parallel$ FreeFem++ script.edp 
-- FreeFem++ v4.12 (Thu Jun  8 11:13:38 AM CST 2023 - git v4.12)
   file : script.edp
 Load: lg_fem lg_mesh lg_mesh3 eigenvalue 
    1 : //  run with MPI:  ff-mpirun -np 4 script.edp
    2 : // NBPROC 4
    3 : // PARAM -n 5
    4 : 
    5 : /*
    6 : to change locatation of daynamic lib. set   freefem++.pref   file like :
    7 : more freefem++.pref
    8 : loadpath = "../../plugin/seq/"
    9 : loadpath += "./"
   10 : */
   11 : 
   12 : load "msh3"
   13 : //load "medit"
   14 : load "MUMPS_mpi"
Load error: MUMPS_mpi
	 fail: 
 dlerror : /home/cyc/local/FreeFem++/lib/ff++/4.12/lib/MUMPS_mpi.so: cannot open shared object file: No such file or directory
list prefix: './' '/home/cyc/local/FreeFem++/lib/ff++/4.12/lib/' list suffix: '' , '.so' 
  current line = 14
Load error : MUMPS_mpi
	line number :14, MUMPS_mpi
error Load error : MUMPS_mpi
	line number :14, MUMPS_mpi
 code = 2 mpirank: 0
  • ff-mpirun
'/usr/local/bin/mpich/bin/mpiexec' -np 4 /home/cyc/local/FreeFem++/bin/FreeFem++-mpi -nw 'script.edp'
-- FreeFem++ v4.12 (Thu Jun  8 11:13:38 AM CST 2023 - git v4.12)
   file : script.edp
 Load: lg_fem lg_mesh lg_mesh3 eigenvalue parallelempi 
    1 : //  run with MPI:  ff-mpirun -np 4 script.edp
    2 : // NBPROC 4
    3 : // PARAM -n 5
    4 : 
    5 : /*
    6 : to change locatation of daynamic lib. set   freefem++.pref   file like :
    7 : more freefem++.pref
    8 : loadpath = "../../plugin/seq/"
    9 : loadpath += "./"
   10 : */
   11 : 
   12 : load "msh3"
   13 : //load "medit"
   14 : load "MUMPS_mpi"
Load error: MUMPS_mpi
	 fail: 
 dlerror : /usr/local/bin/mpich/lib/libmpifort.so.12: undefined symbol: PMPIX_Start_progress_thread
list prefix: './' '/home/cyc/local/FreeFem++/lib/ff++/4.12/lib/mpi/' list suffix: '' , '  current line = 14 mpirank 2 / 4
.so' 
  current line = 14 mpirank 0 / 4
Load error : MUMPS_mpi
	line number :14, MUMPS_mpi
  current line = 14 mpirank 1 / 4
error Load error : MUMPS_mpi
	line number :14, MUMPS_mpi
 code = 2 mpirank: 0
  current line = 14 mpirank 3 / 4
  • sudo ff-mpirun
(base) cyc@cyc-Z10PA-D8-Series:~/Desktop/Fem/FreeFem/FreeFemMODS/test_parallel$ sudo ff-mpirun -np 4 script.edp 
sudo: ff-mpirun: command not found
  • config.log
This file contains any messages produced by compilers while
running configure, to aid debugging if configure makes a mistake.

It was created by FreeFEM configure 4.12, which was
generated by GNU Autoconf 2.71.  Invocation command line was

  $ ./configure --enable-download --enable-optim --with-hdf5=/usr/bin/h5cc --with-mpi=/usr/local/bin/mpich/bin/mpic++ --prefix=/home/cyc/local/FreeFem++

## --------- ##
## Platform. ##
## --------- ##

hostname = cyc-Z10PA-D8-Series
uname -m = x86_64
uname -r = 5.19.0-43-generic
uname -s = Linux
uname -v = #44~22.04.1-Ubuntu SMP PREEMPT_DYNAMIC Mon May 22 13:39:36 UTC 2

/usr/bin/uname -p = x86_64
/bin/uname -X     = unknown

/bin/arch              = x86_64
/usr/bin/arch -k       = unknown
/usr/convex/getsysinfo = unknown
/usr/bin/hostinfo      = unknown
/bin/machine           = unknown
/usr/bin/oslevel       = unknown
/bin/universe          = unknown

PATH: /usr/local/bin/mpich/bin/
PATH: /home/cyc/mambaforge/bin/
PATH: /home/cyc/mambaforge/condabin/
PATH: /usr/local/texlive/2022/texmf-dist/scripts/latexindent/
PATH: /usr/local/texlive/2022/bin/x86_64-linux/
PATH: /usr/local/bin/
PATH: /usr/local/hdf5/bin/
PATH: /home/cyc/.local/bin/
PATH: /home/cyc/bin/
PATH: /usr/local/sbin/
PATH: /usr/local/bin/
PATH: /usr/sbin/
PATH: /usr/bin/
PATH: /sbin/
PATH: /bin/
PATH: /usr/games/
PATH: /usr/local/games/
PATH: /snap/bin/
PATH: /snap/bin/
PATH: /usr/local/go/bin/
PATH: /usr/local/go/bin/
PATH: /usr/local/FreeFem/bin/
PATH: /home/cyc/bin/
PATH: ./
PATH: /usr/lib/
PATH: /home/cyc/lib/
PATH: /usr/local/FreeFem/lib/ff++/4.12/lib$/
PATH: /usr/local/hdf5/lib/
PATH: /usr/local/petsc/lib/
PATH: /usr/local/cuda/lib64/
PATH: /usr/local/cuda/extras/CPUTI/lib64/
PATH: /usr/local/cuda/bin/
8

Please look at the error message. It’s still looking for dlerror : /usr/local/bin/mpich/lib/libmpifort.so.12: undefined symbol: PMPIX_Start_progress_thread, which is not in your prefix directory, so you did not clean things up properly. Furthermore, why do you have stuff like /usr/local/bin/mpich/bin/ or /usr/local/FreeFem/lib/ff++/4.12/lib$/ in your PATH?

9

Thanks for your reply.

  • libmpifort.so.12 not in prefix directory

does it mean that i should recompile and install mpich in a directory without requirement of sudo

  • /usr/local/bin/mpich/bin/ or /usr/local/FreeFem/lib/ff++/4.12/lib$/ in PATH

i just followed the installguide mpich-4.1.1-installguide , with file ~/.bashrc modified for adding export PATH=/mpich-installed/bin:$PATH

10

Please remove everything that you installed by hand. FreeFEM will automatically build PETSc, which itself will build MPICH, and things will work. You are mixing things currently, and since you do not know how to make them work, I’d suggest you stick to the default installation procedure.