Strange behavior with MatLoad

General Discussion
1

Dear FreeFem users,

I would like to use ObjectView and MatLoad for saving the PETSc numbering solution (vectors) in binary, possible to dome post-processing in Matlab, etc. However, when I am loading the results back in FreeFem, I get a very stange behavior: the unknown numbering change per process. The following MWE demonstrates this behavior:

load "PETSc"

{
int n=1000;
real[int] rhs(n+mpirank*n*0.1);

real[int, int] rhsView(rhs.n,1);
rhsView(:,0) = rhs;
ObjectView(rhsView, format = "binary", name = "rhsout");
cout << "writing: rank " << mpirank << "/" << mpisize << ", rhsView.n=" << rhsView.n << ", rhs.n=" << rhs.n << endl;
}

{
real[int, int] rhsView(1, 1);
MatLoad(rhsView, format = "binary", name = "rhsout");
real[int] rhs = rhsView.asarray;  
cout << "reading: rank " << mpirank << "/" << mpisize << ", rhsView.n=" << rhsView.n << ", rhs.n=" << rhs.n << endl;
}

The resulting output is the following:

writing: rank 1/2, rhsView.n=1100, rhs.n=1100
writing: rank 0/2, rhsView.n=1000, rhs.n=1000
reading: rank 1/2, rhsView.n=1050, rhs.n=1050
reading: rank 0/2, rhsView.n=1050, rhs.n=1050

I would expect the numerings to be the same. Is this a bug, or am I misunderstanding something?

2

PETSc does not store the partitioning. This has the big advantage that you can save with, e.g., 1 process, and then load with, e.g., 4 processes. In plain PETSc, you can force the local dimension before loading a Mat so that the partitioning suits your needs, but this is not implemented in the FreeFEM plugin. I could add that if you want, though, I’m not sure when (probably not before the end of this weekend).

1 Like
3

Thanks for the quick reply, this makes complete sense.

I think there is no need for you implement additional features, I think I can just (i) save the vector, (ii) save the numbering accessed via GlobalNumbering (along with the corresponding ranks), (iii) load the vector and and the numbering on rank 0, and (iv) distribute the vector myself among the processes. Please correct me if I am wrong.

A note: the example FreeFem-sources/diffusion-substructuring-2d-PETSc.edp at master · FreeFem/FreeFem-sources · GitHub throws me an error at line 45 I think due to incorrect indexing of numbering[i]. I am not sure what the example exactly does, so I cannot propose a fix unfortunately.

4

I was trying to implement what I described above, however, it did not work. The error can be reproduced in the MWE I posted initially: I would expect it to work with reading only on a single core:

if(mpirank==0){
real[int, int] rhsView(1, 1);
MatLoad(rhsView, format = "binary", name = "rhsout");
real[int] rhs = rhsView.asarray;  
cout << "reading: rank " << mpirank << "/" << mpisize << ", rhsView.n=" << rhsView.n << ", rhs.n=" << rhs.n << endl;
}

However, at MatLoad the script seems to freeze. Is this behavior intended, or is this a bug?
(Again, I think I can circumvent this, I am just curious and I would like to understand the behavior)

5

You probably need to specify communicator = mpiCommSelf in the MatLoad().

1 Like
6

Thanks, now it works perfectly!