# Full parallel adaptmesh

**URL:** https://community.freefem.org/t/full-parallel-adaptmesh/759
**Category:** General Discussion
**Created:** [January 26, 2021, 12:27pm UTC](https://community.freefem.org/t/full-parallel-adaptmesh/759 "2021-01-26T12:27:38Z")
**Posts on this page:** 12
**Page:** 1

<div class="post-metadata">

### Author: ![LucasPa](https://yyz2.discourse-cdn.com/flex030/user_avatar/community.freefem.org/lucaspa/32/347_2.png) [@LucasPa](https://community.freefem.org/u/LucasPa)
#### Post date: [January 26, 2021, 12:27pm UTC](https://community.freefem.org/t/full-parallel-adaptmesh/759/1 "2021-01-26T12:27:38Z")

</div>

Hi!

From several examples, I understood how to perform mesh adaptation. In particular in case of parallel example, I noticed that the following steps are taken:

1. Compute the global solution from solutions on local meshes (with solution .\*= matrice.D, restrict, etc …)
2. Perform adaptmesh on the global solution only on proc 0 and then broadcast the new mesh on all other procs.
3. Interpolate the global solution on the new mesh.
4. Re-create the partitioning.
5. Compute local solutions from the global solution.

I wonder if it would be possible to do something similar fully in parallel, i.e. all procs would run adaptmesh and not only the proc 0. I understand that this is not trivial as it means that we should have after partitioning identical overlapping elements.

Best,

Lucas

---

<div class="post-metadata">

### Author: ![prj](https://avatars.discourse-cdn.com/v4/letter/p/ecae2f/32.png) [@prj](https://community.freefem.org/u/prj)
#### Post date: [January 26, 2021, 12:49pm UTC](https://community.freefem.org/t/full-parallel-adaptmesh/759/2 "2021-01-26T12:49:24Z")

</div>

Hello Lucas,  
You are explicitly mentioning `adaptmesh`, which only works in 2D, so I’m guessing right now you are focusing on a 2D problem. Unfortunately:

1. ParMmg only works in 3D;
2. I’m not sure `adaptmesh` is 100% deterministic, and for sure, it is sequential.

You can try to do a `mpiAllReduce` instead of `mpiReduce`, and call `adaptmesh` on all processes, but please be careful and make sure that the output mesh is the same on all processes, e.g., by checking `ThOutput.nt` and `ThOutput.nv`.  
You can also perform step 5 in parallel, using the `transfer` macro.  
All in all, the adaptation step is supposed to be quite cheap in 2D, this is a completely different story in 3D, is it not the case in your application?  
In 3D, you can do eveything in parallel, see, e.g., [laplace-adapt-dist-3d-PETSc.edp](https://github.com/FreeFem/FreeFem-sources/blob/master/examples/hpddm/laplace-adapt-dist-3d-PETSc.edp). Only the initial mesh is create redundantly on all processes, but this can be bypassed, see, e.g., [reconstructDmesh.edp](https://github.com/FreeFem/FreeFem-sources/blob/master/examples/hpddm/reconstructDmesh.edp).

---

<div class="post-metadata">

### Author: ![LucasPa](https://yyz2.discourse-cdn.com/flex030/user_avatar/community.freefem.org/lucaspa/32/347_2.png) [@LucasPa](https://community.freefem.org/u/LucasPa)
#### Post date: [January 26, 2021, 1:42pm UTC](https://community.freefem.org/t/full-parallel-adaptmesh/759/3 "2021-01-26T13:42:48Z")

</div>

Thank you Pierre for this clear answer.

Indeed I am working on 2D cases, I guess for now I can live with sequential mesh adaptation ^^

---

<div class="post-metadata">

### Author: ![cmd](https://yyz2.discourse-cdn.com/flex030/user_avatar/community.freefem.org/cmd/32/67_2.png) [@cmd](https://community.freefem.org/u/cmd)
#### Post date: [January 9, 2023, 3:46pm UTC](https://community.freefem.org/t/full-parallel-adaptmesh/759/4 "2023-01-09T15:46:35Z")

</div>

Hi Pierre, would it be possible to update the example `newton-adaptmesh-2d-PETSc.edp` to use the `createMat()` and `buildDmesh()` macros (perhaps also showing the use of `transfer()`?) instead of using the older `buildMat()` macro?

I am facing a problem with a parallel code that iteratively calls `adaptmesh`, and I think I am messing up the PETSc numbering somewhere.

---

<div class="post-metadata">

### Author: ![prj](https://avatars.discourse-cdn.com/v4/letter/p/ecae2f/32.png) [@prj](https://community.freefem.org/u/prj)
#### Post date: [January 9, 2023, 3:50pm UTC](https://community.freefem.org/t/full-parallel-adaptmesh/759/5 "2023-01-09T15:50:26Z")

</div>

Would the example [FreeFem-sources/laplace-adapt-3d-PETSc.edp at master · FreeFem/FreeFem-sources · GitHub](https://github.com/FreeFem/FreeFem-sources/blob/master/examples/hpddm/laplace-adapt-3d-PETSc.edp) help you in anyway?

---

<div class="post-metadata">

### Author: ![cmd](https://yyz2.discourse-cdn.com/flex030/user_avatar/community.freefem.org/cmd/32/67_2.png) [@cmd](https://community.freefem.org/u/cmd)
#### Post date: [January 9, 2023, 4:03pm UTC](https://community.freefem.org/t/full-parallel-adaptmesh/759/6 "2023-01-09T16:03:49Z")

</div>

Thank you for pointing that one out. I will play around with this to see if I can figure out my issue.

---

<div class="post-metadata">

### Author: ![cmd](https://yyz2.discourse-cdn.com/flex030/user_avatar/community.freefem.org/cmd/32/67_2.png) [@cmd](https://community.freefem.org/u/cmd)
#### Post date: [January 9, 2023, 4:35pm UTC](https://community.freefem.org/t/full-parallel-adaptmesh/759/7 "2023-01-09T16:35:19Z")

</div>

This did help. It turns out my issue was that I was doing this:

```auto
real[int] q;
changeNumbering(A, u[], q, inverse = true, exchange = false);
Vhg uG, uR;
uR = u;
mpiAllReduce(uR[], uG[], mpiCommWorld, mpiSUM);

```

instead of this:

```auto
real[int] q;
changeNumbering(A, u[], q, inverse = true, exchange = false);
Vhg uG, uR;
for[i, v : restu] uR[][v] = u[][i];
mpiAllReduce(uR[], uG[], mpiCommWorld, mpiSUM);

```

Thanks for the guidance!

---

<div class="post-metadata">

### Author: ![prj](https://avatars.discourse-cdn.com/v4/letter/p/ecae2f/32.png) [@prj](https://community.freefem.org/u/prj)
#### Post date: [January 9, 2023, 4:38pm UTC](https://community.freefem.org/t/full-parallel-adaptmesh/759/8 "2023-01-09T16:38:03Z")

</div>

Best to avoid the interpolator whenever you can 🙂

---

<div class="post-metadata">

### Author: ![cmd](https://yyz2.discourse-cdn.com/flex030/user_avatar/community.freefem.org/cmd/32/67_2.png) [@cmd](https://community.freefem.org/u/cmd)
#### Post date: [January 9, 2023, 4:45pm UTC](https://community.freefem.org/t/full-parallel-adaptmesh/759/9 "2023-01-09T16:45:50Z")

</div>

On that note, is there a better way to update the `fespace` besides using interpolation after mesh adaptation?

Right now, I do this:

```auto
broadcast(processor(0), Thg);
uG = uG;
{
  Th = Thg;
  Mat Adapt;
  createMatu(Th, Adapt, Pk);
  A = Adapt;
}
u = uG;

```

but it would be nice to replace the last line with something like:

```auto
{
  int[int] temprest;
  temprest = restrict(Vh, Vhg, n2o);
  rest.resize(temprest.n);
  rest = temprest;
}
u[] = uG[](rest);

```

---

<div class="post-metadata">

### Author: ![prj](https://avatars.discourse-cdn.com/v4/letter/p/ecae2f/32.png) [@prj](https://community.freefem.org/u/prj)
#### Post date: [January 9, 2023, 5:04pm UTC](https://community.freefem.org/t/full-parallel-adaptmesh/759/10 "2023-01-09T17:04:05Z")

</div>

This gets the job done. I’m not sure this is what you are looking for?

```auto
diff --git a/examples/hpddm/laplace-adapt-3d-PETSc.edp b/examples/hpddm/laplace-adapt-3d-PETSc.edp
index 7f7bb3d6..72c7da7d 100644
--- a/examples/hpddm/laplace-adapt-3d-PETSc.edp
+++ b/examples/hpddm/laplace-adapt-3d-PETSc.edp
@@ -57,6 +57,15 @@ for(int i = 0; i < iMax; ++i) {
         createMat(Th3, Adapt, P1);
         A = Adapt;
         u = 0.0;
+ hReduced = hReduced; // this may be costly, interpolation on the global adapted mesh
+ {
+ int[int] temprest;
+ temprest = restrict(Vh, VhBackup, n2o);
+ rest.resize(temprest.n);
+ rest = temprest;
+ }
+ u[] = hReduced[](rest);
+ plot(u);
         err *= 0.5;
     }
 }

```

---

<div class="post-metadata">

### Author: ![cmd](https://yyz2.discourse-cdn.com/flex030/user_avatar/community.freefem.org/cmd/32/67_2.png) [@cmd](https://community.freefem.org/u/cmd)
#### Post date: [January 9, 2023, 5:19pm UTC](https://community.freefem.org/t/full-parallel-adaptmesh/759/11 "2023-01-09T17:19:59Z")

</div>

This code is still calling the interpolator on the line `u = 0.0;` though, correct? Is there a way to bypass that?

---

<div class="post-metadata">

### Author: ![prj](https://avatars.discourse-cdn.com/v4/letter/p/ecae2f/32.png) [@prj](https://community.freefem.org/u/prj)
#### Post date: [January 9, 2023, 5:21pm UTC](https://community.freefem.org/t/full-parallel-adaptmesh/759/12 "2023-01-09T17:21:56Z")

</div>

Since `0.0` is an analytical function, this is not calling the interpolator, just evaluating the function.
