# Number of processes and memory usage **URL:** \ **Category:** General Discussion\ **Created:** [February 23, 2022, 12:21pm UTC](https://community.freefem.org/t/number-of-processes-and-memory-usage/1551 "2022-02-23T12:21:40Z")\ **Posts on this page:** 16\ **Page:** 1
**Author:** ![aszaboa](https://yyz2.discourse-cdn.com/flex030/user_avatar/community.freefem.org/aszaboa/32/2918_2.png) [@aszaboa](https://community.freefem.org/u/aszaboa)\ **Post date:** [February 23, 2022, 12:21pm UTC](https://community.freefem.org/t/number-of-processes-and-memory-usage/1551/1 "2022-02-23T12:21:40Z")
Dear FreeFem users, when I run FreeFem in parallel on e.g., 5 processes, it also starts 10 sleeping processes. These processes seem to consume memory. Should I be worried about this? That is the reason for this? See also the attached image. I experience this behaviour on ubuntu 18.04, either mounted from oracle virtualbox or on a machine with just ubuntu. I use a recently compiled version of FreeFem (develop branch). I run FreeFem with `mpirun --allow-run-as-root -np 5 FreeFem++-mpi .....` ![sleepingprocess](https://canada1.discourse-cdn.com/flex030/uploads/freefem/original/2X/c/cc807ee8e71f4a91134cd1ddee701a0cb56fd76a.png) ---
**Author:** ![prj](https://avatars.discourse-cdn.com/v4/letter/p/ecae2f/32.png) [@prj](https://community.freefem.org/u/prj)\ **Post date:** [February 23, 2022, 1:18pm UTC](https://community.freefem.org/t/number-of-processes-and-memory-usage/1551/2 "2022-02-23T13:18:17Z")
What BLAS are you using? ---
**Author:** ![aszaboa](https://yyz2.discourse-cdn.com/flex030/user_avatar/community.freefem.org/aszaboa/32/2918_2.png) [@aszaboa](https://community.freefem.org/u/aszaboa)\ **Post date:** [February 23, 2022, 2:45pm UTC](https://community.freefem.org/t/number-of-processes-and-memory-usage/1551/3 "2022-02-23T14:45:50Z")
I must admit, I have no idea what the FreeFem compilation process does, so it is likely what I am not looking at the right place. I am sending you the config.log file from the FreeFem-sources directory. Could you please help me out how can i check the BLAS version? [config.log](https://community.freefem.org/uploads/short-url/a3ud2bVZoOQ9R7kDUVcMmpKmx1r.log) (347.1 KB) ---
**Author:** ![prj](https://avatars.discourse-cdn.com/v4/letter/p/ecae2f/32.png) [@prj](https://community.freefem.org/u/prj)\ **Post date:** [February 23, 2022, 3:20pm UTC](https://community.freefem.org/t/number-of-processes-and-memory-usage/1551/4 "2022-02-23T15:20:44Z")
It’s in the file you sent or in PETSc `configure.log`. But it looks alright. I think it’s OK, are you getting bad performance? ---
**Author:** ![aszaboa](https://yyz2.discourse-cdn.com/flex030/user_avatar/community.freefem.org/aszaboa/32/2918_2.png) [@aszaboa](https://community.freefem.org/u/aszaboa)\ **Post date:** [February 23, 2022, 3:28pm UTC](https://community.freefem.org/t/number-of-processes-and-memory-usage/1551/5 "2022-02-23T15:28:10Z")
The speed of the calculation is fine I suppose. I am trying out [the steady-state mAL preconditioner NS solver](https://github.com/prj-/moulin2019al), and I seems to run out of memory quite fast (at least compared to other commercial CFD software with about the same mesh size). I found that changing the last part of `string paramsV = "-ksp_type gmres -ksp_pc_side right " + "-ksp_rtol " + tolV + " -ksp_gmres_restart 50 -pc_type asm " + "-pc_asm_overlap 1 -sub_pc_type lu -sub_pc_factor_mat_solver_type mumps";` to `-pc_asm_overlap 1 -sub_pc_type ilu` helps. Do you have any suggestion maybe to decrease the memory requirement (even if the computational speed is slowed down)? ---
**Author:** ![prj](https://avatars.discourse-cdn.com/v4/letter/p/ecae2f/32.png) [@prj](https://community.freefem.org/u/prj)\ **Post date:** [February 23, 2022, 3:40pm UTC](https://community.freefem.org/t/number-of-processes-and-memory-usage/1551/6 "2022-02-23T15:40:56Z")
What’s the size of the problem? ---
**Author:** ![aszaboa](https://yyz2.discourse-cdn.com/flex030/user_avatar/community.freefem.org/aszaboa/32/2918_2.png) [@aszaboa](https://community.freefem.org/u/aszaboa)\ **Post date:** [February 23, 2022, 4:56pm UTC](https://community.freefem.org/t/number-of-processes-and-memory-usage/1551/7 "2022-02-23T16:56:10Z")
This run just dpes not run out of ram on a virtual machine with 12 GB ram. number of triangles: 244925,number of vertices: 47183 total ndof: 1183721 (in FreeFem numbering) I am also sending you the output of the `-log_view` option. [Nonlinear-solver\_Re350.log](https://community.freefem.org/uploads/short-url/akG8wNS8rC002PctSMIDsWwK1Sg.log) (14.3 KB) ---
**Author:** ![prj](https://avatars.discourse-cdn.com/v4/letter/p/ecae2f/32.png) [@prj](https://community.freefem.org/u/prj)\ **Post date:** [February 23, 2022, 8:21pm UTC](https://community.freefem.org/t/number-of-processes-and-memory-usage/1551/8 "2022-02-23T20:21:22Z")
This is a tiny problem, why not use plain LU? ---
**Author:** ![aszaboa](https://yyz2.discourse-cdn.com/flex030/user_avatar/community.freefem.org/aszaboa/32/2918_2.png) [@aszaboa](https://community.freefem.org/u/aszaboa)\ **Post date:** [February 24, 2022, 8:24am UTC](https://community.freefem.org/t/number-of-processes-and-memory-usage/1551/9 "2022-02-24T08:24:25Z")
For this mesh size the mAL preconditioned GMRES method seems to outperform plain LU factorization considering lower memory usage, but LU is slightly faster. The thing is, we want to run this on our desktop machine, or even if we will use a cluster, we would like to use only a few processes (4-6/case), and run several cases in parallel (the scaling of other parts of the workflow which take up most of the computation determine the number of processes). This is why we are concerned with the memory usage. Do you have maybe other suggestions to altering the numerics? E.g., use `-pc_type gamg` instead of `-pc_type asm`? Inner iteration tolerance? And referring back to the original question regarding the additional sleeping processes: to you think I can ignore these, or should I investigate the cause? If the latter, where should I start! ---
**Author:** ![prj](https://avatars.discourse-cdn.com/v4/letter/p/ecae2f/32.png) [@prj](https://community.freefem.org/u/prj)\ **Post date:** [February 24, 2022, 4:47pm UTC](https://community.freefem.org/t/number-of-processes-and-memory-usage/1551/10 "2022-02-24T16:47:13Z")
If you’ll stick to low process counts, you could try `PCASM` + `PCLU` for subdomain solvers (maybe with MUMPS instead of the default PETSc LU factorisation). For the sleeping processes, I don’t know, sorry. ---
**Author:** ![aszaboa](https://yyz2.discourse-cdn.com/flex030/user_avatar/community.freefem.org/aszaboa/32/2918_2.png) [@aszaboa](https://community.freefem.org/u/aszaboa)\ **Post date:** [February 24, 2022, 8:46pm UTC](https://community.freefem.org/t/number-of-processes-and-memory-usage/1551/11 "2022-02-24T20:46:28Z")
Thanks for the helpful comments. Just to be sure, you mean by `PCASM`+`PCLU` for the subdomain solver the following: `-pc_type asm -pc_asm_overlap 1 -sub_pc_type asm -sub_sub_pc_type lu -sub_sub_pc_factor_mat_solver_type mumps`? (this seems to run and converge) ---
**Author:** ![prj](https://avatars.discourse-cdn.com/v4/letter/p/ecae2f/32.png) [@prj](https://community.freefem.org/u/prj)\ **Post date:** [February 24, 2022, 8:51pm UTC](https://community.freefem.org/t/number-of-processes-and-memory-usage/1551/12 "2022-02-24T20:51:28Z")
No, it should be `-pc_type asm -pc_asm_overlap 1 -sub_pc_type lu -sub_pc_factor_mat_solver_type [mumps,petsc]`. Always check on a small problem with `-ksp_view` to make sure that what you think you are feeding to PETSc is what it is actually using. ---
**Author:** ![aszaboa](https://yyz2.discourse-cdn.com/flex030/user_avatar/community.freefem.org/aszaboa/32/2918_2.png) [@aszaboa](https://community.freefem.org/u/aszaboa)\ **Post date:** [February 25, 2022, 9:32am UTC](https://community.freefem.org/t/number-of-processes-and-memory-usage/1551/13 "2022-02-25T09:32:41Z")
I see what I mixed up. Thanks for the tip. Just two more questions: as I said previously, for `PCASM`, `-sub_pc_type ilu` seems to outperform `-sub_pc_type lu`. Do you think it is worth experimenting with the `-sub_ksp_type` option? Do you have other suggestion what other method could be chosen for the subdomain? ---
**Author:** ![prj](https://avatars.discourse-cdn.com/v4/letter/p/ecae2f/32.png) [@prj](https://community.freefem.org/u/prj)\ **Post date:** [February 25, 2022, 1:30pm UTC](https://community.freefem.org/t/number-of-processes-and-memory-usage/1551/14 "2022-02-25T13:30:15Z")
Finding the most efficient solver is a job in itself. There are too many options to just list them here, it takes time to adjust and fine tune such a method. ---
**Author:** ![aszaboa](https://yyz2.discourse-cdn.com/flex030/user_avatar/community.freefem.org/aszaboa/32/2918_2.png) [@aszaboa](https://community.freefem.org/u/aszaboa)\ **Post date:** [February 28, 2022, 7:16am UTC](https://community.freefem.org/t/number-of-processes-and-memory-usage/1551/15 "2022-02-28T07:16:27Z")
Ok, thanks for the tips! ---
**Author:** ![aszaboa](https://yyz2.discourse-cdn.com/flex030/user_avatar/community.freefem.org/aszaboa/32/2918_2.png) [@aszaboa](https://community.freefem.org/u/aszaboa)\ **Post date:** [May 20, 2022, 1:16pm UTC](https://community.freefem.org/t/number-of-processes-and-memory-usage/1551/16 "2022-05-20T13:16:33Z")
Regarding the additional sleeping processes, changing openmpi to mpich helped; see [this](https://community.freefem.org/t/sorting-out-memory-leakage/1745/15) post for more detail.