Memory usage during remeshing is much greater than normal pepc-v occupation [pepc-v]
Summary
When starting the remeshing stage the memory usage largely increases, more than one order of magnitude than the occupation during time marching.
Steps to reproduce
Run the case as in vrings.h.
I launched it with mpirun -np 1 pepc-v vrings.h
. The behaviour seems to be the same disregarding the number of ranks or threads and of the value of the np_mult
parameter.
I tried monitoring memory occupation using valgrind mpirun -np 1 valgrind --tool=massif pepc-v vrings.h
but it crashes at some point. Since I'm not an expert of valgrind I gave up and used the more basical htop
tool.
What is the current behaviour?
For this case about 300k particles are involved. During the RK stages the virtual memory is large but resident memory occupation is rather low (400M) When remeshing starts the memory occupation gradually increases up to a maximum of about 10G. Then after remeshing the number of particles is about 180k (about half with respect to the initial configuration) but the memory occupation is around 5G, larger than the value before remeshing but lower with respect to remeshing stage.
Logs and Screenshots
Here logs and htop screenshots are provided (I stopped the process to print the appropriate line from htop). During the RK stages:
When remeshing is running:
After remeshing:
Where does the problem occur
The problem most probably comes from the generation of particle duplicates during remeshing. Indeed, I tested that the memory increase is essentially dependent on the value of mesh_supp
in subroutine remeshing
in manipulate_particles.f90
.
In the current configuration with mesh_supp=4
the number of particles is multiplied by 64 before the collapse of doublets. This seems to be the source of this issue.