[CRIU] Soft dirty pages in expanded VMA

[Mike Rapoport]

Hi,

On Tue, May 14, 2019 at 12:10:26PM -0700, Diyu Zhou wrote:
> Hey Mike,
> 
> Thanks for your reply. I understand what you wrote. However, what I observe is
> that some of the "clean" pages are copied by the CRIU.
> 
> For example, let's say CRIU performs iterative checkpoints (let's say using
> predump) on a dumpee process. After one checkpoint is completed, the dumpee
> process calls sbrk to increase the size of the heap. What I observe is that
> even the dumpee program didn't touch any pages in the heap, all the present
> pages in the heap will be marked as dirty and copied in the next checkpoint
> operation. Is there an explanation for such a behavior? 

Are you running with '--track-mem'?

If I understand correctly, the size of pages*img increases for each
iteration?

Can you check what's the contents of pagemap*img with
'crit show /path/to/pagemap-<pid>.img'
and see if the second and the next iterations contain areas marked as
'PE_PRESENT'?
 
> Thanks in advance for your help.
> 
> Best,
> Diyu

--
Sincerely yours,
Mike.
 
> On Tue, May 14, 2019 at 9:04 AM Mike Rapoport <rppt at linux.ibm.com> wrote:
> 
>     On Mon, May 13, 2019 at 04:10:54PM -0700, Diyu Zhou wrote:
>     > Hey, 
>     >
>     > Firstly, thanks for the great tool! It is very very helpful. I can see
>     the
>     > development of such a tool 
>     > is very challenging and you guys do several clever tricks to make it
>     possible.
>     > I enjoy a lot using 
>     > the tool as well as reading the source code and have learned a lot from
>     it. 
>     >
>     > I have a question regarding the implementation of the soft dirty pages.
>     From
>     > what I can tell, if 
>     > a VMA is expanded, all the pages within it will be marked as soft dirty.
>     This
>     > seems to me will 
>     > result in unnecessarily transferring clean pages when iterative dumps are
>     > performed. 
>     > Is there any reason not just mark the pages in the expanded part of the
>     VMA
>     > soft dirty? 
> 
>     When a VMA is expanded is is marked soft-dirty to ensure it would be
>     scanned for the dirty pages.
>     Each page has its own marking that says whether the page is dirty or not.
>     The per-page information is available at /proc/PID/pagemap and CRIU uses it
>     to actually decide what page should be dumped.
> 
>     > Thanks again for the wonderful tool and your help! 
>     >
>     > Best,
>     > Diyu
>     >
>     >
>     >  
> 
>     > _______________________________________________
>     > CRIU mailing list
>     > CRIU at openvz.org
>     > https://lists.openvz.org/mailman/listinfo/criu
> 
> 
>     --
>     Sincerely yours,
>     Mike.
> 
>