[CRIU] [PATCH 5/9] Handle vmsplice failure for read mode pre-dump

[Andrei Vagin]

On Fri, Sep 27, 2019 at 03:14:03PM +0530, abhishek dubey wrote:
> Yes, this is something for which I had no clue. So I fixed it with minor
> hack by writing 1 page less to pipe.
> 
> Fixing it will require deep debugging.

I think I know where the problem is. A buffer what you give to vmsplice
isn't aligned to the page size.

Could you remove this hack and try out the this patch:

--- a/criu/page-xfer.c
+++ b/criu/page-xfer.c
@@ -6,6 +6,7 @@
 #include <sys/types.h>
 #include <sys/wait.h>
 #include <sys/stat.h>
+#include <sys/mman.h>
 
 #undef LOG_PREFIX
 #define LOG_PREFIX "page-xfer: "
@@ -822,6 +823,12 @@ int page_xfer_predump_pages(int pid, struct page_xfer *xfer,
 
        char *userbuf = xzalloc(BUFFER_SIZE);
 
+       userbuf = mmap(NULL, BUFFER_SIZE, PROT_READ | PROT_WRITE, MAP_ANONYMOUS | MAP_PRIVATE, -1, 0);
+       if (userbuf == MAP_FAILED) {
+               pr_perror("Unable to mmap a buffer");
+               return -1;
+       }
+
        list_for_each_entry(ppb, &pp->bufs, l) {
 
                timing_start(TIME_MEMDUMP);
@@ -883,7 +890,7 @@ int page_xfer_predump_pages(int pid, struct page_xfer *xfer,
                timing_stop(TIME_MEMWRITE);
        }
 
-       xfree(userbuf);
+       munmap(userbuf, BUFFER_SIZE);
        timing_start(TIME_MEMWRITE);
        return dump_holes(xfer, pp, &cur_hole, NULL);
 }

> 
> On 25/09/19 9:57 PM, Andrei Vagin wrote:
> > On Sun, Sep 22, 2019 at 09:54:54AM +0530, Abhishek Dubey wrote:
> > > vmsplice fails to completely splice 2^k pages
> > > from user-buffer to ppb pipes. Sometimes last
> > > page is partially spliced.
> > > To overcome this, page count for splicing is
> > > kept 1 less than the size of pipe.
> > > 
> > > Signed-off-by: Abhishek Dubey <dubeyabhishek777 at gmail.com>
> > > ---
> > >   criu/page-pipe.c | 16 ++++++++++++++++
> > >   1 file changed, 16 insertions(+)
> > > 
> > > diff --git a/criu/page-pipe.c b/criu/page-pipe.c
> > > index a821696..e60dd85 100644
> > > --- a/criu/page-pipe.c
> > > +++ b/criu/page-pipe.c
> > > @@ -40,6 +40,22 @@ static int __ppb_resize_pipe(struct page_pipe_buf *ppb, unsigned long new_size)
> > >   	ret /= PAGE_SIZE;
> > >   	BUG_ON(ret < ppb->pipe_size);
> > > +	/*
> > > +	 * TODO: Investigate and handle
> > > +	 * Sometimes vmsplice fails to splice N pages(where N=2^k)
> > > +	 * from user buffer to pipe, so restricting page-pipe
> > > +	 * capacity to accommodate N-1 pages.
> > > +	 * e.g. Pipe size can be 512 pages, but it's filled till
> > > +	 * 511 pages.
> > > +	 *
> > > +	 * Steps to reproduce issue:
> > > +	 * 1. Comment-out next two lines following this comment
> > > +	 * 2. Run maps007/maps004 from transitive tests with
> > > +	 *    --pre-dump-mode=read option
> > > +	 */
> > > +	if (opts.pre_dump_mode == PRE_DUMP_READ)
> > > +		ret -= 1;
> > This doesn't look nice...
> > 
> > > +
> > >   	pr_debug("Grow pipe %x -> %x\n", ppb->pipe_size, ret);
> > >   	ppb->pipe_size = ret;
> > > -- 
> > > 2.7.4
> > >