[CRIU] [PATCH] zdtm: add ability to execute tests on aufs
Saied Kazemi
saied at google.com
Tue Sep 9 09:19:14 PDT 2014
Thanks for the patch Andrew. I did a quick test and verified that it fixes
the problem we see in /proc/<pid>/map_files. C/R still worked with the
existing AUFS support code in CRIU although I didn't test extensively.
When this kernel patch has made it to all distributions, we can revert the
AUFS patches in CRIU.
Since it will take some time for the kernel patch to be in all distros,
what do you and Pavel think about the mnt_id patch that I sent earlier?
--Saied
On Mon, Sep 8, 2014 at 10:33 PM, Andrew Vagin <avagin at parallels.com> wrote:
> On Mon, Sep 08, 2014 at 09:00:53AM -0700, Saied Kazemi wrote:
> > Sure, and that would be the right solution. As I haven't worked with J.
> R.
> > Okajima, please feel free to let him know about the issues since you
> seem to be
> > working with him already. Alternatively, I can let him know too.
>
> The patch from J. R. Okajima is attached.
>
> >
> > That said, please note that we'd still need logic in CRIU while the AUFS
> fixed
> > isn't available.
> >
> > --Saied
> >
> >
> >
> > On Mon, Sep 8, 2014 at 8:49 AM, Andrew Vagin <avagin at parallels.com>
> wrote:
> >
> > On Sun, Sep 07, 2014 at 11:36:15PM -0700, Saied Kazemi wrote:
> > > Here is an explanation of the problem. As you know, the AUFS
> > implementation in
> > > the kernel has a bug. Specifically, by pointing inside AUFS
> branches,
> > the file
> >
> > Saied, can we ask AUFS developers to fix this bug? I was talking
> with J. R.
> > Okajima about inotify issues and he fixed all this issues very fast.
> >
> > > descriptors of links in /proc/<pid>/map_files create two problems
> (these
> > file
> > > descriptors are saved by ma_get_mapfile() in vma->vm_file_fd):
> > >
> > > 1. When doing readlink() on them, the path inside the branch is not
> > accessible
> > > in the mount namespace.
> > >
> > > 2. When reading their /proc/<pid>/fdinfo, the mnt_id value
> corresponds to
> > a
> > > mountpoint that is not visible in the mount namespace.
> > >
> > > The patch that we applied earlier addressed issue 1 above.
> Attached is a
> > patch
> > > for issue 2 if we need a quick workaround but I would like a better
> > solution.
> > > The patch has a special provision for AUFS to return the root
> mountpoint
> > if
> > > mnt_id cannot be found.
> > >
> > > What do you guys think?
> > >
> > > --Saied
> > >
> > >
> > >
> > > On Wed, Sep 3, 2014 at 12:32 AM, Andrew Vagin <
> avagin at parallels.com>
> > wrote:
> > >
> > > On Tue, Sep 02, 2014 at 03:14:17PM -0700, Saied Kazemi wrote:
> > > > Pertaining to my previous email, I noticed that when I
> upgraded my
> > Ubuntu
> > > > 14.04's default 3.13 kernel to 3.15 using instructions at
> http://
> > > > ubuntuhandbook.org/index.php/2014/06/
> > install-upgrade-linux-kernel-3-15,
> > > it
> > > > lacks AUFS support causing Docker to use VFS.
> > > >
> > > > I need to test the same system as yours with 3.15 kernel and
> AUFS
> > > support.
> > > > What are you using?
> > >
> > > I use the aufs3.16 branch of
> https://github.com/sfjro/aufs3-linux
> > >
> > > >
> > > > --Saied
> > > >
> > > >
> > > >
> > > > On Tue, Sep 2, 2014 at 11:36 AM, Saied Kazemi <
> saied at google.com>
> > wrote:
> > > >
> > > > Andrew,
> > > >
> > > > I installed 3.15 and ran the env00 test but it succeeded
> for me
> > > (please see
> > > > below). I also verified that checkpointing and restoring
> > Docker
> > > containers
> > > > using AUFS still works.
> > > >
> > > > I noticed that you're specifying --aufs which was
> removed in
> > commit
> > > > 102a6778.
> > > >
> > > > Can you please try again with CRIU source from the
> head? If it
> > still
> > > > fails, please send me specific information on how to
> reproduce
> > the
> > > problem.
> > > >
> > > > Thanks,
> > > >
> > > > --Saied
> > > >
> > > > $ uname -a
> > > > Linux saied-docker 3.15.0-031500-generic #201406131105
> SMP Fri
> > Jun 13
> > > > 15:06:46 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
> > > > $ sudo bash test/zdtm.sh ns/static/env00
> > > > ================================= CRIU CHECK
> ==================
> > ======
> > > ======
> > > > ===
> > > > Error (timerfd.c:56): timerfd: No timerfd support for
> c/r:
> > > Inappropriate
> > > > ioctl for device
> > > > ============================= WARNING
> =========================
> > ====
> > > > Not all features needed for CRIU are merged to upstream
> kernel
> > yet,
> > > > so for now we maintain our own branch which can be
> cloned from:
> > > > git://git.kernel.org/pub/scm/linux/kernel/git/gorcunov/
> > linux-cr.git
> > > >
> ===============================================================
> > ====
> > > > Execute zdtm/live/static/env00
> > > > ./env00 --pidfile=env00.pid --outfile=env00.out
> --envname=
> > ENV_00_TEST
> > > > /home/saied/work/crtools.upstream/test
> > > > Dump 6558
> > > > Restore
> > > > Check results 6587
> > > > 18:26:23.831: 5: PASS
> > > > Test: zdtm/live/static/env00, Result: PASS
> > > > ZDTM tests PASS.
> > > >
> > > > $
> > > >
> > > >
> > > > On Tue, Sep 2, 2014 at 9:07 AM, Saied Kazemi <
> saied at google.com>
> > > wrote:
> > > >
> > > > Hi Andrew,
> > > >
> > > > I was kind of afraid of this because none of the
> kernels
> > that I
> > > had
> > > > supported mnt_id. I will install 3.15 to
> investigate the
> > issue
> > > and get
> > > > back to you soon.
> > > >
> > > > --Saied
> > > >
> > > >
> > > >
> > > > On Tue, Sep 2, 2014 at 7:05 AM, Andrew Vagin <
> > > avagin at parallels.com>
> > > > wrote:
> > > >
> > > > Hi Saied,
> > > >
> > > > I found an issue, which blocks testing of aufs.
> > > > We can't dump nothing on a new kernel (3.15),
> which
> > shows
> > > mnt_id in
> > > > /proc/PID/fdinfo/FD. Looks like we need to fixup
> mnt_id
> > for
> > > aufs
> > > > files
> > > > too.
> > > >
> > > > Said, could you look at this issue?
> > > >
> > > >
> > > > [root at avagin-fc19-cr criu]# bash test/zdtm.sh
> --aufs
> > ns/
> > > static/
> > > > env00
> > > > ================================= CRIU CHECK
> ==========
> > ======
> > > ======
> > > > ===========
> > > > Error (timerfd.c:56): timerfd: No timerfd
> support for c
> > /r:
> > > > Inappropriate ioctl for device
> > > > ============================= WARNING
> =================
> > ======
> > > ======
> > > > Not all features needed for CRIU are merged to
> upstream
> > > kernel yet,
> > > > so for now we maintain our own branch which can
> be
> > cloned
> > > from:
> > > > git://
> git.kernel.org/pub/scm/linux/kernel/git/gorcunov/
> > > linux-cr.git
> > > >
> =======================================================
> > ======
> > > ======
> > > > Execute zdtm/live/static/env00
> > > > ./env00 --pidfile=env00.pid --outfile=env00.out
> > --envname=
> > > > ENV_00_TEST
> > > > /root/git/orig/criu/test
> > > > Dump 30437
> > > > WARNING: env00 returned 1 and left running for
> debug
> > needs
> > > > Test: zdtm/live/static/env00, Result: FAIL
> > > > ==================================== ERROR
> ============
> > ======
> > > ======
> > > > ============
> > > > Test: zdtm/live/static/env00, Namespace: 1
> > > > Dump log :
> /root/git/orig/criu/test/dump/static/env00
> > /30437
> > > /1/
> > > > dump.log
> > > > --------------------------------- grep Error
> > > > ---------------------------------
> > > > (00.035741) Error (files-reg.c:629): Unable to
> look up
> > the 33
> > > mount
> > > > (00.035771) Error (cr-dump.c:1665): Dump
> mappings (pid:
> > > 30437)
> > > > failed with -1
> > > > (00.036042) Error (cr-dump.c:1953): Dumping
> FAILED.
> > > > ------------------------------------- END
> > > > -------------------------------------
> > > > ================================= ERROR OVER
> ==========
> > ======
> > > ======
> > > > ===========
> > > >
> > > > [root at avagin-fc19-cr criu]# cat
> /proc/30437/mountinfo
> > > > 68 42 0:33 / / rw,relatime - aufs none rw,si=
> > 46777ba2737567db
> > > > 43 68 0:35 / /proc rw,relatime - proc proc rw
> > > > 44 68 0:36 / /dev/pts rw,relatime - devpts pts
> rw,mode=
> > > 666,ptmxmode
> > > > =666
> > > >
> > > > [root at avagin-fc19-cr criu]# cat
> /proc/self/mountinfo |
> > grep ^
> > > 33
> > > > 33 0 8:3 / / rw,relatime shared:1 - ext4
> /dev/sda3
> > rw,data=
> > > ordered
> > > >
> > > >
> > > >
> > > > On Sat, Aug 30, 2014 at 02:49:22PM +0400, Andrew
> Vagin
> > wrote:
> > > > > And here are a few issues:
> > > > >
> > > > > + cat
> /tmp/criu-root.7GvnTZ/dump/static/cwd01/6970/1/
> > > dump.log
> > > > > (00.012328) Error (cr-dump.c:303): Can't open
> 6974/
> > cwd on
> > > procfs:
> > > > Stale file handle
> > > > > (00.012350) Error (cr-dump.c:1671): Dump fs
> (pid:
> > 6974)
> > > failed
> > > > with -1
> > > > > (00.012528) Error (cr-dump.c:1953): Dumping
> FAILED.
> > > > >
> > > > > + cat /tmp/criu-root.HTTKBE/dump/static/
> > unlink_fstat03/
> > > 14801/1/
> > > > dump.log
> > > > > (00.009870) Error (files-reg.c:455): Can't
> link remap
> > to /
> > > zdtm/
> > > > live/static/unlink_fstat03.test (deleted): No
> such file
> > or
> > > > directory
> > > > > (00.009906) Error (cr-dump.c:1603): Dump files
> (pid:
> > 14805)
> > > > failed with -1
> > > > > (00.010533) Error (cr-dump.c:1953): Dumping
> FAILED.
> > > > >
> > > > > + cat
> /tmp/criu-root.HDf6Nn/dump/static/unlink_mmap00
> > /23065
> > > /1/
> > > > dump.log
> > > > > (00.008996) Error (proc_parse.c:472): Failed
> stat on
> > > 23069's map
> > > > 47071697305600
> (/tmp/criu-root.HDf6Nn/zdtm/live/static/
> > > > unlink_mmap00.test (deleted)): No such file or
> > directory
> > > > > (00.009021) Error (cr-dump.c:1515): Collect
> mappings
> > (pid:
> > > 23069)
> > > > failed with -1
> > > > > (00.009228) Error (cr-dump.c:1953): Dumping
> FAILED.
> > > > >
> > > > > + cat
> /tmp/criu-root.XULszc/dump/static/inotify00/
> > 30616/1/
> > > > dump.log
> > > > > (00.010422) Error (proc_parse.c:1345):
> > parse_fdinfo_pid_s:
> > > error
> > > > parsing [inotify wd:2 ino:1202 sdev:24
> mask:800afff
> > > ignored_mask:0
> > > > > (00.010465) Error (cr-dump.c:1603): Dump files
> (pid:
> > 30620)
> > > > failed with -1
> > > > > (00.011124) Error (cr-dump.c:1953): Dumping
> FAILED.
> > > > >
> > > > > We have problems with unlinked files and
> fsnotifies.
> > > > >
> > > > > Cc: Saied Kazemi <saied at google.com>
> > > > > Signed-off-by: Andrew Vagin <avagin at openvz.org
> >
> > > > > ---
> > > > > test/zdtm.sh | 11 ++++++++++-
> > > > > 1 file changed, 10 insertions(+), 1
> deletion(-)
> > > > >
> > > > > diff --git a/test/zdtm.sh b/test/zdtm.sh
> > > > > index eee3342..f92a63f 100755
> > > > > --- a/test/zdtm.sh
> > > > > +++ b/test/zdtm.sh
> > > > > @@ -412,7 +412,12 @@ start_test()
> > > > > mkdir -p dump
> > > > > ZDTM_ROOT=`mktemp -d
> /tmp/
> > > criu-root.XXXXXX`
> > > > > ZDTM_ROOT=`readlink -f
> > $ZDTM_ROOT`
> > > > > - mount --make-private
> --bind .
> > > $ZDTM_ROOT ||
> > > > return 1
> > > > > + if [ -z "$ZDTM_AUFS" ];
> then
> > > > > + mount
> --make-private
> > --bind .
> > > > $ZDTM_ROOT || return 1
> > > > > + else
> > > > > + mount -t aufs -o
> br=.
> > none
> > > > $ZDTM_ROOT || return 1
> > > > > + cd $ZDTM_ROOT
> > > > > + fi
> > > > > fi
> > > > > construct_root $ZDTM_ROOT $test
> ||
> > return 1
> > > > > export ZDTM_NEWNS=1
> > > > > @@ -901,6 +906,10 @@ while :; do
> > > > > }
> > > > > shift
> > > > > ;;
> > > > > + --aufs)
> > > > > + export ZDTM_AUFS=1
> > > > > + shift
> > > > > + ;;
> > > > > -*)
> > > > > echo "Unrecognized option $1,
> aborting!
> > " 1>&2
> > > > > usage
> > > > > --
> > > > > 1.9.3
> > > > >
> > > >
> > > >
> > > >
> > > >
> > > >
> > > >
> > >
> > >
> >
> >
> >
> >
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openvz.org/pipermail/criu/attachments/20140909/1ca5c8a4/attachment-0001.html>
More information about the CRIU
mailing list