[CRIU] Restoring lxc-1.1.5 centos 7 container with httpd fails

[Andrey Vagin]

On Fri, May 06, 2016 at 02:09:47PM +0200, Adrian Reber wrote:
> On Thu, May 05, 2016 at 06:15:39PM -0700, Andrey Vagin wrote:
> > On Thu, May 05, 2016 at 02:03:17PM +0300, Pavel Emelyanov wrote:
> > > On 05/04/2016 03:41 PM, Adrian Reber wrote:
> > > > I am able to restore a lxc-1.1.5 container running centos 7. But if I
> > > > start apache's httpd in the container I am no longer able to restore the
> > > > container. I get:
> > > > 
> > > > (00.028105)      1: mnt: 148:./ private 0 shared 0 slave 1
> > > > (00.028206)      1: mnt: Start with 148:./
> > > > (00.028209)      1: mnt: 	Mounting unsupported @./ (0)
> > > > (00.028212)      1: mnt: 	Mounting unsupported @.criu.mntns.8pI85T/13/ (0)
> > > > (00.028215)      1: mnt: 	Bind /tmp/cr-tmpfs.JF8cRh/ to .criu.mntns.8pI85T/13/
> > > > (00.028218)      1: mnt: 93:.criu.mntns.8pI85T/13/ private 0 shared 1 slave 0
> > > > (00.028225)      1: mnt: 	Mounting tmpfs @.criu.mntns.8pI85T/13/dev (0)
> > > > (00.030633)      1: mnt: 94:.criu.mntns.8pI85T/13/dev private 0 shared 1 slave 0
> > > > (00.030645)      1: mnt: 	Mounting devpts @.criu.mntns.8pI85T/13/dev/pts (0)
> > > > (00.030725)      1: mnt: 95:.criu.mntns.8pI85T/13/dev/pts private 0 shared 1 slave 0
> > > > (00.030731)      1: mnt: 	Mounting hugetlbfs @.criu.mntns.8pI85T/13/dev/hugepages (0)
> > > > (00.030753)      1: mnt: 98:.criu.mntns.8pI85T/13/dev/hugepages private 0 shared 1 slave 0
> > > > (00.030758)      1: mnt: 	Mounting mqueue @.criu.mntns.8pI85T/13/dev/mqueue (0)
> > > > (00.030769)      1: mnt: 97:.criu.mntns.8pI85T/13/dev/mqueue private 0 shared 1 slave 0
> > > > (00.030773)      1: mnt: 	Mounting tmpfs @.criu.mntns.8pI85T/13/dev/shm (0)
> > > > (00.033125)      1: mnt: 96:.criu.mntns.8pI85T/13/dev/shm private 0 shared 1 slave 0
> > > > (00.033140)      1: mnt: 	Mounting unsupported @.criu.mntns.8pI85T/13/var/tmp (0)
> > > > (00.033143)      1: mnt: 	Bind //var/lib/lxc/c7/rootfs/var/tmp/systemd-private-953f46ab8f04414db2ca1b71ed6212c2-httpd.service-yl6pWp/tmp to .criu.mntns.8pI85T/13/var/tmp
> > > > (00.033165)      1: Error (mount.c:2464): mnt: Can't mount at .criu.mntns.8pI85T/13/var/tmp: No such file or directory
> > > > (00.033542) Error (cr-restore.c:1384): 18067 exited, status=1
> > > > (00.033610) Switching to new ns to clean ghosts
> > > > (00.033743) Error (cr-restore.c:2223): Restoring FAILED.
> > > > 
> > > > Same error with today's master and criu-dev branch. Full log at:
> > > > 
> > > > https://lisas.de/~adrian/restore.log
> > > > 
> > > > The host is centos 7 as well. And ideas?
> > > 
> > > Can you try recent patches from Andrey:
> > > mount: remove an extra condition from mounts_equal()
> > > mount: handle a case when parent and child mounts in the same directory
> > > 
> > > ?
> > >
> > 
> > I think these patches don't affect this issue.
> 
> Those patches did not help. The following is all with those patches
> applied.
> 
> > We've discussed with Adrian in irc and he promised to give more
> > info about this issue.
> > 
> > To investiage this sort of bugs I add sleep(1000) after pr_err() to
> > freeze processes in a moment of the error and try to find what is wrong
> > here via /proc/PID/root.
> 
> With the sleep after the last pr_err() I see two criu processes:
> 
> # ls -la /proc/10183/root
> lrwxrwxrwx. 1 root root 0 May  6 08:47 /proc/10183/root -> /
> # ls -la /proc/10188/root
> lrwxrwxrwx. 1 root root 0 May  6 08:47 /proc/10188/root -> /

I mean that you need to try to resolve source and target argumnets of a
mount syscall which returns an error.

> 
> > Somtimes strace of criu restore can be useful.
> 
> strace -fo strace.log -s 256  lxc-checkpoint -r -n c7 -D /share/cp -vvvv

strace doesn't work for this case, because criu restore is executed
with the --restore-sibling option.

> 
> http://lisas.de/~adrian/strace-restore.log
> http://lisas.de/~adrian/strace.log
> 
> 
> 		Adrian