[CRIU] Restoring lxc-1.1.5 centos 7 container with httpd fails

[Adrian Reber]

On Thu, May 05, 2016 at 06:15:39PM -0700, Andrey Vagin wrote:
> On Thu, May 05, 2016 at 02:03:17PM +0300, Pavel Emelyanov wrote:
> > On 05/04/2016 03:41 PM, Adrian Reber wrote:
> > > I am able to restore a lxc-1.1.5 container running centos 7. But if I
> > > start apache's httpd in the container I am no longer able to restore the
> > > container. I get:
> > > 
> > > (00.028105)      1: mnt: 148:./ private 0 shared 0 slave 1
> > > (00.028206)      1: mnt: Start with 148:./
> > > (00.028209)      1: mnt: 	Mounting unsupported @./ (0)
> > > (00.028212)      1: mnt: 	Mounting unsupported @.criu.mntns.8pI85T/13/ (0)
> > > (00.028215)      1: mnt: 	Bind /tmp/cr-tmpfs.JF8cRh/ to .criu.mntns.8pI85T/13/
> > > (00.028218)      1: mnt: 93:.criu.mntns.8pI85T/13/ private 0 shared 1 slave 0
> > > (00.028225)      1: mnt: 	Mounting tmpfs @.criu.mntns.8pI85T/13/dev (0)
> > > (00.030633)      1: mnt: 94:.criu.mntns.8pI85T/13/dev private 0 shared 1 slave 0
> > > (00.030645)      1: mnt: 	Mounting devpts @.criu.mntns.8pI85T/13/dev/pts (0)
> > > (00.030725)      1: mnt: 95:.criu.mntns.8pI85T/13/dev/pts private 0 shared 1 slave 0
> > > (00.030731)      1: mnt: 	Mounting hugetlbfs @.criu.mntns.8pI85T/13/dev/hugepages (0)
> > > (00.030753)      1: mnt: 98:.criu.mntns.8pI85T/13/dev/hugepages private 0 shared 1 slave 0
> > > (00.030758)      1: mnt: 	Mounting mqueue @.criu.mntns.8pI85T/13/dev/mqueue (0)
> > > (00.030769)      1: mnt: 97:.criu.mntns.8pI85T/13/dev/mqueue private 0 shared 1 slave 0
> > > (00.030773)      1: mnt: 	Mounting tmpfs @.criu.mntns.8pI85T/13/dev/shm (0)
> > > (00.033125)      1: mnt: 96:.criu.mntns.8pI85T/13/dev/shm private 0 shared 1 slave 0
> > > (00.033140)      1: mnt: 	Mounting unsupported @.criu.mntns.8pI85T/13/var/tmp (0)
> > > (00.033143)      1: mnt: 	Bind //var/lib/lxc/c7/rootfs/var/tmp/systemd-private-953f46ab8f04414db2ca1b71ed6212c2-httpd.service-yl6pWp/tmp to .criu.mntns.8pI85T/13/var/tmp
> > > (00.033165)      1: Error (mount.c:2464): mnt: Can't mount at .criu.mntns.8pI85T/13/var/tmp: No such file or directory
> > > (00.033542) Error (cr-restore.c:1384): 18067 exited, status=1
> > > (00.033610) Switching to new ns to clean ghosts
> > > (00.033743) Error (cr-restore.c:2223): Restoring FAILED.
> > > 
> > > Same error with today's master and criu-dev branch. Full log at:
> > > 
> > > https://lisas.de/~adrian/restore.log
> > > 
> > > The host is centos 7 as well. And ideas?
> > 
> > Can you try recent patches from Andrey:
> > mount: remove an extra condition from mounts_equal()
> > mount: handle a case when parent and child mounts in the same directory
> > 
> > ?
> >
> 
> I think these patches don't affect this issue.

Those patches did not help. The following is all with those patches
applied.

> We've discussed with Adrian in irc and he promised to give more
> info about this issue.
> 
> To investiage this sort of bugs I add sleep(1000) after pr_err() to
> freeze processes in a moment of the error and try to find what is wrong
> here via /proc/PID/root.

With the sleep after the last pr_err() I see two criu processes:

# ls -la /proc/10183/root
lrwxrwxrwx. 1 root root 0 May  6 08:47 /proc/10183/root -> /
# ls -la /proc/10188/root
lrwxrwxrwx. 1 root root 0 May  6 08:47 /proc/10188/root -> /

> Somtimes strace of criu restore can be useful.

strace -fo strace.log -s 256  lxc-checkpoint -r -n c7 -D /share/cp -vvvv

http://lisas.de/~adrian/strace-restore.log
http://lisas.de/~adrian/strace.log


		Adrian