[CRIU] [PATCH 00/11] tty: Add c/r for system console

Tue Sep 30 06:32:57 PDT 2014

On Tue, Sep 30, 2014 at 10:47:06AM +0400, Cyrill Gorcunov wrote:
> On Mon, Sep 29, 2014 at 06:31:51PM -0500, Tycho Andersen wrote:
> > Hi Cyrill,
> > 
> > On Mon, Sep 29, 2014 at 08:25:26PM +0400, Cyrill Gorcunov wrote:
> > > Hi guys, here is a series which implements c/r for /dev/console,
> > > I'm not really sure how to test it -- I simply modified our pty00
> > > test case adding
> > > 
> > > 	console = open("/dev/console", O_RDONLY);
> > > 
> > > into it to check that c/r flow takes proper branches (and looked
> > > into image data).
> > > 
> > > I guess some help needed presumably attempting to c/r lxc conatiner
> > > with /dev/console bound to some readable output.
> > > 
> > > Anyway, any feedback would be highly appreciated!
> > 
> > Thanks for your work on this set. This patch does fix the aforementioned bug
> > about not being able to checkpoint containers that have been restored. (I'm not
> > sure which patch does it, but with the entire series applied it works just
> > fine. I can go back and find out if this information would be useful.)
> 
> Hi Tycho! Thanks a lot for giving it a shot (and btw I remember about the
> problem with sequential c/r where second dump failed due to tty layer, simply
> didn't manage yet to investigate, I suspect I need to achieve same error on
> my testing node, once I manage to I'll ping you, hopefully on the week).

What I meant by the above is that this patchset actually fixes that
error for me (somehow, not sure which patch does it :).

> > 
> > However, when I try to checkpoint a container with /dev/console and some ttys, I
> > get:
> > 
> > Error (mount.c:453): 113:./dev/lxc/tty4 doesn't have a proper root mount
> > 
> > On an LXC in my system with ttys enabled, the /proc/self/mountinfo has:
> > 
> > ubuntu at u1:~$ cat /proc/self/mountinfo 
> > 75 76 253:1 /usr/local/var/lib/lxc/u1/rootfs / rw,relatime - ext4 /dev/disk/by-uuid/7fd0b676-46f3-46d6-94a6-5c620e72e7b0 rw,data=ordered
> > 103 75 0:35 / /proc rw,nosuid,nodev,noexec,relatime - proc proc rw
> > 104 75 0:33 / /sys rw,relatime - sysfs sysfs rw
> > 105 104 0:18 / /sys/fs/fuse/connections rw,relatime - fusectl none rw
> > 106 104 0:6 / /sys/kernel/debug rw,relatime - debugfs none rw
> > 107 104 0:10 / /sys/kernel/security rw,relatime - securityfs none rw
> > 108 104 0:22 / /sys/fs/pstore rw,relatime - pstore none rw
> > 109 75 0:12 /7 /dev/lxc/console rw,nosuid,noexec,relatime - devpts devpts rw,gid=5,mode=620,ptmxmode=000
> > 110 75 0:12 /1 /dev/lxc/tty1 rw,nosuid,noexec,relatime - devpts devpts rw,gid=5,mode=620,ptmxmode=000
> > 111 75 0:12 /3 /dev/lxc/tty2 rw,nosuid,noexec,relatime - devpts devpts rw,gid=5,mode=620,ptmxmode=000
> > 112 75 0:12 /5 /dev/lxc/tty3 rw,nosuid,noexec,relatime - devpts devpts rw,gid=5,mode=620,ptmxmode=000
> > 113 75 0:12 /6 /dev/lxc/tty4 rw,nosuid,noexec,relatime - devpts devpts rw,gid=5,mode=620,ptmxmode=000
> > 77 75 0:36 / /dev/pts rw,relatime - devpts devpts rw,gid=5,mode=620,ptmxmode=666
> > 78 75 0:36 /ptmx /dev/ptmx rw,relatime - devpts devpts rw,gid=5,mode=620,ptmxmode=666
> > 79 104 0:37 / /sys/fs/cgroup rw,relatime - tmpfs none rw,size=4k,mode=755
> > 80 75 0:38 / /run rw,nosuid,noexec,relatime - tmpfs none rw,size=200060k,mode=755
> > 81 80 0:39 / /run/lock rw,nosuid,nodev,noexec,relatime - tmpfs none rw,size=5120k
> > 82 80 0:40 / /run/shm rw,nosuid,nodev,relatime - tmpfs none rw
> > 83 80 0:41 / /run/user rw,nosuid,nodev,noexec,relatime - tmpfs none rw,size=102400k,mode=755
> > 
> 
> Mind to share your lxc conatiner details and how you set it up?

Sure, the config I used is:

# Template used to create this container:
# /usr/local/share/lxc/templates/lxc-download
# Parameters passed to the template: -d ubuntu -r trusty -a amd64
# For additional config options, please look at lxc.container.conf(5)

# Distribution configuration
lxc.include = /usr/local/share/lxc/config/ubuntu.common.conf
lxc.arch = x86_64

# Container specific configuration
lxc.rootfs = /usr/local/var/lib/lxc/u1/rootfs
lxc.utsname = u1

# Network configuration
lxc.network.type = veth
lxc.network.flags = up
lxc.network.link = lxcbr0
lxc.network.hwaddr = 00:16:3e:0e:08:34

# hax for criu
#lxc.console = none
#lxc.tty = 0
lxc.cgroup.devices.deny = c 5:1 rwm

And I created it with the command line:

sudo lxc-create -t download -n u1 -- -d ubuntu -r trusty -a amd64

> > When I try to work around that with something like,
> > 
> > diff --git a/mount.c b/mount.c
> > index 8c60693..a28e041 100644
> > --- a/mount.c
> > +++ b/mount.c
> > @@ -433,8 +433,19 @@ static int validate_mounts(struct mount_info *info, bool for_dump)
> >  					ret = run_plugins(DUMP_EXT_MOUNT, m->mountpoint, m->mnt_id);
> >  					if (ret == 0)
> >  						m->need_plugin = true;
> > -					else if (ret == -ENOTSUP)
> > -						ret = try_resolve_ext_mount(m);
> > +					else if (ret == -ENOTSUP) {
> > +						/* inside containers
> > +						 * /dev/console and /dev/tty*
> > +						 * are mounted from paths like
> > +						 * /5 or /6, so we ignore
> > +						 * their path here if things
> > +						 * go bad.
> > +						 */
> > +						if (m->fstype->code == FSTYPE__DEVPTS)
> > +							ret = 0;
> > +						else
> > +							ret = try_resolve_ext_mount(m);
> > +					}
> >  				} else {
> >  					if (m->need_plugin || m->external)
> >  						/*
> > 
> > 
> > I hit:
> > 
> > (00.247031) Error (tty.c:233): tty: Unexpected format on path /dev/lxc/tty4
> > 
> > which looks like it might be a bigger problem :)
> > 
> > Any thoughts are appreciated.
> 
> Show me the stat output on this paths please? (stat /dev/lxc/tty4)

ubuntu at u1:~$ stat /dev/lxc/tty4
  File: ‘/dev/lxc/tty4’
  Size: 0           Blocks: 0          IO Block: 1024   character special file
Device: ch/12d  Inode: 10          Links: 1     Device type: 88,7
Access: (0660/crw-rw----)  Uid: (    0/    root)   Gid: (    5/     tty)
Access: 2014-09-30 13:28:09.359281000 +0000
Modify: 2014-09-30 13:28:09.359281000 +0000
Change: 2014-09-30 13:28:13.239281000 +0000
 Birth: -

Tycho