[CRIU] [PATCH 00/11] tty: Add c/r for system console
Cyrill Gorcunov
gorcunov at gmail.com
Tue Sep 30 11:03:33 PDT 2014
On Tue, Sep 30, 2014 at 08:32:57AM -0500, Tycho Andersen wrote:
> > >
> > > Thanks for your work on this set. This patch does fix the aforementioned bug
> > > about not being able to checkpoint containers that have been restored. (I'm not
> > > sure which patch does it, but with the entire series applied it works just
> > > fine. I can go back and find out if this information would be useful.)
> >
> > Hi Tycho! Thanks a lot for giving it a shot (and btw I remember about the
> > problem with sequential c/r where second dump failed due to tty layer, simply
> > didn't manage yet to investigate, I suspect I need to achieve same error on
> > my testing node, once I manage to I'll ping you, hopefully on the week).
>
> What I meant by the above is that this patchset actually fixes that
> error for me (somehow, not sure which patch does it :).
Actually this patchset only adds support of /dev/console, so in your
report we've had pty pair in first dump, then after restore (for some
unknown reason) instead of ptys we got /dev/console reference and that's
the question who is changed it, I suspect it might be systemd or something
like that.
> > >
> > > However, when I try to checkpoint a container with /dev/console and some ttys, I
> > > get:
> > >
> > > Error (mount.c:453): 113:./dev/lxc/tty4 doesn't have a proper root mount
> > >
> > > On an LXC in my system with ttys enabled, the /proc/self/mountinfo has:
> > >
> > > ubuntu at u1:~$ cat /proc/self/mountinfo
> > > 75 76 253:1 /usr/local/var/lib/lxc/u1/rootfs / rw,relatime - ext4 /dev/disk/by-uuid/7fd0b676-46f3-46d6-94a6-5c620e72e7b0 rw,data=ordered
> > > 103 75 0:35 / /proc rw,nosuid,nodev,noexec,relatime - proc proc rw
> > > 104 75 0:33 / /sys rw,relatime - sysfs sysfs rw
> > > 105 104 0:18 / /sys/fs/fuse/connections rw,relatime - fusectl none rw
> > > 106 104 0:6 / /sys/kernel/debug rw,relatime - debugfs none rw
> > > 107 104 0:10 / /sys/kernel/security rw,relatime - securityfs none rw
> > > 108 104 0:22 / /sys/fs/pstore rw,relatime - pstore none rw
> > > 109 75 0:12 /7 /dev/lxc/console rw,nosuid,noexec,relatime - devpts devpts rw,gid=5,mode=620,ptmxmode=000
> > > 110 75 0:12 /1 /dev/lxc/tty1 rw,nosuid,noexec,relatime - devpts devpts rw,gid=5,mode=620,ptmxmode=000
> > > 111 75 0:12 /3 /dev/lxc/tty2 rw,nosuid,noexec,relatime - devpts devpts rw,gid=5,mode=620,ptmxmode=000
> > > 112 75 0:12 /5 /dev/lxc/tty3 rw,nosuid,noexec,relatime - devpts devpts rw,gid=5,mode=620,ptmxmode=000
> > > 113 75 0:12 /6 /dev/lxc/tty4 rw,nosuid,noexec,relatime - devpts devpts rw,gid=5,mode=620,ptmxmode=000
> > > 77 75 0:36 / /dev/pts rw,relatime - devpts devpts rw,gid=5,mode=620,ptmxmode=666
> > > 78 75 0:36 /ptmx /dev/ptmx rw,relatime - devpts devpts rw,gid=5,mode=620,ptmxmode=666
> > > 79 104 0:37 / /sys/fs/cgroup rw,relatime - tmpfs none rw,size=4k,mode=755
> > > 80 75 0:38 / /run rw,nosuid,noexec,relatime - tmpfs none rw,size=200060k,mode=755
> > > 81 80 0:39 / /run/lock rw,nosuid,nodev,noexec,relatime - tmpfs none rw,size=5120k
> > > 82 80 0:40 / /run/shm rw,nosuid,nodev,relatime - tmpfs none rw
> > > 83 80 0:41 / /run/user rw,nosuid,nodev,noexec,relatime - tmpfs none rw,size=102400k,mode=755
> > >
Looks like this are external moutns which might require --ext-mount-map keys, how this mountinfo
represented on pid 1 of container (ie do "ps afx" on node and the choose pid for init of container,
then show /proc/$pid/mountinfo)
> >
> > Mind to share your lxc conatiner details and how you set it up?
>
> Sure, the config I used is:
>
> # Template used to create this container:
> # /usr/local/share/lxc/templates/lxc-download
> # Parameters passed to the template: -d ubuntu -r trusty -a amd64
> # For additional config options, please look at lxc.container.conf(5)
>
> # Distribution configuration
> lxc.include = /usr/local/share/lxc/config/ubuntu.common.conf
> lxc.arch = x86_64
>
> # Container specific configuration
> lxc.rootfs = /usr/local/var/lib/lxc/u1/rootfs
> lxc.utsname = u1
>
> # Network configuration
> lxc.network.type = veth
> lxc.network.flags = up
> lxc.network.link = lxcbr0
> lxc.network.hwaddr = 00:16:3e:0e:08:34
>
> # hax for criu
> #lxc.console = none
> #lxc.tty = 0
> lxc.cgroup.devices.deny = c 5:1 rwm
>
> And I created it with the command line:
>
> sudo lxc-create -t download -n u1 -- -d ubuntu -r trusty -a amd64
>
> > > When I try to work around that with something like,
> > >
> > > diff --git a/mount.c b/mount.c
> > > index 8c60693..a28e041 100644
> > > --- a/mount.c
> > > +++ b/mount.c
> > > @@ -433,8 +433,19 @@ static int validate_mounts(struct mount_info *info, bool for_dump)
> > > ret = run_plugins(DUMP_EXT_MOUNT, m->mountpoint, m->mnt_id);
> > > if (ret == 0)
> > > m->need_plugin = true;
> > > - else if (ret == -ENOTSUP)
> > > - ret = try_resolve_ext_mount(m);
> > > + else if (ret == -ENOTSUP) {
> > > + /* inside containers
> > > + * /dev/console and /dev/tty*
> > > + * are mounted from paths like
> > > + * /5 or /6, so we ignore
> > > + * their path here if things
> > > + * go bad.
> > > + */
> > > + if (m->fstype->code == FSTYPE__DEVPTS)
> > > + ret = 0;
> > > + else
> > > + ret = try_resolve_ext_mount(m);
> > > + }
> > > } else {
> > > if (m->need_plugin || m->external)
> > > /*
> > >
> > >
> > > I hit:
> > >
> > > (00.247031) Error (tty.c:233): tty: Unexpected format on path /dev/lxc/tty4
> > >
> > > which looks like it might be a bigger problem :)
Aha! We're expecting ptys to have a canonical forms, ie /dev/pts/N, where N is
pty number, need to thhink how to workaround it (you can fix up criu by now
changing parse_index() helper to parse not PTS_FMT but say "/dev/lxc/tty%d"
instead).
> > >
> > > Any thoughts are appreciated.
> >
> > Show me the stat output on this paths please? (stat /dev/lxc/tty4)
>
> ubuntu at u1:~$ stat /dev/lxc/tty4
> File: ‘/dev/lxc/tty4’
> Size: 0 Blocks: 0 IO Block: 1024 character special file
> Device: ch/12d Inode: 10 Links: 1 Device type: 88,7
> Access: (0660/crw-rw----) Uid: ( 0/ root) Gid: ( 5/ tty)
> Access: 2014-09-30 13:28:09.359281000 +0000
> Modify: 2014-09-30 13:28:09.359281000 +0000
> Change: 2014-09-30 13:28:13.239281000 +0000
> Birth: -
Thanks, this is indeed unix98 pty device, so need to fix this
problem somehow...
More information about the CRIU
mailing list