[CRIU] [RFC][PATCH 0/3] Proposal for native (w/o plugins) support for external bind mounts

[Saied Kazemi]

Yes to both questions. 1. The information retrieved from mountinfo during
dump does not include docker daemon's root path (/var/lib/docker), so we
have to prepend it during restore when we recreate the external bind
mounts. 2. My version of CRIU includes: your original two mnt patches to
mount.c, your --ext-mount patches, and my changes to try_ext_bind_mount()
in mount.c.

--Saied



On Wed, Jun 4, 2014 at 4:25 AM, Pavel Emelyanov <xemul at parallels.com> wrote:

> On 06/04/2014 05:07 AM, Saied Kazemi wrote:
> > Pavel,
> >
> > Good news.  Making the following change to try_ext_bind_mount() after
> applying your --ext-mount patches and using
> > -M on dump and -M/var/lib/docker on restore, it's possible to resume a
> Docker container process _without_ any plugins.
>
> Wow! IOW you need an ability to prepend some string to the mount points
> root path, right?
> And, other than this, you must have applied other patches from me to
> mount.c, am I right
> with that too?
>
> >  Please note that as mentioned before, the restore is successful as far
> as CRIU is concerned and is not visible to
> > the Docker daemon.
> >
> > --Saied
> >
> > diff --git a/mount.c b/mount.c
> > index eca3fa5..0f4a807 100644
> > --- a/mount.c
> > +++ b/mount.c
> > @@ -1174,22 +1174,32 @@ static int do_new_mount(struct mount_info *mi)
> >
> >  static int try_ext_bind_mount(struct mount_info *mi)
> >  {
> > -       if (opts.ext_mount_mode == EXT_MOUNT_NONE) {
> > -              pr_err("Restoring external bind mount %s is not
> enabled\n",
> > -                              mi->mountpoint);
> > -              return -1;
> > -       }
> > +       char buf[256];
> > +       char *source;
> >
> > -       if (opts.ext_mount_mode != EXT_MOUNT_OK)
> > -              return -1;
> > +       if (opts.ext_mount_mode == EXT_MOUNT_NONE) {
> > +               pr_err("Restoring external bind mount %s is not
> enabled\n",
> > +                       mi->mountpoint);
> > +               return -1;
> > +       }
> >
> > -       pr_info("Bind-mounting %s to %s\n", mi->root, mi->mountpoint);
> > -       if (mount(mi->root, mi->mountpoint, NULL, MS_BIND, NULL)) {
> > -              pr_perror("Can't bind ext mount");
> > -              return -1;
> > -       }
> > +       if (opts.ext_mount_mode == EXT_MOUNT_PREP) {
> > +               snprintf(buf, 256, "%s/%s", opts.em_arg_1, mi->root);
> > +               source = buf;
> > +       } else
> > +               source = mi->root;
> > +
> > +       if (opts.ext_mount_mode != EXT_MOUNT_OK &&
> > +           opts.ext_mount_mode != EXT_MOUNT_PREP)
> > +               return -1;
> >
> > -       return 0;
> > +       pr_info("Bind-mounting %s to %s\n", source, mi->mountpoint);
> > +       if (mount(source, mi->mountpoint, NULL, MS_BIND, NULL)) {
> > +               pr_perror("Can't bind ext mount");
> > +               return -1;
> > +       }
> > +
> > +       return 0;
> >  }
> >
> >
> >
> > On Tue, Jun 3, 2014 at 1:44 PM, Saied Kazemi <saied at google.com <mailto:
> saied at google.com>> wrote:
> >
> >     Hi Pavel,
> >
> >     It'd make using CRIU much easier :)  For now, I've been using a
> Docker plugin in my experiments to dump and restore a process inside a
> Docker container:
> >
> >     1. On the dump side, I am passing m->root to the Docker plugin (in
> addition to m->mountpoint and m->mnt_id) so that the path can be recorded
> in the img file.
> >
> >             ret = cr_plugin_dump_ext_mount(m->root, m->mountpoint + 1,
> m->mnt_id);
> >
> >     2. On the restore side, I pass opts.root from --root on the command
> line instead of always passing "/" (this was probably your original
> intention).  Then the Docker plugin prepends /var/lib/docker to the path
> that was recorded in the img file during dump.
> >
> >             ret = cr_plugin_restore_ext_mount(mi->mnt_id,
> mi->mountpoint, opts.root ? : "/", NULL);
> >
> >     Like the plugin in the run.sh example, the Docker plugin is using an
> environment variable (CR_EXT_BIND_MOUNTS) to identify its files.  It'd be
> great to get rid of this environment variable.
> >
> >     Finally, I have to manually set up the cgroups (e.g.,
> /sys/fs/cgroup/cpu/docker/<ID>) before restoring:
> >
> >             criu restore -o restore.log --root
> /var/lib/docker/vfs/dir/<ID> -D img -v4 -j -n mnt -n pid --lib
> /path/to/plugin
> >
> >     The end result is that the process is successfully restored and
> resumes operation.  Currently, however, the Docker daemon is totally
> unaware of the process being resurrected and docker ps won't show it.  We
> need supporting code in Docker.
> >
> >     --Saied
> >
> >
> >
> >
> >
> >
> >     On Tue, Jun 3, 2014 at 9:26 AM, Pavel Emelyanov <xemul at parallels.com
> <mailto:xemul at parallels.com>> wrote:
> >
> >         Hi,
> >
> >         I'd like to propose an API to support external mount points w/o
> plugins.
> >         It's a typical scenario when people just do a bind mount before
> going
> >         into chroot/pivot_root and starting a container. Such a
> construction is
> >         not dumpable by CRIU, but I think it's possibly to handle the
> common and
> >         typical scenario.
> >
> >         That said, the proposal is to introduce the --ext-mount option
> with the
> >         syntax:
> >
> >         --ext-mount without arguments on dump says, that mount point
> with the
> >         source sitting outside of the namespace root should be dumped
> "as is".
> >         On restore this will say to try create bind mount with unchanged
> source
> >         path. It's useful for suspend/resume scenario and for
> live-migration
> >         between hosts with identical FS paths.
> >
> >         --ext-mount p<path>, e.g. --ext-mount p/foo on restore will
> cause the
> >         <path> be prepended to any bind mount source. IOW /bar bind
> mounted into
> >         /root will result in /foo/bar get bind mounted into /root on
> restore
> >
> >         --ext-mount s<delim><path1><delim><path2>, e.g. --ext-mount
> s:/foo:/bar
> >         will case the <path1> prefix get substituted with <path2> prefix
> for
> >         every external bind mount on restore. IOW /foo/foobar bind
> mounted into
> >         /root will result in /bar/foobar bind mounted to /root on
> restore.
> >
> >         The patchset is not complete, it just demonstrates the
> intention. Saied,
> >         it seems to handle your scripted case, the external file gets
> mounted into
> >         proper place with plain --ext-mount option, but there's still
> unresolved
> >         issue with proc :(
> >
> >         Comments are welcome.
> >
> >         TODO
> >
> >         1. check that flags (ro, shared) are dumped and restore correctly
> >         2. implement p and s modes for --ext-mount option on restore
> >         3. support ext-mount option in RPC API
> >         4. add option help text and documentation
> >
> >
> >
>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openvz.org/pipermail/criu/attachments/20140604/51a82964/attachment-0001.html>