<div dir="ltr">Yes to both questions. 1. The information retrieved from mountinfo during dump does not include docker daemon's root path (/var/lib/docker), so we have to prepend it during restore when we recreate the external bind mounts. 2. My version of CRIU includes: your original two mnt patches to mount.c, your --ext-mount patches, and my changes to try_ext_bind_mount() in mount.c.<div>
<br></div><div>--Saied<div><br></div></div></div><div class="gmail_extra"><br><br><div class="gmail_quote">On Wed, Jun 4, 2014 at 4:25 AM, Pavel Emelyanov <span dir="ltr"><<a href="mailto:xemul@parallels.com" target="_blank">xemul@parallels.com</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div class="">On 06/04/2014 05:07 AM, Saied Kazemi wrote:<br>
> Pavel,<br>
><br>
> Good news. Making the following change to try_ext_bind_mount() after applying your --ext-mount patches and using<br>
> -M on dump and -M/var/lib/docker on restore, it's possible to resume a Docker container process _without_ any plugins.<br>
<br>
</div>Wow! IOW you need an ability to prepend some string to the mount points root path, right?<br>
And, other than this, you must have applied other patches from me to mount.c, am I right<br>
with that too?<br>
<div class="HOEnZb"><div class="h5"><br>
> Please note that as mentioned before, the restore is successful as far as CRIU is concerned and is not visible to<br>
> the Docker daemon.<br>
><br>
> --Saied<br>
><br>
> diff --git a/mount.c b/mount.c<br>
> index eca3fa5..0f4a807 100644<br>
> --- a/mount.c<br>
> +++ b/mount.c<br>
> @@ -1174,22 +1174,32 @@ static int do_new_mount(struct mount_info *mi)<br>
><br>
> static int try_ext_bind_mount(struct mount_info *mi)<br>
> {<br>
> - if (opts.ext_mount_mode == EXT_MOUNT_NONE) {<br>
> - pr_err("Restoring external bind mount %s is not enabled\n",<br>
> - mi->mountpoint);<br>
> - return -1;<br>
> - }<br>
> + char buf[256];<br>
> + char *source;<br>
><br>
> - if (opts.ext_mount_mode != EXT_MOUNT_OK)<br>
> - return -1;<br>
> + if (opts.ext_mount_mode == EXT_MOUNT_NONE) {<br>
> + pr_err("Restoring external bind mount %s is not enabled\n",<br>
> + mi->mountpoint);<br>
> + return -1;<br>
> + }<br>
><br>
> - pr_info("Bind-mounting %s to %s\n", mi->root, mi->mountpoint);<br>
> - if (mount(mi->root, mi->mountpoint, NULL, MS_BIND, NULL)) {<br>
> - pr_perror("Can't bind ext mount");<br>
> - return -1;<br>
> - }<br>
> + if (opts.ext_mount_mode == EXT_MOUNT_PREP) {<br>
> + snprintf(buf, 256, "%s/%s", opts.em_arg_1, mi->root);<br>
> + source = buf;<br>
> + } else<br>
> + source = mi->root;<br>
> +<br>
> + if (opts.ext_mount_mode != EXT_MOUNT_OK &&<br>
> + opts.ext_mount_mode != EXT_MOUNT_PREP)<br>
> + return -1;<br>
><br>
> - return 0;<br>
> + pr_info("Bind-mounting %s to %s\n", source, mi->mountpoint);<br>
> + if (mount(source, mi->mountpoint, NULL, MS_BIND, NULL)) {<br>
> + pr_perror("Can't bind ext mount");<br>
> + return -1;<br>
> + }<br>
> +<br>
> + return 0;<br>
> }<br>
><br>
><br>
><br>
</div></div><div class="im HOEnZb">> On Tue, Jun 3, 2014 at 1:44 PM, Saied Kazemi <<a href="mailto:saied@google.com">saied@google.com</a> <mailto:<a href="mailto:saied@google.com">saied@google.com</a>>> wrote:<br>
><br>
> Hi Pavel,<br>
><br>
> It'd make using CRIU much easier :) For now, I've been using a Docker plugin in my experiments to dump and restore a process inside a Docker container:<br>
><br>
> 1. On the dump side, I am passing m->root to the Docker plugin (in addition to m->mountpoint and m->mnt_id) so that the path can be recorded in the img file.<br>
><br>
> ret = cr_plugin_dump_ext_mount(m->root, m->mountpoint + 1, m->mnt_id);<br>
><br>
> 2. On the restore side, I pass opts.root from --root on the command line instead of always passing "/" (this was probably your original intention). Then the Docker plugin prepends /var/lib/docker to the path that was recorded in the img file during dump.<br>
><br>
> ret = cr_plugin_restore_ext_mount(mi->mnt_id, mi->mountpoint, opts.root ? : "/", NULL);<br>
><br>
> Like the plugin in the run.sh example, the Docker plugin is using an environment variable (CR_EXT_BIND_MOUNTS) to identify its files. It'd be great to get rid of this environment variable.<br>
><br>
> Finally, I have to manually set up the cgroups (e.g., /sys/fs/cgroup/cpu/docker/<ID>) before restoring:<br>
><br>
> criu restore -o restore.log --root /var/lib/docker/vfs/dir/<ID> -D img -v4 -j -n mnt -n pid --lib /path/to/plugin<br>
><br>
> The end result is that the process is successfully restored and resumes operation. Currently, however, the Docker daemon is totally unaware of the process being resurrected and docker ps won't show it. We need supporting code in Docker.<br>
><br>
> --Saied<br>
><br>
><br>
><br>
><br>
><br>
><br>
</div><div class="HOEnZb"><div class="h5">> On Tue, Jun 3, 2014 at 9:26 AM, Pavel Emelyanov <<a href="mailto:xemul@parallels.com">xemul@parallels.com</a> <mailto:<a href="mailto:xemul@parallels.com">xemul@parallels.com</a>>> wrote:<br>
><br>
> Hi,<br>
><br>
> I'd like to propose an API to support external mount points w/o plugins.<br>
> It's a typical scenario when people just do a bind mount before going<br>
> into chroot/pivot_root and starting a container. Such a construction is<br>
> not dumpable by CRIU, but I think it's possibly to handle the common and<br>
> typical scenario.<br>
><br>
> That said, the proposal is to introduce the --ext-mount option with the<br>
> syntax:<br>
><br>
> --ext-mount without arguments on dump says, that mount point with the<br>
> source sitting outside of the namespace root should be dumped "as is".<br>
> On restore this will say to try create bind mount with unchanged source<br>
> path. It's useful for suspend/resume scenario and for live-migration<br>
> between hosts with identical FS paths.<br>
><br>
> --ext-mount p<path>, e.g. --ext-mount p/foo on restore will cause the<br>
> <path> be prepended to any bind mount source. IOW /bar bind mounted into<br>
> /root will result in /foo/bar get bind mounted into /root on restore<br>
><br>
> --ext-mount s<delim><path1><delim><path2>, e.g. --ext-mount s:/foo:/bar<br>
> will case the <path1> prefix get substituted with <path2> prefix for<br>
> every external bind mount on restore. IOW /foo/foobar bind mounted into<br>
> /root will result in /bar/foobar bind mounted to /root on restore.<br>
><br>
> The patchset is not complete, it just demonstrates the intention. Saied,<br>
> it seems to handle your scripted case, the external file gets mounted into<br>
> proper place with plain --ext-mount option, but there's still unresolved<br>
> issue with proc :(<br>
><br>
> Comments are welcome.<br>
><br>
> TODO<br>
><br>
> 1. check that flags (ro, shared) are dumped and restore correctly<br>
> 2. implement p and s modes for --ext-mount option on restore<br>
> 3. support ext-mount option in RPC API<br>
> 4. add option help text and documentation<br>
><br>
><br>
><br>
<br>
<br>
</div></div></blockquote></div><br></div>