[CRIU] Errors on dump with external mountpoints
Saied Kazemi
saied at google.com
Thu Jun 12 21:46:45 PDT 2014
Bing,
Attached please find instructions on how to dump and restore a process
running in a Docker container. Please note that:
1. I am using Docker version 0.10.0. More recent versions create
/proc/kcore as tmpfs in a container and bind mount it to /dev/null. CRIU
currently assumes that all tmpfs bind mounts are directories so fails to
handle /proc/kcore and exits when dumping.
2. While the process is successfully restored by CRIU, the Docker daemon
currently doesn't have support to integrate the container back into the
list of the active containers it manages.
3. The instructions provided are just for experimentation purposes only
(i.e., as-is with no support).
Hope this helps.
--Saied
On Thu, Jun 12, 2014 at 8:19 PM, Bing X <xiebingbing at gmail.com> wrote:
> So, currently I couldn't use criu to dump docker container?
>
>
> On Wed, Jun 11, 2014 at 4:48 PM, Cyrill Gorcunov <gorcunov at gmail.com>
> wrote:
>
>> On Wed, Jun 11, 2014 at 01:39:51PM -0400, Bing X wrote:
>> > I am using criu to dump my docker container. I'm working the latest
>> > version of criu, which is directly pulled from criu repository; and
>> using
>> > the patch /test/mounts/ext/, in which I changed to rebind my
>> container
>> > root to test_dir, etc. as described in run.sh, run_ns.sh and
>> run_wait.sh.
>> > Please find the error from the attached dump.log
>>
>> From the view of tty code -- the pts point hasn't granted the path
>> /dev/pts/N
>> (where N is the pty number). That said I guess there is a problem in mount
>> points format. Not sure yet how to resolve...
>>
>
>
>
> --
> Thanks
> Bing
>
> _______________________________________________
> CRIU mailing list
> CRIU at openvz.org
> https://lists.openvz.org/mailman/listinfo/criu
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openvz.org/pipermail/criu/attachments/20140612/72efbe62/attachment.html>
-------------- next part --------------
To dump and restore a process running inside a Docker container, you
need the latest CRIU sources along with three patches that Pavel posted
to the mailing lists (Native (w/o plugins) support for external bind
mounts (v3)) and identified in the following commit message from my
local working CRIU directory:
$ git log -n 3
commit 79b2e17fca0a60e94fee32a43114a07a837c70e4
Author: Saied Kazemi <saied at google.com>
Date: Thu Jun 12 13:40:39 2014 -0700
Native (w/o plugins) support for external bind mounts (v3)
This commit incudes the following 3 patches from Pavel sent out on
June 9th.
mnt: Tossing bits around in validate_mounts
crtools: Introduce the --ext-mount-map option (v3)
mnt: Handle external bind mounts according to --ext-mount option (v3)
commit 43c96be7980385a4f1a795291a75498c5ff37297
Author: Tycho Andersen <tycho.andersen at canonical.com>
Date: Tue Jun 10 18:13:00 2014 +0400
Allow dumping of pstore, securityfs, fusectl, debugfs
These are mounted by default in ubuntu containers, so criu should know about
them and remount them on restore.
Signed-off-by: Tycho Andersen <tycho.andersen at canonical.com>
Signed-off-by: Pavel Emelyanov <xemul at parallels.com>
commit 72a9372aff2e4679a58047d5b481efbc2b74736f
Author: Pavel Emelyanov <xemul at parallels.com>
Date: Mon Jun 9 16:51:16 2014 +0400
fs: Opening FE-s after fchdir doesn't work
It uses absolute file names, so any open-s should happen _before_
we change tasks' root.
Signed-off-by: Pavel Emelyanov <xemul at parallels.com>
Acked-by: Andrew Vagin <avagin at parallels.com>
Then you need the following patch for mount.c:
$ git diff
diff --git a/mount.c b/mount.c
index b0838ab..5ac5a6d 100644
--- a/mount.c
+++ b/mount.c
@@ -1657,9 +1657,14 @@ static int collect_mnt_from_image(struct mount_info **pms, struct ns_id *nsid)
pr_err("No mapping for %s mountpoint\n", me->mountpoint);
goto err;
}
+ if (strcmp(me->mountpoint, em->val)) {
+ pr_err("Key %s matches root but val %s does not match mountpoint %s\n",
+ me->root, em->val, me->mountpoint);
+ goto err;
+ }
pm->external = em;
- pm->root = em->val;
+ pm->root = em->key;
pr_debug("Mountpoint %s will have root from %s\n",
me->mountpoint, pm->root);
Once you have built and installed CRIU , proceed with the following
instructions as an example to dump and restore a process running in a
Docker container.
TERMINAL A
A.1. Make sure Docker daemon is up and running:
$ ps -efl | grep docker
4 S root 666 1 0 80 0 - 51957 futex_ 13:50 ? 00:00:00 /usr/bin/docker -d
$
A.2. Start a Docker container:
$ sudo docker run -i busybox:latest /bin/sh -c 'i=0; while true; do echo $i >> /foo; i=`expr $i + 1`; sleep 3; done'
TERMINAL B
B.1. Verify /bin/sh inside the container is running:
# docker ps -q --no-trunc
b361c2a156953e518bf7c3443cc91b8ccb0a9d42e3b7143ca613290d157b648a
# ID=b361c2a156953e518bf7c3443cc91b8ccb0a9d42e3b7143ca613290d157b648a
# tail -f
/var/lib/docker/vfs/dir/$ID/foo
12
13
14
...
TERMINAL C
C.1. Before dumping, bind mount container's root (required by CRIU):
# R=/var/lib/docker/vfs/dir/$ID
# mount --rbind $R $R
#
C.2. Find PID of /bin/sh inside the container:
# ps -efl | grep /bin/sh
4 S root 1412 1319 0 80 0 - 16182 poll_s 13:51 pts/0 00:00:00 sudo docker run -i busybox:latest /bin/sh -c i=0; while true; do echo $i >> /foo; i=`expr $i + 1`; sleep 3; done
4 S root 1413 1412 0 80 0 - 66349 futex_ 13:51 pts/0 00:00:00 docker run -i busybox:latest /bin/sh -c i=0; while true; do echo $i >> /foo; i=`expr $i + 1`; sleep 3; done
4 S root 1426 666 0 80 0 - 791 wait 13:51 ? 00:00:00 /bin/sh -c i=0; while true; do echo $i >> /foo; i=`expr $i + 1`; sleep 3; done
0 S root 2116 2046 0 80 0 - 3412 pipe_w 13:56 pts/4 00:00:00 grep --color=auto /bin/sh
C.3. Dump /bin/sh:
# criu dump -D img -o dump.log -v4 -j -n mnt -n pid -t 1426 \
--ext-mount-map /.dockerinit:/var/lib/docker/init/dockerinit-0.10.0-dev \
--ext-mount-map /.dockerenv:/var/lib/docker/containers/$ID/config.env \
--ext-mount-map /etc/resolv.conf:/var/lib/docker/containers/$ID/resolv.conf \
--ext-mount-map /etc/hostname:/var/lib/docker/containers/$ID/hostname \
--ext-mount-map /etc/hosts:/var/lib/docker/containers/$ID/hosts
# echo $?
0
#
C.4. Before restoring, set up container's control groups:
# mkdir -p /sys/fs/cgroup/blkio/docker/$ID
# mkdir -p /sys/fs/cgroup/cpu/docker/$ID
# mkdir -p /sys/fs/cgroup/cpuacct/docker/$ID
# mkdir -p /sys/fs/cgroup/devices/docker/$ID
# mkdir -p /sys/fs/cgroup/freezer/docker/$ID
# mkdir -p /sys/fs/cgroup/memory/docker/$ID
# mkdir -p /sys/fs/cgroup/perf_event/docker/$ID
#
C.5. Restore /bin/sh inside the container:
# criu restore -D img -o restore.log -v4 -j -n mnt -n pid --root /var/lib/docker/vfs/dir/$ID -d \
--ext-mount-map /var/lib/docker/init/dockerinit-0.10.0-dev:/.dockerinit \
--ext-mount-map /var/lib/docker/containers/$ID/config.env:/.dockerenv \
--ext-mount-map /var/lib/docker/containers/$ID/resolv.conf:/etc/resolv.conf \
--ext-mount-map /var/lib/docker/containers/$ID/hostname:/etc/hostname \
--ext-mount-map /var/lib/docker/containers/$ID/hosts:/etc/hosts
# echo $?
0
#
TERMINAL B
B.2. Verify /bin/sh inside the container is running again:
...
22
23
24
...
More information about the CRIU
mailing list