[CRIU] system configuration for dumping/restoring LXC

Thu Sep 11 10:50:45 PDT 2014

The bottom line is that I can dump but I can't restore.  I'm still confused how rootfs should be used.  I still would like to know what your system configuration was in the example described on CRIU/LXC.  I'm using 64-bit Ubuntu desktop 14.04 but I'm willing to switch to whatever worked for you.  Same goes for the LXC tools.

Krystof

The following are some of the steps I performed to get to that point:
1. apt-get install lxc installs bridge and changes interfaces file (there are no ifcfg-* files in Ubuntu) it also installs lxcbr0 instead of br0.  
2. lxc-checkconfig shows everything is enabled so mounting cgroup is not required
3. lxc-create -t ubuntu -n cn_01 and lxc-start -n cn_01 work
4. I downloaded centos-6-x86_64.tar.gz and extracted it to /root/test-lxc
5. I copied the test-lxc.conf and fstab from the website.
6. lxc-create -n test-lxc -f test-lxc.conf fails; there is no specific error but I'm assuming it is because of the wrong rootfs = /root/test-lxc-root
7. When I created /root/test-lxc-root I was able to create the container but it wouldn't start
8. When I changed rootfs to /root/test-lxc, I was able to create the container and to start it
9. lxc-stop took a long time to stop the container
10. Since I didn't have to execute mount --bind test-lxc test-lxc-root/ to run the container I thought that maybe that step should be executed with original configuration but before the container is created so I destroyed the container
11. I restored test-lxc.conf to what is on the website, restored test-lxc folder, created test-lxc-root and executed mount --bind test-lxc test-lxc-root/ 
12. I created the container again and it worked this time
13. I started the container and it also worked
14. However, this time, after lxc-attach -n test-lxc, mount produced this output:
/dev/disk/by-uuid/a1b2c830-cce3-4373-a598-9702ef6d8f69 on / type ext4 (rw,relatime,errors=remount-ro,data=ordered)
none on /dev/pts type devpts (rw,relatime,mode=600,ptmxmode=000)
none on /proc type proc (rw,relatime)
none on /sys type sysfs (rw,relatime)
none
 on /dev/shm type tmpfs (rw,relatime)
none on /dev type tmpfs (rw,relatime,mode=755)
none on /proc/sys/fs/binfmt_misc type binfmt_misc (rw,relatime)

While originally it only produced this:
/dev/disk/by-uuid/a1b2c830-cce3-4373-a598-9702ef6d8f69 on / type ext4 (rw,relatime,errors=remount-ro,data=ordered)
none on /proc type proc (rw,nosuid,nodev,noexec,relatime)
none on /sys type sysfs (rw,nosuid,nodev,noexec,relatime)
none on /dev type tmpfs (rw,relatime,mode=755)
none on /proc/sys/fs/binfmt_misc type binfmt_misc (rw,relatime)

15. lxc-stop still takes long time

16. Installing criu-1.3
17. apt-get
install libprotobuf-dev libprotoc-dev protobuf-c-compiler libprotobuf-c0
libprotobuf-c0-dev
18. make
19. cd test/app-emu/lxc
20. /run.sh test-lxc produces an error:
Name:           test-lxc
State:          RUNNING
PID:            25189
CPU use:        0.64 seconds
BlkIO use:      56.00 KiB
Memory use:     8.33 MiB
KMem use:       0 bytes
Link:           vethJ8MR0R
 TX bytes:      648 bytes
 RX bytes:      4.18 KiB
 Total bytes:   4.81 KiB
The CT's "init" process has PID of 
kill: usage: kill [-s sigspec | -n signum | -sigspec] pid | jobspec ... or kill -l [sigspec]
root at se:~/criu-1.3/test/app-emu/lxc# 

21. I changed pid to PID in  pid=$(lxc-info --name $name | awk '/PID:/ { print $2 }')
22. This time I got: bridge br0 does not exist!
23. I edited network-script.sh accordingly
24.  Dump doesn't work.  See attached dump1.log.
25. So I went back to the way I originally created the container.  I changed rootfs in test-lxc.conf to /root/test-lxc
26. I created the container and started it and I was able to dump it; see attached dump2.log
27 However, the restore hangs.  ps a shows these processes but there is no init; see attached restore.log

  413 pts/1    S+     0:00 /bin/bash ./run.sh test-lxc
  498 pts/1    S+     0:00 /root/criu-1.3/criu restore --evasive-devices --tcp-e
  499 pts/1    t+     0:00 /root/criu-1.3/criu restore --evasive-devices --tcp-e

 From: Krystof Zmudzinski <krystof_zmudzinski at yahoo.com>
To: Andrey Wagin <avagin at gmail.com> 
Cc: "criu at openvz.org" <criu at openvz.org> 
Sent: Thursday, September 11, 2014 7:57 AM
Subject: Re: system configuration for dumping/restoring LXC

Thanks for the fast reply.  I will reinstall Ubuntu 14.04, LXC tools and criu-1.3 as before.  I will let you know what I had to do along the way when I encounter problems.  Maybe at the end you will be able to post this somewhere on your website for somebody like me who is not a Linux expert. :-)

Speaking of not being a Linux expert, I'm not sure if I understand the point about the rootfs mounts.  I do recall that one of the error messages on restore was something about same rootfs can't be used so I can see how it makes sense what you said about different rootfs there.  But I don't understand how the configuration on the website can work when lxc-create is executed.

Krystof

 From: Andrey Wagin <avagin at gmail.com>
To: Krystof Zmudzinski <krystof_zmudzinski at yahoo.com> 
Cc: "criu at openvz.org" <criu at openvz.org> 
Sent: Wednesday, September 10, 2014 11:59 PM
Subject: Re: system configuration for dumping/restoring LXC

Hi Krystof,

I added criu at openvz.org in CC

2014-09-11 5:03 GMT+04:00 Krystof Zmudzinski <krystof_zmudzinski at yahoo.com>:

Andrew,
>
>
>I've spent some time trying to build a system, which would allow me to dump and restore a Linux container using criu.  I started by reading instructions on this page LXC but the information seems buggy.  For example, test-lxc.conf has these 2 seemingly incompatible lines in it:
>lxc.mount = /root/test-lxc/etc/fstab
>lxc.rootfs = /root/test-lxc-root/
>
>
>So is the rootfs in /root/test-lxc or /root/test-lxc-root>

It can't be buggy, because I have copied this config from my test node. I think you can use  /root/test-lxc and don't bind-mount /root/test-lxc to /root/test-lxc-root, but you will need to do this bind-mount to restore the CT.

>
>When I tried executing lxc-create -n test-lxc -f test-lxc.conf with the configuration as presented on that page, lxc-create failed.  When I changed rootfs to /root/test-lxc everywhere, it worked.  I was even able to dump the container using the criu-1.3 but restore never worked.  That was on Ubuntu 14.04.  When I switched to Fedora 20, even dump didn't work.

If you are able to dump the CT, you must able to restore it. If you can't to do that, it's bug and we are ready to fix it. So could you show how you execute criu and logs from "criu dump" and "criu restore"?

>
>I don't really care what system I have to use and how simple my container is.  I can tell that there are other people on the net who talk about successfully dumping and restoring containers but I can't figure out what system/container configurations they use.
>
>
>Could you provide me with a pointer to the host configuration and container configuration -- even if I have to go some older versions -- that allow for a successful dump/restore of a Linux container?
>
>
>BTW, I have been able to use criu-1.3 to dump/restore single processes.
>
>
>Thanks in adavance,
>
>
>Krystof
>
>
>
>
>
> 
>
> 
> 
> 
> 
> 
>LXC
>This article describes how to perform checkpoint-restore for an LXC container.  
>View on criu.org Preview by Yahoo  
>
> 
> 
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openvz.org/pipermail/criu/attachments/20140911/81bd915b/attachment-0001.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: dump1.log
Type: application/octet-stream
Size: 861218 bytes
Desc: not available
URL: <http://lists.openvz.org/pipermail/criu/attachments/20140911/81bd915b/attachment-0003.obj>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: restore.log
Type: application/octet-stream
Size: 2787 bytes
Desc: not available
URL: <http://lists.openvz.org/pipermail/criu/attachments/20140911/81bd915b/attachment-0004.obj>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: dump2.log
Type: application/octet-stream
Size: 872134 bytes
Desc: not available
URL: <http://lists.openvz.org/pipermail/criu/attachments/20140911/81bd915b/attachment-0005.obj>