[CRIU] LXC checkpoint/restore HOWTO using upstream tools

[Zmudzinski, Krystof C]

Tycho,

After following the instructions on http://criu.org/LXC, I wanted to install the latest source for LXC and make some changes.  How can I build and install the new lxc-* so I overwrite what sudo apt-get install lxc did?  Right now, following the instructions on http://bazaar.launchpad.net/~tycho-s/+junk/snapshot-instructions/view/head:/README everything gets installed in different directories and things stop working completely.  

I always get this error when I try to start a container:
 >       lxc-start 1410804731.572 ERROR    lxc_cgfs - Could not find writable mount point for cgroup hierarchy 3 while trying to create cgroup.
>       lxc-start 1410804731.572 ERROR    lxc_start - failed creating cgroups
>       lxc-start 1410804731.599 ERROR    lxc_start - failed to spawn 'u1'
>       lxc-start 1410804736.604 ERROR    lxc_start_ui - The container failed to start.

Krystof

-----Original Message-----
From: Tycho Andersen [mailto:tycho.andersen at canonical.com] 
Sent: Friday, September 19, 2014 10:53 AM
To: Zmudzinski, Krystof C
Cc: CRIU
Subject: Re: [CRIU] LXC checkpoint/restore HOWTO using upstream tools

Hi Krystof,

On Fri, Sep 19, 2014 at 05:18:32PM +0000, Zmudzinski, Krystof C wrote:
> lxc-checkpoint fails.  I did a fresh install of ubuntu 14.04 and followed your instructions.  I also installed criu-1.3.1.  
> 
> From dump.log:
> (00.377632) Error (mount.c:805): fusectl isn't empty: 8388625

Ah, that is a good point. Right now CRIU doesn't support dumping any fuse filesystems (i.e., /sys/fs/fuse/connections needs to be empty). I guess stock desktop ubuntu might have some of these enabled. You can either uninstall any fuse modules or try ubuntu-server (or better yet, the cloud images) as a host.

I guess maybe we should have lxc-checkpoint look for some of this stuff too, vs. just config.

> The container does contain ttys and console:
> 5784 ?        Ss     0:00              \_ lxc-start -n cn_01
>  5804 ?        Ss     0:01                  \_ /sbin/init
>  5998 ?        S      0:00                      \_ upstart-udev-bridge --daemon
>  6009 ?        Ss     0:00                      \_ /lib/systemd/systemd-udevd --daemon
>  6077 ?        S      0:00                      \_ upstart-socket-bridge --daemon
>  6079 ?        Ssl    0:00                      \_ rsyslogd
>  6085 ?        S      0:00                      \_ upstart-file-bridge --daemon
>  6117 ?        Ss     0:00                      \_ dhclient -1 -v -pf /run/dhclient.eth0.pid -lf /var/lib/dhcp/dhclient.eth0.leases eth0
>  6202 ?        Ss     0:00                      \_ cron
>  6207 ?        Ss     0:00                      \_ /usr/sbin/sshd -D
>  7083 ?        Ss     0:00                      \_ /sbin/getty -8 38400 tty2
>  7084 ?        Ss     0:00                      \_ /sbin/getty -8 38400 tty4
>  7085 ?        Ss     0:00                      \_ /sbin/getty -8 38400 tty3
>  7086 ?        Ss     0:00                      \_ /sbin/getty -8 38400 console
>  7087 ?        Ss     0:00                      \_ /sbin/getty -8 38400 tty1
> 
> I don't think it's enough to just add this to the config file:
> # hax for criu
> lxc.console = none
> lxc.tty = 0
> lxc.cgroup.devices.deny = c 5:1 rwm
> 
> because there is this at the very beginning:
> # Common configuration
> lxc.include = /usr/share/lxc/config/ubuntu.common.conf

Why is that a problem? The later values in the config override any earlier ones. lxc-checkpoint will also complain and refuse to dump a container that doesn't have the right configuration bits set, so if it tried to dump, that means it thinks the config is valid.

Tycho