[CRIU] LXC checkpoint/restore HOWTO using upstream tools
Zmudzinski, Krystof C
krystof.c.zmudzinski at intel.com
Wed Sep 24 09:14:00 PDT 2014
Tycho,
I have finally succeeded to configure my system so I'm able to suspend and resume my container using LXC built from source.
I started with a fresh Ubuntu 14.04 LTS.
The trick that did it for me was to first follow the instructions from
http://bazaar.launchpad.net/~tycho-s/+junk/snapshot-instructions/view/head:/README
This, by itself, didn't quite work as I was unable to start my container. The error messages had to do first with missing cgroups; after I fixed that with missing apparmor profiles.
So I just installed the default lxc package (i.e., sudo apt-get install lxc)
And everything works now.
I think that the development package still wants to use
/etc/apparmor.d/lxc/
Instead of
/usr/local/etc/apparmor.d/lxc/
because that was the last thing that installing the default lxc fixed.
What I still don't understand is that I can't find lxc-container-default-with-mounting anywhere on my machine.
I could be completely wrong here so I will try repeating these steps again to confirm.
Krystof
-----Original Message-----
From: Tycho Andersen [mailto:tycho.andersen at canonical.com]
Sent: Tuesday, September 23, 2014 1:36 PM
To: Zmudzinski, Krystof C
Cc: CRIU
Subject: Re: [CRIU] LXC checkpoint/restore HOWTO using upstream tools
Hi Krystof,
On Tue, Sep 23, 2014 at 08:29:30PM +0000, Zmudzinski, Krystof C wrote:
> In fact, the last time everything seems to be working is just before ldconfig -v. I can start, attach and stop. But as soon as I execute ldconfig, lxc-start says that lxc-start: Executing '/sbin/init' with no configuration file may crash the host and lxc-info says that the container doesn't exist. Reverting ldconfig doesn't change anything.
Yes, that's (presumably) because it is looking at a new lxcpath (/usr/local/var/lib/lxc vs. /var/lib/lxc) when you're using the right liblxc.so. You have a few options here:
1. just re-create your container under /usr/local (or just copy it) 2. change your lxcpath to point to the old container 3. re-compile with --prefix /usr so that it installs over the packaged
lxc (I don't recommend this :)
You can verify which lxcpath things are pointed to via:
sudo lxc-config lxc.lxcpath
Tycho
>
> Krystof
>
> -----Original Message-----
> From: Tycho Andersen [mailto:tycho.andersen at canonical.com]
> Sent: Monday, September 22, 2014 4:32 PM
> To: Zmudzinski, Krystof C
> Cc: CRIU
> Subject: Re: [CRIU] LXC checkpoint/restore HOWTO using upstream tools
>
> Hi Krystof,
>
> On Mon, Sep 22, 2014 at 09:44:24PM +0000, Zmudzinski, Krystof C wrote:
> > Tycho,
> >
> > After following the instructions on http://criu.org/LXC, I wanted to install the latest source for LXC and make some changes. How can I build and install the new lxc-* so I overwrite what sudo apt-get install lxc did? Right now, following the instructions on http://bazaar.launchpad.net/~tycho-s/+junk/snapshot-instructions/view/head:/README everything gets installed in different directories and things stop working completely.
>
> What's the output of ldd `which lxc-checkpoint`? Did you do the bit in there about modifying the ld.so.conf?
>
> Tycho
>
> > I always get this error when I try to start a container:
> > > lxc-start 1410804731.572 ERROR lxc_cgfs - Could not find writable mount point for cgroup hierarchy 3 while trying to create cgroup.
> > > lxc-start 1410804731.572 ERROR lxc_start - failed creating cgroups
> > > lxc-start 1410804731.599 ERROR lxc_start - failed to spawn 'u1'
> > > lxc-start 1410804736.604 ERROR lxc_start_ui - The container failed to start.
> >
> > Krystof
> >
> > -----Original Message-----
> > From: Tycho Andersen [mailto:tycho.andersen at canonical.com]
> > Sent: Friday, September 19, 2014 10:53 AM
> > To: Zmudzinski, Krystof C
> > Cc: CRIU
> > Subject: Re: [CRIU] LXC checkpoint/restore HOWTO using upstream
> > tools
> >
> > Hi Krystof,
> >
> > On Fri, Sep 19, 2014 at 05:18:32PM +0000, Zmudzinski, Krystof C wrote:
> > > lxc-checkpoint fails. I did a fresh install of ubuntu 14.04 and followed your instructions. I also installed criu-1.3.1.
> > >
> > > From dump.log:
> > > (00.377632) Error (mount.c:805): fusectl isn't empty: 8388625
> >
> > Ah, that is a good point. Right now CRIU doesn't support dumping any fuse filesystems (i.e., /sys/fs/fuse/connections needs to be empty). I guess stock desktop ubuntu might have some of these enabled. You can either uninstall any fuse modules or try ubuntu-server (or better yet, the cloud images) as a host.
> >
> > I guess maybe we should have lxc-checkpoint look for some of this stuff too, vs. just config.
> >
> > > The container does contain ttys and console:
> > > 5784 ? Ss 0:00 \_ lxc-start -n cn_01
> > > 5804 ? Ss 0:01 \_ /sbin/init
> > > 5998 ? S 0:00 \_ upstart-udev-bridge --daemon
> > > 6009 ? Ss 0:00 \_ /lib/systemd/systemd-udevd --daemon
> > > 6077 ? S 0:00 \_ upstart-socket-bridge --daemon
> > > 6079 ? Ssl 0:00 \_ rsyslogd
> > > 6085 ? S 0:00 \_ upstart-file-bridge --daemon
> > > 6117 ? Ss 0:00 \_ dhclient -1 -v -pf /run/dhclient.eth0.pid -lf /var/lib/dhcp/dhclient.eth0.leases eth0
> > > 6202 ? Ss 0:00 \_ cron
> > > 6207 ? Ss 0:00 \_ /usr/sbin/sshd -D
> > > 7083 ? Ss 0:00 \_ /sbin/getty -8 38400 tty2
> > > 7084 ? Ss 0:00 \_ /sbin/getty -8 38400 tty4
> > > 7085 ? Ss 0:00 \_ /sbin/getty -8 38400 tty3
> > > 7086 ? Ss 0:00 \_ /sbin/getty -8 38400 console
> > > 7087 ? Ss 0:00 \_ /sbin/getty -8 38400 tty1
> > >
> > > I don't think it's enough to just add this to the config file:
> > > # hax for criu
> > > lxc.console = none
> > > lxc.tty = 0
> > > lxc.cgroup.devices.deny = c 5:1 rwm
> > >
> > > because there is this at the very beginning:
> > > # Common configuration
> > > lxc.include = /usr/share/lxc/config/ubuntu.common.conf
> >
> > Why is that a problem? The later values in the config override any earlier ones. lxc-checkpoint will also complain and refuse to dump a container that doesn't have the right configuration bits set, so if it tried to dump, that means it thinks the config is valid.
> >
> > Tycho
More information about the CRIU
mailing list