[CRIU] CRIU LXC Container Live Migration Concerns

[Deepak Vij (A)]

Hi folks, I am back with the following few additional questions. I would look forward to your response/feedback on these. Thanks.

Regards,
Deepak K. Vij

=================================

1) Live Migration & Network Addressing: Hypervisor based VM Mobility environment is typically restricted to a Layer 3 subnet and a Layer 2 domain (for VLANs) because the underlying network will support the VM operating outside of the local scope of those addresses. Needless to say, the network addressing scheme in a cloud operated by an entirely different service provider is not only a different subnet but a different class B or class A network altogether. At the time of live migration of VMs, routers and switches simply would not know how to cope with the "rogue" running system at the time of live migration. One alternative is to extend the L2 segment using L2overL3 tunneling/SDN based protocols.

Although container based virtualization (LXC etc.) has a notion of Network Namespaces, wouldn't container based live migration have the same issues as the above mentioned hypervisor based VM Mobility across subnet. From that standpoint, shouldn't this be the same for any container environment, be it LXC/Docker, OpenVZ, Parallels, BSD Jails, or Solaris Zones etc.

2) Live Migration & Storage/Databases: For any Live Migration (Hypervisor based VM Mobility or Container based Live Migration), storage replication across LAN & WAN environments needs to go along with the live migration process - such as SAN storage (LAN) or maybe EMC VPLEX like replicated storage (WAN). One could also choose to move just the VM state while keeping storage in the original data center, but that will impact application performance.

I would look forward to your response on these two observations.

-----Original Message-----
From: James Bottomley [mailto:jbottomley at parallels.com] 
Sent: Wednesday, April 30, 2014 3:49 PM
To: Deepak Vij (A)
Cc: Pavel Emelianov; criu at openvz.org
Subject: Re: [CRIU] CRIU LXC Container Live Migration Concerns

On Tue, 2014-04-29 at 17:45 +0000, Deepak Vij (A) wrote:
> Thanks Pavel & James for your detail response to my earlier email. I
> appreciate it. 
> 
> Based on our back-and-forth discussion so far, I am fully convinced
> that you folks, as part of the CRIU "Live Migration" effort, are
> definitely on the right track. I am convinced that "Live Migration"
> design which you have put forward using Kernel level API should work.
> 
> Although, the concern I had was at a higher LXC/Container level
> itself. You also concur based on the following response from Pavel.
> 
> ====================================
> Hm... Good question -- what if after migration all the system
> libraries have changed. Yes, this _is_ a problem. E.g. I've seen that
> applications crash after migration between two nodes with glibcs of
> equal versions, but "fixed" by prelink in two different ways.
> ====================================
> 
> Now, first and foremost, I am also a big proponent of LXC-container
> and do want this to succeed as this is the right unit-of-work from
> portability standpoint versus traditional heavyweight hypervisor based
> unit-of-work VM model.

Just to be clear, we at Parallels and OpenVZ aren't really LXC people.
There are LXC people who hang out on this list, but we're mostly
concerned about OpenVZ which uses templates to solve this problem in a
mechanism similar to Docker.  In our commercial product, we also do
template migration, which means we never really have to worry about
this.

>  To that regards, Docker folks are trying to solve this problem by
> enabling LXC-Container capabilities across various flavors of Linux
> based operating systems and dependency issues etc. However, there
> needs to be more work done so that Docker like abstraction can provide
> OVF (Open Virtualization Format for hypervisor based VMs) like
> seamless compatibility across various LXC-Containers environment,
> supporting all kinds of dependencies, injections etc. 

My impression is that if you read the documents, docker thinks they've
got this pretty much solved.

https://www.docker.io/the_whole_story/

The overlay diffs look quite compelling.  However, you probably need to
take it up with them.  Right at the moment, we've barely got to the
stage where we can migrate a docker container.

> This is something very near and dear to my heart from IEEE Intercloud
> standards development perspectives. As part of the IEEE Intercloud
> standardization effort, so far we have made a good progress on the
> Interoperability standards across clouds (using Ontology definitions,
> Trust Management, etc). However, the portability issue we have kind of
> shoved it under the rug as traditional VM-Mobility like portability is
> not really viable across geographically dispersed cloud computing
> environment.

I don't quite see why you think the docker or OpenVZ templates wouldn't
work inter-cloud.  We think they do.

> This is a great opportunity for us to define standardization around
> lightweight container based unit-of-work, something like OVF, to make
> this whole thing work. I am not sure, who to contact to this regards.
> Maybe we should loop in Docker folks into all this as well. Or maybe
> they are already working towards that, I am not sure.

Docker doesn't really do email per se, the closest is probably google
groups.  This is how you contact them:

https://www.docker.io/community/

James