[Users] New Kernel Patch

[Michael H. Warfield]

On Sat, 2010-01-16 at 15:07 -0500, Scott Dowdle wrote: 
> Michael,

: - Very long SNIP...

> > If I had the maturity and stability of the OpenVZ utilities running on
> > the mainline kernel using namespaces and cgroups and no custom patch,
> > that would be my ideal combination right now.

> Wow, I'm really glad you gave the overview of LXC's current status. I
> am constantly asked about it and have yet to find a good source of
> information. I guess the mainline LXC developers have a mailing list
> but I was under the impression that it would be full of implimentation
> type discussions so I haven't joined it. Your posting is the most
> informative I've seen to date.

Thanks.  I figured it wouldn't be too far off base, given the degree to
which the OpenVZ developers have contributed to and participated in the
containers effort.  That's just about all you see on the openvz-devel
list.

Ok...  You got me.  I missed the biggest disadvantage to LXC of them
all.  To call the documentation and information on it primitive would be
being generous.  That's a major problem.  Right now, I honestly see
learning OpenVZ first and then seeing how to do similar things on LXC is
the way to learn.  I forget what exactly finally prodded me into trying
to migrate a few test VM's over (maybe it was the inclusion in F11 and
F12), but that's really what got me started.  Tried creating a few
machines from scratch using their lxc-fedora script and did not have a
joyful time with that.  Even now, if I want to create an LXC machine,
I'm going to start with an OpenVZ template.

> Previously I've only seen:

> IBM DeveloperWorks (from last Feb and quite outdated now
> http://www.ibm.com/developerworks/linux/library/l-lxc-containers/

Yeah, I've read that and it's good for a technical background so I
already had an understanding of what was going on.  But it's far far
from a decent cookbook, like we really need.

> A few rather brief postings by one Fedora Planet blogger (which refers
> to you)
> http://prefetch.net/blog/index.php/2009/06/21/installing-lxc-containers-on-fedora-hosts/

Aaaaaa!!!  Ok...  Don't mind me while I pick my jaw up off the floor.
Yeah, I remember Ryan's presentation on KVM and QEMU back in June of
last year for the Atlanta Linux Enthusiasts (ALE).  I remember the QA
session getting into a discussion of libvirt and virsh where I mentioned
both OpenVZ and LXC.  Didn't realize that made such an impression on
him.  I'm one of the original founding members of ALE and I typically
give one or two presentations in front of that group every year on
things from security to data recovery to IPv6.  Take that "world famous"
comment with a big grain of salt.  Feels WEIRD when I run into things
like that unexpectedly.

> A webpage on OpenSUSE 
> http://en.opensuse.org/LXC
> 
> I'd really like to see a practical guide that applies to the distros
> you mentioned... but as you said the code is under heavy development.
> Ideally LWN would make a nice article about this. I'll email them and
> give them a link to this thread to see if they'd be interested.

Yeah, we need some practical guides and some cookbooks, which makes me
think I may have a topic for my NEXT ALE presentation.  :-)

> Anyway, for me the main reasons I like OpenVZ are because of the ease
> of install and use (especially on my preferred distros RHEL/CentOS),
> the resource management features, and the checkpointing and migration
> features. I have a lot of respect for Linux-VServer's Unification
> feature and the work they do to adapt to newer mainline kernels. And
> now, with the information you have provided about LXC... it just makes
> me wish would could wrap all three up into a matured LXC in the
> mainline with the added benefit of KVM in mainline as well. Of course
> we'd need a mature tool that could manage all containers and KVM VMs
> in a sane way and yes libvirt / virsh seem to be what will evolve...
> although I'd love to see Kir pick up the fork of vzctl with LXC
> support he abandoned because of lack of time a while back.

Ooo...  I wasn't aware of that LXC fork of vzctl.  See...  Now there is
an excellent example of just what I mean.  The same amount of work into
that fork as what it might take to cut another patched kernel could well
give us an implementation of OpenVZ utilizing the Linux Containers in
the mainline kernel.  IMHO, that would be a very good thing and a more
productive use of limited developer time and cut way down on long term
development, support, and maintenance down the road.

> I think I've seen one or two blog postings somewhere where people were
> using LXC containers as separators of KVM VMs so they could apply
> cgroup resource management on them. Sorry for not providing the links
> to those as I don't have them handy... and can't seem to engage the
> google-fu.

> I'd love to give LXC a try... and I may do so using your
> recommendations but the problems you covered still apply:

Yup...  Set up a test bed now to start coming up to speed but don't
begin production migrations until you are very happy with what you're
doing.

> 1) It is a big moving target and will continue to be so for some time

Yeah but at least most of the cats are now moving in one direction.

> 2) Lack of features particularly in resource controls (possible with
> manual cgroup tools?)

Actually, they have support for cgroups in the LXC configs and an
lxc-cgroup command.  So support of cgroups is not a problem.  Support IN
cgroups for all the bean_counter options may be.  I honestly have not
tried to do a 1:1 mapping yet.  I also haven't tried to play with quotas
for the VMs in it yet either.  Another todo.

> 3) The management tool immaturity because of #1

> 4) The needed distros (Fedora or Ubuntu)... too rapid of a release
> cycle making them inappropriate for server use and medium to long-term
> deployments

It's actually not all that bad.  My major production servers are all
remote and I've got remote management down to a science to the extent
that I actually PREFER Fedora over RHEL/CentOS.  Some of my servers
started out on Fedora 1 and have been upgraded using the "yum update"
method the whole time and are now F11's.  You can't do that with RHEL or
CentOS.  My IT department is weeping and nashing teeth over upgrades
from RHEL 4 to RHEL 5.  I may upgrade 3 times as often but they're
working 10 times as hard each time.  I also use NST (Network Security
Toolkit) as a run-live version of Fedora that I keep in the CD drives of
those remote machines in case I have to do serious diagnostics or
forensics.  They have cross connected serial cables and serial consoles
where I can interrupt the boot process and boot the machines purely from
CD and analyze the live drives from trusted RO media.  If I don't
interrupt the boot process, it chain loads to the live drives and boots
normally.  That's all Fedora based.  I'm doing some work with that group
and looking to integrate some of this into that distro (another reason
why I wanted to migrate to LXC).  Then I could boot from the CD and run
the VM's from the main drives.

Using a stripped down Fedora on the host engine is relatively painless
to upgrade and then run CentOS or RHEL in the guest VM's for services,
if you like, for stability.  Both OpenVZ and LXC then make upgrading
THOSE critters vastly simpler as well.  In those cases, I can create new
VM's with the new OS and cross mount the data directories and do the
migrations of databases and applications between two live machines
sharing common directories.  This makes it a dream to me.

> While it is exciting that LXC is somewhat usable now... I still think
> it'll be at least two years before those four points are resolved...
> but that is just a guess. I certainly hope it takes less time than
> that.

I don't think it's going to take that long and I'm trying to dig into it
more and contribute in that arena.  I don't know what their roadmap is
yet.

> Mike, you are the closest thing I've found to an LXC "expert" and I'd
> love to talk with you more about LXC and possibly do an interview with
> you for my website (MontanaLinux.org) if you'd be interested.

Yeah, OK.  We can get together off line and sort that out.  Might even
help me put together some documentation and presentations myself as
well.

> My friends were asking me if I was going to do a presentation at LFNW
> this year. The last two years I've done one on OpenVZ and didn't want
> to do yet another one on OpenVZ. You appear to live in Georgia so I
> doubt you'd be interested in going to the state of Washington for a
> Linux conference to give a presentation having to foot the expenses
> yourself? Maybe I could learn enough about LXC by the end of April to
> give it a go. Anyway, thanks again for the information!

Yeah, I live here in Georgia but I have done talks all over the world,
although my office has cut back on my travel budget the last couple of
years.  Been to Washington state several times (last time on my way to
Vancouver where I was speaking) and I have several friends who work for
the evil empire.  I use to be a regular speaker at Linux World in San
Jose, San Francisco, and New York but my travels been cut back pretty
heavily.  April would be too short of notice for me as well.  We'll get
together outside of this forum.

I just will concur with you and hope this discussion spurs Kir into
picking up and running with that LXC branch of vzctl again.

> TYL,

Mike
-- 
Michael H. Warfield (AI4NB) | (770) 985-6132 |  mhw at WittsEnd.com
   /\/\|=mhw=|\/\/          | (678) 463-0932 |  http://www.wittsend.com/mhw/
   NIC whois: MHW9          | An optimist believes we live in the best of all
 PGP Key: 0x674627FF        | possible worlds.  A pessimist is sure of it!
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 482 bytes
Desc: This is a digitally signed message part
Url : http://openvz.org/pipermail/users/attachments/20100116/dab7a7cd/attachment.bin