[Users] Shortest guide about running OpenVZ containers on top of ZFS

[Pavel Snajdr]

Oh, again, this debate always goes on and on :)

Guys, try ZFS yourselves and come back here :)

You obviously haven't seen ARC caching in action. You haven't played
with snapshots. You haven't seen what the online compression can do.

Etc., etc., etc.

There's lots to ZFS, which neither BTRFS will ever even remotely approach.

Try having this config:

- 300+ containers on a single node
- 128G RAM
- 6 spindles, 2 SSDs
- run MySQL on at least 50 of the containers

Not only it is way too faster than anything you could do with ext4 even
if it's split via ploop into smaller filesystems, it is also much, much
easier to manage. ZFS has a tree structure of filesystems with property
inheritance.

It's designed to be The Solution for situations exactly like this one.

The only shortcoming I can really see and mention from my experience of
running an OpenVZ based hosting with 850 active CTs on top of ZFS, is
that it lacks the support for dquota.

I've looked into integrating dquota with ZFS, but it's such an utter
mess of an invention, that I have quickly changed my mind and instead
we're just doing more datasets (== subvolumes in BTRFS). They are really
inexpensive (16kB each), can have own size limits (quotas in ZFS lingo)
and thanks to the tree structure with inheritance it's easy to manage them.

Also, forget about rsync and all that crap. Send/receive kills it with ease.

/snajpa

On 11/12/2014 09:48 PM, Scott Dowdle wrote:
> Greetings,
> 
> ----- Original Message -----
>> Ploop is really useless for ZFS because it solves ext4 troubles and
>> ZFS haven't this issues by design. Quotes maybe problems, good
>> addition. I just added remark about quotes to comparison table.
> 
> Performance issues aren't the only problem ploop solves... it also solves the changing inode issue.  When a container is migrated from one host to another with simfs, inodes will change... and some services don't like that.  Also because the size of a ploop disk image is fixed (although changeable), the fixed size acts as a quota... so you get your quota back if you turned it off.
> 
> For me, unless something changes, ZFS isn't a starter because almost no one ships with it because of licensing issues.
> 
> How about btrfs?  I don't think btrfs is available easily in the existing OpenVZ kernels... nor in a modular format (like ZFS) so we might have to wait until the availability of a RHEL7-based OpenVZ branch.  Red Hat still considers btrfs experimental but that may change with upcoming RHEL7 updates.  Both SUSE and Oracle have been using btrfs for some time although they do not support btrfs' entire feature set... they stick with the basic features and avoid the less mature ones.  Luckily that includes mirror, checksums, snapshoting, subvolumes, etc.
> 
> I wouldn't put simfs and ploop in the same column as the underlying filesystems.
> 
> I'm not sure why the chart says that simfs has issues with migration.  Other than the inode issue, which isn't an issue with the services I run, simfs actually migrates faster because it doesn't have to transfer the entire disk image... and if the host has been migrated before and has a previous copy of its filesystem available, only the changed files have to be transferred... saving a lot of time.
> 
> TYL,
>