[CRIU] rewriting images via criu (or crit?)

[Ruslan Kuprieiev]

On 27.10.2014 18:42, Tycho Andersen wrote:
> On Mon, Oct 27, 2014 at 11:32:18AM -0500, Tycho Andersen wrote:
>> On Mon, Oct 27, 2014 at 06:04:39PM +0200, Ruslan Kuprieiev wrote:
>>> On 27.10.2014 17:32, Tycho Andersen wrote:
>>>> On Mon, Oct 27, 2014 at 05:10:33PM +0200, Ruslan Kuprieiev wrote:
>>>>> Hi all,
>>>>>
>>>>> I was recently thinking about converting/modifying images and i want to
>>>>> share some of my thoughts.
>>>>> For now I see 4 main ways:
>>>>>
>>>>> 1) Use existing patch set for crit written in python.
>>>>> Pros:
>>>>>      1. Already works.
>>>>>      2. Python project is easy to support.
>>>>>      3. Adding xpath-like language for modifying images should be easy in
>>>>> Python.
>>>>>      4. All needed dependencies are available in (any?) repo.
>>>>> Cons:
>>>>>      1. Performance(maybe we could use Cython to make it better?).
>>>>>
>>>>> 2) Rewrite crit in C
>>>>> Pros:
>>>>>      1. Performance.
>>>>> Cons:
>>>>>      1. C project is hard to support.
>>>>>      3. Adding xpath-like language for modifying images is not as easy as in
>>>>> Python.
>>>>>      4. Requires protobuf-c-text, which is quite new and isn't available in
>>>>> repos.
>>>>>
>>>>> 3) Add crit functionality to criu ("criu convert", "criu modify").
>>>>> Pros:
>>>>>      1. Performance
>>>>>      2. Allows us to reuse a bunch of existing code.
>>>>>      3. Having such tool shipped along with "criu show" looks like a nice
>>>>> feature.
>>>>>      4. We can easily teach criu how to use images in text format, which
>>>>> seems
>>>>>          quite handy.
>>>>> Cons:
>>>>>      1. criu is already complex enough and adding more stuff might be too
>>>>> much,
>>>>>          especially xpath-like language engine.
>>>>>      2. Hard to support.
>>>>>      3. Requires protobuf-c-text, which is quite new and isn't available in
>>>>> repos.
>>>>>      4. Requires heavy patching to existing code.
>>>>>
>>>>> 4) Use 1), but for _high-speed_ image modification use criu plugins and
>>>>> hooks
>>>>>      to modify images on restore.
>>>>> Pros:
>>>>>      1. Performance.
>>>>>      2. No heavy patching.
>>>>> Cons:
>>>>>      Can't see any.
>>>>>
>>>>> I prefer 4) because it seems like a good compromise.
>>>>> What do you guys think?
>>>> Does that mean we'd need to expose all the protobuf headers? Will
>>>> users then need to link against libprotobuf?
>>> We would need to expose proto files with crit anyway(just the way we are
>>> shipping rpc.proto), so exposing protobuf headers shouldn't be a problem.
>> Why do we need to expose the proto files if we have the language to
>> modify the images?

Well, for example, for developer needs. So one can recompile proto files 
using protoc.

>>> Plugin indeed should be linked against libprotobuf-c. Is that a problem?
>> It would take some amount of non-trivial work for lxc to link against
>> libprotobuf-c (lxc is in the main ubuntu archive, so all of its
>> dependencies must be too; the main archive has extra reviews and
>> restrictions). Not to say it couldn't be done, but it would be nice to
>> avoid it.

Why lxc even should link to libprotobuf-c? If you are able to call tool 
that is
linked to libprotobuf-c (criu, for example), then you should be able to 
just pass
plugin to criu.

>> Another major benefit is that the proto files are essentially an
>> internal API for criu. When we make changes to the proto files, we can
>> do that as long as we write some code that handles the old style
>> images. If we make these "public", then we have to have some way to
>> version them.
>>
>>> I mean, yes, plugin isn't very flexible tool, but it should be faster than
>>> calling
>>> a separate tool to modify image. For flexibility one could use crit written
>>> in python.
>> Isn't it mostly just saving you an extra fork()? Assuming the
>> performance critical bits of crit were written in C, shouldn't it be
>> essentially equivalent to doing it in criu?
> Ah, I guess not. You only have to read the images once in this case.
> That may be a significant enough difference to warrant the change, I'm
> not sure.

For plugin:
1) call criu with plugin
     1. open img
     2. modify img in RAM
     3. restore

For tool:
1) call tool to modify image
     1. open img
     2. parse xpath-like cmd
     2. modfiy img
     3. write img
2) call criu
     1. open img
     2. restore

Not sure how slow dlopen is, but it looks like using plugin should be
significantly faster.

I'm assuming you need to modify images of migrating process, right?
So you just need to fix some values in image as fast as possible, without
saving the image somewhere on disk, right?

And other people who use criu, for example, for testing, could use
"crit modify" to modify image, save it somewhere and restore a lot
of times.

> Tycho