[CRIU] Project for GSoC 2019: Optimize logging engine

[Vaibhav Gupta]

On Tue, Mar 12, 2019 at 9:57 PM Pavel Emelianov <xemul at virtuozzo.com> wrote:

> On 3/12/19 2:14 PM, Vaibhav Gupta wrote:
> > Hello everyone,
> > .
> > I was exploring CRIU - Suggested Ideas for GSoC 2019, and have got
> interested in the following project:
> >
> >   * Optimize logging engine -
> https://criu.org/Google_Summer_of_Code_Ideas#Optimize_logging_engine
>
> That's great!
>
> > .
> > I am already exploring this topic and would like to discuss with mentors.
>
> Sure! Do you have questions about the gsoc process itself, or can we just
> proceed to the
> technical discussion?
>
No, I don't have nay query regarding GSoC Procedures.

>
> > .
> > It would be great if mentors would like to suggest any specific
> resources, which can be help me to know more about the project.
>
> So, the doc to start with is
> https://criu.org/GSoC_Students_Recommendations, it will
> let you start running criu instantly. Then glance through the
> sources/criu/log.c where
> the logging code sits.
>
> As far as the task itself is concerned -- once you're ready to run criu
> you can try to
> even just checkpoint some zdtm test with -v0 (make criu quiet) and -v4
> (make criu print
> everything it wants) CLI option 1000 times and compare the time taken. You
> should notice
> that verbose logging gives you a noticeable penalty on the runtime (and
> thus the freeze
> time as well, the freeze time is reported via stats:
> https://criu.org/Statistics).
>
> So the goal is to have the best of two worlds -- have as many messages as
> with -v4, but
> at the maximum possible speed (i.e. -- like it's with -v0). Our research
> showed that the
> logging slow-down sits in sprintf() call that scans the formatting string
> and formats
> the arguments, so the first thing to do is to avoid doing this by logging
> binary data
> and providing a tool for decoding the resulting blob into text.
>
> Thus the task roughly falls into 3 sub-tasks:
>
> 1. The code that keeps the binary messages around and flushes them into
> file
> 2. The tool to decode the file with binary messages
> 3. Turn existing pr_foo() calls into binary form
>
> The latter can be a routine task where one sits and fixes the pr_foo()
> arguments into
> binary versions, but we have a dream to make it smarter. Like this.
> Consider the simple
> logging code
>
>         int bar;
>         ...
>         pr_debug("Foo %d\n", bar);
>
> If order to make it binary we need to do this build-time trick:
>
> 1. detect this pr_foo message
> 2. get the format string and auto-assign an ID to it
> 3. get the arguments and their types and generate a code looking smth like
>
>     mem = binary_log_get_buffer(size_of_all_arguments);
>     *((int *)mem) = /* ID of the message */; mem += sizeof(int);
>     *((typeof(bar) *)mem) = bar; mem += sizeof(bar);
>
> So far we haven't found good solution for it, but hopefully we will.
>
Okay, I am gonna follow the steps you said, will be my first task. I will
revert back once i complete this.

>
> -- Pavel
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openvz.org/pipermail/criu/attachments/20190314/0fd37858/attachment.html>