[CRIU] Project for GSoC 2019: Optimize logging engine

[Pavel Emelianov]

On 3/12/19 2:14 PM, Vaibhav Gupta wrote:
> Hello everyone,
> .
> I was exploring CRIU - Suggested Ideas for GSoC 2019, and have got interested in the following project:
> 
>   * Optimize logging engine -  https://criu.org/Google_Summer_of_Code_Ideas#Optimize_logging_engine

That's great!

> .
> I am already exploring this topic and would like to discuss with mentors.

Sure! Do you have questions about the gsoc process itself, or can we just proceed to the
technical discussion?

> .
> It would be great if mentors would like to suggest any specific resources, which can be help me to know more about the project.

So, the doc to start with is https://criu.org/GSoC_Students_Recommendations, it will
let you start running criu instantly. Then glance through the sources/criu/log.c where
the logging code sits.

As far as the task itself is concerned -- once you're ready to run criu you can try to
even just checkpoint some zdtm test with -v0 (make criu quiet) and -v4 (make criu print
everything it wants) CLI option 1000 times and compare the time taken. You should notice
that verbose logging gives you a noticeable penalty on the runtime (and thus the freeze
time as well, the freeze time is reported via stats: https://criu.org/Statistics).

So the goal is to have the best of two worlds -- have as many messages as with -v4, but
at the maximum possible speed (i.e. -- like it's with -v0). Our research showed that the
logging slow-down sits in sprintf() call that scans the formatting string and formats
the arguments, so the first thing to do is to avoid doing this by logging binary data
and providing a tool for decoding the resulting blob into text.

Thus the task roughly falls into 3 sub-tasks:

1. The code that keeps the binary messages around and flushes them into file
2. The tool to decode the file with binary messages
3. Turn existing pr_foo() calls into binary form

The latter can be a routine task where one sits and fixes the pr_foo() arguments into
binary versions, but we have a dream to make it smarter. Like this. Consider the simple
logging code

	int bar;
	...
	pr_debug("Foo %d\n", bar);

If order to make it binary we need to do this build-time trick:

1. detect this pr_foo message
2. get the format string and auto-assign an ID to it
3. get the arguments and their types and generate a code looking smth like

    mem = binary_log_get_buffer(size_of_all_arguments);
    *((int *)mem) = /* ID of the message */; mem += sizeof(int);
    *((typeof(bar) *)mem) = bar; mem += sizeof(bar);

So far we haven't found good solution for it, but hopefully we will.

-- Pavel