[CRIU] Project for GSoC 2019: Optimize logging engine

[Vaibhav Gupta]

On Thu, Mar 14, 2019 at 11:10 AM Vaibhav Gupta <vaibhavgupta40 at gmail.com>
wrote:

>
>
> On Tue, Mar 12, 2019 at 9:57 PM Pavel Emelianov <xemul at virtuozzo.com>
> wrote:
>
>> On 3/12/19 2:14 PM, Vaibhav Gupta wrote:
>> > Hello everyone,
>> > .
>> > I was exploring CRIU - Suggested Ideas for GSoC 2019, and have got
>> interested in the following project:
>> >
>> >   * Optimize logging engine -
>> https://criu.org/Google_Summer_of_Code_Ideas#Optimize_logging_engine
>>
>> That's great!
>>
>> > .
>> > I am already exploring this topic and would like to discuss with
>> mentors.
>>
>> Sure! Do you have questions about the gsoc process itself, or can we just
>> proceed to the
>> technical discussion?
>>
> No, I don't have nay query regarding GSoC Procedures.
>
>>
>> > .
>> > It would be great if mentors would like to suggest any specific
>> resources, which can be help me to know more about the project.
>>
>> So, the doc to start with is
>> https://criu.org/GSoC_Students_Recommendations, it will
>> let you start running criu instantly. Then glance through the
>> sources/criu/log.c where
>> the logging code sits.
>>
> I was following the above guide. when I run:
$ sudo python zdtm.py run -t static/env00
.
I get following error in the last:
.
zdtm.py:21: DeprecationWarning: the imp module is deprecated in favour of
importlib; see the module's documentation for alternative uses
  import imp
The kernel is tainted: '12289\n'
Traceback (most recent call last):
  File "zdtm.py", line 2306, in <module>
    opts['action'](opts)
  File "zdtm.py", line 1975, in run_tests
    launcher = Launcher(opts, len(torun))
  File "zdtm.py", line 1671, in __init__
    raise Exception("The kernel is tainted: %r" % self.__taint)
Exception: The kernel is tainted: '12289\n'


>> As far as the task itself is concerned -- once you're ready to run criu
>> you can try to
>> even just checkpoint some zdtm test with -v0 (make criu quiet) and -v4
>> (make criu print
>> everything it wants) CLI option 1000 times and compare the time taken.
>> You should notice
>> that verbose logging gives you a noticeable penalty on the runtime (and
>> thus the freeze
>> time as well, the freeze time is reported via stats:
>> https://criu.org/Statistics).
>>
>> So the goal is to have the best of two worlds -- have as many messages as
>> with -v4, but
>> at the maximum possible speed (i.e. -- like it's with -v0). Our research
>> showed that the
>> logging slow-down sits in sprintf() call that scans the formatting string
>> and formats
>> the arguments, so the first thing to do is to avoid doing this by logging
>> binary data
>> and providing a tool for decoding the resulting blob into text.
>>
>> Thus the task roughly falls into 3 sub-tasks:
>>
>> 1. The code that keeps the binary messages around and flushes them into
>> file
>> 2. The tool to decode the file with binary messages
>> 3. Turn existing pr_foo() calls into binary form
>>
>> The latter can be a routine task where one sits and fixes the pr_foo()
>> arguments into
>> binary versions, but we have a dream to make it smarter. Like this.
>> Consider the simple
>> logging code
>>
>>         int bar;
>>         ...
>>         pr_debug("Foo %d\n", bar);
>>
>> If order to make it binary we need to do this build-time trick:
>>
>> 1. detect this pr_foo message
>> 2. get the format string and auto-assign an ID to it
>> 3. get the arguments and their types and generate a code looking smth like
>>
>>     mem = binary_log_get_buffer(size_of_all_arguments);
>>     *((int *)mem) = /* ID of the message */; mem += sizeof(int);
>>     *((typeof(bar) *)mem) = bar; mem += sizeof(bar);
>>
>> So far we haven't found good solution for it, but hopefully we will.
>>
> Okay, I am gonna follow the steps you said, will be my first task. I will
> revert back once i complete this.
>
>>
>> -- Pavel
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openvz.org/pipermail/criu/attachments/20190314/e5340b92/attachment-0001.html>