[CRIU] Project for GSoC 2019: Optimize logging engine

[Vaibhav Gupta]

I figured it out, '--ignore-taint' flag was required. The tests passed
successfully.

On Thu, Mar 14, 2019 at 11:24 AM Vaibhav Gupta <vaibhavgupta40 at gmail.com>
wrote:

>
>
> On Thu, Mar 14, 2019 at 11:10 AM Vaibhav Gupta <vaibhavgupta40 at gmail.com>
> wrote:
>
>>
>>
>> On Tue, Mar 12, 2019 at 9:57 PM Pavel Emelianov <xemul at virtuozzo.com>
>> wrote:
>>
>>> On 3/12/19 2:14 PM, Vaibhav Gupta wrote:
>>> > Hello everyone,
>>> > .
>>> > I was exploring CRIU - Suggested Ideas for GSoC 2019, and have got
>>> interested in the following project:
>>> >
>>> >   * Optimize logging engine -
>>> https://criu.org/Google_Summer_of_Code_Ideas#Optimize_logging_engine
>>>
>>> That's great!
>>>
>>> > .
>>> > I am already exploring this topic and would like to discuss with
>>> mentors.
>>>
>>> Sure! Do you have questions about the gsoc process itself, or can we
>>> just proceed to the
>>> technical discussion?
>>>
>> No, I don't have nay query regarding GSoC Procedures.
>>
>>>
>>> > .
>>> > It would be great if mentors would like to suggest any specific
>>> resources, which can be help me to know more about the project.
>>>
>>> So, the doc to start with is
>>> https://criu.org/GSoC_Students_Recommendations, it will
>>> let you start running criu instantly. Then glance through the
>>> sources/criu/log.c where
>>> the logging code sits.
>>>
>> I was following the above guide. when I run:
> $ sudo python zdtm.py run -t static/env00
> .
> I get following error in the last:
> .
> zdtm.py:21: DeprecationWarning: the imp module is deprecated in favour of
> importlib; see the module's documentation for alternative uses
>   import imp
> The kernel is tainted: '12289\n'
> Traceback (most recent call last):
>   File "zdtm.py", line 2306, in <module>
>     opts['action'](opts)
>   File "zdtm.py", line 1975, in run_tests
>     launcher = Launcher(opts, len(torun))
>   File "zdtm.py", line 1671, in __init__
>     raise Exception("The kernel is tainted: %r" % self.__taint)
> Exception: The kernel is tainted: '12289\n'
>
>
>>> As far as the task itself is concerned -- once you're ready to run criu
>>> you can try to
>>> even just checkpoint some zdtm test with -v0 (make criu quiet) and -v4
>>> (make criu print
>>> everything it wants) CLI option 1000 times and compare the time taken.
>>> You should notice
>>> that verbose logging gives you a noticeable penalty on the runtime (and
>>> thus the freeze
>>> time as well, the freeze time is reported via stats:
>>> https://criu.org/Statistics).
>>>
>>> So the goal is to have the best of two worlds -- have as many messages
>>> as with -v4, but
>>> at the maximum possible speed (i.e. -- like it's with -v0). Our research
>>> showed that the
>>> logging slow-down sits in sprintf() call that scans the formatting
>>> string and formats
>>> the arguments, so the first thing to do is to avoid doing this by
>>> logging binary data
>>> and providing a tool for decoding the resulting blob into text.
>>>
>>> Thus the task roughly falls into 3 sub-tasks:
>>>
>>> 1. The code that keeps the binary messages around and flushes them into
>>> file
>>> 2. The tool to decode the file with binary messages
>>> 3. Turn existing pr_foo() calls into binary form
>>>
>>> The latter can be a routine task where one sits and fixes the pr_foo()
>>> arguments into
>>> binary versions, but we have a dream to make it smarter. Like this.
>>> Consider the simple
>>> logging code
>>>
>>>         int bar;
>>>         ...
>>>         pr_debug("Foo %d\n", bar);
>>>
>>> If order to make it binary we need to do this build-time trick:
>>>
>>> 1. detect this pr_foo message
>>> 2. get the format string and auto-assign an ID to it
>>> 3. get the arguments and their types and generate a code looking smth
>>> like
>>>
>>>     mem = binary_log_get_buffer(size_of_all_arguments);
>>>     *((int *)mem) = /* ID of the message */; mem += sizeof(int);
>>>     *((typeof(bar) *)mem) = bar; mem += sizeof(bar);
>>>
>>> So far we haven't found good solution for it, but hopefully we will.
>>>
>> Okay, I am gonna follow the steps you said, will be my first task. I will
>> revert back once i complete this.
>>
>>>
>>> -- Pavel
>>>
>>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openvz.org/pipermail/criu/attachments/20190314/08d93aec/attachment.html>