[CRIU] Restore failed. Exit code: 43

[Christopher Covington]

Hi Paschalis,

If you're interested in run time "fast forwarding" approaches as alternatives
to modifying the benchmark source, I've experimented with two approaches:
instruction counts from perf events and breakpoints with GDB or uprobes. While
my original implementation was built in to CRIU, I've now settled on small
separate tools that send the benchmark SIGSTOP at the right moment, after
which I can dump a checkpoint. You can then run CRIU restore on the same or a
different system, take your time setting up perf record, memory tracing, or
whatever, and then send the benchmark SIGCONT. I'm hoping the SIGSTOP, CRIU
dump, CRIU restore, SIGCONT sequence doesn't perturb most benchmarks, but I've
only done modest testing. I can't guarantee anything, but if the fast
forwarding tools sound interesting, I can try to publish them as part of perf,
CRIU, or standalone.

On 01/27/2015 07:57 AM, Pavel Emelyanov wrote:
> On 01/27/2015 03:48 PM, Paschalis Mpeis wrote:
>>
>>
>> On Tue, Jan 27, 2015 at 12:05 PM, Pavel Emelyanov <xemul at parallels.com <mailto:xemul at parallels.com>> wrote:
>>
>>     On 01/26/2015 08:05 PM, Paschalis Mpeis wrote:
>>     > My initial thought was to fork the initial process, so as to exploit the copy-on-write fork implementation on Linux.
>>     >
>>     > Then the forked process will call the function, and the parent will make the dump, by providing the PID of its child.
>>
>>     This would work, but will anyway dump the whole process. In order to dump only what it relevant
>>     to a single function... hmm... you should either explore the binary with disassembler or...
>>     maybe the memory tracked would help. You reset the tracker, then start working. Then you
>>     call criu to dump the process and dump only what memory has changed. This will include only
>>     what was touched by the function.

I was thinking you'd have to run the restored benchmark under valgrind or QEMU
or set a bunch of breakpoints/watchpoints with GDB in order to obtain a memory
trace, but page granularity tracking with the software dirty bit is probably
fine and that's already being used for related functionality.

>> ​So, I should reset the bits of the used memory pages, so CRIU will dump only the "dirty" ones?
> 
> Yes and no. Currently CRIU knows how to work with memory tracked, but the
> implementation of it is pretty strict -- you should provide criu with the
> "parent" image. In your case you do the single dump, so there's no parent
> image and criu will always dump all the memory.
> 
>> Or I should modify parts also parts of the dump functionality?​
> 
> Yes. Look at the mem.c generate_iovs() function. It generates the bitmap
> of pages that should be taken into the image.

I wonder if using the new CRIT tool to remove the unused pages from the CRIU
checkpoint might be easier than modifying CRIU.

>>     And on restore time you should create the process and ask CRIU to repopulate only part of
>>     it. But this functionality is not yet there, criu should be patched to allow for this.
>>
>>
>> ​Since
>>  the images include only what is relevant to the function, and also the code after the dump
>> just executes the function and terminates, why it would require extra patches for restore?
> 
> Because criu on restore recreates the whole process from scratch. We currently
> don't have the functionality to restore only a part of the process.

I'm excited to see more folks using CRIU for what looks like simulation and
performance analysis purposes.

Regards,
Chris

-- 
Qualcomm Innovation Center, Inc.
The Qualcomm Innovation Center, Inc. is a member of the Code Aurora Forum,
a Linux Foundation Collaborative Project