[CRIU] Restore failed. Exit code: 43

[Pavel Emelyanov]

On 01/22/2015 07:45 PM, Paschalis Mpeis wrote:
> 
> 
> On Wed, Jan 21, 2015 at 5:30 PM, Pavel Emelyanov <xemul at parallels.com <mailto:xemul at parallels.com>> wrote:
> 
>     On 01/21/2015 07:46 PM, Paschalis Mpeis wrote:
>     > A "return" at line 77 <https://gist.github.com/Paschalis/a96b2747ed85b8e5a796#file-crlib-c-L77> could do the
>     > trick. However, the restored program has standard input, output, and error closed.
> 
>     Of course. In line 60 of crlib.c  you close them before dumping.
> 
> 
>     Since they are terminals there's no easy way to do it. If you could
>     redirect stdios to files, then you could just leave them open on
>     dump, then restore would re-create them back.
> 
> 
> ​Yes, I do, but it's because otherwise CRIU won't dump the application.
> ​O​​n one hand, it makes sense to have them closed, but on the other CRIU could just 
> handle them like the rest of the files. ​

Well, I don't disagree :) In your case these files are terminal pairs, but just
one end of it. CRIU tries hard not to break connections between tasks and if we
dump one half of a tty we would do it. So for this the "shell-job" option was
introduced. It tries to handle such cases.

> 
>     > I tried backing them using dup, and restoring them using dup2. But this causes CRIU dump to fail. What is the
>     > proper way to do it? I have googled your mailing lists but I didn't found anything!
> 
>     Well, the shell job C/R (http://criu.org/Advanced_usage#Shell_jobs_C.2FR)
>     is pretty close to what you try to achieve.
> 
> 
> I have tried with files, and I stucked on a segmentation fault

Was that CRIU who segfaulted?

> So,
> ​ ​
> you are suggesting to use "--shell-job". I guess there must be a way to pass it on "criu_init_opts"?

criu_set_shell_job(true)

> 
>      *Capture:*
>     > Process A is my program. Then, it is forked, so we have B,
> 
>     Which is the complete copy of A.
> 
>     > in which you do your magic, so my program is captured. B is a "capturer". Right?
> 
>     Not exactly. The capturer in your case is criu service. And B is what is being captured.
> 
>     > So, when B continues, it does staff unrelated to my program, maybe some CRIU staff, and then it finally exits.
> 
>     No, it does what's written in your dumpApplication() function.
> 
>     > Then, process A, waits for the dump to be finished, and when this happens, it continues execution.
> 
>     Yes.
> 
> 
> So we fork B, so as to have all memory pages, etc copied to the new process, and then simply write all these data to images.

Well, yes, this is what happens, but it's not necessarily to fork B for that. It's your
dumpApplication() that does fork(), not libcriu :)

>     > Specifically, A will continue executing from line 30 here <https://gist.github.com/Paschalis/a96b2747ed85b8e5a796#file-linpack_h1_-c-L30>.
>     > ​Is that correct?
> 
>     No. The A will continue execution here: https://gist.github.com/Paschalis/a96b2747ed85b8e5a796#file-crlib-c-L86
> 
> 
> ​Yes, I meant that A will wait for the child B (crlib.c:86), make the chk_exit, and then return where it was before 
> the call of dumpApplication, and finally continue execution.​

Yes.

>     > Also, I have a question regarding command "criu_set_leave_running(true)".
>     > It will be executed by child B, right?
> 
>     Yes.
> 
>     > Why should I bother setting this, since B is the capturer?
> 
>     B is what being captured. Well, I think you can avoid setting this in B, in this
>     case criu would kill B after dump, A will wake up and continue running. After restore
>     everything should look the same.
> 
> 
> Can you provide a very simple use case scenario?

Sorry, scenario of what?

> For example, if I set leave_running, and instead of exit I do a return, then both A and B will execute the remaining
> of the execution? (the code after the dump)​

No, since A calls waitpid() it will wait for B to exit.

>     > Is C the "restorer"? (I am bit confused about this)
> 
>     No. C forks criu, criu forks D as C's child and restores B's state into D. So after
>     restore D is C's child and is 100% clone of B.
> 
> 
> ​Just to see whether I got this correctly.
> I launch C. Then C it is forked into CRIU.

Yes.

> Then CRIU it is forked into D, but its parent is C.

Yes.

> Then the dumped state (of B), is restored to D.

Exactly.

> Sorry for the multiple messages, and a big big thank you Pavel.
> As for the function-only capturing, I might need some direction, just to point me to the right source
> files so I can implement such feature.

I would be happy to help, but I don't understand what "function-only" capturing means. Can
you describe your scenario in more details?

Thanks,
Pavel