[CRIU] criu_restore() in Open MPI problems

[Pavel Emelyanov]

On 04/29/2014 03:30 PM, Adrian Reber wrote:

>>> Another thought. I am running mpirun which starts opal-restart which
>>> execvp()'s 'criu restore'. opal-restart's FDs (stdout/stderr) are
>>> already connected to the correct pipes of mpirun. Is there a way for
>>> criu to inherit those FDs. So instead of re-opening stdout/stderr it
>>> just uses the FDs 'criu restore' already uses? So maybe the plugin just
>>> detects that at the same FD there is already a pipe and just re-uses
>>> that pipe?
>>
>> I think that plugin should detect that we're having a PIPE via environment
>> and use one on restore. What do you think, would that be possible?
> 
> I am now exporting the IDs of the pipes from Open MPI and can read
> it in criu via getenv().

Great!

> I see pipes.c:305 open_pipe() in which criu wants to restore the
> pipes with the old ID.. Would it be enough to replace pi->pe->pipe_id
> with the new IDs of the pipes or does it require more. 

I'm not sure that _replacing_ would work as these IDs are used to
bind shared pipes together (the pipe_list list) to properly resolve
sharing of file descriptors. Also the pipe_id is also used by the
restore_pipe_date.

If we handle both it, then it should be enough.

> At what place would a plugin need to be placed to detect that an
> environment variable dos exist and replace the old pipe IDs with the
> new pipe IDs specified in the environment variable.

The beginning of the open_pipe() seem to be the correct place. Can we
mark pipe restored with plugins with a special boolean bit in the image
to be 100% sure we do things right, as we do for files and mounts?

Thanks,
Pavel