[CRIU] [PATCH 0/5] pipes: support for packetized mode (with O_DIRECT)

[Stanislav Kinsburskiy]

15.12.2015 21:16, Pavel Emelyanov пишет:
> On 12/15/2015 11:11 PM, Stanislav Kinsburskiу wrote:
>> 15 дек. 2015 г. 20:39 пользователь Pavel Emelyanov <xemul at parallels.com> написал:
>>> On 12/15/2015 08:55 PM, Stanislav Kinsburskiy wrote:
>>>> There is something, I would like to discuss.
>>>> Currently, only _write_ end as O_DIRECT flag. Criu uses _read_ end to
>>>> tee data from it to local pipe. There is a problem, _how_ to discover,
>>>> whether pipe is "packetized" or not. This knowledge is required, because
>>>> splice() can be used only for non-packetized pipes (otherwise all
>>>> individual packets will be merged to one in image file).
>>>>
>>>> But, I'm afraid, it's not enough... Because "packetized" pipe mode is
>>>> not something stable, because it's represented by file flag. Regular
>>>> pipe can be reopened with O_DIRECT flag or vise versa, but packets,
>>>> which were sent, _when_ pipe was _"packetized"_, are already _marked_ as
>>>> _"non-mergeable"_.
>>>> So, probably, in generic case, it's not enough to create a pipe on
>>>> restore, fill it with data and set correct file flags, because pipe can
>>>> contain both "packets" and regular content.
>>> I wouldn't say it's typical use for pipes. Can we support only those
>>> that have all ends in the same state -- either packetized or not?
>>>
>> No, it's not a typical way for sure.
>> But I don't know yet, how to distinguish...
>> One of major problems, as I mentioned, is that only _write_ end is marked with O_DIRECT.
>> What to do, if we have a bunch of pipe ends (inherited, whatever)?
>> Should we gather them somehow on dump and check, that all write ends are either with O_DIRECT or without?
> Yes.


I need some help with this. To make the above check, all the pipes have 
to be collected, somehow organized and checked.
To do so, all the processes have to infected first, then pipes have to 
be collected.
With current code, all this pipes are collected during dump.
There are some problems because of it:
1) Pipe data is dumped before all the pipes are collected. Thus, if we 
have pipe with O_DIRECT, but we found read end first, we can't discover, 
whether it's packetized pipe or not.
2) There could be packetized pipe with only read end. In this case we 
screwed.

The first problem can be solved by collecting all the pipes before 
actual dump. But then it's a significant rework of the existent code, 
and I'm not sure it worth it...
The second problem I don't know how to fix yet and would be glad to hear 
any proposals.

Maybe it's better to stop this O_DIRECT pipes migration quest, leave it 
as unsupported and in case of autofs pipes collect them somewhere on 
"parse" mount call, and use this information to dump only those 
packetized pipes (because we anyway have a restriction, that they must 
be empty)?

>> ?Probably, it makes sense to send a patch, which also marks read end with O_DIRECT on creation?
> No, read-end seems to be unaffected by this.
>
>> If it will be aacepted, then, at least, we can put a restriction, that we treat pipes mode as
>> static (either always paketized of never packetized). What do you think?
>> Or we need some way to ask, which packet type is the next in the queue... This will allow to
>> handle any pipes, but then we can't use splice and have to change pipe type during its content restore if it contains different buffers.
>>
>> But frankly, from my POW, packet pipe design was broken from early beginning.
>> Man page says, that if pipe is packetized, reader can be sure, that incomming packet won't be
>> bigger than page size. I can easily violate this rule by reopening pipe without O_DIRECT, and
>> writing more than page into it...
> Sure. As glibc former maintainer once said: "giving people a rope to shoot themselves
> is a Unix tradition".
>
> -- Pavel