[CRIU] race dumping fds in wily LXC containers

[Pavel Emelyanov]

On 07/16/2015 05:06 PM, Tycho Andersen wrote:
> On Thu, Jul 16, 2015 at 12:41:40PM +0300, Pavel Emelyanov wrote:
>> On 07/15/2015 11:39 PM, Tycho Andersen wrote:
>>> On Wed, Jul 15, 2015 at 01:01:15PM +0300, Pavel Emelyanov wrote:
>>>>
>>>>> Right, this fstat() fails because the previous readlink() returned a
>>>>> "(deleted)", i.e. rpath in the fstatat() call in check_path_remap()
>>>>> has this "(deleted)".
>>>>
>>>> No, we don't check for "(deleted)" to take any decisions. Only stat/fstat
>>>> results comparisons. The only thing we do with it is strip one from the
>>>> file path if it's there :)
>>>
>>> Not to make any decisions, but the problem here is that the call in
>>> check_path_remap() looks like:
>>>
>>> fstatat(mntns_root, "/sys/fs/cgroup/systemd/lxc/w1-4 (deleted)", &pst, 0);
>>>
>>> which doesn't work.
>>
>> Exactly! The fstatat() reports there's no such file _name_ in the system.
>> Since there's the "(deleted)" suffix in the name it means, that kernel
>> sees the respective dentry as unhashed. This happens when one unlink-s
>> the file.
>>
>> But if you say that the name wl-4 is there, this means that there's _one_
>> _more_ file with the same name in the tree.
>>
>> You can see this situation with this steps:
>>
>> term-1 $ cat > x
>> term-2 $ ls -l /proc/$(pidof cat)/fd
>> ...
>> ... 1 -> /home/x
>>
>> # so the cat has 1 pointing to /home/x, that's correct.
>>
>> term-2 $ ln x y
>> term-2 $ rm -f x
>> term-2 $ ls -l /proc/$(pidof cat)/fd
>> ...
>> ... 1 -> /home/x (deleted)
>>
>> # now x is deleted and criu will dump it as link remap since there's y
>> # name holding the inode and providing the n_link being 1 to the file
>>
>> term-2 $ touch x
>> #or you can do
>> term-2 $ ln y x
>>
>> # now we have created another file named x, in the former case -- just
>> # a new file, in the latter -- another name for the same inode as old
>> # x used to have %)
>>
>> term-2 $ ls -l /proc/$(pidof cat)fd
>> ...
>> ... 1 -> /home/x (deleted)
>>
>> # See? The cat's x file is still considered to be deleted as the dentry
>> # that is used by cat was unhashed with the rm -f command. And even if
>> # there's a new x file in /home, the cat's 1 descriptor should still be
>> # treated as unlinked.
> 
> Right, except in this case if I readlink the fd that criu is failing
> on from /proc after criu fails, it doesn't have a "(deleted)", so I

Wait, in my example the "(deleted)" is always there. And in your case
you also see the "(deleted)" one:

   fstatat(mntns_root, "/sys/fs/cgroup/systemd/lxc/w1-4 (deleted)", &pst, 0);

> think it's not been unlinked (indeed it can't be unlinked, lxcfs
> doesn't implement unlink). Truly perplexing :(

Well, it's not necessarily has to be unlink(). Anything that unhashes the dentry,
it can also be rename() or some internal filesystem thing that replaces one
dentry with another (d_revalidate for example).

-- Pavel