[CRIU] irmap: Can't stat /no-such-path: No such file or directory

[Tycho Andersen]

On Wed, Sep 09, 2015 at 02:50:22PM +0300, Pavel Emelyanov wrote:
> On 09/08/2015 07:05 PM, Tycho Andersen wrote:
> > On Tue, Sep 08, 2015 at 06:59:01PM +0300, Pavel Emelyanov wrote:
> >> On 09/08/2015 05:49 PM, Tycho Andersen wrote:
> >>> On Tue, Sep 08, 2015 at 05:34:47PM +0300, Pavel Emelyanov wrote:
> >>>> On 09/08/2015 04:47 PM, Tycho Andersen wrote:
> >>>>
> >>>>>>> http://paste.ubuntu.com/12279228/
> >>>>>>>
> >>>>>>> It seems to me like we shouldn't be trying to stat no-such-path, but
> >>>>>>> that seems like the symptom of a larger problem which is that we can't
> >>>>>>> otherwise resolve the handle.
> >>>>>>>
> >>>>>>> I resolved the inode using debugfs, and it says it's /var/log in the
> >>>>>>> container, which is not one of our heuristic paths we check:
> >>>>>>>
> >>>>>>> # debugfs -R 'ncheck 525799' /dev/vda1
> >>>>>>> debugfs 1.42.12 (29-Aug-2014)
> >>>>>>> Inode Pathname
> >>>>>>> 525799  /var/lib/lxd/containers/unpriv/rootfs/var/log
> >>>>>>>
> >>>>>>> If I add this path (and several others, see the attached patch) to the
> >>>>>>> list of irmap hints, it works fine. Obviously this isn't sustainable
> >>>>>>> though; is there some nicer solution to this?
> >>>>>>
> >>>>>> Well, right now we don't have any :( The problem here is that *notify
> >>>>>> objects are assigned on inode and kernel can drop all the path information
> >>>>>> from it. So once we meet an *notified inode we _try_ to ask kernel for
> >>>>>> path and, if it fails, have no other ways to resolve it back.
> >>>>>
> >>>>> Hmm, ok. Can we not ask debugfs like I did above? Or is it possible
> >>>>> that debugfs would also not know?
> >>>>
> >>>> Debugfs would also do what we do in irmap -- it will scan the paths
> >>>> from root comparing the inode numbers. But unlike debugfs irmap scans
> >>>> the tree only ones keeping the path-to-inode mappings, so it (in
> >>>> theory) can be asked several times.
> >>>>
> >>>>> I can see two potential solutions in that case: 1. we could just
> >>>>> hardcode these paths (as they're needed by systemd bits) or 2. we
> >>>>> could add a --irmap-path argument, to allow users to specify which
> >>>>> paths are interesting and need to be checked. I'm happy to send a
> >>>>> patch for whatever you think is appropriate.
> >>>>
> >>>> An option would be good :) But the paths are better to eventually be
> >>>> shared between you, us and others, so they will slowly get hardcoded
> >>>> into irmap-cache.c.
> >>>
> >>> What about another option, --irmap-deep-scan which sets /.'s nr_kids
> >>> to -1, so that the whole fs will be scanned if there is one of these
> >>> unknown paths? If we take the initial patch I posted, things will
> >>> still be fast for current systemd, but with --irmap-deep-scan a dump
> >>> will still succeed, it will just be slower.
> >>
> >> Yes, this is possible. But...
> >>
> >>> We can have --irmap-deep-scan print out a warning recommending that
> >>> any new path potentially be added to the hardcoded list if it is
> >>> system wide?
> >>
> >> ... I would disable this deep scan for dump, only for pre-dump. Scanning
> >> the whole tree can be really really slow operation, doing it having processes
> >> frozen is not nice.
> > 
> > I wonder if we can make it a fallback, so that if it's not found in
> > one of the hinted locations, we can do a full scan if
> > --irmap-deep-scan is passed. The situation I'm trying to avoid is that
> > someone's dump fails because we didn't do the full scan when it was
> > necessary, because it would be slow in the common case where.
> 
> Doing full-scan with tasks frozen can become a destructive operation (e.g. sockets
> can start to time-out). So doing full scan at pre-dump time is OK even w/o the
> option, doing full scan dump time -- only under non recommended option :)

I see. Sounds like we should really implement pre-dump support in LXC
:)

> >> And the option to add hint (until it's hard-coded) is also useful to let
> >> people live w/o full-scan between releases.
> > 
> > Ok, I'll add a --irmap-scan-loc. Should we also apply the first patch
> > as well?
> 
> The patch looks nice, I'll apply one.

Ok, I'll send the one about --irmap-scan-path soon, haven't had time
to actually test it yet.

Tycho