[CRIU] link_remap_ok
beproject criu
beprojectcriu at gmail.com
Fri Feb 27 13:50:30 PST 2015
I have edited mount.c. Patch is attached with the mail.
1. Where really is your container's root? Is it node's / or is it
/usr/local/lib/lxc/rootfs/L
or w/o "L" at the end of previous path? Or where? And how does it get
there -- with the
chroot() call or with the pivot_root() one?
----- host root: "/" container's root: "/sdcard/L/rootfs" -> for pivot
root it is mounted at "/usr/local/lib/lxc/rootfs"
2. How did you manage to get several rootfs mountpoints inside container?
Do you use
somehow patched kernel?
not sure. may be host's mounpoints are visible inside the container? There
was a discussion about this a while back on CRIU
https://lists.linuxcontainers.org/pipermail/lxc-devel/2014-October/010571.html
/sdcard # ls -ld /proc/self/root/
drwxrwxr-x 15 1001 1001 340 Feb 27 13:15 /proc/self/root/
/sdcard # lxc-attach -n L -- ls -ld /proc/self/root/
drwxrwxr-x 15 1001 1001 340 Feb 27 13:15 /proc/self/root/
On Sat, Feb 28, 2015 at 2:13 AM, Pavel Emelyanov <xemul at parallels.com>
wrote:
> On 02/27/2015 11:09 PM, beproject criu wrote:
> > Is this ok?
>
> Yes, that's OK and this ... is even more strange than I expected %)
> But anyway, let's try to debug that.
>
> > /sdcard # lxc-attach -n L -- cat /proc/self/mountinfo
> > 16 16 0:2 / / rw - rootfs rootfs rw,size=372900k,nr_inodes=93225
> > 17 16 0:4 / /proc rw,relatime - proc proc rw
> > 18 16 0:11 / /sys rw,relatime - sysfs sysfs rw
> > 19 18 0:12 / /sys/fs/cgroup rw,relatime - cgroup none
> rw,cpuset,debug,cpu,cpuacct,memory,devices,freezer,blkio,perf_event,clone_children
> > 20 18 0:6 / /sys/kernel/debug rw,relatime - debugfs none rw
> > 21 16 0:10 / /dev/pts rw,relatime - devpts devpts
> rw,mode=600,ptmxmode=000
> > 22 16 179:0 / /sdcard rw,relatime - ext2 /dev/mmcblk0 rw
> > 24 16 0:2 /usr/local/lib/lxc/rootfs /usr/local/lib/lxc/rootfs rw -
> rootfs rootfs rw,size=372900k,nr_inodes=93225
> > 25 24 0:14 / /usr/local/lib/lxc/rootfs rw,relatime - tmpfs none
> rw,size=12k,mode=755
> > 35 25 179:0 /L/rootfs /usr/local/lib/lxc/rootfs/root rw,relatime
> shared:1 - ext2 /dev/mmcblk0 rw
> > 36 35 0:13 / /usr/local/lib/lxc/rootfs/root/proc rw,relatime shared:2 -
> proc none rw
> > 37 35 0:15 / /usr/local/lib/lxc/rootfs/root/sys rw,relatime shared:3 -
> sysfs none rw
> > 38 35 0:2 /dev /usr/local/lib/lxc/rootfs/root/dev rw,relatime shared:6 -
> rootfs rootfs rw,size=372900k,nr_inodes=93225
> > 39 38 0:10 /1 /usr/local/lib/lxc/rootfs/root/dev/console rw,relatime
> shared:7 - devpts devpts rw,mode=600,ptmxmode=000
> > 26 37 0:6 / /usr/local/lib/lxc/rootfs/root/sys/kernel/debug rw,relatime
> shared:4 - debugfs debugfs rw
> > 27 37 0:16 / /usr/local/lib/lxc/rootfs/root/sys/fs/cgroup rw,relatime
> shared:5 - tmpfs none rw,mode=750,gid=1000
> > 28 35 0:17 / /usr/local/lib/lxc/rootfs/root/mnt/asec rw,relatime
> shared:8 - tmpfs tmpfs rw,mode=755,gid=1000
> > 29 35 0:18 / /usr/local/lib/lxc/rootfs/root/mnt/obb rw,relatime shared:9
> - tmpfs tmpfs rw,mode=755,gid=1000
> > 30 35 31:0 / /usr/local/lib/lxc/rootfs/root/system ro,relatime shared:10
> - ext4 /dev/block/mtdblock0 ro,data=ordered
> > 31 35 31:1 / /usr/local/lib/lxc/rootfs/root/data rw,nosuid,nodev,noatime
> shared:11 - ext4 /dev/block/mtdblock1 rw,data=ordered
> > 32 35 31:2 / /usr/local/lib/lxc/rootfs/root/cache
> rw,nosuid,nodev,noatime shared:12 - ext4 /dev/block/mtdblock2
> rw,data=ordered
>
> OK, so if we put all bits together I see that some process inside
> container has a file
> named /usr/local/lib/lxc/rootfs/root/dev/__properties__ opened and this
> file is really
> present inside container. According to the mountpoints this path should
> exist.
>
> Next, CRIU tries to stat this file relative to what it thinks is your
> container's root
> (the 1017-th descriptor). And gets the ENOENT error, which in turn means,
> that the
> 1017 points not to container's root.
>
> Need to find out why. But things that I don't understand and need your
> help (since I
> don't have your box's console at hands) are:
>
> 1. Where really is your container's root? Is it node's / or is it
> /usr/local/lib/lxc/rootfs/L
> or w/o "L" at the end of previous path? Or where? And how does it get
> there -- with the
> chroot() call or with the pivot_root() one?
>
> 2. How did you manage to get several rootfs mountpoints inside container?
> Do you use
> somehow patched kernel?
>
> 3. In theory this fd should point to container's namespace's root, which
> _should_
> be invisible in criu's namespace and, thus, have the path "/". But it's
> not such.
> What if you find your container's root task as seen from host and do
> this:
>
> ls -ld /proc/$the-pid-in-question/root
>
> ? What path would you see?
>
> Thanks,
> Pavel
>
> > /sdcard # cat /proc/self/mountinfo
> > 1 1 0:2 / / rw - rootfs rootfs rw,size=372900k,nr_inodes=93225
> > 10 1 0:4 / /proc rw,relatime - proc proc rw
> > 11 1 0:11 / /sys rw,relatime - sysfs sysfs rw
> > 12 11 0:12 / /sys/fs/cgroup rw,relatime - cgroup none
> rw,cpuset,debug,cpu,cpuacct,memory,devices,freezer,blkio,perf_event,clone_children
> > 13 11 0:6 / /sys/kernel/debug rw,relatime - debugfs none rw
> > 14 1 0:10 / /dev/pts rw,relatime - devpts devpts rw,mode=600,ptmxmode=000
> > 15 1 179:0 / /sdcard rw,relatime - ext2 /dev/mmcblk0 rw
> > /sdcard #
> >
> >
> > I have this line in my lxc.conf
> > lxc.mount.entry=/dev /sdcard/L/rootfs/dev none defaults,bind 0 0
> >
> >
> > Thanks.
> >
> > On Sat, Feb 28, 2015 at 1:20 AM, Pavel Emelyanov <xemul at parallels.com
> <mailto:xemul at parallels.com>> wrote:
> >
> > On 02/27/2015 08:05 PM, beproject criu wrote:
> > > path [/usr/local/lib/lxc/rootfs/root] read_fd_link() return : 30
> > > I have mounted the /dev of host on /dev inside my container.
> >
> > That's interesting. How did you do it? Can you show me the mount
> namespaces
> > layout on your node and inside the container? It's just
> /proc/$pid/mountinfo
> > file contents on some process on host and some process in CT.
> >
> > > (03.347637) Dumping path for 4 fd via self 43
> [/usr/local/lib/lxc/rootfs/root/dev/__properties__]
> >
> > OK, this is quite strange. The /usr/local/lib/lxc/rootfs is the path
> on HOST.
> > How did this path become visible inside container?
> >
> > > (03.347847) [nyc_fd] The required root is already opened.
> get_service_fd() returns : 1017
> > > (03.348197) [nyc_fd] Going into fstatat()-> [mntns_root :
> 1017],[rpath : ./usr/local/lib/lxc/rootfs/root/dev/__properties__]
> > > (03.348393) [nyc_fd] Out of fstatat()-> [ret : -1],[rpath :
> ./usr/local/lib/lxc/rootfs/root/dev/__properties__]
> > > (03.348583) [nyc_fd] Going into dump_linked_remap()
> > > (03.349177) [nyc_fd] The required root is already opened.
> get_service_fd() returns : 1017
> > > (03.349432) [nyc_fd] Doing linkat() [mntns_root : 1017],[link_name
> : ./usr/local/lib/lxc/rootfs/root/dev/link_remap.4]
> > >
> > > (03.349756) [nyc_fd] bad_path [/usr/local/lib/lxc/rootfs/root]
> ,read_fd_link() return : 30
> > >
> > > (03.350027) Error (files-reg.c:515): Can't link remap to
> /usr/local/lib/lxc/rootfs/root/dev/__properties__: No such file or directory
> > > (03.351097) [nyc_fd] Error in dump_one_reg_file->check_path_remap()
> > > (03.351266) [nyc_fd] Going into dump_task_files_seized() :
> dump_one_file()
> > >
> > >
> > > On Fri, Feb 27, 2015 at 8:10 PM, Pavel Emelyanov <
> xemul at parallels.com <mailto:xemul at parallels.com> <mailto:
> xemul at parallels.com <mailto:xemul at parallels.com>>> wrote:
> > >
> > > On 02/27/2015 05:25 PM, beproject criu wrote:
> > > > This is the flow before error,could you get what's wrong :
> > > >
> > > > (02.352922) Dumping path for 4 fd via self 43
> [/usr/local/lib/lxc/rootfs/root/dev/__properties__]
> > > > (02.353079) [nyc_fd] The required root is already opened.
> get_service_fd() returns : 1017
> > > > (02.353216) [nyc_fd] Going into fstatat()-> [mntns_root :
> 1017],[rpath : ./usr/local/lib/lxc/rootfs/root/dev/__properties__]
> > > > (02.353398) [nyc_fd] Out of fstatat()-> [ret : -1],[rpath :
> ./usr/local/lib/lxc/rootfs/root/dev/__properties__]
> > > > (02.353574) [nyc_fd] Going into dump_linked_remap()
> > > > (02.354191) [nyc_fd] The required root is already opened.
> get_service_fd() returns : 1017
> > > > (02.354431) [nyc_fd] Doing linkat() [mntns_root :
> 1017],[link_name : ./usr/local/lib/lxc/rootfs/root/dev/link_remap.4]
> > > > (02.354747) Error (files-reg.c:510): Can't link remap to
> /usr/local/lib/lxc/rootfs/root/dev/__properties__: No such file or directory
> > > > (02.355915) [nyc_fd] Error in
> dump_one_reg_file->check_path_remap()
> > > > (02.356066) [nyc_fd] Going into dump_task_files_seized() :
> dump_one_file()
> > >
> > > OK, so in both cases we access file af fd 1017 and sub-path
> "./user/local/lib/lxc/roootfs/root/dev/__properties__"
> > > and you tell that this file actually exists in container,
> right?
> > >
> > > The failing fstatat then means, that the 1017 descriptor
> points to some bad path. Can you check where? We
> > > have a helper called read_fd_link() helper for that, see the
> fsnotify.c line 455 for code example.
> > >
> > > And one more question -- does your container live in another
> mount namespace, or shared one with host?
> > >
> > > Thanks,
> > > Pavel
> > >
> > >
> >
> >
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openvz.org/pipermail/criu/attachments/20150228/b88f294f/attachment-0001.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: chroot-ns.patch.tar.gz
Type: application/x-gzip
Size: 948 bytes
Desc: not available
URL: <http://lists.openvz.org/pipermail/criu/attachments/20150228/b88f294f/attachment-0001.gz>
More information about the CRIU
mailing list