<div dir="ltr">Hi, Tycho<br><div><div class="gmail_extra"><br><div class="gmail_quote">On Wed, Aug 5, 2015 at 9:39 AM, Tycho Andersen <span dir="ltr"><<a href="mailto:tycho.andersen@canonical.com" target="_blank">tycho.andersen@canonical.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">Hi Hui,<br>
<span class=""><br>
On Tue, Aug 04, 2015 at 11:15:25PM -0400, Hui Kang wrote:<br>
> Hi, Tycho<br>
</span><span class="">> I created a patch<br>
> (<a href="http://lists.openvz.org/pipermail/criu/2015-August/021576.html" rel="noreferrer" target="_blank">http://lists.openvz.org/pipermail/criu/2015-August/021576.html</a>) and<br>
> pass cgrou_manage as CG_MODE_FULL to the criu service. From the<br>
> restore.log, I can see that criu will smartly create all the cgroup<br>
> directories. However, the restore still fails at this cpu of wrting to<br>
><br>
> (cgroup.c:901): cg: Can't move into cpuset//[PID]/tasks (-1/0): No<br>
> space left on device<br>
> (00.082249) Error (cr-restore.c:1919): Restoring FAILED.<br>
><br>
> As Tycho pointed that:<br>
><br>
> >> > ENOSPC Attempted to write(2) the process ID (PID) of a process<br>
> >> > to a cpuset<br>
> >> > tasks file when the cpuset had an empty cpuset.cpus or empty<br>
> >> > cpuset.mems<br>
> >> > setting.<br>
> >> ><br>
> >> > ENOSPC Attempted to write(2) an empty cpuset.cpus or cpuset.mems<br>
> >> > setting to a<br>
> >> > cpuset that has tasks attached.<br>
<br>
</span>There are a few different things going on here,<br>
<span class=""><br>
> I verify that cpuset.cpus (created by CRIU full cg manage) in the<br>
> destination host is empty, while cpuset.cpus in the source host is<br>
> "0-3".<br>
><br>
> Further, when I attemped to write "0-3" to the cpuset.cpus in the<br>
> destination host before restoring, it gave permission error.<br>
><br>
> echo "0-3" > /sys/fs/cgroup/cpuset/[PID]/cpuset.cpus<br>
> -bash: echo: write error: Permission denied<br>
<br>
</span>You're probably not root here, so you need to do something like:<br>
<br>
$ echo "0-3" | sudo tee /sys/fs/cgroup/cpuset/foo/cpuset.cpus<br>
0-3<br>
<br>
otherwise you'll get EPERM.<br></blockquote><div><br></div><div>I tried both sudo and root to restore and write to the cpuset.cpus file; both failed and gave me the error<br><br>tee: /sys/fs/cgroup/cpuset/foo/cpuset.cpus: Permission denied<br></div><div> </div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<span class=""><br>
> Should CRIU write some default value to the cpuset.cpus file when<br>
> creating the director? This could help restoring processes with cgroup<br>
> across host. If so, I would be more than happy to work on a patch.<br>
> Thanks.</span> </blockquote><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><span class="">
<br>
</span>It should already try to do this in restore_special_cpuset_props(),<br>
although it seems that bit isn't working for you. If you want to<br>
debug, that's probably a good place to start.<br></blockquote><div><br></div><div>In fact the restoration of cgroup on a different host fails in the function restore_cgroup_prop. The error message I got is <br><br>prepare_cgroup_dirs<br>(00.001362) cg: Created cgroup dir cpuset/foo<br>(00.001429) Error (cgroup.c:978): cg: Failed closing cpuset/foo/cpuset.cpus: Permission denied<br>(00.001441) Error (cgroup.c:1085): cg: Restoring special cpuset props failed!<br><br></div><div> Then I changed chmod 777 to cpuset/foot and got those "permission denied" or "no space left" error.<br><br><br></div><div>I am guessing that current criu implementation failed to propogate the cgroup dir across host. Have you ever test this? Thanks.<br></div><div><br></div><div>- Hui<br></div><div><br></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<span class="HOEnZb"><font color="#888888"><br>
Tycho<br>
</font></span></blockquote></div><br></div></div></div>