<div dir="ltr"><br><div class="gmail_extra"><br><div class="gmail_quote">On Tue, Aug 18, 2015 at 11:13 AM, Pavel Emelyanov <span dir="ltr"><<a href="mailto:xemul@parallels.com" target="_blank">xemul@parallels.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div class="HOEnZb"><div class="h5">On 08/18/2015 12:55 AM, Hui Kang wrote:<br>
> On Mon, Aug 17, 2015 at 10:48 AM, Pavel Emelyanov <<a href="mailto:xemul@parallels.com">xemul@parallels.com</a>> wrote:<br>
>> On 08/17/2015 08:07 AM, Hui Kang wrote:<br>
>>> On Sun, Aug 16, 2015 at 9:20 PM, Pavel Emelyanov <<a href="mailto:xemul@parallels.com">xemul@parallels.com</a>> wrote:<br>
>>>> On 08/13/2015 07:41 PM, Hui Kang wrote:<br>
>>>>> Skip these top cgroup directories: cpu, cpuset, cpuacct, memory<br>
>>>>><br>
>>>>> Signed-off-by: Hui Kang <<a href="mailto:hkang.sunysb@gmail.com">hkang.sunysb@gmail.com</a>><br>
>>>>> ---<br>
>>>>> cgroup.c | 4 ++++<br>
>>>>> 1 file changed, 4 insertions(+)<br>
>>>>><br>
>>>>> diff --git a/cgroup.c b/cgroup.c<br>
>>>>> index a4e0146..94749b1 100644<br>
>>>>> --- a/cgroup.c<br>
>>>>> +++ b/cgroup.c<br>
>>>>> @@ -992,6 +992,10 @@ static int prepare_cgroup_dir_properties(char *path, int off, CgroupDirEntry **e<br>
>>>>> CgroupDirEntry *e = ents[i];<br>
>>>>> size_t off2 = off;<br>
>>>>><br>
>>>>> + if (strcmp(e->dir_name, "") == 0 &&<br>
>>>>> + ((strstr(path, "cpu")) || (strstr(path, "memory"))))<br>
>>>>> + continue;<br>
>>>>> +<br>
>>>><br>
>>>> With this "continue" won't we skip all the properties restore? The e-s we<br>
>>>> have here represent all the tree, starting from root and descending. If you<br>
>>>> skip the root one, then the whole subtree is skipped too.<br>
>>><br>
>>> No. Although the e-s represent the tree, the for loops will iterate<br>
>>> all the directories.<br>
>><br>
>> The loop iterates top-level directories, then dives into children. Can you show<br>
>> the contents of your cgroup.img file?<br>
><br>
> {<br>
> "magic": "CGROUP",<br>
> "entries": [<br>
<br>
</div></div>Ah, I see. The thing is that in your case you don't have nested directories. In<br>
particular -- cpuset is only the "foo/foo1" dir, so your check doesn't match<br>
one and it's restored. The cpu/cpuacct are both "/" only, so you check matches<br>
them, skips, but we notice nothing as there are no sub-directories.<br></blockquote><div><br></div><div>HI, Pavel,<br></div><div>can you give me an example which has nested directories? I added the process to foo/foo1/tasks and then dump the process. So no value in foo is dumped. <br><br></div><div>For a nested cgroup directory, do you mean I should also add a parent tasks to foo and then dump both parent and child?<br><br> <br></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<br>
And for the same reason restore-special-props works OK -- the cpuset controller<br>
is "foo/foo1", so it gets restored successfully.<br>
<br>
So my concern about the wrong place for strcmp(name, "") check still stands.<br></blockquote><div><br></div><div>If you could give me an counter example like nested directory, it will helpful to fix this check. Thanks.<br><br></div><div>- Hui<br></div><div> </div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<span class="HOEnZb"><font color="#888888"><br>
-- Pavel<br>
</font></span><div><div class="h5"><br>
> {<br>
> "sets": [<br>
> {<br>
> "id": 2,<br>
> "ctls": [<br>
> {<br>
> "name": "blkio",<br>
> "path": "/"<br>
> },<br>
> {<br>
> "name": "cpu",<br>
> "path": "/"<br>
> },<br>
> {<br>
> "name": "cpuacct",<br>
> "path": "/"<br>
> },<br>
> {<br>
> "name": "cpuset",<br>
> "path": "/foo/foo1"<br>
> },<br>
> {<br>
> "name": "devices",<br>
> "path": "/"<br>
> },<br>
> {<br>
> "name": "freezer",<br>
> "path": "/"<br>
> },<br>
> {<br>
> "name": "hugetlb",<br>
> "path": "/"<br>
> },<br>
> {<br>
> "name": "memory",<br>
> "path": "/"<br>
> },<br>
> {<br>
> "name": "name=systemd",<br>
> "path": "/"<br>
> },<br>
> {<br>
> "name": "perf_event",<br>
> "path": "/"<br>
> }<br>
> ]<br>
> }<br>
> ],<br>
> "controllers": [<br>
> {<br>
> "cnames": [<br>
> "cpuset"<br>
> ],<br>
> "dirs": [<br>
> {<br>
> "dir_name": "foo/foo1",<br>
> "properties": [<br>
> {<br>
> "name": "cpuset.cpus",<br>
> "value": "1"<br>
> },<br>
> {<br>
> "name": "cpuset.mems",<br>
> "value": "0"<br>
> },<br>
> {<br>
> "name": "cpuset.memory_migrate",<br>
> "value": "0"<br>
> },<br>
> {<br>
> "name": "cpuset.cpu_exclusive",<br>
> "value": "0"<br>
> },<br>
> {<br>
> "name": "cpuset.mem_exclusive",<br>
> "value": "0"<br>
> },<br>
> {<br>
> "name": "cpuset.mem_hardwall",<br>
> "value": "0"<br>
> },<br>
> {<br>
> "name": "cpuset.memory_spread_page",<br>
> "value": "0"<br>
> },<br>
> {<br>
> "name": "cpuset.memory_spread_slab",<br>
> "value": "0"<br>
> },<br>
> {<br>
> "name": "cpuset.sched_load_balance",<br>
> "value": "1"<br>
> },<br>
> {<br>
> "name": "cpuset.sched_relax_domain_level",<br>
> "value": "-1"<br>
> },<br>
> {<br>
> "name": "notify_on_release",<br>
> "value": "0"<br>
> }<br>
> ]<br>
> }<br>
> ]<br>
> },<br>
> {<br>
> "cnames": [<br>
> "cpu"<br>
> ],<br>
> "dirs": [<br>
> {<br>
> "dir_name": "",<br>
> "properties": [<br>
> {<br>
> "name": "cpu.shares",<br>
> "value": "1024"<br>
> },<br>
> {<br>
> "name": "cpu.cfs_period_us",<br>
> "value": "100000"<br>
> },<br>
> {<br>
> "name": "cpu.cfs_quota_us",<br>
> "value": "-1"<br>
> },<br>
> {<br>
> "name": "notify_on_release",<br>
> "value": "0"<br>
> }<br>
> ]<br>
> }<br>
> ]<br>
> },<br>
> {<br>
> "cnames": [<br>
> "cpuacct"<br>
> ],<br>
> "dirs": [<br>
> {<br>
> "dir_name": ""<br>
> }<br>
> ]<br>
> },<br>
> {<br>
> "cnames": [<br>
> "memory"<br>
> ],<br>
> "dirs": [<br>
> {<br>
> "dir_name": "",<br>
> "properties": [<br>
> {<br>
> "name": "memory.limit_in_bytes",<br>
> "value": "-1"<br>
> },<br>
> {<br>
> "name": "memory.use_hierarchy",<br>
> "value": "0"<br>
> },<br>
> {<br>
> "name": "notify_on_release",<br>
> "value": "0"<br>
> }<br>
> ]<br>
> }<br>
> ]<br>
> },<br>
> {<br>
> "cnames": [<br>
> "devices"<br>
> ],<br>
> "dirs": [<br>
> {<br>
> "dir_name": ""<br>
> }<br>
> ]<br>
> },<br>
> {<br>
> "cnames": [<br>
> "freezer"<br>
> ],<br>
> "dirs": [<br>
> {<br>
> "dir_name": "",<br>
> "properties": [<br>
> {<br>
> "name": "notify_on_release",<br>
> "value": "0"<br>
> }<br>
> ]<br>
> }<br>
> ]<br>
> },<br>
> {<br>
> "cnames": [<br>
> "blkio"<br>
> ],<br>
> "dirs": [<br>
> {<br>
> "dir_name": "",<br>
> "properties": [<br>
> {<br>
> "name": "blkio.weight",<br>
> "value": "1000"<br>
> },<br>
> {<br>
> "name": "notify_on_release",<br>
> "value": "0"<br>
> }<br>
> ]<br>
> }<br>
> ]<br>
> },<br>
> {<br>
> "cnames": [<br>
> "perf_event"<br>
> ],<br>
> "dirs": [<br>
> {<br>
> "dir_name": ""<br>
> }<br>
> ]<br>
> },<br>
> {<br>
> "cnames": [<br>
> "hugetlb"<br>
> ],<br>
> "dirs": [<br>
> {<br>
> "dir_name": ""<br>
> }<br>
> ]<br>
> },<br>
> {<br>
> "cnames": [<br>
> "name=systemd"<br>
> ],<br>
> "dirs": [<br>
> {<br>
> "dir_name": ""<br>
> }<br>
> ]<br>
> }<br>
> ]<br>
> }<br>
> ]<br>
> }<br>
><br>
>><br>
>>> With "full" mode, the top "cpu" and "mem"<br>
>>> directories should be skipped. The first condition<br>
>>> (strcmp(e->dir_name, "") guarantees that non-root directory will be<br>
>>> restored. For a dumped process with cgroup tree "cpuset/foo/foo1". The<br>
>>> restore process with "full" mode is like<br>
>>><br>
>>> (00.046731) cg: Restored cgroup property value 1 to cpuset/foo/foo1/cpuset.cpus<br>
>>><br>
>>> (00.046803) cg: Restored cgroup property value 0 to cpuset/foo/foo1/cpuset.mems<br>
>>><br>
>>> ... (after all foo1 properties are restore, the it moves to the top level)<br>
>>><br>
>>> (00.047287) cg: ---> Skip Empty name for top path cpu , top dir<br>
>>><br>
>>> As you can see, the patch will only skip those root directories.<br>
>>><br>
>>> One side observations is that the property values of the intermediate<br>
>>> level (e.g., cpuset/foo) is not dumpped. I confirmed this by running<br>
>>> crit decode -i cgroup.img --pretty. So when restoring, I have to<br>
>>> manually create cpuset/foo and populate the value. Is this a bug of<br>
>>> CRIU?<br>
>><br>
>> CRIU only messes with the cgroups tasks live in, so if they don't live<br>
>> in foo/ then foo will not be restored. And now it seems strange to me<br>
>> that the root one is being restored by criu.<br>
><br>
> When you use "--mange-cgroup=full", criu will try to restore<br>
> /sys/fs/cgroup/cpusets/cpuset.cpus, which is not supposed to be<br>
> written.<br>
> But this will not happen with the default "soft" mode; the code sets<br>
> n_properties = 0.<br>
><br>
> if (opts.manage_cgroups & (CG_MODE_SOFT | CG_MODE_NONE)) {<br>
> pr_info("Skip restoring properties on<br>
> cgroup dir %s\n", paux);<br>
> if (e->n_properties > 0) {<br>
> xfree(e->properties);<br>
> e->properties = NULL;<br>
> e->n_properties = 0;<br>
> }<br>
> }<br>
><br>
> - Hui<br>
><br>
>><br>
>> -- Pavel<br>
>><br>
</div></div>> .<br>
><br>
<br>
</blockquote></div><br></div></div>