On 08/18/2015 12:55 AM, Hui Kang wrote:
> On Mon, Aug 17, 2015 at 10:48 AM, Pavel Emelyanov <xemul at parallels.com> wrote:
>> On 08/17/2015 08:07 AM, Hui Kang wrote:
>>> On Sun, Aug 16, 2015 at 9:20 PM, Pavel Emelyanov <xemul at parallels.com> wrote:
>>>> On 08/13/2015 07:41 PM, Hui Kang wrote:
>>>>> Skip these top cgroup directories: cpu, cpuset, cpuacct, memory
>>>>>
>>>>> Signed-off-by: Hui Kang <hkang.sunysb at gmail.com>
>>>>> ---
>>>>> cgroup.c | 4 ++++
>>>>> 1 file changed, 4 insertions(+)
>>>>>
>>>>> diff --git a/cgroup.c b/cgroup.c
>>>>> index a4e0146..94749b1 100644
>>>>> --- a/cgroup.c
>>>>> +++ b/cgroup.c
>>>>> @@ -992,6 +992,10 @@ static int prepare_cgroup_dir_properties(char *path, int off, CgroupDirEntry **e
>>>>> CgroupDirEntry *e = ents[i];
>>>>> size_t off2 = off;
>>>>>
>>>>> + if (strcmp(e->dir_name, "") == 0 &&
>>>>> + ((strstr(path, "cpu")) || (strstr(path, "memory"))))
>>>>> + continue;
>>>>> +
>>>>
>>>> With this "continue" won't we skip all the properties restore? The e-s we
>>>> have here represent all the tree, starting from root and descending. If you
>>>> skip the root one, then the whole subtree is skipped too.
>>>
>>> No. Although the e-s represent the tree, the for loops will iterate
>>> all the directories.
>>
>> The loop iterates top-level directories, then dives into children. Can you show
>> the contents of your cgroup.img file?
>
> {
> "magic": "CGROUP",
> "entries": [
Ah, I see. The thing is that in your case you don't have nested directories. In
particular -- cpuset is only the "foo/foo1" dir, so your check doesn't match
one and it's restored. The cpu/cpuacct are both "/" only, so you check matches
them, skips, but we notice nothing as there are no sub-directories.
And for the same reason restore-special-props works OK -- the cpuset controller
is "foo/foo1", so it gets restored successfully.
So my concern about the wrong place for strcmp(name, "") check still stands.
-- Pavel
> {
> "sets": [
> {
> "id": 2,
> "ctls": [
> {
> "name": "blkio",
> "path": "/"
> },
> {
> "name": "cpu",
> "path": "/"
> },
> {
> "name": "cpuacct",
> "path": "/"
> },
> {
> "name": "cpuset",
> "path": "/foo/foo1"
> },
> {
> "name": "devices",
> "path": "/"
> },
> {
> "name": "freezer",
> "path": "/"
> },
> {
> "name": "hugetlb",
> "path": "/"
> },
> {
> "name": "memory",
> "path": "/"
> },
> {
> "name": "name=systemd",
> "path": "/"
> },
> {
> "name": "perf_event",
> "path": "/"
> }
> ]
> }
> ],
> "controllers": [
> {
> "cnames": [
> "cpuset"
> ],
> "dirs": [
> {
> "dir_name": "foo/foo1",
> "properties": [
> {
> "name": "cpuset.cpus",
> "value": "1"
> },
> {
> "name": "cpuset.mems",
> "value": "0"
> },
> {
> "name": "cpuset.memory_migrate",
> "value": "0"
> },
> {
> "name": "cpuset.cpu_exclusive",
> "value": "0"
> },
> {
> "name": "cpuset.mem_exclusive",
> "value": "0"
> },
> {
> "name": "cpuset.mem_hardwall",
> "value": "0"
> },
> {
> "name": "cpuset.memory_spread_page",
> "value": "0"
> },
> {
> "name": "cpuset.memory_spread_slab",
> "value": "0"
> },
> {
> "name": "cpuset.sched_load_balance",
> "value": "1"
> },
> {
> "name": "cpuset.sched_relax_domain_level",
> "value": "-1"
> },
> {
> "name": "notify_on_release",
> "value": "0"
> }
> ]
> }
> ]
> },
> {
> "cnames": [
> "cpu"
> ],
> "dirs": [
> {
> "dir_name": "",
> "properties": [
> {
> "name": "cpu.shares",
> "value": "1024"
> },
> {
> "name": "cpu.cfs_period_us",
> "value": "100000"
> },
> {
> "name": "cpu.cfs_quota_us",
> "value": "-1"
> },
> {
> "name": "notify_on_release",
> "value": "0"
> }
> ]
> }
> ]
> },
> {
> "cnames": [
> "cpuacct"
> ],
> "dirs": [
> {
> "dir_name": ""
> }
> ]
> },
> {
> "cnames": [
> "memory"
> ],
> "dirs": [
> {
> "dir_name": "",
> "properties": [
> {
> "name": "memory.limit_in_bytes",
> "value": "-1"
> },
> {
> "name": "memory.use_hierarchy",
> "value": "0"
> },
> {
> "name": "notify_on_release",
> "value": "0"
> }
> ]
> }
> ]
> },
> {
> "cnames": [
> "devices"
> ],
> "dirs": [
> {
> "dir_name": ""
> }
> ]
> },
> {
> "cnames": [
> "freezer"
> ],
> "dirs": [
> {
> "dir_name": "",
> "properties": [
> {
> "name": "notify_on_release",
> "value": "0"
> }
> ]
> }
> ]
> },
> {
> "cnames": [
> "blkio"
> ],
> "dirs": [
> {
> "dir_name": "",
> "properties": [
> {
> "name": "blkio.weight",
> "value": "1000"
> },
> {
> "name": "notify_on_release",
> "value": "0"
> }
> ]
> }
> ]
> },
> {
> "cnames": [
> "perf_event"
> ],
> "dirs": [
> {
> "dir_name": ""
> }
> ]
> },
> {
> "cnames": [
> "hugetlb"
> ],
> "dirs": [
> {
> "dir_name": ""
> }
> ]
> },
> {
> "cnames": [
> "name=systemd"
> ],
> "dirs": [
> {
> "dir_name": ""
> }
> ]
> }
> ]
> }
> ]
> }
>
>>
>>> With "full" mode, the top "cpu" and "mem"
>>> directories should be skipped. The first condition
>>> (strcmp(e->dir_name, "") guarantees that non-root directory will be
>>> restored. For a dumped process with cgroup tree "cpuset/foo/foo1". The
>>> restore process with "full" mode is like
>>>
>>> (00.046731) cg: Restored cgroup property value 1 to cpuset/foo/foo1/cpuset.cpus
>>>
>>> (00.046803) cg: Restored cgroup property value 0 to cpuset/foo/foo1/cpuset.mems
>>>
>>> ... (after all foo1 properties are restore, the it moves to the top level)
>>>
>>> (00.047287) cg: ---> Skip Empty name for top path cpu , top dir
>>>
>>> As you can see, the patch will only skip those root directories.
>>>
>>> One side observations is that the property values of the intermediate
>>> level (e.g., cpuset/foo) is not dumpped. I confirmed this by running
>>> crit decode -i cgroup.img --pretty. So when restoring, I have to
>>> manually create cpuset/foo and populate the value. Is this a bug of
>>> CRIU?
>>
>> CRIU only messes with the cgroups tasks live in, so if they don't live
>> in foo/ then foo will not be restored. And now it seems strange to me
>> that the root one is being restored by criu.
>
> When you use "--mange-cgroup=full", criu will try to restore
> /sys/fs/cgroup/cpusets/cpuset.cpus, which is not supposed to be
> written.
> But this will not happen with the default "soft" mode; the code sets
> n_properties = 0.
>
> if (opts.manage_cgroups & (CG_MODE_SOFT | CG_MODE_NONE)) {
> pr_info("Skip restoring properties on
> cgroup dir %s\n", paux);
> if (e->n_properties > 0) {
> xfree(e->properties);
> e->properties = NULL;
> e->n_properties = 0;
> }
> }
>
> - Hui
>
>>
>> -- Pavel
>>
> .
>