[CRIU] [PATCH v2 12/14] files: Make tasks set their own service_fd_base

[Andrei Vagin]

> > diff --git a/test/zdtm/static/env00.c b/test/zdtm/static/env00.c
> > index 1feabfa9f..d2517b835 100644
> > --- a/test/zdtm/static/env00.c
> > +++ b/test/zdtm/static/env00.c
> > @@ -12,10 +12,23 @@ TEST_OPTION(envname, string, "environment variable name", 1);
> >  
> >  int main(int argc, char **argv)
> >  {
> > +	int i;
> > +	pid_t pid;
> >  	char *env;
> >  
> >  	test_init(argc, argv);
> >  
> > +	for (i = 0; i < 10; i++) {
> > +		pid = fork();
> > +		if (pid == 0) {
> > +			while (1)
> > +				sleep(1);
> > +			return 0;
> > +		}
> > +	}
> > +
> > +//	dup2(1, 1000000);
> > +
> >  	if (setenv(envname, test_author, 1)) {
> >  		pr_perror("Can't set env var \"%s\" to \"%s\"", envname, test_author);
> >  		exit(1);
> > 
> > before dump:
> > root at pro:/home/kirill# cat /sys/fs/cgroup/memory/test/memory.kmem.usage_in_bytes
> > 11849728
> > 
> > after restore:
> > root at pro:/home/kirill# cat /sys/fs/cgroup/memory/test/memory.kmem.usage_in_bytes
> > 20877312
> > 
> > Criu restore changes many in-kernel structures, and this is a global problem, not
> > about sfds.
> 
> In this case, the delta is 10MB. If you uncomment dup2(), the delta will
> be 100MB. This is not a global problem, it is the problem that we move
> service descriptors in child processes.

I create a small path for zdtm.py to get kmem before and after c/r:
git clone https://github.com/avagin/criu -b 2195

And here are steps how I do my experiments:

[root at fc24 criu]# rmdir /sys/fs/cgroup/memory/test/xxx
[root at fc24 criu]# ulimit -n 1024
[root at fc24 criu]# python test/zdtm.py run -t zdtm/static/env00 --sbs -f h --memcg /sys/fs/cgroup/memory/test/xxx
=== Run 1/1 ================ zdtm/static/env00

========================== Run zdtm/static/env00 in h ==========================
Start test
./env00 --pidfile=env00.pid --outfile=env00.out --envname=ENV_00_TEST
memory.kmem.usage_in_bytes = 9920512
Pause at pre-dump. Press any key to continue.
Run criu dump

Unable to kill 36: [Errno 3] No such process
Pause at pre-restore. Press any key to continue.Run criu restore
Pause at post-restore. Press any key to continue.
memory.kmem.usage_in_bytes = 22913024
before: memory.kmem.usage_in_bytes =              9920512
after:  memory.kmem.usage_in_bytes =             22913024 (230.97%)
Send the 15 signal to  36
Wait for zdtm/static/env00(36) to die for 0.100000
Unable to kill 36: [Errno 3] No such process
Removing dump/zdtm/static/env00/36
========================= Test zdtm/static/env00 PASS ==========================

[root at fc24 criu]# 
[root at fc24 criu]# ulimit -n $((1 << 20))
[root at fc24 criu]# rmdir /sys/fs/cgroup/memory/test/xxx
[root at fc24 criu]# python test/zdtm.py run -t zdtm/static/env00 --sbs -f h --memcg /sys/fs/cgroup/memory/test/xxx
=== Run 1/1 ================ zdtm/static/env00

========================== Run zdtm/static/env00 in h ==========================
Start test
./env00 --pidfile=env00.pid --outfile=env00.out --envname=ENV_00_TEST
memory.kmem.usage_in_bytes = 19714048
Pause at pre-dump. Press any key to continue.
Run criu dump
Unable to kill 36: [Errno 3] No such process
Pause at pre-restore. Press any key to continue.
Run criu restore
Pause at post-restore. Press any key to continue.
memory.kmem.usage_in_bytes = 914178048
before: memory.kmem.usage_in_bytes =             19714048
after:  memory.kmem.usage_in_bytes =            914178048 (4637.19%)
Send the 15 signal to  36
Wait for zdtm/static/env00(36) to die for 0.100000
Unable to kill 36: [Errno 3] No such process
Removing dump/zdtm/static/env00/36
========================= Test zdtm/static/env00 PASS ==========================

When we call ulimit -n 1024, dup2(1, 1000000) returns an error and
the test has only descriptors with small numbers. In this case,
a kmem consumption increases on 130%. We need to investigate this fact,
but it isn't so critical, if we compare it with the next case.

When we call ulimit -n $((1 << 20)) before running the test, dup2(1, 1000000)
creates the 1000000 descriptor, and a kmem consumption after c/r increases on
4537.19%. This is a real problem, what we have to solve with a high priority.

> 
> > 
> > >>  
> > >>  	/* Dup sfds in memmove() style: they may overlap */
> > >> -	if (get_service_fd(LOG_FD_OFF) > __get_service_fd(LOG_FD_OFF, id))
> > >> +	if (get_service_fd(LOG_FD_OFF) > new_base - LOG_FD_OFF - SERVICE_FD_MAX * id)
> > >>  		for (i = SERVICE_FD_MIN + 1; i < SERVICE_FD_MAX; i++)
> > >>  			move_service_fd(me, i, id, new_base);
> > >>  	else
> > >>  		for (i = SERVICE_FD_MAX - 1; i > SERVICE_FD_MIN; i--)
> > >>  			move_service_fd(me, i, id, new_base);
> > >>  
> > >> +	service_fd_base = new_base;
> > >>  	service_fd_id = id;
> > >>  	ret = 0;
> > 
> > Kirill
> _______________________________________________
> CRIU mailing list
> CRIU at openvz.org
> https://lists.openvz.org/mailman/listinfo/criu