[CRIU] open files issue

[Sanidhya Kashyap]

Hello everyone,

I have been using criu to checkpoint memcached which works quite well but there
is an issue with the dumping of the open files - func: dump_task_files_seized in
cr-dump.c, specially dump_one_file in files.c. 	

The function dump_one_file actually takes malmost 99% of the time when doing
either single dump or multiple pre-dumps followed by a single dump. I have
attached the dump with this email.

I have some questions related to the dumping of the data:

1) Why does the open-files takes a lot of time in case of open connections? Is
this case only with socket connection or even others as well?

2) I keep on getting the error message like these:

(00.097203) Error (parasite-syscall.c:388): si_code=1 si_pid=2643 si_status=0
(00.111058) Error (parasite-syscall.c:388): si_code=1 si_pid=2644 si_status=0

(00.125807) Error (parasite-syscall.c:388): si_code=1 si_pid=2645 si_status=0
(00.138801) Error (parasite-syscall.c:388): si_code=1 si_pid=2646 si_status=0



What is the meaning of above messages?

3) As you can see that the total dump time is around 8.69 seconds out of which
the open files dumping lasted from 0.09 second to 8.29 second, which is VERY
VERY high. Thus, it will be very difficult for anyone to checkpoint for network
based applications as lot of sockets connections can be opened and this will
degrade the performance and the dump_one_file is saving the socket info and
writing appending iptables rules, specially in the live migration and seamless
kernel upgrade cases.
What are the optimal approaches to solve this issue either in the userspace or
even at the kernelspace level?

I am running this benchmark on 32 core machine (Intel(R) Xeon(R) CPU E5-2630 v3
@ 2.40GHz) with fedora 20 running kernel version 3.17.4-200.fc20.x86_64. The
memcached is running with 64GB RAM with 8 threads. The requests are generated
using memaslap with 4 threads and 256 concurrent connections on a gigabit
network. I have changed the ip address in the log for the sake of security.

I run the following command:

sudo criu dump -t `pgrep memcached` --tcp-established -j -D criu-dump -o
dump.log -v4

I shall be grateful if anyone can provide me some insight into this issue and
how to resolve this.

Thanks,
Sanidhya
-------------- next part --------------
A non-text attachment was scrubbed...
Name: dump.log
Type: text/x-log
Size: 779085 bytes
Desc: not available
URL: <http://lists.openvz.org/pipermail/criu/attachments/20141215/78ff594a/attachment-0001.bin>