<div dir="ltr">Sending pages to page server is the longest part of predump iteration and only here(and short time between criu dumps in p.haul) checkpointed program is not freezed so I think that's all we have.<br><br>To estimate: Walking one time across 0,5Gb of pages takes aproximately ~0.01 sec; on 10 Gbit/sec network the transfer time will be ~ 0.4 sec. So we will have approximately 3 such measurements. On my node for local transfer I got >= 10 measurements in test: "test/zdtm.py run --page-server --pre 1 -k always -t zdtm/static/maps04".<br></div><div class="gmail_extra"><br clear="all"><div><div class="gmail_signature"><div dir="ltr"><br>Best Regards, Tikhomirov Pavel.</div></div></div>
<br><div class="gmail_quote">2016-04-12 14:12 GMT+03:00 Pavel Emelyanov <span dir="ltr"><<a href="mailto:xemul@virtuozzo.com" target="_blank">xemul@virtuozzo.com</a>></span>:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div class="HOEnZb"><div class="h5">On 04/07/2016 04:28 PM, Pavel Tikhomirov wrote:<br>
> On memory predumping target process is unfreezed so it can change<br>
> its memory, criu uses kernel soft-dirty tracking mechanism to find<br>
> regions of unchanged memory and can skip dumping unchanged pages<br>
> in iterative migration. Introduce dirty_logger_pthread to get<br>
> number of dirty memory pages and all pages. Thus we will be<br>
> able to analize memory iterative migration flow.<br>
><br>
> move free_pstree to the end of cr_pre_dump_finish as will need it in<br>
> dirty_logger_pthread<br>
><br>
> Signed-off-by: Pavel Tikhomirov <<a href="mailto:ptikhomirov@virtuozzo.com">ptikhomirov@virtuozzo.com</a>><br>
> ---<br>
> criu/Makefile.crtools | 1 +<br>
> criu/cr-dump.c | 36 +++++++++++++++++++++++++++++++++++-<br>
> criu/dirty-logger.c | 6 ++++++<br>
> criu/include/dirty-logger.h | 6 ++++++<br>
> 4 files changed, 48 insertions(+), 1 deletion(-)<br>
> create mode 100644 criu/dirty-logger.c<br>
> create mode 100644 criu/include/dirty-logger.h<br>
><br>
> diff --git a/criu/Makefile.crtools b/criu/Makefile.crtools<br>
> index ef152d2..666a906 100644<br>
> --- a/criu/Makefile.crtools<br>
> +++ b/criu/Makefile.crtools<br>
> @@ -13,6 +13,7 @@ obj-y += cr-exec.o<br>
> obj-y += cr-restore.o<br>
> obj-y += cr-service.o<br>
> obj-y += crtools.o<br>
> +obj-y += dirty-logger.o<br>
> obj-y += eventfd.o<br>
> obj-y += eventpoll.o<br>
> obj-y += fault-injection.o<br>
> diff --git a/criu/cr-dump.c b/criu/cr-dump.c<br>
> index ade2f31..f188c1e 100644<br>
> --- a/criu/cr-dump.c<br>
> +++ b/criu/cr-dump.c<br>
> @@ -22,6 +22,8 @@<br>
> #include <sched.h><br>
> #include <sys/resource.h><br>
><br>
> +#include <pthread.h><br>
> +<br>
> #include "protobuf.h"<br>
> #include "images/fdinfo.pb-c.h"<br>
> #include "images/fs.pb-c.h"<br>
> @@ -80,6 +82,7 @@<br>
> #include "seccomp.h"<br>
> #include "seize.h"<br>
> #include "fault-injection.h"<br>
> +#include "dirty-logger.h"<br>
><br>
> #include "asm/dump.h"<br>
><br>
> @@ -1426,15 +1429,25 @@ static int setup_alarm_handler()<br>
> static int cr_pre_dump_finish(struct list_head *ctls, int ret)<br>
> {<br>
> struct parasite_ctl *ctl, *n;<br>
> + pthread_t dirty_logger;<br>
> + int pthread_ret, arg = 0;<br>
><br>
> pstree_switch_state(root_item, TASK_ALIVE);<br>
> - free_pstree(root_item);<br>
><br>
> timing_stop(TIME_FROZEN);<br>
><br>
> if (ret < 0)<br>
> goto err;<br>
><br>
> + if (opts.log_dirty) {<br>
> + pr_info("Start dirty memory logger on pre-dumping tasks' memory\n");<br>
> + pthread_ret = pthread_create(&dirty_logger, NULL, dirty_logger_pthread, (void *)&arg);<br>
> + if (pthread_ret) {<br>
> + pr_err("Pthread creation of dirty_logger_pthread failed: %d\n", pthread_ret);<br>
> + return -1;<br>
> + }<br>
> + }<br>
> +<br>
> pr_info("Pre-dumping tasks' memory\n");<br>
> list_for_each_entry_safe(ctl, n, ctls, pre_list) {<br>
> struct page_xfer xfer;<br>
> @@ -1477,6 +1490,27 @@ static int cr_pre_dump_finish(struct list_head *ctls, int ret)<br>
> write_stats(DUMP_STATS);<br>
> pr_info("Pre-dumping finished successfully\n");<br>
> }<br>
> +<br>
> + if (opts.log_dirty) {<br>
> + pr_info("Stop dirty memory logger\n");<br>
> + pthread_ret = pthread_cancel(dirty_logger);<br>
> + if (pthread_ret && pthread_ret != ESRCH) {<br>
> + pr_err("Pthread cancel of dirty_logger_pthread failed: %d\n", pthread_ret);<br>
> + ret = -1;<br>
> + goto exit;<br>
> + }<br>
> + pthread_ret = pthread_join(dirty_logger, NULL);<br>
> + if (pthread_ret) {<br>
> + pr_perror("Pthread join of dirty_logger_pthread failed: %d\n", pthread_ret);<br>
> + ret = -1;<br>
> + goto exit;<br>
> + }<br>
> + if (arg == -1)<br>
> + pr_perror("Pthread dirty_logger_pthread failed\n");<br>
<br>
</div></div>You kill this thread right after pre-dump finish, so this thread only works when<br>
the pre-dump is in action. This is quite short period of time, is it enough?<br>
<div class="HOEnZb"><div class="h5"><br>
> + }<br>
> +<br>
> +exit:<br>
> + free_pstree(root_item);<br>
> return ret;<br>
> }<br>
><br>
> diff --git a/criu/dirty-logger.c b/criu/dirty-logger.c<br>
> new file mode 100644<br>
> index 0000000..6620626<br>
> --- /dev/null<br>
> +++ b/criu/dirty-logger.c<br>
> @@ -0,0 +1,6 @@<br>
> +#include <pthread.h><br>
> +<br>
> +void *dirty_logger_pthread(void *arg)<br>
> +{<br>
> + pthread_exit(NULL);<br>
> +}<br>
> diff --git a/criu/include/dirty-logger.h b/criu/include/dirty-logger.h<br>
> new file mode 100644<br>
> index 0000000..312c44b<br>
> --- /dev/null<br>
> +++ b/criu/include/dirty-logger.h<br>
> @@ -0,0 +1,6 @@<br>
> +#ifndef __CR_DIRTY_LOGGER_H__<br>
> +#define __CR_DIRTY_LOGGER_H__<br>
> +<br>
> +extern void *dirty_logger_pthread(void *arg);<br>
> +<br>
> +#endif<br>
><br>
<br>
</div></div><div class="HOEnZb"><div class="h5">_______________________________________________<br>
CRIU mailing list<br>
<a href="mailto:CRIU@openvz.org">CRIU@openvz.org</a><br>
<a href="https://lists.openvz.org/mailman/listinfo/criu" rel="noreferrer" target="_blank">https://lists.openvz.org/mailman/listinfo/criu</a><br>
</div></div></blockquote></div><br></div>