[CRIU] [PATCH CRIU 2/7] pre-dump: add dirty memory statistics logger thread

[Pavel Tikhomirov]

Sending pages to page server is the longest part of predump iteration and
only here(and short time between criu dumps in p.haul) checkpointed program
is not freezed so I think that's all we have.

To estimate: Walking one time across 0,5Gb of pages takes aproximately
~0.01 sec; on 10 Gbit/sec network the transfer time will be ~ 0.4 sec. So
we will have approximately 3 such measurements. On my node for local
transfer I got >= 10 measurements in test: "test/zdtm.py run --page-server
--pre 1 -k always -t zdtm/static/maps04".


Best Regards, Tikhomirov Pavel.

2016-04-12 14:12 GMT+03:00 Pavel Emelyanov <xemul at virtuozzo.com>:

> On 04/07/2016 04:28 PM, Pavel Tikhomirov wrote:
> > On memory predumping target process is unfreezed so it can change
> > its memory, criu uses kernel soft-dirty tracking mechanism to find
> > regions of unchanged memory and can skip dumping unchanged pages
> > in iterative migration. Introduce dirty_logger_pthread to get
> > number of dirty memory pages and all pages. Thus we will be
> > able to analize memory iterative migration flow.
> >
> > move free_pstree to the end of cr_pre_dump_finish as will need it in
> > dirty_logger_pthread
> >
> > Signed-off-by: Pavel Tikhomirov <ptikhomirov at virtuozzo.com>
> > ---
> >  criu/Makefile.crtools       |  1 +
> >  criu/cr-dump.c              | 36 +++++++++++++++++++++++++++++++++++-
> >  criu/dirty-logger.c         |  6 ++++++
> >  criu/include/dirty-logger.h |  6 ++++++
> >  4 files changed, 48 insertions(+), 1 deletion(-)
> >  create mode 100644 criu/dirty-logger.c
> >  create mode 100644 criu/include/dirty-logger.h
> >
> > diff --git a/criu/Makefile.crtools b/criu/Makefile.crtools
> > index ef152d2..666a906 100644
> > --- a/criu/Makefile.crtools
> > +++ b/criu/Makefile.crtools
> > @@ -13,6 +13,7 @@ obj-y                       += cr-exec.o
> >  obj-y                        += cr-restore.o
> >  obj-y                        += cr-service.o
> >  obj-y                        += crtools.o
> > +obj-y                        += dirty-logger.o
> >  obj-y                        += eventfd.o
> >  obj-y                        += eventpoll.o
> >  obj-y                        += fault-injection.o
> > diff --git a/criu/cr-dump.c b/criu/cr-dump.c
> > index ade2f31..f188c1e 100644
> > --- a/criu/cr-dump.c
> > +++ b/criu/cr-dump.c
> > @@ -22,6 +22,8 @@
> >  #include <sched.h>
> >  #include <sys/resource.h>
> >
> > +#include <pthread.h>
> > +
> >  #include "protobuf.h"
> >  #include "images/fdinfo.pb-c.h"
> >  #include "images/fs.pb-c.h"
> > @@ -80,6 +82,7 @@
> >  #include "seccomp.h"
> >  #include "seize.h"
> >  #include "fault-injection.h"
> > +#include "dirty-logger.h"
> >
> >  #include "asm/dump.h"
> >
> > @@ -1426,15 +1429,25 @@ static int setup_alarm_handler()
> >  static int cr_pre_dump_finish(struct list_head *ctls, int ret)
> >  {
> >       struct parasite_ctl *ctl, *n;
> > +     pthread_t dirty_logger;
> > +     int pthread_ret, arg = 0;
> >
> >       pstree_switch_state(root_item, TASK_ALIVE);
> > -     free_pstree(root_item);
> >
> >       timing_stop(TIME_FROZEN);
> >
> >       if (ret < 0)
> >               goto err;
> >
> > +     if (opts.log_dirty) {
> > +             pr_info("Start dirty memory logger on pre-dumping tasks'
> memory\n");
> > +             pthread_ret = pthread_create(&dirty_logger, NULL,
> dirty_logger_pthread, (void *)&arg);
> > +             if (pthread_ret) {
> > +                     pr_err("Pthread creation of dirty_logger_pthread
> failed: %d\n", pthread_ret);
> > +                     return -1;
> > +             }
> > +     }
> > +
> >       pr_info("Pre-dumping tasks' memory\n");
> >       list_for_each_entry_safe(ctl, n, ctls, pre_list) {
> >               struct page_xfer xfer;
> > @@ -1477,6 +1490,27 @@ static int cr_pre_dump_finish(struct list_head
> *ctls, int ret)
> >               write_stats(DUMP_STATS);
> >               pr_info("Pre-dumping finished successfully\n");
> >       }
> > +
> > +     if (opts.log_dirty) {
> > +             pr_info("Stop dirty memory logger\n");
> > +             pthread_ret = pthread_cancel(dirty_logger);
> > +             if (pthread_ret && pthread_ret != ESRCH) {
> > +                     pr_err("Pthread cancel of dirty_logger_pthread
> failed: %d\n", pthread_ret);
> > +                     ret = -1;
> > +                     goto exit;
> > +             }
> > +             pthread_ret = pthread_join(dirty_logger, NULL);
> > +             if (pthread_ret) {
> > +                     pr_perror("Pthread join of dirty_logger_pthread
> failed: %d\n", pthread_ret);
> > +                     ret = -1;
> > +                     goto exit;
> > +             }
> > +             if (arg == -1)
> > +                     pr_perror("Pthread dirty_logger_pthread failed\n");
>
> You kill this thread right after pre-dump finish, so this thread only
> works when
> the pre-dump is in action. This is quite short period of time, is it
> enough?
>
> > +     }
> > +
> > +exit:
> > +     free_pstree(root_item);
> >       return ret;
> >  }
> >
> > diff --git a/criu/dirty-logger.c b/criu/dirty-logger.c
> > new file mode 100644
> > index 0000000..6620626
> > --- /dev/null
> > +++ b/criu/dirty-logger.c
> > @@ -0,0 +1,6 @@
> > +#include <pthread.h>
> > +
> > +void *dirty_logger_pthread(void *arg)
> > +{
> > +     pthread_exit(NULL);
> > +}
> > diff --git a/criu/include/dirty-logger.h b/criu/include/dirty-logger.h
> > new file mode 100644
> > index 0000000..312c44b
> > --- /dev/null
> > +++ b/criu/include/dirty-logger.h
> > @@ -0,0 +1,6 @@
> > +#ifndef __CR_DIRTY_LOGGER_H__
> > +#define __CR_DIRTY_LOGGER_H__
> > +
> > +extern void *dirty_logger_pthread(void *arg);
> > +
> > +#endif
> >
>
> _______________________________________________
> CRIU mailing list
> CRIU at openvz.org
> https://lists.openvz.org/mailman/listinfo/criu
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openvz.org/pipermail/criu/attachments/20160526/c04b8f11/attachment-0001.html>