[CRIU] [P.haul + Docker] Missing Running States During Live Migration of Container

[Lele Ma]

On Mon, Feb 20, 2017 at 2:37 PM, Pavel Emelyanov <xemul at virtuozzo.com> wrote:
> On 02/20/2017 10:23 PM, Lele Ma wrote:
>>
>> On Mon, Feb 20, 2017 at 1:16 PM, Pavel Emelyanov <xemul at virtuozzo.com <mailto:xemul at virtuozzo.com>> wrote:
>>
>>     On 02/19/2017 10:50 PM, Lele Ma wrote:
>>     > Hi All,
>>     >
>>     > I am testing container live migration with this github repos <https://github.com/boucher/docker/tree/v1.10_2-16-16-experimental <https://github.com/boucher/docker/tree/v1.10_2-16-16-experimental>> for docker-1.10-dev. I found the container not restored exactly where it's checkpointed. For example:
>>     >
>>     > The container I run
>>     >      docker run  -d busybox  /bin/sh -c 'echo > /foo; max=1000000; i=0; while [ $i -lt $max ] ; do date >> /foo; date +%s >> /foo; echo "i=$i" >> /foo; i=$(expr $i + 1 ); sleep 0.0001; done'
>>     >
>>     > After migrated using p.haul, I got the /foo in target node:
>>     > .....
>>     > Sun Feb 19 03:23:13 UTC 2017
>>     > 1487474593
>>     > i=4247
>>     > Sun Feb 19 03:23:13 UTC 2017
>>     > 1487474593
>>     > i=4248                       -----> before migration
>>     > i=7545                       -----> after migartion ( it is supposed to be i=4249 )
>>     > Sun Feb 19 03:23:20 UTC 2017
>>     > 1487474600
>>     > i=7546
>>     > Sun Feb 19 03:23:20 UTC 2017
>>     > 1487474600
>>     > i=7547
>>     > ......
>>     > The printed numbers jump from 'i=4248' to 'i=7545' instead of increasing by one. It seems that it ignores
>>     > some computation status of the docker containers. But I am not sure where it goes wrong. However, when I
>>     > checkpoint and restore the container locally, the number increase continuously with no such jumping.
>>
>>     Where do you get these numbers from? Docker console or some file on disk?
>>
>>
>> It's from the file '/foo' inside container. ( The container is running /bin/sh -c 'echo > /foo;
>> max=1000000; i=0; while [ $i -lt $max ] ; do date >> /foo; date +%s >> /foo; echo "i=$i" >> /foo;
>> i=$(expr $i + 1 ); sleep 0.0001; done' )
>
> Then this is likely a race between images sync and filesystem sync.
> You can check your /foo file on the source node right after container
> migration, it should contain the missing numbers :)
>
> What p.haul do you use, btw?

Thank you. But how can we avoid the race? I am using this repo from
Ross Boucher: https://github.com/boucher/p.haul/tree/docker-1.10

Best,
Lele