<div dir="ltr">Hi Pavel,<div><br></div><div>my test env and test results are as follows</div><div><div style="font-size:12.8000001907349px"><div style="overflow:hidden"><div lang="ZH-CN" link="#0563C1" vlink="#954F72"><p class="MsoNormal" style="margin:0cm 0cm 0.0001pt;text-align:justify;font-size:10.5pt;font-family:Calibri,sans-serif"><span lang="EN-US">ENV:<u></u><u></u></span></p><p class="MsoNormal" style="margin:0cm 0cm 0.0001pt;text-align:justify;font-size:10.5pt;font-family:Calibri,sans-serif"><span lang="EN-US">tcp client on host: 192.168.9.170<u></u><u></u></span></p><p class="MsoNormal" style="margin:0cm 0cm 0.0001pt;text-align:justify;font-size:10.5pt;font-family:Calibri,sans-serif"><span lang="EN-US">tcp server in docker: 192.168.9.129<u></u><u></u></span></p><p class="MsoNormal" style="margin:0cm 0cm 0.0001pt;text-align:justify;font-size:10.5pt;font-family:Calibri,sans-serif"><span lang="EN-US">udp client in docker: 192.168.9.129<u></u><u></u></span></p><p class="MsoNormal" style="margin:0cm 0cm 0.0001pt;text-align:justify;font-size:10.5pt;font-family:Calibri,sans-serif"><span lang="EN-US">udp server on host: 192.168.9.170<u></u><u></u></span></p><p class="MsoNormal" style="margin:0cm 0cm 0.0001pt;text-align:justify;font-size:10.5pt;font-family:Calibri,sans-serif"><span lang="EN-US"><u></u> <u></u></span></p><p class="MsoNormal" style="margin:0cm 0cm 0.0001pt;text-align:justify;font-size:10.5pt;font-family:Calibri,sans-serif"><span lang="EN-US">migrate docker container from host 192.168.9.170 to host 192.168.9.180<u></u><u></u></span></p><p class="MsoNormal" style="margin:0cm 0cm 0.0001pt;text-align:justify;font-size:10.5pt;font-family:Calibri,sans-serif"><span lang="EN-US"><u></u> <u></u></span></p><p class="MsoNormal" style="margin:0cm 0cm 0.0001pt;text-align:justify;font-size:10.5pt;font-family:Calibri,sans-serif"><span lang="EN-US">Summarize:<u></u><u></u></span></p><p class="MsoNormal" style="margin:0cm 0cm 0.0001pt;text-align:justify;font-size:10.5pt;font-family:Calibri,sans-serif"><span lang="EN-US">migration time: checkpoint+restore = 0.569+1.709 = 2.278s<u></u><u></u></span></p><p class="MsoNormal" style="margin:0cm 0cm 0.0001pt;text-align:justify;font-size:10.5pt;font-family:Calibri,sans-serif"><span lang="EN-US">existing socket hang time: 3.6s, actually is 3.6-2.278 = <b>1.322s</b>, except migration downtime<u></u><u></u></span></p><p class="MsoNormal" style="margin:0cm 0cm 0.0001pt;text-align:justify;font-size:10.5pt;font-family:Calibri,sans-serif"><span lang="EN-US"><u></u> <u></u></span></p><p class="MsoNormal" style="margin:0cm 0cm 0.0001pt;text-align:justify;font-size:10.5pt;font-family:Calibri,sans-serif"><span lang="EN-US">In tcpdump file timestamp is 3.941260 - 7.619094 for tcp, timestamp 4.012874 - 7.619208 for udp<u></u><u></u></span></p><p class="MsoNormal" style="margin:0cm 0cm 0.0001pt;text-align:justify;font-size:10.5pt;font-family:Calibri,sans-serif"><span lang="EN-US"><u></u> <u></u></span></p><p class="MsoNormal" style="margin:0cm 0cm 0.0001pt;text-align:justify;font-size:10.5pt;font-family:Calibri,sans-serif"><span lang="EN-US">---<u></u><u></u></span></p><p class="MsoNormal" style="margin:0cm 0cm 0.0001pt;text-align:justify;font-size:10.5pt;font-family:Calibri,sans-serif"><span lang="EN-US"><a href="mailto:root@192.168.9.170:~#" target="_blank">root@192.168.9.170:~#</a> time ./dockercrnfs.sh 192.168.9.180 00dd5314605f<u></u><u></u></span></p><p class="MsoNormal" style="margin:0cm 0cm 0.0001pt;text-align:justify;font-size:10.5pt;font-family:Calibri,sans-serif"><span lang="EN-US">start checkpoint: 2015-07-23 00:44:26.057426748<u></u><u></u></span></p><p class="MsoNormal" style="margin:0cm 0cm 0.0001pt;text-align:justify;font-size:10.5pt;font-family:Calibri,sans-serif"><span lang="EN-US">00dd5314605f1952a727d662860ca38b9d4c4a46761f74d445a8a4a2d8343ee2<u></u><u></u></span></p><p class="MsoNormal" style="margin:0cm 0cm 0.0001pt;text-align:justify;font-size:10.5pt;font-family:Calibri,sans-serif"><span lang="EN-US"><u></u> <u></u></span></p><p class="MsoNormal" style="margin:0cm 0cm 0.0001pt;text-align:justify;font-size:10.5pt;font-family:Calibri,sans-serif"><span lang="EN-US">real 0m0.569s<u></u><u></u></span></p><p class="MsoNormal" style="margin:0cm 0cm 0.0001pt;text-align:justify;font-size:10.5pt;font-family:Calibri,sans-serif"><span lang="EN-US">user 0m0.011s<u></u><u></u></span></p><p class="MsoNormal" style="margin:0cm 0cm 0.0001pt;text-align:justify;font-size:10.5pt;font-family:Calibri,sans-serif"><span lang="EN-US">sys 0m0.000s<u></u><u></u></span></p><p class="MsoNormal" style="margin:0cm 0cm 0.0001pt;text-align:justify;font-size:10.5pt;font-family:Calibri,sans-serif"><span lang="EN-US"><u></u> <u></u></span></p><p class="MsoNormal" style="margin:0cm 0cm 0.0001pt;text-align:justify;font-size:10.5pt;font-family:Calibri,sans-serif"><span lang="EN-US">end checkpoint: 2015-07-23 00:44:26.637416792<u></u><u></u></span></p><p class="MsoNormal" style="margin:0cm 0cm 0.0001pt;text-align:justify;font-size:10.5pt;font-family:Calibri,sans-serif"><span lang="EN-US"><u></u> <u></u></span></p><p class="MsoNormal" style="margin:0cm 0cm 0.0001pt;text-align:justify;font-size:10.5pt;font-family:Calibri,sans-serif"><span lang="EN-US">start restore on host <a href="http://192.168.9.180/" target="_blank">192.168.9.180</a>: 2015-07-23 00:44:27.893451353<u></u><u></u></span></p><p class="MsoNormal" style="margin:0cm 0cm 0.0001pt;text-align:justify;font-size:10.5pt;font-family:Calibri,sans-serif"><span lang="EN-US">00dd5314605f1952a727d662860ca38b9d4c4a46761f74d445a8a4a2d8343ee2<u></u><u></u></span></p><p class="MsoNormal" style="margin:0cm 0cm 0.0001pt;text-align:justify;font-size:10.5pt;font-family:Calibri,sans-serif"><span lang="EN-US"><u></u> <u></u></span></p><p class="MsoNormal" style="margin:0cm 0cm 0.0001pt;text-align:justify;font-size:10.5pt;font-family:Calibri,sans-serif"><span lang="EN-US">real 0m1.709s<u></u><u></u></span></p><p class="MsoNormal" style="margin:0cm 0cm 0.0001pt;text-align:justify;font-size:10.5pt;font-family:Calibri,sans-serif"><span lang="EN-US">user 0m0.011s<u></u><u></u></span></p><p class="MsoNormal" style="margin:0cm 0cm 0.0001pt;text-align:justify;font-size:10.5pt;font-family:Calibri,sans-serif"><span lang="EN-US">sys 0m0.008s<u></u><u></u></span></p><p class="MsoNormal" style="margin:0cm 0cm 0.0001pt;text-align:justify;font-size:10.5pt;font-family:Calibri,sans-serif"><span lang="EN-US"><u></u> <u></u></span></p><p class="MsoNormal" style="margin:0cm 0cm 0.0001pt;text-align:justify;font-size:10.5pt;font-family:Calibri,sans-serif"><span lang="EN-US">end restore on host <a href="http://192.168.9.180/" target="_blank">192.168.9.180</a>: 2015-07-23 00:44:29.604002575<u></u><u></u></span></p><p class="MsoNormal" style="margin:0cm 0cm 0.0001pt;text-align:justify;font-size:10.5pt;font-family:Calibri,sans-serif"><span lang="EN-US">---<u></u><u></u></span></p><p class="MsoNormal" style="margin:0cm 0cm 0.0001pt;text-align:justify;font-size:10.5pt;font-family:Calibri,sans-serif"><span lang="EN-US"><u></u> <u></u></span></p><div><span lang="EN-US"><br></span></div><div></div></div></div></div><div style="font-size:12.8000001907349px"></div></div></div><div class="gmail_extra"><br><div class="gmail_quote">On Tue, Jul 21, 2015 at 11:54 PM, Pavel Emelyanov <span dir="ltr"><<a href="mailto:xemul@parallels.com" target="_blank">xemul@parallels.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><span class="">On 07/21/2015 06:25 PM, Yanbao Cui wrote:<br>
> thanks Saied!<br>
> The delay is happened after restore. The -existing- socket(s) cannot work, but at<br>
> the same time ping the container IP is OK.<br>
><br>
> So I am confused that the delay is caused by Docker or CRIU? Because it seems that<br>
> it only happened in Docker migration with CRIU<br>
<br>
</span>I would say that this is related to TCP checkpoint-restore. The restored connection<br>
starts with zero window and sends the probe to remote side to get it's actual value.<br>
This can be the reason for delay.<br>
<br>
Can you collect tcpdump for your connection with timestamps starting from the moment<br>
of restore so we could look at what packets are traveling?<br>
<span class="HOEnZb"><font color="#888888"><br>
-- Pavel<br>
<br>
</font></span></blockquote></div><br><br clear="all"><div><br></div>-- <br><div class="gmail_signature"><font color="#3333ff"><div><font color="#3333ff">Best Regards</font></div><div><font color="#3333ff">Cui Yanbao | 崔言宝</font></div><div><font color="#3333ff">--</font></div>龍生玖天，豈能安於凡塵！</font></div>
</div>