<div dir="ltr">I know what you mean. But I just want to migrate some containers to another host, all of those application can deal tcp reconnection.<div>You say if there are some packets cannot deliver into new container's location, the restore will broken.</div><div>So, If there is<b> a method to skip this progress, just to bind the old ip address to the container on the new host.</b></div><div>Everytime I checkpoint a container with tcp connection, scp to another host, then restore, the restore log always say that</div><div><br></div><div><div>(00.294516) 1: Found id pipe:[6392267] (fd 1) in inherit fd list</div><div>(00.294520) 1: File pipe:[6392267] will be restored from fd 3 dumped from inherit fd 1</div><div>(00.294535) 1: Found fd 1 (id pipe:[6392267]) in inherit fd list (caller inherit_fd_resolve_clash)</div><div>(00.294538) 1: Inherit fd 1 moved to 3 to resolve clash</div><div>(00.294541) 1: Create fd for 1</div><div>(00.294544) 1: Restoring fd 2 (state -> create)</div><div>(00.294545) 1: Creating pipe pipe_id=0x6189cc id=0x3</div><div>(00.294547) 1: Found id pipe:[6392268] (fd 2) in inherit fd list</div><div>(00.294548) 1: File pipe:[6392268] will be restored from fd 4 dumped from inherit fd 2</div><div>(00.294554) 1: Found fd 2 (id pipe:[6392268]) in inherit fd list (caller inherit_fd_resolve_clash)</div><div>(00.294555) 1: Inherit fd 2 moved to 4 to resolve clash</div><div>(00.294558) 1: Create fd for 2</div><div>(00.294560) 1: Restoring fd 4 (state -> create)</div><div>(00.294569) 1: Restore: family 10 type 1 proto 6 port 6379 state 10 src_addr ::</div><div>(00.294592) 1: 5 restore sndbuf 16384 rcv buf 87380</div><div>(00.294595) 1: restore priority 0 for socket</div><div>(00.294596) 1: restore rcvlowat 1 for socket</div><div>(00.294598) 1: restore mark 0 for socket</div><div>(00.294611) 1: Found fd 4 (id pipe:[6392268]) in inherit fd list (caller inherit_fd_resolve_clash)</div><div>(00.294616) 1: Inherit fd 4 moved to 6 to resolve clash</div><div>(00.294619) 1: Create fd for 4</div><div>(00.294622) 1: Restoring fd 5 (state -> create)</div><div>(00.294625) 1: Restore: family 2 type 1 proto 6 port 6379 state 10 src_addr 0.0.0.0</div><div>(00.294633) 1: 5 restore sndbuf 16384 rcv buf 87380</div><div>(00.294635) 1: restore priority 0 for socket</div><div>(00.294636) 1: restore rcvlowat 1 for socket</div><div>(00.294638) 1: restore mark 0 for socket</div><div>(00.294640) 1: Create fd for 5</div><div>(00.294642) 1: Restoring fd 6 (state -> create)</div><div>(00.294644) 1: Restore: family 2 type 1 proto 6 port 6379 state 1 src_addr 172.17.128.2</div><div>(00.294646) 1: Restoring TCP connection</div><div>(00.294649) 1: Restoring TCP connection id 7 ino 619526</div><div>(00.294657) 1: Setting 1 queue seq to 2786935679</div><div>(00.294659) 1: Setting 2 queue seq to 919486667</div><div>(00.294679) 1: Error (sk-inet.c:721): Can't bind inet socket (id 7): Cannot assign requested address</div></div><div><br></div><div>You see, 172.17.128.2 is the old ip, criu tried to bind this ip to new container, but failed.</div></div><div class="gmail_extra"><br clear="all"><div><div class="gmail_signature"><div dir="ltr"><div><div dir="ltr"><div><span style="font-family:Calibri;color:rgb(31,73,125);font-size:medium">Regards,</span><br></div><div><p style="margin:0cm 0cm 0pt"><font size="3"><span lang="EN-US" style="color:rgb(31,73,125)"><font face="Calibri">David Sun </font></span><span style="color:rgb(31,73,125)">孙炜程</span></font><span lang="EN-US"><br><span style="color:rgb(31,73,125)"><font size="3"><font face="Calibri">School of Software Engineering, Shanghai Jiaotong University</font></font></span></span></p><p style="margin:0cm 0cm 0pt"><span lang="EN-US" style="color:rgb(31,73,125)"><font size="3"><font face="Calibri">Mobile: <a value="+8615921638078" style="color:rgb(17,85,204)">15921638078</a></font></font></span></p><p style="margin:0cm 0cm 0pt"><span lang="EN-US" style="color:rgb(31,73,125)"><font size="3" face="Calibri">E-mail: </font><a href="mailto:SunWeicheng0001@gmail.com" style="color:rgb(17,85,204)" target="_blank"><font color="#0000ff" size="3" face="Calibri">SunWeicheng0001@gmail.com</font></a></span></p></div></div></div></div></div></div>
<br><div class="gmail_quote">2016-04-28 18:33 GMT+08:00 Pavel Emelyanov <span dir="ltr"><<a href="mailto:xemul@virtuozzo.com" target="_blank">xemul@virtuozzo.com</a>></span>:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><span class="">On 04/28/2016 08:01 AM, Weicheng SUN wrote:<br>
> Hi Everyone,<br>
><br>
> The latest release of cr-combined said that it supports network. Then I do some experiments. I use<br>
> 1. docker run -d -v demo:/data --name=redis redis:latest<br>
> 2. docker run -d -e USE_REDIS_HOST=redis --link redis:redis -p 80:80 --name=app binocarlos/moby-counter:latest<br>
> which is a demo of simple website and redis is its storage. I first run them on host A<br>
><br>
> I try to checkpoint them and copy to another host(host B). I create two containers with same configuration<br>
> of them and applied the change of file system to the new created container. Then I tried to restore them,<br>
> which sometimes came out a wrong result.<br>
> I notice that when redis and app have same ip address on host A and B, then the restore will success. But,<br>
> when they have different IP on two hosts, the restore progress will certainly broken.<br>
><br>
> The reason I guess is that the state of tcp connection in checkpoint image didn't match the new environment<br>
> in host B. For example, redis in host A is 17.172.20.1 and app in host A is 17.172.20.2, then for app there<br>
> is a connection whose remote host is 17.172.20.1. Then come to host B, if redis in host B is 17.172.20.3 and<br>
> when I try to restore app, criu will confuse where is 17.172.20.1(redis)?<br>
<br>
</span>After migration all the sockets get restored with the IP addresses they had on source node,<br>
so if for some reason packets to former container IP cannot be delivered into container's<br>
new location, the container would get broken. This is unavoidable limitation of TCP protocol<br>
itself. There can be workarounds to this (DNAT-like for example), but this is out of CRIU's<br>
scope and should be done on Docker level.<br>
<span class=""><br>
> Generally, is there always a problem to checkpoint and restore a container, who has a link with other<br>
> containers, on the another host?<br>
><br>
> And who knows that how to use a specific IP address while start a new container? For the latest release of<br>
> cr-combined doesn't support docker run --ip. Even --ip is supported, it requires a user defined network,<br>
> default network is not supported.<br>
><br>
> Hopes for your kindly response & wish all of you a great day!<br>
><br>
> Regards,<br>
><br>
> David Sun<br>
><br>
><br>
><br>
</span>> _______________________________________________<br>
> CRIU mailing list<br>
> <a href="mailto:CRIU@openvz.org">CRIU@openvz.org</a><br>
> <a href="https://lists.openvz.org/mailman/listinfo/criu" rel="noreferrer" target="_blank">https://lists.openvz.org/mailman/listinfo/criu</a><br>
><br>
<br>
</blockquote></div><br></div>