[CRIU] Options when restoring a socket

[Pavel Emelyanov]

On 04/19/2015 04:48 AM, Ross Boucher wrote:
> Hey everyone,

Hi, Ross.

> I've been trying to figure out both what happens when you checkpoint an open socket 
> and what my options are for restoring that socket (or maybe doing something else at
> that point in time). It might be best to just describe the program I have and what
> I want to accomplish.
> 
> I have two programs communicating over a socket. Program A opens a socket and listens 
> for connections, and then program B connects to it. They essentially exchange messages 
> forever in a pattern something like:
> 
> A -> B send next message
> B -> A ok, here's the next message
> 
> Obviously, in between, A performs some actions. The goal is to checkpoint A after each
> message is processed and before the next is received (while leaving the process
> running), so that we can restore to any previous state and reprocess possibly changed 
> messages.

First thing that comes to mind is that --track-mem thing definitely makes sense for
such frequent C/R-s. But that's a side note :)

> It's completely fine for our use case to have to re-establish that socket connection, 
> we don't actually need or want to try and magically use the same socket (since program
> B has probably moved on to other things in between).

Hm.. OK.

> Is this a use case for a criu plugin? 

Well, I would say it is, but there are two things about it. First is that we don't have any
hooks in CRIU code for sockets, so patching will be required. And the second is -- many 
people are asking for handling the connected unix socket, so I think we'd better patch
criu itself to provide some sane API rather than make everybody invent their own plugins :)

So, I see two options for such an API.

The first is to extend the --ext-unix-sk to accept socket ID as an argument that would 
force even stream sockets to be marked as "external" and not block the dump. On restore
the --ext-unix-sk with $ID would make CRIU connect the restored unix socket back to its
original path. Optionally we can make the argument look like $ID:$PATH and connect to
$PATH instead.

The other option would be to still teach dump accept the --ext-unix-sk $ID parameter. On
restore you can create the connection yourself and then pass it into CRIU with --inherit-fd
argument. We already have fd inheritance for pipes and files [1], but we can patch CRIU
to support it for sockets too.

[1] http://criu.org/Inheriting_FDs_on_restore

> I've tried playing around with the ext-unix-sk flag but I haven't quite figured anything 
> out yet.

The --ext-unix-sk is for datagram sockets as they are stateless and we can just close and
re-open one back w/o any risk that the peer notices it.

> Any help would be appreciated. Thanks!

-- Pavel