[CRIU] implementing some kind of --leave-frozen option for c/r in CRIU

[Andrew Vagin]

On Mon, May 16, 2016 at 06:25:51PM -0600, Tycho Andersen wrote:
> On Mon, May 16, 2016 at 05:20:25PM -0700, Andrew Vagin wrote:
> > On Mon, May 16, 2016 at 04:39:43PM -0600, Tycho Andersen wrote:
> > > On Mon, May 16, 2016 at 03:34:59PM -0700, Andrew Vagin wrote:
> > > > On Tue, May 10, 2016 at 11:04:56AM -0600, Tycho Andersen wrote:
> > > > > Hi guys,
> > > > > 
> > > > > I'm looking at implementing some kind of --leave-frozen option in
> > > > > CRIU, so that we can have a basic UX in LXD where we can wait for the
> > > > > restore to be successful before we kill the checkpointed container. I
> > > > > know p.haul does this by just using a callback, but it would be sort
> > > > > of painful to absorb just the callback part without doing a lot of
> > > > > extra engineering. We'll get LXD using p.haul someday, though :)
> > > > > 
> > > > > The actual --leave-frozen patch is not so bad (see attached), but I'm
> > > > > not sure what to do about the network locking/unlocking bits.
> > > > > 
> > > > > It seems like it is always safe to do the bits in
> > > > > cpt_unlock_tcp_connections() since that's just disabling tcp repair
> > > > > mode, but all of the iptables rules seem necessary in order to keep
> > > > > the network locked.
> > > > > 
> > > > > So my question is: is there a nice way we can "tag" these rules so
> > > > > that something can come by and delete them later? I was thinking about
> > > > > having criu add a comment (via -m comment --comment "CRIU-LOCK-RULE")
> > > > > to each rule it adds, but I'm not sure if there's a better way, or if
> > > > > I've missed something entirely.
> > > > > 
> > > > > Thanks!
> > > > 
> > > > We create a separate CRIU table, when we dump/restore a network
> > > > namespace. Maybe we need to do the same for a case when we don't dump
> > > > netns and collect all rules in this table.
> > > > 
> > > > In this case, we need only drop this table to unlock network.
> > > 
> > > I see that we create the separate table for local connections, but for
> > > tcp connections they still go in the "filter" chain right?
> > 
> > Yes. But we can add the CRIU table for this case too.
> 
> Ok. In that case I don't actually need a `criu gc`, we could just have
> the semantics of --leave-frozen be that it leaves the CRIU chain in
> iptables and that users delete that to unlock the network.
> 
> Pavel, Eugene, does that sound ok? If so, I can send an updated
> --leave-frozen patch.

Yesterday I tried to use a separate table in a case when netns isn't
dumped:
https://github.com/avagin/criu/commit/a3b3ae4e7db40f5b0d4456b27062484c9e27ad8a

here are two problem:
* we need two tables to block input and output trafic
* we need to generate uniq names for these tables. We can't use fixed
  names, because we may want to make checkpoint/restore a few times
  simulteniously.

The profit from this approach is that:
* we can unlock network very fast and don't need to enumerate sockets
  for that.
* It's easy to understand which rules were added by criu and an user
  can easy clean up them himself

> 
> Tycho
> 
> > > 
> > > static const char *nf_conn_cmd = "%s -t filter %s %s --protocol tcp "
> > >         "--source %s --sport %d --destination %s --dport %d -j DROP";
> > > 
> > > Tycho