[CRIU] [PATCH RFC 00/30] Rework file opening scheme to make it asynchronous

[Kirill Tkhai]

This series rework file opening scheme. Currently, we have 4 stages of opening:
1)prepare
2)create
3)receive
4)post_create

It's became too complex, but we bumped in the situation with unix sockets,
when even 4 stages is not enough. So, it will became more complex in this way.

Also, the problem is in the scheme is synchronous. You should wait receiving
of master fle (or pipe fd) and just to sleep, even if you have to restore another
files, which are not depend on receiving foreign fd. So, the restored tasks stand
in a chain and this is not good for performance.

The patchset aims to solve these problems. It makes open scheme asynchronous,
and not using blocking syscalls like receive. The restorer iterates over
file list and call ->open method for each file. If file is ready to be restored,
it restores. Otherwise, "repeat" answer (ORV_AGAIN) is returned, and we try
to restore it after out other files are restored (or changed their stage),
or some extenal event is received.

External event receiving is implemented using global futex task_st. It's
organized similar to the way the try_to_wake_up() is made in linux kernel.

If you change a stage of a fle, which may be interested for other task,
you change the stage and set FDS_EVENT bit to the other task's task_st.
If there was no the bit, you wake the futex up (it's possible to say,
FDS_EVENT is similar to TASK_RUNNING).

The other task follows the next scheme. It clears FDS_EVENT in its task_st,
and polls the fles, which it's interested in. If nothing changes it sleep
on futex again. If there is a changes, it handles it using ->open. See
"files: Implement {set,get}_fds_event()" for "try_to_wake_up" details.
Full polling is implemented in "files: Kill struct file_desc_ops::post_open".

It's possible to make restore async further, but now it's a moment,
when patches is too more and the main goal is implemented. Further, is
possible to make send_fd_to_peer() receive ORV_AGAIN also and merge
different types of files in the single list (fds, eventpoll, tty slaves and ctty).

Pasha, I need your comments on this all.
---

Kirill Tkhai (30):
      pie: Optimize send_fds() and recv_fds() with opts
      files: Create per-process transport sock and bind it
      files: Wait transport_fd before sending a fd to peer
      pie: Allow to transfer pointers on struct fdinfo_list_entry together with fds
      files: Send/receive fds with pointers on their struct fdinfo_list_entry in receiver
      files: Move closing fle's transport_fd to recv_fd_from_peer()
      files: Move derefferencing of TRANSPORT_FD_OFF to send_fd_to_peer()
      files: Add fle_init() for initialization new fdinfo_list_entry
      files: Allow to receive further fds
      files: Make sending fds to peers async
      files: Receive real fd numbers using recv_fd_from_peer()
      files: Kill file_desc_ops::want_transport()
      files: Kill fd parameter of transport_name_gen()
      files: Use collect_gen_fd() in tty_collect_fd()
      files: Kill want_post_open_stage()
      files: Kill fd_open_state::receive_fd stage
      files: Merge states iteration into open_fdinfos()
      files: Implement {set,get}_fds_event()
      files: Add fdinfo_list_entry::stage
      unix: Link connected to peer sockets to its list
      unix: Wait a peer using task_st futex
      inet: Link sockets in port_type list
      inet: Use task_st futex for notification instead of per-port
      timerfd: Kill post_open stage
      files: Add open stages enum
      eventpoll: Make post_open stage may fail
      files: Set fds event after sending fd to peer
      files: Add new_fd parameter to file_desc_ops::open()
      files: Make recv_fd_from_peer() non-block
      files: Kill struct file_desc_ops::post_open


 criu/autofs.c           |   29 +++
 criu/cr-restore.c       |    6 -
 criu/eventfd.c          |    5 -
 criu/eventpoll.c        |   39 ++++
 criu/fifo.c             |    9 +
 criu/files-ext.c        |    5 -
 criu/files-reg.c        |   10 +
 criu/files.c            |  430 +++++++++++++++++++++++------------------------
 criu/fsnotify.c         |   10 +
 criu/include/files.h    |   44 +++--
 criu/include/lock.h     |    5 +
 criu/include/pipes.h    |    1 
 criu/include/pstree.h   |    8 +
 criu/include/sk-inet.h  |    1 
 criu/include/tty.h      |    2 
 criu/include/util-pie.h |   11 +
 criu/namespaces.c       |    5 -
 criu/parasite-syscall.c |    2 
 criu/pie/parasite.c     |    2 
 criu/pie/util-fd.c      |   52 ++++--
 criu/pipes.c            |   46 ++---
 criu/pstree.c           |    1 
 criu/signalfd.c         |    5 -
 criu/sk-inet.c          |   52 ++++--
 criu/sk-netlink.c       |    5 -
 criu/sk-packet.c        |    5 -
 criu/sk-unix.c          |  131 +++++++++-----
 criu/timerfd.c          |   25 +--
 criu/tty.c              |   67 +++----
 criu/tun.c              |    9 +
 30 files changed, 583 insertions(+), 439 deletions(-)

--
Signed-off-by: Kirill Tkhai <ktkhai at virtuozzo.com>