[Devel] [PATCH hci-8.1 0/4] dm-ploop: fix the pio submission stop around cmds

[Denis V. Lunev]

A cmd stops pio submission and waits for the inflight pios before it
rewrites the BAT levels or deltas[], and expects nothing to touch them
until it resumes. Konstantin found three ways this does not hold while
reviewing the VSTOR-151979 series (VSTOR-152985, VSTOR-152984,
VSTOR-152981):

- a submitter racing the stop either runs without the cmd waiting for
  it, or is parked after resume and never completes (patch 1);

- the ploop_allocator() thread is not stopped at all. It truncates the
  top delta while merge or flip moves it in deltas[], cutting the live
  image to the preallocation size (patch 3). The thread can also miss a
  wakeup, which stalls the device by itself and would hang the new wait,
  so patch 2 fixes that first;

- a grow failing to copy a cluster resumes submission with that
  cluster's BAT page still busy, and writes to it hang (patch 4).

Patch 3 is the one losing data: a snapshot delete under write load is
enough.

Tested on dev-cl04, stock 6.12.0-211.52.1.22.11.vz10, dm-ploop built
out of tree:
- each bug reproduced on the stock module, the races with a test-only
  kprobe module widening the window: a pio running past the suspend of
  a cmd; a pio never completing after resume; the top image truncated
  from 512M to 128M by a merge and to 256M by a flip; after a grow
  failing with ENOSPC on a full host fs, a writer stuck in D state, an
  unsuspendable device and a later cmd failing with ETIMEDOUT. None of
  them reproduce with the series;
- fio 4k random reads and writes, O_DIRECT, 1 to 48 jobs, on a fully
  allocated image over brd: IOPS and system CPU stay within the
  run-to-run spread of the stock module. This setup tops out at about
  420K IOPS elsewhere in dm-ploop and would not show a lock taken on
  submission either, so patch 1 keeps that path lockless;
- the ploop pytest suite, ploop and qcow2 formats, passes. A test for
  patch 4 goes to the ploop tests separately.

Denis V. Lunev (4):
  dm-ploop: fix pio submission racing with the suspend and resume of a
    cmd
  dm-ploop: fix a lost wakeup of the ploop_allocator() thread
  dm-ploop: wait for the ploop_allocator() thread before changing
    deltas[]
  dm-ploop: fix a failed cluster copy on grow leaving the BAT page busy

 drivers/md/dm-ploop-cmd.c    | 18 ++++++++++++++++--
 drivers/md/dm-ploop-map.c    | 22 +++++++++++++++-------
 drivers/md/dm-ploop-target.c |  4 ++++
 drivers/md/dm-ploop.h        |  1 +
 4 files changed, 36 insertions(+), 9 deletions(-)


base-commit: 09eb9a82fe872048d082391415d1cdeda2ac6879
-- 
2.53.0