[Devel] [PATCH hci-8.1 v4 00/11] dm-ploop: give back unused tail of top delta on suspend

[Alexander Ivanov]

Preallocated space and clusters at the end of the top delta freed by
discard are never given back, so the image file never shrinks. The
last patch truncates the top delta to the end of its last used cluster
on suspend.

The next resumed ploop must see the new file size, so the previous
patch reloads all the metadata (header, BAT, file sizes) on resume.
This also fixes "dmsetup reload", after which the new ploop worked with
metadata read in ploop_ctr() while the old one kept changing images.

Patches 1-5 fix header validation and a lost wakeup in the allocator,
which the reload would otherwise hit. Patches 6-9 prepare the reload
and fix races on md pages and inflight_md that the new waits on
suspend expose.

The corresponding ploop userspace changes:
https://bitbucket.org/openvz/ploop.ovz/pull-requests/486

https://virtuozzo.atlassian.net/browse/VSTOR-148171

v4:
- Patch 1: was sent separately as "[PATCH hci-8.1] dm-ploop: fix u32
  overflow of the delta BAT byte size", unchanged.
- Patch 2 (was 1): drop the comment above set_current_state(), add
  Reviewed-by.
- Patches 3, 4: new. Reject a top delta header whose BAT does not
  cover the target, and reject empty deltas, instead of skipping the
  mappings of an empty delta in patch 5.
- Patch 5 (was 2): say that raw deltas have never worked since the
  driver was added; drop the empty delta handling, see patches 3, 4;
  add Reviewed-by.
- Patch 6 (was 3): fix the stale comment of ploop_add_delta() about
  the order deltas are added in.
- Patches 7, 8: split from patch 10. Patch 7 takes bat_lock around md
  page lookup in ploop_llseek_hole() to protect it from resize, so
  llseek doesn't wait for long control commands.
- Patch 9: new, fixes a use-after-free of the ploop on the last
  inflight_md decrement from BAT write completion.
- Patch 10 (was 4):
  - test MD_* flags with test_bit(), they are bit numbers;
  - keep the in-memory m_DiskInUse only after flip_upper_deltas, take
    it from disk otherwise, so the CBT written on umount isn't lost;
  - don't wait for preallocation on postsuspend in standby, and make
    the wait killable; skip the reload and prealloc on resume in
    standby or while preallocation is pending;
  - track standby per ploop, as ploop_ctr() of a new table clears the
    queue bit before the old table is suspended;
  - drop ctl_rwsem and the suspended check from ploop_llseek_hole():
    dm core doesn't call it on a suspended device;
  - describe the new preresume failures in the commit message.
- Patch 11 (was 5): don't truncate in standby, if the wait for
  preallocation was killed, or if the file size differs from
  top->file_size, i.e. another ploop writes to the top delta.
  Shorten the comments of ploop_calc_used_end(), which repeated the
  commit message.

v3:
- Split from the common series with dm-qcow2: dm-qcow2 patches are
  sent as a separate series. Patches 3-6 of v2 are patches 2-5 now.
- Patch 1: new patch.
- Patch 2: also skip applying the mappings of an empty delta, as BAT
  iterators can't handle nr_be == 0.
- Patch 4: wait for inflight_md on postsuspend, as data bios are
  completed before BAT writeback is finished, and use wake_up() for
  inflight_md_wq; reload metadata under memalloc_noio_save(), as the
  device is suspended; restore m_Sig, m_Type and m_Sectors on reload
  failure.
- Patch 5: don't truncate anything if m_FormatExtensionOffset is not
  cluster-aligned or out of range; truncate under memalloc_noio_save().

v2:
- Add dm-ploop support (patches 3-6), as requested in v1 review.
- dm-qcow2: move get_md_page_sync() to a separate patch as
  qcow2_md_page_read_sync() and use it in qcow2_parse_header() as well.
- dm-qcow2: start the refcount scan below file_preallocated_area_start
  instead of the end of the file, and truncate to the used end directly
  instead of min(file_preallocated_area_start, used end).

Alexander Ivanov (11):
  dm-ploop: fix u32 overflow of the delta BAT byte size
  dm-ploop: fix lost wakeup in ploop_allocator()
  dm-ploop: reject a delta header whose BAT does not cover the target
  dm-ploop: reject empty deltas
  dm-ploop: set delta is_raw before applying its mappings
  dm-ploop: factor out ploop_read_delta() from ploop_add_delta()
  dm-ploop: take bat_lock around md page lookup in ploop_llseek_hole()
  dm-ploop: use wake_up() for inflight_md_wq and dispatcher_wq_prealloc
  dm-ploop: fix use-after-free on the last inflight_md decrement
  dm-ploop: reload metadata on resume
  dm-ploop: truncate unused tail of top delta on suspend

 drivers/md/dm-ploop-bat.c    | 280 ++++++++++++++++++++++++++++++++---
 drivers/md/dm-ploop-cmd.c    |  26 ++--
 drivers/md/dm-ploop-map.c    |  79 ++++++----
 drivers/md/dm-ploop-target.c | 152 +++++++++++++++----
 drivers/md/dm-ploop.h        |  22 ++-
 5 files changed, 469 insertions(+), 90 deletions(-)

-- 
2.53.0