[Devel] [PATCH hci-8.1 v2 3/4] dm-ploop: wait for the ploop_allocator() thread before changing deltas[]

[Denis V. Lunev]

From: Denis V. Lunev <den at openvz.org>

An allocating pio requests the preallocation of the top delta ahead of
itself and does not wait when its cluster already fits, so the
allocator can still be working when submission is suspended or the
device is dm suspended. It looks up the top delta without a lock and
keeps the pointer and the file across its truncate or fallocate, while
a cmd rewrites deltas[] under neither of its locks:

- merge and notify_merged move the top delta one slot down and zero the
  old slot. The allocator then reads a zero file_size from it and
  truncates the live top image down to the preallocation size, or reads
  a NULL file and oopses;

- flip_upper_deltas swaps the two upper slots of a dm suspended device.
  The allocator computes the new length from the size of the delta now
  on top and truncates the old top one, now holding data below it, to
  that length.

Wait for the allocator to finish in ploop_suspend_submitting_pios()
after the inflight pios are gone, and in ploop_postsuspend() before the
device is marked suspended, which is what the flip checks. No pio can
request a new preallocation after either point, but a resize of a dm
suspended device can, as the grow allocates clusters, so wait in
ploop_flip_upper_deltas() too. Wake the waiters with wake_up(), as the
one in ploop_postsuspend() sleeps uninterruptibly.

Fixes: cf489cf22cd5 ("dm-ploop: make filespace preallocations async")
Reported-by: Konstantin Khorenko <khorenko at virtuozzo.com>
https://virtuozzo.atlassian.net/browse/VSTOR-152984
Feature: dm-ploop: ploop target driver
Signed-off-by: Denis V. Lunev <den at openvz.org>
---
 drivers/md/dm-ploop-cmd.c    | 11 +++++++++++
 drivers/md/dm-ploop-map.c    |  4 ++--
 drivers/md/dm-ploop-target.c |  4 ++++
 drivers/md/dm-ploop.h        |  1 +
 4 files changed, 18 insertions(+), 2 deletions(-)

diff --git a/drivers/md/dm-ploop-cmd.c b/drivers/md/dm-ploop-cmd.c
index d85f4d803b61..29f9a7ee7540 100644
--- a/drivers/md/dm-ploop-cmd.c
+++ b/drivers/md/dm-ploop-cmd.c
@@ -159,6 +159,11 @@ static int ploop_suspend_submitting_pios(struct ploop *ploop)
 
 	ret = wait_event_interruptible(ploop->inflight_md_wq,
 				       !atomic_read(&ploop->inflight_md));
+	if (ret)
+		goto out;
+
+	ret = wait_event_interruptible(ploop->dispatcher_wq_prealloc,
+				       ploop_no_pending_prealloc(ploop));
 out:
 	if (ret)
 		ploop_resume_submitting_pios(ploop);
@@ -1176,6 +1181,12 @@ static int ploop_flip_upper_deltas(struct ploop *ploop)
 	if (ret)
 		return ret;
 
+	/* A resize of the suspended device may have restarted the allocator */
+	ret = wait_event_interruptible(ploop->dispatcher_wq_prealloc,
+				       ploop_no_pending_prealloc(ploop));
+	if (ret)
+		return ret;
+
 	return process_flip_upper_deltas(ploop);
 }
 
diff --git a/drivers/md/dm-ploop-map.c b/drivers/md/dm-ploop-map.c
index e64fd8965d46..ae3e287604b7 100644
--- a/drivers/md/dm-ploop-map.c
+++ b/drivers/md/dm-ploop-map.c
@@ -1233,7 +1233,7 @@ void ploop_should_prealloc(struct ploop *ploop)
 	spin_unlock_irqrestore(&ploop->bat_lock, flags);
 }
 
-static int ploop_no_pending_prealloc(struct ploop *ploop)
+int ploop_no_pending_prealloc(struct ploop *ploop)
 {
 	int ret;
 	unsigned long flags;
@@ -2342,7 +2342,7 @@ int ploop_allocator(void *data)
 					PL_FMT("failed to allocate space: %d\n"),
 					       ploop_device_name(ploop), ret);
 			}
-			wake_up_interruptible(&ploop->dispatcher_wq_prealloc);
+			wake_up(&ploop->dispatcher_wq_prealloc);
 			current_restore_flags(old_flags, PLOOP_PFLAGS);
 		}
 		if (kthread_should_stop()) {
diff --git a/drivers/md/dm-ploop-target.c b/drivers/md/dm-ploop-target.c
index 3b589579741c..d8279c79ef56 100644
--- a/drivers/md/dm-ploop-target.c
+++ b/drivers/md/dm-ploop-target.c
@@ -743,6 +743,10 @@ static void ploop_presuspend_undo(struct dm_target *ti)
 }
 static void ploop_postsuspend(struct dm_target *ti)
 {
+	struct ploop *ploop = ti->private;
+
+	wait_event(ploop->dispatcher_wq_prealloc,
+		   ploop_no_pending_prealloc(ploop));
 	ploop_set_suspended(ti, true);
 }
 static int ploop_preresume(struct dm_target *ti)
diff --git a/drivers/md/dm-ploop.h b/drivers/md/dm-ploop.h
index 7d0d94e7b266..5ae02ef5ffc2 100644
--- a/drivers/md/dm-ploop.h
+++ b/drivers/md/dm-ploop.h
@@ -652,6 +652,7 @@ extern void ploop_disable_writeback_delay(struct ploop *ploop);
 extern void ploop_enable_writeback_delay(struct ploop *ploop);
 
 extern void ploop_should_prealloc(struct ploop *ploop);
+extern int ploop_no_pending_prealloc(struct ploop *ploop);
 extern void ploop_resubmit_enospc_pios(struct ploop *ploop);
 
 #endif /* __DM_PLOOP_H */
-- 
2.53.0