[Devel] [PATCH RHEL7 COMMIT] ploop: fix fsync_reqs accounting

[Konstantin Khorenko]

The commit is pushed to "branch-rh7-3.10.0-327.18.2.vz7.15.x-ovz" and will appear at https://src.openvz.org/scm/ovz/vzkernel.git
after rh7-3.10.0-327.18.2.vz7.14.27
------>
commit 452d6836237fe1ffb72ae265bdb4301ac34e0a5a
Author: Maxim Patlasov <mpatlasov at virtuozzo.com>
Date:   Tue Jul 19 13:43:35 2016 +0400

    ploop: fix fsync_reqs accounting
    
    Patchset description:
    ploop: fix free_list starvation
    
    The first patch of the patch-set fixes a minor unrelated
    problem. It is trivial.
    
    The remaining three patches try to solve the following
    problem:
    
    Under high load, and when push_backup is in progress, it is
    possible that all preq-s from free_list will be consumed by
    either incoming bio-s waiting for backup tool out-of-band
    processing, or some incoming bio-s blocked on the former ones.
    
    Then, ploop reaches maximum possible preq->active_reqs and
    goes to sleep waiting for something. But this something is
    actually the backup tool who is blocked on reading from
    the ploop device. Deadlock.
    
    See per-patch descriptions for details.
    
    https://jira.sw.ru/browse/PSBM-49454
    
    Maxim Patlasov (4):
          ploop: fix fsync_reqs accounting
          ploop: introduce plo->free_qlen counter
          ploop: introduce plo->blockable_reqs counter
          ploop: fix free_list starvation
    
    ======================================================
    This patch description:
    
    io->fsync_qlen stands for the number of ploop requests waiting for processing
    by io fsync thread.
    
    The fix is obvious: each time we add preq to fsync thread queue, we have to
    increment fsync_reqs; each time we delete it from the queue, we have to
    decrement it.
    
    The fix should not affect anything because currently nobody cares about the
    value of io->fsync_qlen. The patch is useful because we expose the value
    as /sys/block/ploopN/pstate/fsync_reqs.
    
    Signed-off-by: Maxim Patlasov <mpatlasov at virtuozzo.com>
---
 drivers/block/ploop/dev.c       | 1 +
 drivers/block/ploop/io_direct.c | 3 ++-
 2 files changed, 3 insertions(+), 1 deletion(-)

diff --git a/drivers/block/ploop/dev.c b/drivers/block/ploop/dev.c
index 3dc94ca..0711fb5 100644
--- a/drivers/block/ploop/dev.c
+++ b/drivers/block/ploop/dev.c
@@ -2055,6 +2055,7 @@ ploop_entry_request(struct ploop_request * preq)
 		    !test_bit(PLOOP_REQ_FSYNC_DONE, &preq->state)) {
 			spin_lock_irq(&plo->lock);
 			list_add_tail(&preq->list, &top_io->fsync_queue);
+			top_io->fsync_qlen++;
 			if (waitqueue_active(&top_io->fsync_waitq))
 				wake_up_interruptible(&top_io->fsync_waitq);
 			spin_unlock_irq(&plo->lock);
diff --git a/drivers/block/ploop/io_direct.c b/drivers/block/ploop/io_direct.c
index 1086850..c12e3c8 100644
--- a/drivers/block/ploop/io_direct.c
+++ b/drivers/block/ploop/io_direct.c
@@ -477,9 +477,10 @@ try_again:
 		ploop_acc_flush_skip_locked(plo, preq->req_rw);
 		preq->iblock = iblk;
 		list_add_tail(&preq->list, &io->fsync_queue);
+		io->fsync_qlen++;
 		plo->st.bio_syncwait++;
 		if ((test_bit(PLOOP_REQ_SYNC, &preq->state) ||
-		     ++io->fsync_qlen >= plo->tune.fsync_max) &&
+		     io->fsync_qlen >= plo->tune.fsync_max) &&
 		    waitqueue_active(&io->fsync_waitq))
 			wake_up_interruptible(&io->fsync_waitq);
 		else if (!timer_pending(&io->fsync_timer))