[Devel] [PATCH hci-8.1 v6 3/7] block/blk-cbt: read a map slot once, publish a map page with release

[Pavel Tikhomirov]

CBT_PAGE() loads a map slot twice. A lockless reader racing with the
cache flush, which marks an empty slot CBT_PAGE_MISSED, can get NULL from
the first load and the marker from the second, and then use
(struct page *)0x1 as a page.

A page is also installed with a plain store under cbt->lock, but readers
don't take that lock: on a weakly ordered CPU a reader may see the
pointer before the page's zeroing, and the late zeroing can then clear a
bit the reader has just set.

Load the slot once with READ_ONCE(), whose address dependency orders the
reader's accesses to the page, and install a page with
smp_store_release().

https://virtuozzo.atlassian.net/browse/VSTOR-149268
Feature: cbt: changed block tracking (for backup)
Signed-off-by: Pavel Tikhomirov <ptikhomirov at virtuozzo.com>
---
 block/blk-cbt.c | 15 ++++++++++-----
 1 file changed, 10 insertions(+), 5 deletions(-)

diff --git a/block/blk-cbt.c b/block/blk-cbt.c
index 1066b6a33b6b5..73df03c6c7e74 100644
--- a/block/blk-cbt.c
+++ b/block/blk-cbt.c
@@ -27,8 +27,11 @@
 #define BITS_PER_PAGE		(1UL << (PAGE_SHIFT + 3))
 
 #define CBT_PAGE_MISSED (struct page *)(0x1)
-#define CBT_PAGE(cbt, idx) (cbt->map[idx] == CBT_PAGE_MISSED ? \
-			    NULL : cbt->map[idx])
+/* Load the slot once: the cache flush may mark it missing meanwhile */
+#define CBT_PAGE(cbt, idx) ({						\
+	struct page *__page = READ_ONCE((cbt)->map[idx]);		\
+	__page == CBT_PAGE_MISSED ? NULL : __page;			\
+})
 
 static __cacheline_aligned_in_smp DEFINE_MUTEX(cbt_mutex);
 
@@ -148,10 +151,12 @@ static int cbt_page_alloc(struct cbt_info  **cbt_pp, unsigned long idx,
 		return -ENOMEM;
 	}
 
-	if (likely(CBT_PAGE(cbt, idx) == NULL))
-		cbt->map[idx] = page;
-	else
+	if (likely(CBT_PAGE(cbt, idx) == NULL)) {
+		/* Release: lockless readers use the page once they see it */
+		smp_store_release(&cbt->map[idx], page);
+	} else {
 		__free_page(page);
+	}
 
 	page = NULL;
 	spin_unlock_irq(&cbt->lock);

-- 
2.55.0