[Devel] [PATCH VZ10] fs/ext4: critical suboptimality in ext4

[Alexey Kuznetsov]

Classic example of ignorant programming. test_and_set
is very expensive atomic operation, it must be avoided
unless it is strictly necessary.

The bit READ_BIT is set-only, it is set when group is
read the first time and never cleared, so that it
is safe to test it before set.

Due to this bug even on just 4 allocating threads
ext4 wastes 50% of cpu time on this test_and_set,
none with the patch. The patch has been issued
as private RK patch before and its effect has been tested
in production.

NOTE: ext4 has other performance issues,
but we do not address them for now, because they
do not have a bright signature in profiles. Examples are:
bad alignment of struct ext4_group_info, frequently accessed
read-only data share cache line with frequent writable data,
even worse some of bits are atomic. WAS_TRIMMED bit also
seems to be optimizable to be flipped less frequently.

Affects: #VSTOR-137147

Signed-off-by: Alexey Kuznetsov <kuznet at virtuozzo.com>
---
 fs/ext4/ext4.h    | 2 ++
 fs/ext4/mballoc.c | 4 +++-
 2 files changed, 5 insertions(+), 1 deletion(-)

diff --git a/fs/ext4/ext4.h b/fs/ext4/ext4.h
index bf7637785e07..ca6498609e2e 100644
--- a/fs/ext4/ext4.h
+++ b/fs/ext4/ext4.h
@@ -3514,6 +3514,8 @@ struct ext4_group_info {
 	(clear_bit(EXT4_GROUP_INFO_WAS_TRIMMED_BIT, &((grp)->bb_state)))
 #define EXT4_MB_GRP_TEST_AND_SET_READ(grp)	\
 	(test_and_set_bit(EXT4_GROUP_INFO_BBITMAP_READ_BIT, &((grp)->bb_state)))
+#define EXT4_MB_GRP_TEST_READ(grp)	\
+	(test_bit(EXT4_GROUP_INFO_BBITMAP_READ_BIT, &((grp)->bb_state)))
 
 #define EXT4_MAX_CONTENTION		8
 #define EXT4_CONTENTION_THRESHOLD	2
diff --git a/fs/ext4/mballoc.c b/fs/ext4/mballoc.c
index aaf2c24c514c..b50aa3405796 100644
--- a/fs/ext4/mballoc.c
+++ b/fs/ext4/mballoc.c
@@ -2836,7 +2836,9 @@ ext4_group_t ext4_mb_prefetch(struct super_block *sb, ext4_group_t group,
 		 * prefetch once, so we avoid getblk() call, which can
 		 * be expensive.
 		 */
-		if (gdp && grp && !EXT4_MB_GRP_TEST_AND_SET_READ(grp) &&
+		if (gdp && grp &&
+		    !EXT4_MB_GRP_TEST_READ(grp) &&
+		    !EXT4_MB_GRP_TEST_AND_SET_READ(grp) &&
 		    EXT4_MB_GRP_NEED_INIT(grp) &&
 		    ext4_free_group_clusters(sb, gdp) > 0 ) {
 			bh = ext4_read_block_bitmap_nowait(sb, group, true);
-- 
2.43.0