[Devel] [PATCH RH7] cgroup: fix mem_cgroup_start_move hang on synchronize_rcu

[Pavel Tikhomirov]

We've faced situation with high cgroup_mutex contention, further
investigating we see that there are a lot of mem_cgroup_start_move()
calls on massive ve stop, which is called under cgroup_mutex and can
hold it for more than one second (on 128 cpu node).

We can try to fix this exact "most oftern" case by switching from rcu
syncronization to simple spinlock. Downside of this solution is that
while probably increasing perfomance of mem_cgroup_start_move we
decrease perfomance of mem_cgroup_begin_update_page_stat path.

Original patch where this rcu_read_lock+synchronize_rcu was added:
32047e2a85f0 ("memcg: avoid lock in updating file_mapped (Was fix race
in file_mapped accouting flag management")

https://jira.sw.ru/browse/PSBM-139206

Signed-off-by: Pavel Tikhomirov <ptikhomirov at virtuozzo.com>
---
 include/linux/memcontrol.h | 5 +++--
 mm/memcontrol.c            | 4 +++-
 2 files changed, 6 insertions(+), 3 deletions(-)

diff --git a/include/linux/memcontrol.h b/include/linux/memcontrol.h
index cffdd621d7b9..cdad4934a49c 100644
--- a/include/linux/memcontrol.h
+++ b/include/linux/memcontrol.h
@@ -188,6 +188,7 @@ void __mem_cgroup_begin_update_page_stat(struct page *page, bool *locked,
 					 unsigned long *flags);
 
 extern atomic_t memcg_moving;
+extern spinlock_t mem_cgroup_start_move_lock;
 
 static inline void mem_cgroup_begin_update_page_stat(struct page *page,
 					bool *locked, unsigned long *flags)
@@ -195,7 +196,7 @@ static inline void mem_cgroup_begin_update_page_stat(struct page *page,
 	if (mem_cgroup_disabled())
 		return;
 
-	rcu_read_lock();
+	spin_lock(&mem_cgroup_start_move_lock);
 	*locked = false;
 	if (atomic_read(&memcg_moving))
 		__mem_cgroup_begin_update_page_stat(page, locked, flags);
@@ -210,7 +211,7 @@ static inline void mem_cgroup_end_update_page_stat(struct page *page,
 		return;
 	if (*locked)
 		__mem_cgroup_end_update_page_stat(page, flags);
-	rcu_read_unlock();
+	spin_unlock(&mem_cgroup_start_move_lock);
 }
 
 void mem_cgroup_update_page_stat(struct page *page,
diff --git a/mm/memcontrol.c b/mm/memcontrol.c
index aa933c25733f..863f6a732a8e 100644
--- a/mm/memcontrol.c
+++ b/mm/memcontrol.c
@@ -2267,12 +2267,14 @@ int mem_cgroup_swappiness(struct mem_cgroup *memcg)
 
 /* for quick checking without looking up memcg */
 atomic_t memcg_moving __read_mostly;
+DEFINE_SPINLOCK(mem_cgroup_start_move_lock);
 
 static void mem_cgroup_start_move(struct mem_cgroup *memcg)
 {
+	spin_lock(&mem_cgroup_start_move_lock);
 	atomic_inc(&memcg_moving);
 	atomic_inc(&memcg->moving_account);
-	synchronize_rcu();
+	spin_unlock(&mem_cgroup_start_move_lock);
 }
 
 static void mem_cgroup_end_move(struct mem_cgroup *memcg)
-- 
2.35.1