[Devel] [PATCH RHEL7 COMMIT] memcg: optimize the "Search everything else" loop in mm_update_next_owner()

[Konstantin Khorenko]

The commit is pushed to "branch-rh7-3.10.0-957.12.2.vz7.96.x-ovz" and will appear at https://src.openvz.org/scm/ovz/vzkernel.git
after rh7-3.10.0-957.12.2.vz7.96.15
------>
commit 79185c03e4397c1e58aeb026f3bc97a4c5ee6b8d
Author: Oleg Nesterov <oleg at redhat.com>
Date:   Fri Jun 14 19:29:16 2019 +0300

    memcg: optimize the "Search everything else" loop in mm_update_next_owner()
    
    for_each_process_thread() is sub-optimal. All threads share the same
    ->mm, we can swicth to the next process once we found a thread with
    ->mm != NULL and ->mm != mm.
    
    Signed-off-by: Oleg Nesterov <oleg at redhat.com>
    Reviewed-by: Michal Hocko <mhocko at suse.cz>
    Cc: Balbir Singh <bsingharora at gmail.com>
    Cc: Johannes Weiner <hannes at cmpxchg.org>
    Cc: KAMEZAWA Hiroyuki <kamezawa.hiroyu at jp.fujitsu.com>
    Cc: Michal Hocko <mhocko at suse.cz>
    Cc: Peter Chiang <pchiang at nvidia.com>
    Signed-off-by: Andrew Morton <akpm at linux-foundation.org>
    Signed-off-by: Linus Torvalds <torvalds at linux-foundation.org>
    
    https://pmc.acronis.com/browse/VSTOR-23975
    (cherry picked from commit 39af1765f1255b2bbadc3064e16270781abf24a1)
    Signed-off-by: Andrey Ryabinin <aryabinin at virtuozzo.com>
---
 kernel/exit.c | 12 +++++++++---
 1 file changed, 9 insertions(+), 3 deletions(-)

diff --git a/kernel/exit.c b/kernel/exit.c
index 659f0286096c..af6d9c54119b 100644
--- a/kernel/exit.c
+++ b/kernel/exit.c
@@ -493,9 +493,15 @@ void mm_update_next_owner(struct mm_struct *mm)
 	/*
 	 * Search through everything else, we should not get here often.
 	 */
-	for_each_process_thread(g, c) {
-		if (!(c->flags & PF_KTHREAD) && c->mm == mm)
-			goto assign_new_owner;
+	for_each_process(g) {
+		if (g->flags & PF_KTHREAD)
+			continue;
+		for_each_thread(g, c) {
+			if (c->mm == mm)
+				goto assign_new_owner;
+			if (c->mm)
+				break;
+		}
 	}
 
 	qread_unlock(&tasklist_lock);