[Devel] [PATCH RHEL COMMIT] ve: get_curr_ve: first try getting ve with rcu lock

[Konstantin Khorenko]

The commit is pushed to "branch-rh9-5.14.vz9.1.x-ovz" and will appear at https://src.openvz.org/scm/ovz/vzkernel.git
after ark-5.14
------>
commit e0d03bf51b10e2c03e8dacce8ca146858138b7bc
Author: Pavel Tikhomirov <ptikhomirov at virtuozzo.com>
Date:   Tue Oct 12 16:45:10 2021 +0300

    ve: get_curr_ve: first try getting ve with rcu lock
    
    By holding rcu lock we can have valid ve pointer. Next using css_tryget
    we can get reference on ve cgroup if it is not yet started to destroy.
    In case cgroup is destroying retry with cgroup_mutex.
    
    https://jira.sw.ru/browse/PSBM-123766
    
    Signed-off-by: Pavel Tikhomirov <ptikhomirov at virtuozzo.com>
    
    ====================
    cgroup: ifpriomap virtualization
    
    I've also added get_curr_ve() helper as it looks like in many places we
    rely that get_exec_env() gives us ve which would not free under us, but
    all processes can be moved easily from this ve in parallel and ve can be
    freed AFAICS.
    
    https://jira.sw.ru/browse/PSBM-123766
    
    Signed-off-by: Kirill Tkhai <ktkhai at virtuozzo.com>
---
 kernel/cgroup/cgroup.c | 25 +++++++++++++++++++++++--
 kernel/ve/ve.c         |  2 +-
 2 files changed, 24 insertions(+), 3 deletions(-)

diff --git a/kernel/cgroup/cgroup.c b/kernel/cgroup/cgroup.c
index 04a5e1effbaf..05fe9436a9a3 100644
--- a/kernel/cgroup/cgroup.c
+++ b/kernel/cgroup/cgroup.c
@@ -1929,8 +1929,29 @@ struct ve_struct *get_curr_ve(void)
 	struct ve_struct *ve;
 
 	/*
-	 * Under cgroup_mutex both current tasks ve cgroup and ->task_ve
-	 * pointer can't change. Corresponding cgroup_mutex around
+	 * If first thread loads current->task_ve pointer, and if just after
+	 * that current is moved by other thread from this ve cgroup to some
+	 * other and this ve cgroup gets destroyed, ve pointer gets freed, so
+	 * first thread can't use such ve pointer safely.
+	 */
+
+	/*
+	 * Fast path: Let's make it safe with rcu lock, though current can be
+	 * moved to other ve cgroup and our ve cgroup can start destroying, ve
+	 * pointer would be still valid. As it is freed in ve_destroy. And
+	 * ve_destroy is called from rcu callback after task_ve had changed.
+	 */
+	rcu_read_lock();
+	ve = rcu_dereference(current->task_ve);
+	if (css_tryget(&ve->css)) {
+		rcu_read_unlock();
+		return ve;
+	}
+	rcu_read_unlock();
+
+	/*
+	 * Slow path: Under cgroup_mutex both current tasks ve cgroup and
+	 * task_ve pointer can't change. Corresponding cgroup_mutex around
 	 * cgroup_attach_task() protects us from it.
 	 */
 	mutex_lock(&cgroup_mutex);
diff --git a/kernel/ve/ve.c b/kernel/ve/ve.c
index 4b4310dd3bfc..f9aaf135f630 100644
--- a/kernel/ve/ve.c
+++ b/kernel/ve/ve.c
@@ -903,7 +903,7 @@ static void ve_attach(struct cgroup_taskset *tset)
 		if (cpuid_override_on())
 			set_tsk_thread_flag(task, TIF_CPUID_OVERRIDE);
 
-		task->task_ve = ve;
+		rcu_assign_pointer(task->task_ve, ve);
 	}
 }