[Devel] [PATCH RHEL8 COMMIT] ve: get_curr_ve: first try getting ve with rcu lock

[Konstantin Khorenko]

The commit is pushed to "branch-rh8-4.18.0-305.3.1.vz8.7.x-ovz" and will appear at https://src.openvz.org/scm/ovz/vzkernel.git
after rh8-4.18.0-305.3.1.vz8.7.10
------>
commit bb2c76daa92dd88fbce12d880b11a393864c3ac7
Author: Pavel Tikhomirov <ptikhomirov at virtuozzo.com>
Date:   Wed Aug 18 14:00:04 2021 +0300

    ve: get_curr_ve: first try getting ve with rcu lock
    
    By holding rcu lock we can have valid ve pointer. Next using css_tryget
    we can get reference on ve cgroup if it is not yet started to destroy.
    In case cgroup is destroying retry with cgroup_mutex.
    
    https://jira.sw.ru/browse/PSBM-123766
    
    Signed-off-by: Pavel Tikhomirov <ptikhomirov at virtuozzo.com>
    
    ====================
    cgroup: ifpriomap virtualization
    
    I've also added get_curr_ve() helper as it looks like in many places we
    rely that get_exec_env() gives us ve which would not free under us, but
    all processes can be moved easily from this ve in parallel and ve can be
    freed AFAICS.
    
    https://jira.sw.ru/browse/PSBM-123766
---
 kernel/cgroup/cgroup.c | 25 +++++++++++++++++++++++--
 kernel/ve/ve.c         |  2 +-
 2 files changed, 24 insertions(+), 3 deletions(-)

diff --git a/kernel/cgroup/cgroup.c b/kernel/cgroup/cgroup.c
index bff15d2d42e0..7ddac084a462 100644
--- a/kernel/cgroup/cgroup.c
+++ b/kernel/cgroup/cgroup.c
@@ -2043,8 +2043,29 @@ struct ve_struct *get_curr_ve(void)
 	struct ve_struct *ve;
 
 	/*
-	 * Under cgroup_mutex both current tasks ve cgroup and ->task_ve
-	 * pointer can't change. Corresponding cgroup_mutex around
+	 * If first thread loads current->task_ve pointer, and if just after
+	 * that current is moved by other thread from this ve cgroup to some
+	 * other and this ve cgroup gets destroyed, ve pointer gets freed, so
+	 * first thread can't use such ve pointer safely.
+	 */
+
+	/*
+	 * Fast path: Let's make it safe with rcu lock, though current can be
+	 * moved to other ve cgroup and our ve cgroup can start destroying, ve
+	 * pointer would be still valid. As it is freed in ve_destroy. And
+	 * ve_destroy is called from rcu callback after task_ve had changed.
+	 */
+	rcu_read_lock();
+	ve = rcu_dereference(current->task_ve);
+	if (css_tryget(&ve->css)) {
+		rcu_read_unlock();
+		return ve;
+	}
+	rcu_read_unlock();
+
+	/*
+	 * Slow path: Under cgroup_mutex both current tasks ve cgroup and
+	 * task_ve pointer can't change. Corresponding cgroup_mutex around
 	 * cgroup_attach_task() protects us from it.
 	 */
 	mutex_lock(&cgroup_mutex);
diff --git a/kernel/ve/ve.c b/kernel/ve/ve.c
index 017435152689..9bccdd90037d 100644
--- a/kernel/ve/ve.c
+++ b/kernel/ve/ve.c
@@ -1134,7 +1134,7 @@ static void ve_attach(struct cgroup_taskset *tset)
 			set_tsk_thread_flag(task, TIF_CPUID_OVERRIDE);
 
 		ve_set_task_start_time(ve, task);
-		task->task_ve = ve;
+		rcu_assign_pointer(task->task_ve, ve);
 	}
 }