[Devel] [PATCH RHEL7 COMMIT] ve: get_curr_ve: first try getting ve with rcu lock

[Vasily Averin]

The commit is pushed to "branch-rh7-3.10.0-1160.36.2.vz7.182.x-ovz" and will appear at https://src.openvz.org/scm/ovz/vzkernel.git
after rh7-3.10.0-1160.36.2.vz7.182.1
------>
commit d5e8cbbf52cbfd6e94df2c819324c6c3f81d5c97
Author: Pavel Tikhomirov <ptikhomirov at virtuozzo.com>
Date:   Tue Aug 24 14:40:06 2021 +0300

    ve: get_curr_ve: first try getting ve with rcu lock
    
    By holding rcu lock we can have valid ve pointer. Next using css_tryget
    we can get reference on ve cgroup if it is not yet started to destroy.
    In case cgroup is destroying retry with cgroup_mutex.
    
    https://jira.sw.ru/browse/PSBM-123766
    
    Signed-off-by: Pavel Tikhomirov <ptikhomirov at virtuozzo.com>
---
 kernel/cgroup.c | 27 ++++++++++++++++++++++++---
 kernel/ve/ve.c  |  2 +-
 2 files changed, 25 insertions(+), 4 deletions(-)

diff --git a/kernel/cgroup.c b/kernel/cgroup.c
index 8f01cb9..846e226 100644
--- a/kernel/cgroup.c
+++ b/kernel/cgroup.c
@@ -4655,10 +4655,31 @@ struct ve_struct *get_curr_ve(void)
 	struct ve_struct *ve;
 
 	/*
-	 * Under cgroup_mutex both current tasks ve cgroup and ->task_ve
-	 * pointer can't change. Corresponding cgroup_mutex around
-	 * cgroup_attach_task() protects us from it.
+	 * If first thread loads current->task_ve pointer, and if just after
+	 * that current is moved by other thread from this ve cgroup to some
+	 * other and this ve cgroup gets destroyed, ve pointer gets freed, so
+	 * first thread can't use such ve pointer safely.
+	 */
+
+	/*
+	 * Fast path: Let's make it safe with rcu lock, though current can be
+	 * moved to other ve cgroup and our ve cgroup can start destroying, ve
+	 * pointer would be still valid. As it is freed in ve_destroy. And
+	 * ve_destroy is called from rcu callback after task_ve had changed.
 	 */
+	rcu_read_lock();
+	ve = rcu_dereference(current->task_ve);
+	if (css_tryget(&ve->css)) {
+		rcu_read_unlock();
+		return ve;
+	}
+	rcu_read_unlock();
+
+	/*
+	 * Slow path: Under cgroup_mutex both current tasks ve cgroup and
+	 * task_ve pointer can't change. Corresponding cgroup_mutex around
+	 * cgroup_attach_task() protects us from it.
+	*/
 	mutex_lock(&cgroup_mutex);
 	ve = get_ve(current->task_ve);
 	mutex_unlock(&cgroup_mutex);
diff --git a/kernel/ve/ve.c b/kernel/ve/ve.c
index 4480476..68c3e91 100644
--- a/kernel/ve/ve.c
+++ b/kernel/ve/ve.c
@@ -1099,7 +1099,7 @@ static void ve_attach(struct cgroup *cg, struct cgroup_taskset *tset)
 
 		ve_try_set_task_start_time(ve, task);
 
-		task->task_ve = ve;
+		rcu_assign_pointer(task->task_ve, ve);
 	}
 
 	/* Adjust cpuid faulting */