[CRIU] [RFC PATCH 08/20] criu/plugin: Implement restore late hook for kfd

[Felix Kuehling]

From: Rajneesh Bhardwaj <rajneesh.bhardwaj at amd.com>

For restarting MMU notifiers and to kick-start the evicted queues that
were created during a previuos criu restore plugin operation, we need an
external trigger that runs outside the target process context i.e. from
within master restorer context for each target (restored) pid. This
external trigger is required because we need to wait till criu pie
finalizes the VMAs relocation within the target process's address space.

When kfd recieves this ioctl, it starts the userptr worker and resumes
the MMU notifiers.

Signed-off-by: Rajneesh Bhardwaj <rajneesh.bhardwaj at amd.com>
---
 test/others/ext-kfd/kfd_plugin.c | 28 +++++++++++++++++++++++++++-
 1 file changed, 27 insertions(+), 1 deletion(-)

diff --git a/test/others/ext-kfd/kfd_plugin.c b/test/others/ext-kfd/kfd_plugin.c
index 8f9670c54..4c55892ff 100644
--- a/test/others/ext-kfd/kfd_plugin.c
+++ b/test/others/ext-kfd/kfd_plugin.c
@@ -784,7 +784,7 @@ clean:
 	xfree(bo_bucket_ptr);
 	xfree(buf);
 	criu_kfd__free_unpacked(e, NULL);
-	pr_info("kfd_plugin: returning kfd fd from plugin\n");
+	pr_info("kfd_plugin: returning kfd fd from plugin, fd = %d\n", fd);
 	return fd;
 }
 CR_PLUGIN_REGISTER_HOOK(CR_PLUGIN_HOOK__RESTORE_EXT_FILE, kfd_plugin_restore_file)
@@ -819,3 +819,29 @@ int kfd_plugin_update_vmamap(const char *old_path, char *new_path, const uint64_
 	return 0;
 }
 CR_PLUGIN_REGISTER_HOOK(CR_PLUGIN_HOOK__UPDATE_VMA_MAP, kfd_plugin_update_vmamap)
+
+int kfd_plugin_resume_devices_late(int target_pid)
+{
+	struct kfd_ioctl_criu_resume_args args = {0};
+	int fd, ret = 0;
+
+	pr_info("kfd_plugin: Inside %s for target pid = %d\n", __func__, target_pid);
+
+	fd = open("/dev/kfd", O_RDWR | O_CLOEXEC);
+	if (fd < 0) {
+		pr_perror("failed to open kfd in plugin");
+		return -1;
+	}
+
+	args.pid = target_pid;
+	pr_info("kfd_plugin: Calling IOCTL to start notifiers and queues\n");
+	if (kmtIoctl(fd, AMDKFD_IOC_CRIU_RESUME, &args) == -1) {
+		pr_perror("restore late ioctl failed\n");
+		ret = -1;
+	}
+
+	close(fd);
+	return ret;
+}
+
+CR_PLUGIN_REGISTER_HOOK(CR_PLUGIN_HOOK__RESUME_DEVICES_LATE, kfd_plugin_resume_devices_late)
-- 
2.17.1