[Devel] [PATCH RFC hci-8.1 0/4] mm: keep DMA-pinned MADV_DOEXEC memory working across exec

[Andrey Drobyshev]

QEMU live update (qemu-update, cpr-exec) keeps guest RAM across exec with
MADV_DOEXEC.  vma_dup() duplicates such VMAs the same way fork does: the
pages get shared and write-protected in both mms.

That's fine for ordinary guest RAM, but not for a VM with a VFIO device.
VFIO holds long-term pins on all guest pages, so after exec the first
guest write to a pinned page can't reuse it and copies it instead.  The
guest and the new QEMU use the copy, while the IOMMU keeps pointing the
device at the old pinned page, and device DMA silently diverges.  After
qemu-update a passed-through GPU fails.

This series moves preserved anonymous VMAs into the new mm instead of
copying them.  Page table entries, physical pages and anon-exclusive state
move as is, so nothing gets copied on write, and the device and the guest
keep seeing the same memory.  Exec is refused while other tasks still share
the old mm, since moving its memory away would break them.

Tested with qemu-update with a whole GPU passed through (RTX 6000 Ada):
the GPU keeps working after qemu-update.

https://virtuozzo.atlassian.net/browse/VSTOR-103860

Signed-off-by: Andrey Drobyshev <andrey.drobyshev at virtuozzo.com>
---
Andrey Drobyshev (4):
      mm: support moving anonymous page tables between address spaces
      exec: protect preserved memory against surviving shared-mm tasks
      mm: move MADV_DOEXEC memory instead of copying it
      selftests/mm: test memory preservation across exec

 fs/exec.c                                 | 102 +++-
 include/linux/binfmts.h                   |   2 +
 include/linux/huge_mm.h                   |   3 +-
 include/linux/mm.h                        |   4 -
 include/linux/mm_types.h                  |   5 +-
 include/linux/rmap.h                      |   7 +-
 kernel/fork.c                             |  34 ++
 mm/huge_memory.c                          |  28 +-
 mm/internal.h                             |   4 +
 mm/madvise.c                              |   2 +-
 mm/memory.c                               |  81 +---
 mm/mlock.c                                |   2 +-
 mm/mmap.c                                 |  64 ++-
 mm/mremap.c                               | 140 +++++-
 tools/testing/selftests/mm/Makefile       |   1 +
 tools/testing/selftests/mm/doexec.c       | 774 ++++++++++++++++++++++++++++++
 tools/testing/selftests/mm/run_vmtests.sh |   1 +
 17 files changed, 1112 insertions(+), 142 deletions(-)
---
base-commit: f5d1fdcfffe101fe2bafa7df09d95d2e811c5c5e
change-id: 20261007-vfio_v1-c812d56db752

Best regards,
--  
Andrey Drobyshev <andrey.drobyshev at virtuozzo.com>