[Devel] [PATCH 06/11] ms/x86/mm: Introduce arch_rnd() to compute 32/64 mmap random base

[Dmitry Safonov]

The compat (32bit) mmap() sycall issued by a 64-bit task results in a
mapping above 4GB. That's outside the compat mode address space and
prevents CRIU to restore 32bit processes from a 64bit application.

As a first step to address this, split out the address base randomizing
calculation from arch_mmap_rnd() into a helper function, which can be used
independent of mmap_ia32() based decisions.

[ tglx: Massaged changelog ]

Suggested-by: Thomas Gleixner <tglx at linutronix.de>
Signed-off-by: Dmitry Safonov <dsafonov at virtuozzo.com>
Cc: 0x7f454c46 at gmail.com
Cc: linux-mm at kvack.org
Cc: Andy Lutomirski <luto at kernel.org>
Cc: Cyrill Gorcunov <gorcunov at openvz.org>
Cc: Borislav Petkov <bp at suse.de>
Cc: "Kirill A. Shutemov" <kirill.shutemov at linux.intel.com>
Link: http://lkml.kernel.org/r/20170306141721.9188-2-dsafonov@virtuozzo.com
Signed-off-by: Thomas Gleixner <tglx at linutronix.de>

[ms commit 6a0b41d1e23d
 1 skipped commit d07e22597d1d ("mm: mmap: add new /proc tunable for
   mmap_base ASLR"), which adds mmap_rnd_bits to procfs
 2 skipped commit 9e08f57d684a ("x86: mm: support ARCH_MMAP_RND_BITS"),
   which refactors this to (1) commit]
Signed-off-by: Dmitry Safonov <dsafonov at virtuozzo.com>

Signed-off-by: Dmitry Safonov <dsafonov at virtuozzo.com>
---
 arch/x86/mm/mmap.c | 27 +++++++++++++++------------
 1 file changed, 15 insertions(+), 12 deletions(-)

diff --git a/arch/x86/mm/mmap.c b/arch/x86/mm/mmap.c
index cb7cd6e3e686..83a986ef7651 100644
--- a/arch/x86/mm/mmap.c
+++ b/arch/x86/mm/mmap.c
@@ -54,6 +54,14 @@ static unsigned long stack_maxrandom_size(void)
 #define MIN_GAP (128*1024*1024UL + stack_maxrandom_size())
 #define MAX_GAP (TASK_SIZE/6*5)
 
+#ifdef CONFIG_COMPAT
+# define mmap32_rnd_bits  8
+# define mmap64_rnd_bits  28
+#else
+# define mmap32_rnd_bits  28
+# define mmap64_rnd_bits  28
+#endif
+
 static int mmap_is_legacy(void)
 {
 	if (current->personality & ADDR_COMPAT_LAYOUT)
@@ -65,20 +73,15 @@ static int mmap_is_legacy(void)
 	return sysctl_legacy_va_layout;
 }
 
-unsigned long arch_mmap_rnd(void)
+static unsigned long arch_rnd(unsigned int rndbits)
 {
-	unsigned long rnd;
-
-	/*
-	 *  8 bits of randomness in 32bit mmaps, 20 address space bits
-	 * 28 bits of randomness in 64bit mmaps, 40 address space bits
-	 */
-	if (mmap_is_ia32())
-		rnd = (unsigned long)get_random_int() % (1<<8);
-	else
-		rnd = (unsigned long)get_random_int() % (1<<28);
+	return ((unsigned long)get_random_int() &
+			((1UL << rndbits) - 1)) << PAGE_SHIFT;
+}
 
-	return rnd << PAGE_SHIFT;
+unsigned long arch_mmap_rnd(void)
+{
+	return arch_rnd(mmap_is_ia32() ? mmap32_rnd_bits : mmap64_rnd_bits);
 }
 
 static unsigned long mmap_base(unsigned long rnd)
-- 
2.12.2