[Devel] [PATCH] selftests/core: fix unshare_test with large fs.nr_open

[Eva Kurchatova]

The test assumes fs.nr_open is close to the default 1048576, but some
systems set it much higher (e.g. 1073741816). In that case, dup2() to
nr_open + 64 requires the kernel to allocate a file descriptor table
with ~1 billion entries, which fails with ENOMEM.

Fix by setting fs.nr_open to a known reasonable base value (1048576)
for the duration of the test, rather than using whatever the system
happens to have.

Signed-off-by: Eva Kurchatova <eva.kurchatova at virtuozzo.com>

https://virtuozzo.atlassian.net/browse/VSTOR-131560
Feature: fix kselftests
---
 tools/testing/selftests/core/unshare_test.c | 4 ++++
 1 file changed, 4 insertions(+)

diff --git a/tools/testing/selftests/core/unshare_test.c b/tools/testing/selftests/core/unshare_test.c
index 7fec9dfb1b0e..f4a4aac6afc4 100644
--- a/tools/testing/selftests/core/unshare_test.c
+++ b/tools/testing/selftests/core/unshare_test.c
@@ -40,6 +40,10 @@ TEST(unshare_EMFILE)
 
 	ASSERT_EQ(sscanf(buf, "%d", &nr_open), 1);
 
+	/* Cap nr_open to avoid ENOMEM from a huge fd table allocation */
+	if (nr_open > 1024 * 1024)
+		n = sprintf(buf, "%d\n", (nr_open = 1024 * 1024));
+
 	ASSERT_EQ(0, getrlimit(RLIMIT_NOFILE, &rlimit));
 
 	/* bump fs.nr_open */
-- 
2.54.0