Skip to contentSkip to conversation

syscalls: unlimit the process's virtual memory (address space)

PATCH In review Li Wang via ltp 2 participants 3 messages Jan 30 – Feb 1

lore
Share & tools
lore
#0 lore raw

PATCH 1/1 syscalls: unlimit the process's virtual memory (address space)

Recently our test environment changes RLIMIT_AS before LTP runs, so these tests hit the per-process virtual-memory ceiling long before they reach the kernel limits they are intended to exercise.

munmap04 exits after only ~36k VMAs and the partial munmap() succeeds:

==== munmap04 ====
command: munmap04
tst_test.c:1846: TINFO: Overall timeout per run is 0h 05m 24s munmap04.c:59: TINFO: Mapped 36280 regions munmap04.c:26: TFAIL: munmap(maps[2] + page_sz, page_sz) succeeded

cve-2017-17052’s MAP_POPULATE loop hits ENOMEM and TBROKs:

==== cve-2017-17052 ====
command: cve-2017-17052
cve-2017-17052.c:48: TBROK: mmap((nil),16777216,PROT_READ(1),32802,-1,0) failed: ENOMEM (12) tst_test.c:479: TINFO: Child process reported TBROK killing the test tst_test.c:1909: TINFO: Killed the leftover descendant processes

cve-2017-1000364 (stack_clash) fails to allocate the guard gap region:

==== cve-2017-1000364 ====
command: stack_clash
tst_kconfig.c:629: TINFO: stack_guard_gap is not found in /proc/cmdline stack_clash.c:293: TINFO: STACK_GROWSDOWN = 1 == 0x7ffc2455d7c0 > 0x7ffc2455d730 stack_clash.c:243: TINFO: Stack:0x7ffc2445d000+0x103000 mmap:0x7ffc2435c000+0x1000 stack_clash.c:127: TBROK: mmap(0x1326000,140444440563712,PROT_NONE(0),1048610,-1,0) failed: ENOMEM (12) tst_test.c:479: TINFO: Child process reported TBROK killing the test
tst_test.c:1909: TINFO: Killed the leftover descendant processes

Explicitly set RLIMIT_AS=RLIM_INFINITY inside each test via struct tst_test.ulimit, ensuring they always have unlimited virtual address space regardless of test env ulimit settings.

Follow-up: c075f6c073 ("cve-2017-17052: tolerate ENOMEM during test")

---

Notes:
When I debugging munmap04 and stack_clash today, I suddenly realized that the cve-2017-17052 error was also closely related to the rlimit change, which was intentionally changed by Xiubo (my colleague) to stabilize some memory pressure scenarios.

However, this change unexpectedly exposed the need to pay attention to RLIMIT_AS in these tests, which also should be fixed in LTP upstream.

I think this patch is valuable to be merged into the new version.

 testcases/cve/cve-2017-17052.c              | 4 ++++
 testcases/cve/stack_clash.c                 | 4 ++++
 testcases/kernel/syscalls/munmap/munmap04.c | 4 ++++
 3 files changed, 12 insertions(+)
testcases/cve/cve-2017-17052.c +4 -0
index f1bff3db2..4d9448c27 100644
@@ -130,6 +130,10 @@ static struct tst_test test = {
 	.cleanup = cleanup,
 	.setup = setup,
 	.test_all = run,
+	.ulimit = (const struct tst_ulimit_val[]) {
+		{RLIMIT_AS, RLIM_INFINITY},
+		{}
+	},
 	.tags = (const struct tst_tag[]) {
 		{"linux-git", "2b7e8665b4ff"},
 		{"CVE", "2017-17052"},
testcases/cve/stack_clash.c +4 -0
index 0584b2f7a..e9dd0736a 100644
@@ -331,6 +331,10 @@ static struct tst_test test = {
 	.needs_root = 1,
 	.setup = setup,
 	.test_all = stack_clash_test,
+	.ulimit = (const struct tst_ulimit_val[]) {
+		{RLIMIT_AS, RLIM_INFINITY},
+		{}
+	},
 	.tags = (const struct tst_tag[]) {
 		{"CVE", "2017-1000364"},
 		{"linux-git", "58c5d0d6d522"},
testcases/kernel/syscalls/munmap/munmap04.c +4 -0
index e1b67aeee..6ccf25b3b 100644
@@ -75,6 +75,10 @@ static struct tst_test test = {
 	.cleanup = cleanup,
 	.needs_root = 1,
 	.min_kver = "4.17",
+	.ulimit = (const struct tst_ulimit_val[]) {
+		{RLIMIT_AS, RLIM_INFINITY},
+		{}
+	},
 	.save_restore = (const struct tst_path_val[]){
 		{ "/proc/sys/vm/max_map_count", TST_TO_STR(MAP_MAX_COUNT), TST_SR_SKIP },
 		{},
Signature · 5 lines
2.52.0


-- 
Mailing list info: https://lists.linux.it/listinfo/ltp
#2 lore raw

Hi Petr,

Patch merged, thanks!

Signature · 2 lines
Regards,
Li Wang
Quoted 9 lines · Petr Vorel

Hi Li, all,

LGTM.

Kind regards,
Petr

Signature · 5 lines
Regards,
Li Wang

-- 
Mailing list info: https://lists.linux.it/listinfo/ltp