Skip to contentSkip to conversation

tst_pollute_memory(): Set minimal safety margin to 64MB

PATCH New Martin Doucha 3 participants 9 messages Jan 18, 2021 – Jan 21, 2021 Library code

lore
Share & tools
lore
#0 lore raw

PATCH 1/1 tst_pollute_memory(): Set minimal safety margin to 64MB

4096 pages amounts to 16MB on x86_64 or 256MB on PPC64. 16MB is not enough to avoid OOM killer on some systems so set the safety margin to either 64MB or 4096 pages, whichever is higher.

---

Here's my proposed fix for the OOM issues with ioctl_sg01. I can't reproduce the issue on my x86_64 VMs/machines so please confirm that it works.

Don't forget to run make clean after applying this patch because ioctl_sg01 will not be rebuilt automatically due to weak dependency on libltp.a.

 lib/tst_memutils.c | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)
lib/tst_memutils.c +2 -1
index f134d90c9..c20c94a00 100644
@@ -20,7 +20,8 @@ void tst_pollute_memory(size_t maxsize, int fillchar)
 	struct sysinfo info;
 
 	SAFE_SYSINFO(&info);
-	safety = 4096 * SAFE_SYSCONF(_SC_PAGESIZE) / info.mem_unit;
+	safety = MAX(4096 * SAFE_SYSCONF(_SC_PAGESIZE), 64 * 1024 * 1024);
+	safety /= info.mem_unit;
 
 	if (info.freeswap > safety)
 		safety = 0;
Signature · 1 line
2.29.2
#1 lore raw

PATCH 1/1 tst_pollute_memory(): Set minimal safety margin to 64MB

Hi!

lib/tst_memutils.c void tst_pollute_memory(size_t maxsize, int fillchar) · line 24

Quoted 4 lines
lib/tst_memutils.c +2 -1
 	SAFE_SYSINFO(&info);
-	safety = 4096 * SAFE_SYSCONF(_SC_PAGESIZE) / info.mem_unit;
+	safety = MAX(4096 * SAFE_SYSCONF(_SC_PAGESIZE), 64 * 1024 * 1024);
+	safety /= info.mem_unit;

I guess that this is safe enough for the release, since it will only increase the safety margin.

Naresh can you please test this patch ASAP?

Signature · 2 lines
Cyril Hrubis
[email protected]
#2 lore raw

PATCH 1/1 tst_pollute_memory(): Set minimal safety margin to 64MB

Quoted 11 lines · Cyril Hrubis

Hi!

lib/tst_memutils.c void tst_pollute_memory(size_t maxsize, int fillchar) · line 24

Quoted 4 lines
      SAFE_SYSINFO(&info);
lib/tst_memutils.c +2 -1
-     safety = 4096 * SAFE_SYSCONF(_SC_PAGESIZE) / info.mem_unit;
+     safety = MAX(4096 * SAFE_SYSCONF(_SC_PAGESIZE), 64 * 1024 * 1024);
+     safety /= info.mem_unit;

I guess that this is safe enough for the release, since it will only increase the safety margin.

Naresh can you please test this patch ASAP?

I have applied your patch and rebuilt completely and retested ioctl_sg01 test case in a loop on three different devices.

1 PASS out of 20 runs with overcommit_memory 0 on x86_64. 1 PASS out of 20 runs with overcommit_memory 1 on x86_64.

Which means 19 times the test case triggered oom-killer and the test was broken.

- Naresh

#3 lore raw

PATCH 1/1 tst_pollute_memory(): Set minimal safety margin to 64MB

Hi!

Quoted 9 lines
Quoted 8 lines

lib/tst_memutils.c void tst_pollute_memory(size_t maxsize, int fillchar) · line 24

Quoted 3 lines
lib/tst_memutils.c +2 -1
-     safety = 4096 * SAFE_SYSCONF(_SC_PAGESIZE) / info.mem_unit;
+     safety = MAX(4096 * SAFE_SYSCONF(_SC_PAGESIZE), 64 * 1024 * 1024);
+     safety /= info.mem_unit;

I guess that this is safe enough for the release, since it will only increase the safety margin.

Naresh can you please test this patch ASAP?

Quoted 7 lines

I have applied your patch and rebuilt completely and retested ioctl_sg01 test case in a loop on three different devices.

1 PASS out of 20 runs with overcommit_memory 0 on x86_64. 1 PASS out of 20 runs with overcommit_memory 1 on x86_64.

Which means 19 times the test case triggered oom-killer and the test was broken.

So it looks like 64MB is not enough in your case.

Does it work with 128MB or 256MB?

i.e.:

safety = MAX(4096 * SAFE_SYSCONF(_SC_PAGESIZE), 128 * 1024 * 1024); safety = MAX(4096 * SAFE_SYSCONF(_SC_PAGESIZE), 256 * 1024 * 1024);

Signature · 2 lines
Cyril Hrubis
[email protected]
#4 lore raw

PATCH 1/1 tst_pollute_memory(): Set minimal safety margin to 64MB

Quoted 19 lines · Cyril Hrubis

Hi!

Quoted 9 lines
Quoted 8 lines

lib/tst_memutils.c void tst_pollute_memory(size_t maxsize, int fillchar) · line 24

Quoted 3 lines
lib/tst_memutils.c +2 -1
-     safety = 4096 * SAFE_SYSCONF(_SC_PAGESIZE) / info.mem_unit;
+     safety = MAX(4096 * SAFE_SYSCONF(_SC_PAGESIZE), 64 * 1024 * 1024);
+     safety /= info.mem_unit;

I guess that this is safe enough for the release, since it will only increase the safety margin.

Naresh can you please test this patch ASAP?

Quoted 7 lines

I have applied your patch and rebuilt completely and retested ioctl_sg01 test case in a loop on three different devices.

1 PASS out of 20 runs with overcommit_memory 0 on x86_64. 1 PASS out of 20 runs with overcommit_memory 1 on x86_64.

Which means 19 times the test case triggered oom-killer and the test was broken.

Quoted 8 lines

So it looks like 64MB is not enough in your case.

Does it work with 128MB or 256MB?

i.e.:

safety = MAX(4096 * SAFE_SYSCONF(_SC_PAGESIZE), 128 * 1024 * 1024); safety = MAX(4096 * SAFE_SYSCONF(_SC_PAGESIZE), 256 * 1024 * 1024);

I will test with these changes and get back to you.

Meanwhile,

20 PASS out of 20 runs with overcommit_memory 2 on x86_64.

- Naresh

#5 lore raw

PATCH 1/1 tst_pollute_memory(): Set minimal safety margin to 64MB

Hi!

Quoted 18 lines
Quoted 8 lines
Quoted 7 lines

I have applied your patch and rebuilt completely and retested ioctl_sg01 test case in a loop on three different devices.

1 PASS out of 20 runs with overcommit_memory 0 on x86_64. 1 PASS out of 20 runs with overcommit_memory 1 on x86_64.

Which means 19 times the test case triggered oom-killer and the test was broken.

Quoted 8 lines

So it looks like 64MB is not enough in your case.

Does it work with 128MB or 256MB?

i.e.:

safety = MAX(4096 * SAFE_SYSCONF(_SC_PAGESIZE), 128 * 1024 * 1024); safety = MAX(4096 * SAFE_SYSCONF(_SC_PAGESIZE), 256 * 1024 * 1024);

Quoted 5 lines

I will test with these changes and get back to you.

Meanwhile,

20 PASS out of 20 runs with overcommit_memory 2 on x86_64.

That is to be expected, since with overcommit_memory=2 we fail the mmap() when the kernel does not think there is enough free memory. See the part where we do:

	if (map_blocks[i] == MAP_FAILED) {
		map_count = i;
		break;
	}

It would be interesting to print the map_count and the value of i in this fucntion and see where it breaks with overcommit_memory=2.

i.e.

	if (map_blocks[i] == MAP_FAILED) {
		tst_res(TINFO, "mmap() failed at %zu expected %zu",
		        i, map_count);
		map_count = i;
		break;
	}
Signature · 2 lines
Cyril Hrubis
[email protected]
#6 lore raw

PATCH 1/1 tst_pollute_memory(): Set minimal safety margin to 64MB

Quoted 17 lines · Cyril Hrubis

Hi!

Quoted 15 lines
Quoted 15 lines
Quoted 7 lines

I have applied your patch and rebuilt completely and retested ioctl_sg01 test case in a loop on three different devices.

1 PASS out of 20 runs with overcommit_memory 0 on x86_64. 1 PASS out of 20 runs with overcommit_memory 1 on x86_64.

Which means 19 times the test case triggered oom-killer and the test was broken.

So it looks like 64MB is not enough in your case.

Does it work with 128MB or 256MB?

i.e.:

safety = MAX(4096 * SAFE_SYSCONF(_SC_PAGESIZE), 128 * 1024 * 1024);

After changing to 128MB the test got PASS with overcommit_memory 0 on x86_64.

- Naresh

#7 lore raw

PATCH 1/1 tst_pollute_memory(): Set minimal safety margin to 64MB

Quoted 11 lines · Naresh Kamboju
Quoted 8 lines · Cyril Hrubis
Quoted 8 lines
Quoted 8 lines

So it looks like 64MB is not enough in your case.

Does it work with 128MB or 256MB?

i.e.:

safety = MAX(4096 * SAFE_SYSCONF(_SC_PAGESIZE), 128 * 1024 * 1024);

After changing to 128MB the test got PASS with overcommit_memory 0 on x86_64.

OK, let's go with 128MB then.

Signature · 7 lines
Martin Doucha   [email protected]
QA Engineer for Software Maintenance
SUSE LINUX, s.r.o.
CORSO IIa
Krizikova 148/34
186 00 Prague 8
Czech Republic