Skip to contentSkip to conversation

mem: child alloc memory should larger than memory.max + memory.swap.max if lite==1

PATCH Superseded Li Wang 2 participants 7 messages Jun 22, 2021 – Jun 23, 2021 Library code

v2 of 3 Superseded Latest is v3 →

lore
Share & tools
lore

Li Wang

author
#0 lore raw

PATCH v2 1/3 mem: child alloc memory should larger than memory.max + memory.swap.max if lite==1

oom03 often gets fail while setting 'memory.swap.max = TESTMEM' in CGroup V2, as in that scenario (lite == 1), child_alloc only start a single process to dirty 'TESTMEM + MB' anonymous memory for testing:

testoom(, lite == 1, ,)
? oom(, lite == 1, ,)
? ? child_alloc(, lite == 1,)
? ? ? ? alloc_mem(TESTMEM + MB, )

mem.c:224: TINFO: start normal OOM testing.
mem.c:146: TINFO: expected victim is 80466.
mem.c:38: TINFO: thread (7f411c69d740), allocating 1074790400 bytes.
mem.c:64: TINFO: swapped is 25546752 bytes. <------- swap occuring -----
mem.c:164: TFAIL: victim unexpectedly ended with retcode: 0, expected: 12

TBH, this can not really test the 'memory.swap.max' as expected, since in the kernel side mem_cgroup_out_of_memory split OOM margin into two-part, one for memory.max limit, another for memory.swap.max, if any of them get overflow, then invoke out_of_memory to kill victim-process.

Theoretically, alloc_mem(TESTMEM + MB, ) should work while 'memory.max' is equal to TESTMEM, but Cgroup v2 tracks memory and swap in separate, which splits memory and swap counter. So with swappiness enable (default value is 60 on RHEL), it likely has part of memory swapping out during the allocating, upon that the two limit loss effect@the same time. Unless disable swap completely then memory.max will take effect in precisely.

To get more opportunities to reach the swap limitation, let's scale down the value of 'memory.swap.max' to only 1MB for CGroup v2.

But for CGroup v1, the memory.memsw.limit_in_bytes disallow to less than memory.limit_in_bytes, so we'd better raise the child_alloc to the twifold of TESTMEM.

---

Notes:
v1 --> v2
* add comments for explaining why set 'memory.swap.max' to 1MB
* Scale down the value of 'memory.swap.max' to only 1MB for CGroup v2.

 testcases/kernel/mem/lib/mem.c   |  2 +-
 testcases/kernel/mem/oom/oom03.c | 17 ++++++++++++++++-
 2 files changed, 17 insertions(+), 2 deletions(-)
testcases/kernel/mem/lib/mem.c +1 -1
index 9f946b5c9..ac890491c 100644
@@ -78,7 +78,7 @@ static void child_alloc(int testcase, int lite, int threads)
 	pthread_t *th;
 
 	if (lite) {
-		int ret = alloc_mem(TESTMEM + MB, testcase);
+		int ret = alloc_mem(TESTMEM * 2 + MB, testcase);
 		exit(ret);
 	}
 
testcases/kernel/mem/oom/oom03.c +16 -1
index 939413744..89d7711a5 100644
@@ -46,7 +46,22 @@ static void verify_oom(void)
 	testoom(0, 0, ENOMEM, 1);
 
 	if (SAFE_CGROUP_HAS(cg, "memory.swap.max")) {
-		SAFE_CGROUP_PRINTF(cg, "memory.swap.max", "%lu", TESTMEM);
+		/*
+		 * Cgroup v2 tracks memory and swap in separate, which splits
+		 * memory and swap counter. So with swappiness enable (default
+		 * value is 60 on RHEL), it likely has part of memory swapping
+		 * out during the allocating, upon that the two limit loss
+		 * effect at the same time.
+		 *
+		 * To get more opportunities to reach the swap limitation,
+		 * let's scale down the value of 'memory.swap.max' to only
+		 * 1MB for CGroup v2.
+		 */
+		if (TST_CGROUP_VER(cg, "memory") != TST_CGROUP_V1)
+			SAFE_CGROUP_PRINTF(cg, "memory.swap.max", "%lu", MB);
+		else
+			SAFE_CGROUP_PRINTF(cg, "memory.swap.max", "%lu", TESTMEM);
+
 		testoom(0, 1, ENOMEM, 1);
 	}
 
Signature · 1 line
2.31.1
author
#1 lore raw

PATCH v2 2/3 oom: enable lite == 1 for memory.swap.max testing

oom03: adding print info and restore memory.swap.max after testing
oom05: enable lite == 1 for memory.swap.max testing

---
 testcases/kernel/mem/oom/oom03.c | 6 ++++++
 testcases/kernel/mem/oom/oom05.c | 8 ++++++--
 2 files changed, 12 insertions(+), 2 deletions(-)
testcases/kernel/mem/oom/oom03.c +6 -0
index 89d7711a5..6728e1ddb 100644
@@ -46,6 +46,7 @@ static void verify_oom(void)
 	testoom(0, 0, ENOMEM, 1);
 
 	if (SAFE_CGROUP_HAS(cg, "memory.swap.max")) {
+		tst_res(TINFO, "OOM on MEMCG with special memswap limitation:");
 		/*
 		 * Cgroup v2 tracks memory and swap in separate, which splits
 		 * memory and swap counter. So with swappiness enable (default
@@ -63,6 +64,11 @@ static void verify_oom(void)
 			SAFE_CGROUP_PRINTF(cg, "memory.swap.max", "%lu", TESTMEM);
 
 		testoom(0, 1, ENOMEM, 1);
+
+		if (TST_CGROUP_VER(cg, "memory") == TST_CGROUP_V1)
+			SAFE_CGROUP_PRINTF(cg, "memory.swap.max", "%lu", ~0UL);
+		else
+			SAFE_CGROUP_PRINT(cg, "memory.swap.max", "max");
 	}
 
 	/* OOM for MEMCG with mempolicy */
testcases/kernel/mem/oom/oom05.c +6 -2
index 9c9bba7f6..9e1cff312 100644
@@ -63,8 +63,12 @@ static void verify_oom(void)
 	if (SAFE_CGROUP_HAS(cg, "memory.swap.max")) {
 		tst_res(TINFO, "OOM on CPUSET & MEMCG with "
 				"special memswap limitation:");
-		SAFE_CGROUP_PRINTF(cg, "memory.swap.max", "%lu", TESTMEM);
-		testoom(0, 0, ENOMEM, 1);
+		if (TST_CGROUP_VER(cg, "memory") != TST_CGROUP_V1)
+			SAFE_CGROUP_PRINTF(cg, "memory.swap.max", "%lu", MB);
+		else
+			SAFE_CGROUP_PRINTF(cg, "memory.swap.max", "%lu", TESTMEM);
+
+		testoom(0, 1, ENOMEM, 1);
 
 		tst_res(TINFO, "OOM on CPUSET & MEMCG with "
 				"disabled memswap limitation:");
Signature · 1 line
2.31.1
#2 lore raw

PATCH v2 2/3 oom: enable lite == 1 for memory.swap.max testing

Hello Li,

Quoted 38 lines · Li Wang

oom03: adding print info and restore memory.swap.max after testing
oom05: enable lite == 1 for memory.swap.max testing

---
 testcases/kernel/mem/oom/oom03.c | 6 ++++++
 testcases/kernel/mem/oom/oom05.c | 8 ++++++--
 2 files changed, 12 insertions(+), 2 deletions(-)
testcases/kernel/mem/oom/oom03.c +6 -0
index 89d7711a5..6728e1ddb 100644
@@ -46,6 +46,7 @@ static void verify_oom(void)
 	testoom(0, 0, ENOMEM, 1);
 
 	if (SAFE_CGROUP_HAS(cg, "memory.swap.max")) {
+		tst_res(TINFO, "OOM on MEMCG with special memswap limitation:");
 		/*
 		 * Cgroup v2 tracks memory and swap in separate, which splits
 		 * memory and swap counter. So with swappiness enable (default
@@ -63,6 +64,11 @@ static void verify_oom(void)
 			SAFE_CGROUP_PRINTF(cg, "memory.swap.max", "%lu", TESTMEM);
 
 		testoom(0, 1, ENOMEM, 1);
+
+		if (TST_CGROUP_VER(cg, "memory") == TST_CGROUP_V1)
+			SAFE_CGROUP_PRINTF(cg, "memory.swap.max", "%lu", ~0UL);
+		else
+			SAFE_CGROUP_PRINT(cg, "memory.swap.max", "max");
 	}
 
 	/* OOM for MEMCG with mempolicy */
testcases/kernel/mem/oom/oom05.c +0 -0
index 9c9bba7f6..9e1cff312 100644

testcases/kernel/mem/oom/oom05.c static void verify_oom(void) · line 69

Quoted 9 lines
testcases/kernel/mem/oom/oom05.c +4 -2
@@ -63,8 +63,12 @@ static void verify_oom(void)
 	if (SAFE_CGROUP_HAS(cg, "memory.swap.max")) {
 		tst_res(TINFO, "OOM on CPUSET & MEMCG with "
 				"special memswap limitation:");
-		SAFE_CGROUP_PRINTF(cg, "memory.swap.max", "%lu", TESTMEM);
-		testoom(0, 0, ENOMEM, 1);
+		if (TST_CGROUP_VER(cg, "memory") != TST_CGROUP_V1)
+			SAFE_CGROUP_PRINTF(cg, "memory.swap.max", "%lu", MB);
+		else
+			SAFE_CGROUP_PRINTF(cg, "memory.swap.max", "%lu", TESTMEM);

Again; TESTMEM + MB?

Quoted 7 lines
testcases/kernel/mem/oom/oom05.c +2 -0
+
+		testoom(0, 1, ENOMEM, 1);
 
 		tst_res(TINFO, "OOM on CPUSET & MEMCG with "
 				"disabled memswap limitation:");
Signature · 1 line
2.31.1
Signature · 2 lines
Thank you,
Richard.
author
#3 lore raw

PATCH v2 3/3 tst_cgroup: make use of alias in safe_cgroup_has

In reply to Li Wang

This is to fix the check issue of 'memory.swap.max' on CGroup v1.

Cc Richard Palethorpe <[email protected]>
---
 lib/tst_cgroup.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
lib/tst_cgroup.c +1 -1
index 18e3b6169..61cc02fa7 100644
@@ -1010,7 +1010,7 @@ int safe_cgroup_has(const char *const file, const int lineno,
 		if (!(alias = cgroup_file_alias(cfile, *dir)))
 		    continue;
 
-		if (!faccessat((*dir)->dir_fd, file_name, F_OK, 0))
+		if (!faccessat((*dir)->dir_fd, alias, F_OK, 0))
 			return 1;
 
 		if (errno == ENOENT)
Signature · 1 line
2.31.1
#4 lore raw

PATCH v2 3/3 tst_cgroup: make use of alias in safe_cgroup_has

Hello,

Quoted 13 lines · Li Wang

This is to fix the check issue of 'memory.swap.max' on CGroup v1.

Cc Richard Palethorpe <[email protected]>
---
 lib/tst_cgroup.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
lib/tst_cgroup.c +0 -0
index 18e3b6169..61cc02fa7 100644

lib/tst_cgroup.c int safe_cgroup_has(const char *const file, const int lineno, · line 1013

Quoted 4 lines
lib/tst_cgroup.c +0 -1
@@ -1010,7 +1010,7 @@ int safe_cgroup_has(const char *const file, const int lineno,
 		if (!(alias = cgroup_file_alias(cfile, *dir)))
 		    continue;
 
-		if (!faccessat((*dir)->dir_fd, file_name, F_OK, 0))

Ah, dumb mistake by me!

lib/tst_cgroup.c int safe_cgroup_has(const char *const file, const int lineno, · line 1016

Quoted 4 lines
lib/tst_cgroup.c +1 -0
+		if (!faccessat((*dir)->dir_fd, alias, F_OK, 0))
 			return 1;
 
 		if (errno == ENOENT)
Signature · 2 lines
Thank you,
Richard.
#5 lore raw

PATCH v2 1/3 mem: child alloc memory should larger than memory.max + memory.swap.max if lite==1

In reply to Li Wang

Hello Li,

Quoted 27 lines · Li Wang

oom03 often gets fail while setting 'memory.swap.max = TESTMEM' in CGroup V2, as in that scenario (lite == 1), child_alloc only start a single process to dirty 'TESTMEM + MB' anonymous memory for testing:

testoom(, lite == 1, ,)
oom(, lite == 1, ,)
child_alloc(, lite == 1,)
alloc_mem(TESTMEM + MB, )

mem.c:224: TINFO: start normal OOM testing.
mem.c:146: TINFO: expected victim is 80466.
mem.c:38: TINFO: thread (7f411c69d740), allocating 1074790400 bytes.
mem.c:64: TINFO: swapped is 25546752 bytes. <------- swap occuring -----
mem.c:164: TFAIL: victim unexpectedly ended with retcode: 0, expected: 12

TBH, this can not really test the 'memory.swap.max' as expected, since in the kernel side mem_cgroup_out_of_memory split OOM margin into two-part, one for memory.max limit, another for memory.swap.max, if any of them get overflow, then invoke out_of_memory to kill victim-process.

Theoretically, alloc_mem(TESTMEM + MB, ) should work while 'memory.max' is equal to TESTMEM, but Cgroup v2 tracks memory and swap in separate, which splits memory and swap counter. So with swappiness enable (default value is 60 on RHEL), it likely has part of memory swapping out during the allocating, upon that the two limit loss effect at the same time. Unless disable swap completely then memory.max will take effect in precisely.

Quoted 6 lines

To get more opportunities to reach the swap limitation, let's scale down the value of 'memory.swap.max' to only 1MB for CGroup v2.

But for CGroup v1, the memory.memsw.limit_in_bytes disallow to less than memory.limit_in_bytes, so we'd better raise the child_alloc to the twifold

^twofold

Quoted 1 line

of TESTMEM.

Ah, this means "memory.swap.x" and "memory.memsw.x" are not really the same thing. This seems to be common pattern, so maybe we could translate V2 values to V1 in the library.

If I understand correctly `memory.swap.max = memory.memsw.limit_in_bytes
- memory.limit_in_bytes`? Also "max" can be mapped to ~0UL or maybe ~0ULL when -m32 is used.

This is not important for the current patch.

Quoted 41 lines

---

Notes:
v1 --> v2
* add comments for explaining why set 'memory.swap.max' to 1MB
* Scale down the value of 'memory.swap.max' to only 1MB for CGroup v2.

 testcases/kernel/mem/lib/mem.c   |  2 +-
 testcases/kernel/mem/oom/oom03.c | 17 ++++++++++++++++-
 2 files changed, 17 insertions(+), 2 deletions(-)
testcases/kernel/mem/lib/mem.c +1 -1
index 9f946b5c9..ac890491c 100644
@@ -78,7 +78,7 @@ static void child_alloc(int testcase, int lite, int threads)
 	pthread_t *th;
 
 	if (lite) {
-		int ret = alloc_mem(TESTMEM + MB, testcase);
+		int ret = alloc_mem(TESTMEM * 2 + MB, testcase);
 		exit(ret);
 	}
 
testcases/kernel/mem/oom/oom03.c +6 -1
index 939413744..89d7711a5 100644
@@ -46,7 +46,22 @@ static void verify_oom(void)
 	testoom(0, 0, ENOMEM, 1);
 
 	if (SAFE_CGROUP_HAS(cg, "memory.swap.max")) {
-		SAFE_CGROUP_PRINTF(cg, "memory.swap.max", "%lu", TESTMEM);
+		/*
+		 * Cgroup v2 tracks memory and swap in separate, which splits
+		 * memory and swap counter. So with swappiness enable (default
+		 * value is 60 on RHEL), it likely has part of memory swapping
+		 * out during the allocating, upon that the two limit loss
+		 * effect at the same time.

testcases/kernel/mem/oom/oom03.c static void verify_oom(void) · line 64

Quoted 10 lines
testcases/kernel/mem/oom/oom03.c +10 -0
+		 *
+		 * To get more opportunities to reach the swap limitation,
+		 * let's scale down the value of 'memory.swap.max' to only
+		 * 1MB for CGroup v2.
+		 */
+		if (TST_CGROUP_VER(cg, "memory") != TST_CGROUP_V1)
+			SAFE_CGROUP_PRINTF(cg, "memory.swap.max", "%lu", MB);
+		else
+			SAFE_CGROUP_PRINTF(cg, "memory.swap.max", "%lu", TESTMEM);
+

To be consistent with V2; should this be TESTMEM + MB?

Quoted 5 lines
testcases/kernel/mem/oom/oom03.c +0 -0
 		testoom(0, 1, ENOMEM, 1);
 	}
 
Signature · 1 line
2.31.1
Signature · 2 lines
Thank you,
Richard.
author
#6 lore raw

PATCH v2 1/3 mem: child alloc memory should larger than memory.max + memory.swap.max if lite==1

Hi Richard,

Thanks for the review.

Quoted 50 lines · Richard Palethorpe

Hello Li,

Quoted 1 line · Li Wang

oom03 often gets fail while setting 'memory.swap.max = TESTMEM' in

CGroup V2,

Quoted 1 line

as in that scenario (lite == 1), child_alloc only start a single process

to

Quoted 11 lines

dirty 'TESTMEM + MB' anonymous memory for testing:

testoom(, lite == 1, ,)
oom(, lite == 1, ,)
child_alloc(, lite == 1,)
alloc_mem(TESTMEM + MB, )

mem.c:224: TINFO: start normal OOM testing.
mem.c:146: TINFO: expected victim is 80466.
mem.c:38: TINFO: thread (7f411c69d740), allocating 1074790400 bytes.
mem.c:64: TINFO: swapped is 25546752 bytes. <------- swap occuring

-----

Quoted 1 line

mem.c:164: TFAIL: victim unexpectedly ended with retcode: 0, expected:

12

Quoted 2 lines

TBH, this can not really test the 'memory.swap.max' as expected, since

in the

Quoted 1 line

kernel side mem_cgroup_out_of_memory split OOM margin into two-part, one

for

Quoted 1 line

memory.max limit, another for memory.swap.max, if any of them get

overflow,

Quoted 3 lines

then invoke out_of_memory to kill victim-process.

Theoretically, alloc_mem(TESTMEM + MB, ) should work while 'memory.max'

is equal

Quoted 1 line

to TESTMEM, but Cgroup v2 tracks memory and swap in separate, which

splits memory

Quoted 1 line

and swap counter. So with swappiness enable (default value is 60 on

RHEL), it

Quoted 1 line

likely has part of memory swapping out during the allocating, upon that

the two

Quoted 1 line

limit loss effect at the same time. Unless disable swap completely then

memory.max

Quoted 3 lines

will take effect in precisely.

To get more opportunities to reach the swap limitation, let's scale down

the

Quoted 5 lines

value of 'memory.swap.max' to only 1MB for CGroup v2.

But for CGroup v1, the memory.memsw.limit_in_bytes disallow to less than memory.limit_in_bytes, so we'd better raise the child_alloc to the twifold

Quoted 7 lines

^twofold

Quoted 1 line

of TESTMEM.

Ah, this means "memory.swap.x" and "memory.memsw.x" are not really the same thing. This seems to be common pattern, so maybe we could translate V2 values to V1 in the library.

+1
We can consider doing that in a new separate patch. And better to check more parameters to guarantee we have the correct understanding in use it:).

Quoted 4 lines

If I understand correctly `memory.swap.max = memory.memsw.limit_in_bytes
- memory.limit_in_bytes`? Also "max" can be mapped to ~0UL or maybe ~0ULL when -m32 is used.

From the definition, yes, the memory.memsw.limit_in_bytes parameter represents the sum of memory and swap.

Quoted 58 lines

This is not important for the current patch.

Quoted 8 lines

---

Notes:
v1 --> v2
* add comments for explaining why set 'memory.swap.max' to 1MB
* Scale down the value of 'memory.swap.max' to only 1MB for

CGroup v2.

Review

Quoted 6 lines
 testcases/kernel/mem/lib/mem.c   |  2 +-
 testcases/kernel/mem/oom/oom03.c | 17 ++++++++++++++++-
 2 files changed, 17 insertions(+), 2 deletions(-)
(quoted diff) +0 -0

b/testcases/kernel/mem/lib/mem.c

testcases/kernel/mem/lib/mem.c static void child_alloc(int testcase, int lite, int

Quoted 4 lines

index 9f946b5c9..ac890491c 100644

testcases/kernel/mem/lib/mem.c +0 -0
@@ -78,7 +78,7 @@ static void child_alloc(int testcase, int lite, int

threads)

Review

Quoted 9 lines
testcases/kernel/mem/lib/mem.c +1 -1
      pthread_t *th;
 
      if (lite) {
-             int ret = alloc_mem(TESTMEM + MB, testcase);
+             int ret = alloc_mem(TESTMEM * 2 + MB, testcase);
              exit(ret);
      }
 
(quoted diff) +0 -0

b/testcases/kernel/mem/oom/oom03.c

testcases/kernel/mem/oom/oom03.c static void verify_oom(void) · line 50

Quoted 10 lines

index 939413744..89d7711a5 100644

testcases/kernel/mem/oom/oom03.c +2 -1
@@ -46,7 +46,22 @@ static void verify_oom(void)
      testoom(0, 0, ENOMEM, 1);
 
      if (SAFE_CGROUP_HAS(cg, "memory.swap.max")) {
-             SAFE_CGROUP_PRINTF(cg, "memory.swap.max", "%lu", TESTMEM);
+             /*
+              * Cgroup v2 tracks memory and swap in separate, which

splits

testcases/kernel/mem/oom/oom03.c static void verify_oom(void) · line 51

Quoted 1 line
testcases/kernel/mem/oom/oom03.c +1 -0
+              * memory and swap counter. So with swappiness enable

(default

testcases/kernel/mem/oom/oom03.c static void verify_oom(void) · line 52

Quoted 1 line
testcases/kernel/mem/oom/oom03.c +1 -0
+              * value is 60 on RHEL), it likely has part of memory

swapping

testcases/kernel/mem/oom/oom03.c static void verify_oom(void)

Quoted 9 lines
testcases/kernel/mem/oom/oom03.c +9 -0
+              * out during the allocating, upon that the two limit loss
+              * effect at the same time.
+              *
+              * To get more opportunities to reach the swap limitation,
+              * let's scale down the value of 'memory.swap.max' to only
+              * 1MB for CGroup v2.
+              */
+             if (TST_CGROUP_VER(cg, "memory") != TST_CGROUP_V1)
+                     SAFE_CGROUP_PRINTF(cg, "memory.swap.max", "%lu",
Quoted 8 lines

MB);

testcases/kernel/mem/oom/oom03.c static void verify_oom(void) · line 63

Quoted 2 lines
testcases/kernel/mem/oom/oom03.c +2 -0
+             else
+                     SAFE_CGROUP_PRINTF(cg, "memory.swap.max", "%lu",

TESTMEM);

testcases/kernel/mem/oom/oom03.c static void verify_oom(void) · line 64

Quoted 1 line
testcases/kernel/mem/oom/oom03.c +1 -0
+

To be consistent with V2; should this be TESTMEM + MB?

Yes, that should be better.

Quoted 12 lines
Quoted 5 lines
testcases/kernel/mem/oom/oom03.c +0 -1
              testoom(0, 1, ENOMEM, 1);
      }
 
--

2.31.1

--
Thank you,
Richard.

Signature · 5 lines
Regards,
Li Wang
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.linux.it/pipermail/ltp/attachments/20210623/087d0ef6/attachment-0001.htm>