mem/lib: keep allocating memory until get an error in single process

PATCH New Li Wang 2 participants 5 messages Jun 20, 2016 – Jun 21, 2016

lore

Share & tools

lore

Li Wang

author

2016-06-20 09:53 UTC #0 lore raw

PATCH 1/1 mem/lib: keep allocating memory until get an error in single process

We occasionally catch errors like:

oom03       0  TINFO  :  start OOM testing for KSM pages.
oom03       0  TINFO  :  expected victim is 3490.
oom03       6  TFAIL  :  mem.c:163: victim unexpectedly ended with retcode: 0, expected: 12
oom03       0  TINFO  :  set overcommit_memory to 0

It cames from the caller testoom(0, 1, ENOMEM, 1). The full reason is that function child_alloc() go into single process mode, then successfully finish the memory allocation and return 0.

In this patch, let's make it (in single mode) keep allocating memory with an incresed length in order to avoid 0 returned.

Signed-off-by Li Wang <[email protected]>

---
 testcases/kernel/mem/lib/mem.c | 10 ++++++++--
 1 file changed, 8 insertions(+), 2 deletions(-)

testcases/kernel/mem/lib/mem.c +8 -2

index 3d853a3..c1f91ce 100644
@@ -70,11 +70,17 @@ static void *child_alloc_thread(void *args)
 
 static void child_alloc(int testcase, int lite, int threads)
 {
-	int i;
+	int i, ret;
 	pthread_t *th;
 
 	if (lite) {
-		int ret = alloc_mem(TESTMEM + MB, testcase);
+		long length = TESTMEM + MB;
+
+		/* keep allocating until there's an error */
+		while (!ret) {
+			ret = alloc_mem(length, testcase);
+			length += MB;
+		}
 		exit(ret);
 	}

Signature · 1 line

1.8.3.1

Jan Stancek

2016-06-20 10:43 UTC #1 lore raw

PATCH 1/1 mem/lib: keep allocating memory until get an error in single process

----- Original Message -----

Quoted 10 lines

From: "Li Wang" <[email protected]>
To: [email protected]
Cc: [email protected]
Sent: Monday, 20 June, 2016 11:53:05 AM
Subject: [PATCH] mem/lib: keep allocating memory until get an error in single process
We occasionally catch errors like:
oom03       0  TINFO  :  start OOM testing for KSM pages.
oom03       0  TINFO  :  expected victim is 3490.
oom03       6  TFAIL  :  mem.c:163: victim unexpectedly ended with retcode:

Quoted 6 lines

0, expected: 12
oom03 0 TINFO : set overcommit_memory to 0
It cames from the caller testoom(0, 1, ENOMEM, 1). The full reason is that function child_alloc() go into single process mode, then successfully finish the memory allocation and return 0.

Description above doesn't explain why you get 0, when oom03 is set to run in cgroup with memory.memsw.limit_in_bytes == TESTMEM, and then allocates TESTMEM + MB.

My guess is a KSM scan merged some pages before you have hit the limit. Do yo get these failures always during KSM test?

Regards,
Jan

Quoted 3 lines

In this patch, let's make it (in single mode) keep allocating memory with an incresed length in order to avoid 0 returned.

Li Wang

author

2016-06-20 11:34 UTC #2 lore raw

PATCH 1/1 mem/lib: keep allocating memory until get an error in single process

Quoted 20 lines · Jan Stancek

----- Original Message -----
Quoted 10 lines
From: "Li Wang" <[email protected]>
To: [email protected]
Cc: [email protected]
Sent: Monday, 20 June, 2016 11:53:05 AM
Subject: [PATCH] mem/lib: keep allocating memory until get an error in single process
We occasionally catch errors like:
oom03       0  TINFO  :  start OOM testing for KSM pages.
oom03       0  TINFO  :  expected victim is 3490.
oom03       6  TFAIL  :  mem.c:163: victim unexpectedly ended with retcode:
Quoted 6 lines
0, expected: 12
oom03 0 TINFO : set overcommit_memory to 0
It cames from the caller testoom(0, 1, ENOMEM, 1). The full reason is that function child_alloc() go into single process mode, then successfully finish the memory allocation and return 0.

Quoted 5 lines

Description above doesn't explain why you get 0, when oom03 is set to run in cgroup with memory.memsw.limit_in_bytes == TESTMEM, and then allocates TESTMEM + MB.
My guess is a KSM scan merged some pages before you have hit the limit.

That might be.

Quoted 1 line

Do yo get these failures always during KSM test?

No, I just get the failures only once during my test.

Quoted 3 lines

Quoted 3 lines
In this patch, let's make it (in single mode) keep allocating memory with an incresed length in order to avoid 0 returned.

Hmm, now I think these codes have two problems.

1. As you said the KSM merge same pages and let the oom03 failed as above.

2. The child_alloc() probably should also do memory allocation with an infinite loop in single process mode. Because if someone has a caller testoom(0, 1, ENOMEM, 1) at other place in future, that'll be easily get fauilures.

e.g. change the testoom(...) as 'testoom(0, 1, ENOMEM, 1)' in oom01.c, it failed like that.

# ./oom01 
oom01       0  TINFO  :  set overcommit_memory to 2
oom01       0  TINFO  :  expected victim is 20068.
oom01       0  TINFO  :  thread (7f05a5051700), allocating 1074790400 bytes.
oom01       1  TFAIL  :  mem.c:165: victim unexpectedly ended with retcode: 0, expected: 12
oom01       0  TINFO  :  set overcommit_memory to 0
oom01       0  TINFO  :  expected victim is 20069.

Or, can we solve them in one method?

Li Wang

Jan Stancek

2016-06-20 13:03 UTC #3 lore raw

PATCH 1/1 mem/lib: keep allocating memory until get an error in single process

----- Original Message -----

Quoted 45 lines

From: "Li Wang" <[email protected]>
To: "Jan Stancek" <[email protected]>
Cc: [email protected]
Sent: Monday, 20 June, 2016 1:34:54 PM
Subject: Re: [PATCH] mem/lib: keep allocating memory until get an error in single process
Quoted 24 lines · Jan Stancek
----- Original Message -----
Quoted 15 lines
From: "Li Wang" <[email protected]>
To: [email protected]
Cc: [email protected]
Sent: Monday, 20 June, 2016 11:53:05 AM
Subject: [PATCH] mem/lib: keep allocating memory until get an error in single process
We occasionally catch errors like:
oom03       0  TINFO  :  start OOM testing for KSM pages.
oom03       0  TINFO  :  expected victim is 3490.
oom03       6  TFAIL  :  mem.c:163: victim unexpectedly ended with
retcode:
0, expected: 12
oom03 0 TINFO : set overcommit_memory to 0
Quoted 5 lines
It cames from the caller testoom(0, 1, ENOMEM, 1). The full reason is that
function child_alloc() go into single process mode, then successfully finish
the memory allocation and return 0.
Quoted 5 lines
Description above doesn't explain why you get 0, when oom03 is set to run in cgroup with memory.memsw.limit_in_bytes == TESTMEM, and then allocates TESTMEM + MB.
My guess is a KSM scan merged some pages before you have hit the limit.
That might be.
Quoted 1 line
Do yo get these failures always during KSM test?
No, I just get the failures only once during my test.
Quoted 3 lines
Quoted 3 lines
In this patch, let's make it (in single mode) keep allocating memory with an incresed length in order to avoid 0 returned.

Quoted 7 lines

Hmm, now I think these codes have two problems.
1. As you said the KSM merge same pages and let the oom03 failed as above.
2. The child_alloc() probably should also do memory allocation with an infinite loop

At the time 'lite' was introduced, everything was single threaded, the only difference was that lite == 0 would keep allocating in infinite loop.

Your patch is effectively eliminating lite == 1, which means that oom03 will pass even if cgroup limits are broken.

Quoted 3 lines

in single process mode. Because if someone has a caller testoom(0, 1, ENOMEM,
1) at other
place in future, that'll be easily get fauilures.

Likely because author is expecting behaviour other than what we have now:

* @lite: if non-zero, child makes only single TESTMEM+MB allocation
* if zero, child keeps allocating memory until it gets killed
* or some operation fails

Quoted 1 line

Or, can we solve them in one method?

We could skip oom(KSM) if lite == 1 in testoom(), since limit_in_bytes may vary from run to run, which isn't reliable for oom03 cgroup test.

Or we could change alloc_mem for lite == 1 && testcase == KSM, such that each page would have unique content (to prevent merging).

Regards,
Jan

Li Wang

author

2016-06-21 03:26 UTC #4 lore raw

PATCH 1/1 mem/lib: keep allocating memory until get an error in single process

Quoted 10 lines · Jan Stancek

Quoted 3 lines
2. The child_alloc() probably should also do memory allocation with an infinite loop
At the time 'lite' was introduced, everything was single threaded, the only difference was that lite == 0 would keep allocating in infinite loop.
Your patch is effectively eliminating lite == 1, which means that oom03 will pass even if cgroup limits are broken.

You're right! My previous meothod didn't solve the essential problem.

Quoted 11 lines

Quoted 4 lines
in single process mode. Because if someone has a caller testoom(0, 1, ENOMEM,
1) at other
place in future, that'll be easily get fauilures.
Likely because author is expecting behaviour other than what we have now:
* @lite: if non-zero, child makes only single TESTMEM+MB allocation
* if zero, child keeps allocating memory until it gets killed
* or some operation fails

Okay, now just keep it this way.

Quoted 8 lines

Quoted 2 lines
Or, can we solve them in one method?
We could skip oom(KSM) if lite == 1 in testoom(), since limit_in_bytes may vary from run to run, which isn't reliable for oom03 cgroup test.
Or we could change alloc_mem for lite == 1 && testcase == KSM, such that each page would have unique content (to prevent merging).

Yes, and we clould also keep the KSM scan off while lite == 1.

	if (lite != 1) SAFE_FILE_PRINTF(cleanup, PATH_KSM "run", "1");

Let me format a new patch for this.

Regards,
Li WANG