Skip to contentSkip to conversation

mem/oom: don't fail if pthread_create hits EAGAIN

PATCH New Jan Stancek 2 participants 3 messages Nov 25, 2015 – Nov 26, 2015

lore
Share & tools
lore

Jan Stancek

author
#0 lore raw

PATCH 1/1 mem/oom: don't fail if pthread_create hits EAGAIN

In a rare situation on system with many CPUs, small RAM and overcommit_memory == 2, threads that are already started can create memory pressure, that causes pthread_create to hit EAGAIN. Don't fail the test, keep going with threads that are already spawned.

Signed-off-by Jan Stancek <[email protected]>
---
 testcases/kernel/mem/lib/mem.c | 7 ++++++-
 1 file changed, 6 insertions(+), 1 deletion(-)
testcases/kernel/mem/lib/mem.c +6 -1
index 215c0527d068..cee4e91fb89f 100644
@@ -90,7 +90,12 @@ static void child_alloc(int testcase, int lite, int threads)
 			(void *)((long)testcase)));
 		if (TEST_RETURN) {
 			tst_resm(TINFO | TRERRNO, "pthread_create");
-			goto out;
+			/*
+			 * Keep going if thread other than first fails to
+			 * spawn due to lack of resources.
+			 */
+			if (i == 0 || TEST_RETURN != EAGAIN)
+				goto out;
 		}
 	}
Signature · 1 line
1.8.3.1
#1 lore raw

PATCH 1/1 mem/oom: don't fail if pthread_create hits EAGAIN

Hi!

Quoted 5 lines

In a rare situation on system with many CPUs, small RAM and overcommit_memory == 2, threads that are already started can create memory pressure, that causes pthread_create to hit EAGAIN. Don't fail the test, keep going with threads that are already spawned.

It took me a while to figure out what are trying to fix, but I think I understand now.

The thing is that if we have many processors the threads running on background allocating and faulting memory would counsume available memory before the for() loop has chance to finish and the ptread_create() starts to fail with EAGAIN, right?

In that case the fix looks good, acked.

Signature · 2 lines
Cyril Hrubis
[email protected]
author
#2 lore raw

PATCH 1/1 mem/oom: don't fail if pthread_create hits EAGAIN

----- Original Message -----

Quoted 13 lines

From: "Cyril Hrubis" <[email protected]>
To: "Jan Stancek" <[email protected]>
Cc: [email protected]
Sent: Thursday, 26 November, 2015 3:26:12 PM
Subject: Re: [LTP] [PATCH] mem/oom: don't fail if pthread_create hits EAGAIN

Hi!

Quoted 5 lines

In a rare situation on system with many CPUs, small RAM and overcommit_memory == 2, threads that are already started can create memory pressure, that causes pthread_create to hit EAGAIN. Don't fail the test, keep going with threads that are already spawned.

Quoted 7 lines

It took me a while to figure out what are trying to fix, but I think I understand now.

The thing is that if we have many processors the threads running on background allocating and faulting memory would counsume available memory before the for() loop has chance to finish and the ptread_create() starts to fail with EAGAIN, right?

Correct, from my experience it's very rarely seen on powerpc (probably because of its large page size).

I can trigger it easily on x86 KVM guest (1GB RAM, 4cpus, no swap) if I modify the test slightly:

testcases/kernel/mem/lib/mem.c +11 -1
index 118b6c0..18a1aa1 100644
@@ -32,8 +32,10 @@ static int alloc_mem(long int length, int testcase)
        long i, pagesz = getpagesize();
        int loop = 10;
 
+/*
        tst_resm(TINFO, "thread (%lx), allocating %ld bytes.",
                (unsigned long) pthread_self(), length);
+*/
 
        s = mmap(NULL, length, PROT_READ | PROT_WRITE,
                 MAP_ANONYMOUS | MAP_PRIVATE, -1, 0);

@@ -65,7 +67,14 @@ static void *child_alloc_thread(void *args)
 
        /* keep allocating until there's an error */
        while (!ret)
-               ret = alloc_mem(LENGTH, (long)args);
+               ret = alloc_mem(1024*1024, (long)args);
+       ret = 0;
+       while (!ret)
+               ret = alloc_mem(4096, (long)args);
+
+       tst_resm(TINFO, "thread (%lx), done allocating");
+
+       sleep(5);
        exit(ret);
 }
 

@@ -92,6 +101,7 @@ static void child_alloc(int testcase, int lite, int threads)
                        tst_resm(TINFO | TRERRNO, "pthread_create");
                        goto out;
                }
+               sleep(5);
        }
 
        /* wait for one of threads to exit whole process */

# ./oom01 
oom01       0  TINFO  :  set overcommit_memory to 2
oom01       0  TINFO  :  expected victim is 30038.
oom01       0  TINFO  :  thread (ffffffff), done allocating
oom01       0  TINFO  :  pthread_create: TEST_RETURN=EAGAIN/EWOULDBLOCK(11): Resource temporarily unavailable
oom01       1  TFAIL  :  mem.c:163: victim unexpectedly ended with retcode: 1, expected: 12
...
Quoted 2 lines

In that case the fix looks good, acked.

Pushed.

Regards,
Jan

Quoted 5 lines

--
Cyril Hrubis
[email protected]