Skip to contentSkip to conversation

pthread_cond_brodcast/1-2.c hung when mem left is not enough

PATCH Superseded DAN LI 3 participants 4 messages Aug 10, 2012 – Oct 17, 2012 1 message not in the archive

v4 of 6 Superseded Latest is v6 →

lore
Share & tools
lore

DAN LI

author
#0 lore raw

PATCH v4 1/1 pthread_cond_brodcast/1-2.c hung when mem left is not enough

pthread_cond_brodcast/1-2.c try to create as many childrens(thread or process) as possible,however,if current free memory is exhausted before all childrens are started,the case will timeout and hung since it's stated with program "t0".

The patch adjust count of childrens according to judgement of size of current free memory.

---
 .../interfaces/pthread_cond_broadcast/1-2.c        |   47 ++++++++++++++++++--
 1 file changed, 43 insertions(+), 4 deletions(-)
testcases/open_posix_testsuite/conformance/interfaces/pthread_cond_broadcast/1-2.c +43 -4
index f6ad9df..897448c 100644
@@ -48,6 +48,7 @@
 #include <sys/mman.h>
 #include <sys/wait.h>
 #include <semaphore.h>
+#include <sys/sysinfo.h>
 
 #include "../testfrmw/testfrmw.h"
 #include "../testfrmw/testfrmw.c"
@@ -82,8 +83,8 @@
 #endif
 
 /* Do not create more than this amount of children: */
-#define MAX_PROCESS_CHILDREN  (200)
-#define MAX_THREAD_CHILDREN   (1000)
+static int max_process_children  = 200;
+static int max_thread_children = 1000;
 
 #define TIMEOUT  (180)
 
@@ -243,6 +244,42 @@ void *timer(void *arg)
 	return NULL; /* For compiler */
 }
 
+#ifdef __linux__
+static void children_number(void)
+{
+	struct sysinfo sysinformation;
+	int ret;
+	int avail_number;
+	unsigned long per_process;
+	unsigned long min_stack;
+
+	min_stack = sysconf(_SC_THREAD_STACK_MIN);
+
+	ret = sysinfo(&sysinformation);
+	if (ret != 0)
+		UNRESOLVED(ret, "Failed to get system information.");
+
+	per_process = min_stack * max_thread_children;
+	if (per_process > sysinformation.freeram)
+		UNTESTED("Not enough memory.");
+
+	avail_number = sysinformation.freeram / per_process;
+
+	if (avail_number < 10)
+		UNTESTED("Not enough memory.");
+
+	max_process_children = (avail_number < max_process_children ?
+					avail_number : max_process_children);
+
+	return;
+}
+#else 
+static void children_number(void)
+{
+	return;
+}
+#endif
+
 int main(int argc, char *argv[])
 {
 	int ret;
@@ -268,6 +305,8 @@ int main(int argc, char *argv[])
 
 	output_init();
 
+	children_number();
+
 	/* check the system abilities */
 	pshared = sysconf(_SC_THREAD_PROCESS_SHARED);
 	cs = sysconf(_SC_CLOCK_SELECTION);
@@ -476,7 +515,7 @@ int main(int argc, char *argv[])
 				} else {
 					ret = errno;
 				}
-			} while ((ret == 0) && (child_count < MAX_THREAD_CHILDREN));
+			} while ((ret == 0) && (child_count < max_thread_children));
 			#if VERBOSE > 2
 			output("[parent] Created %i children threads\n", child_count);
 			#endif
@@ -504,7 +543,7 @@ int main(int argc, char *argv[])
 				} else {
 					ret = errno;
 				}
-			} while ((ret == 0) && (child_count < MAX_PROCESS_CHILDREN));
+			} while ((ret == 0) && (child_count < max_process_children));
 			#if VERBOSE > 2
 			output("[parent] Created %i children processes\n", child_count);
 			#endif
Signature · 12 lines
1.7.10.2

------------------------------------------------------------------------------
Live Security Virtual Conference
Exclusive live event will cover all the ways today's security and 
threat landscape has changed and how IT managers can respond. Discussions 
will include endpoint security, mobile security and the latest in malware 
threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
_______________________________________________
Ltp-list mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/ltp-list
#1 lore raw
Quoted 11 lines · DAN LI

pthread_cond_brodcast/1-2.c try to create as many childrens(thread or process) as possible,however,if current free memory is exhausted before all childrens are started,the case will timeout and hung since it's stated with program "t0".

The patch adjust count of childrens according to judgement of size of current free memory.

---
 .../interfaces/pthread_cond_broadcast/1-2.c        |   47 ++++++++++++++++++--
 1 file changed, 43 insertions(+), 4 deletions(-)

Pushed with removing the tailing spaces. thank you.

Wanlong Gao

------------------------------------------------------------------------------
Live Security Virtual Conference
Exclusive live event will cover all the ways today's security and threat landscape has changed and how IT managers can respond. Discussions will include endpoint security, mobile security and the latest in malware threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
_______________________________________________
Ltp-list mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/ltp-list

#2 lore raw
In reply to DAN LI

Hi!

Quoted 31 lines

/* Do not create more than this amount of children: */

testcases/open_posix_testsuite/conformance/interfaces/pthread_cond_broadcast/1-2.c +21 -2
-#define MAX_PROCESS_CHILDREN  (200)
-#define MAX_THREAD_CHILDREN   (1000)
+static int max_process_children  = 200;
+static int max_thread_children = 1000;
 
 #define TIMEOUT  (180)
@@ -243,6 +244,42 @@ void *timer(void *arg)
 	return NULL; /* For compiler */
 }
 
+#ifdef __linux__
+static void children_number(void)
+{
+	struct sysinfo sysinformation;
+	int ret;
+	int avail_number;
+	unsigned long per_process;
+	unsigned long min_stack;
+
+	min_stack = sysconf(_SC_THREAD_STACK_MIN);
+
+	ret = sysinfo(&sysinformation);
+	if (ret != 0)
+		UNRESOLVED(ret, "Failed to get system information.");
+
+	per_process = min_stack * max_thread_children;
+	if (per_process > sysinformation.freeram)
+		UNTESTED("Not enough memory.");
+

testcases/open_posix_testsuite/conformance/interfaces/pthread_cond_broadcast/1-2.c void *timer(void *arg) · line 275

Quoted 10 lines
testcases/open_posix_testsuite/conformance/interfaces/pthread_cond_broadcast/1-2.c +10 -0
+	avail_number = sysinformation.freeram / per_process;
+
+	if (avail_number < 10)
+		UNTESTED("Not enough memory.");
+
+	max_process_children = (avail_number < max_process_children ?
+					avail_number : max_process_children);
+
+	return;
+}

I generally don't like to see linux-only solutions in the open posix testsuite and advice to use them only when there is no POSIX interface for them.

And this looks like the only thing that is not in the scope of POSIX is size of the available memory. What about creating header in open_posix_testsuite/include/ that would define interface to get free ram in different systems and using this code in general case?

And actually I think that free ram itself is enough to figoure out available memory as the kernel tends to eat as many memory for buffers as available and reclaims it back only when needed.

Signature · 13 lines
Cyril Hrubis
[email protected]

------------------------------------------------------------------------------
Don't let slow site performance ruin your business. Deploy New Relic APM
Deploy New Relic app performance management and know exactly
what is happening inside your Ruby, Python, PHP, Java, and .NET app
Try New Relic at no cost today and get our sweet Data Nerd shirt too!
http://p.sf.net/sfu/newrelic-dev2dev
_______________________________________________
Ltp-list mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/ltp-list

A message that is not in the archive (sent off-list or not yet received): [email protected]

#4 lore raw

Hi!

Quoted 7 lines
Quoted 4 lines

And actually I think that free ram itself is enough to figoure out available memory as the kernel tends to eat as many memory for buffers as available and reclaims it back only when needed.

Unfortunately i'v not got other good ideas.
Do you have any suggestion?

There are several ways how to fix this.

One is to assume usable ram as free + buffers/N where N > 1

Another is to assume usable ram from total amount of RAM as (total - M)/N where M statically reserved size and N > 1.

The problem is to choose the constats carefully so that we excersize the system enough but don't overload it.

After you figure out this part, the patch with sysinfo() in header in include is fine.

Signature · 12 lines
Cyril Hrubis
[email protected]

------------------------------------------------------------------------------
Everyone hates slow websites. So do we.
Make your web apps faster with AppDynamics
Download AppDynamics Lite for free today:
http://p.sf.net/sfu/appdyn_sfd2d_oct
_______________________________________________
Ltp-list mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/ltp-list