Skip to contentSkip to conversation

EVL: hectic test fails when to many CPUs are available

PATCH Applied Hannes Diethelm 2 participants 5 messages Jul 7 – Jul 13

lore
Share & tools
lore
#0 lore raw

Hello

This is the reason for the patch for evl-test. I guess a beginners mistake / edge case but if you want to look into it:

If I don't set evl.oobcpus in my VM or just use to many OOB CPUs, the hectic test crashes and never exits. I have to use kill -9 to stop it.

evl_attach(): Resource temporarily unavailable: This is probably not handled correctly.

evl.oobcpus=0-15 -> Freeze
evl.oobcpus=1-15 -> Freeze
evl.oobcpus=2-15 -> Ok

/usr/libexec/evl/tests/hectic -T 5

== Testing FPU check routines...
== FPU check routines: OK.
== Threads: switcher_ufps0-0 rtk0-1 rtk0-2 rtup0-3 rtup0-4 rtup_ufpp0-5 rtup_ufpp0-6 rtus0-7 rtus0-8 rtus_ufps0-9 rtus_ufps0-10 rtuo0-11 rtuo0-12 rtuo_ufpp0-13 rtuo_ufpp0-14 rtuo_ufps0-15 rtuo_ufps0-16 rtuo_ufpp_ufps0-17 rtuo_ufpp_ufps0-18 fpu_stress_ufps0-19 switcher_ufps1-0 rtk1-1 rtk1-2 rtup1-3 rtup1-4 rtup_ufpp1-5 rtup_ufpp1-6 rtus1-7 rtus1-8 rtus_ufps1-9 rtus_ufps1-10 rtuo1-11 rtuo1-12 rtuo_ufpp1-13 rtuo_ufpp1-14 rtuo_ufps1-15 rtuo_ufps1-16 rtuo_ufpp_ufps1-17 rtuo_ufpp_ufps1-18 fpu_stress_ufps1-19 switcher_ufps2-0 rtk2-1 rtk2-2 rtup2-3 rtup2-4 rtup_ufpp2-5 rtup_ufpp2-6 rtus2-7 rtus2-8 rtus_ufps2-9 rtus_ufps2-10 rtuo2-11 rtuo2-12 rtuo_ufpp2-13 rtuo_ufpp2-14 rtuo_ufps2-15 rtuo_ufps2-16 rtuo_ufpp_ufps2-17 rtuo_ufpp_ufps2-18 fpu_stress_ufps2-19 switcher_ufps3-0 rtk3-1 rtk3-2 rtup3-3 rtup3-4 rtup_ufpp3-5 rtup_ufpp3-6 rtus3-7 rtus3-8 rtus_ufps3-9 rtus_ufps3-10 rtuo3-11 rtuo3-12 rtuo_ufpp3-13 rtuo_ufpp3-14 rtuo_ufps3-15 rtuo_ufps3-16 rtuo_ufpp_ufps3-17 rtuo_ufpp_ufps3-18 fpu_stress_ufps3-19 switcher_ufps4-0 rtk4-1 rtk4-2 rtup4-3 rtup4-4 rtup_ufpp4-5 rtup_ufpp4-6 rtus4-7 rtus4-8 rtus_ufps4-9 rtus_ufps4-10 rtuo4-11 rtuo4-12 rtuo_ufpp4-13 rtuo_ufpp4-14 rtuo_ufps4-15 rtuo_ufps4-16 rtuo_ufpp_ufps4-17 rtuo_ufpp_ufps4-18 fpu_stress_ufps4-19 switcher_ufps5-0 rtk5-1 rtk5-2 rtup5-3 rtup5-4 rtup_ufpp5-5 rtup_ufpp5-6 rtus5-7 rtus5-8 rtus_ufps5-9 rtus_ufps5-10 rtuo5-11 rtuo5-12 rtuo_ufpp5-13 rtuo_ufpp5-14 rtuo_ufps5-15 rtuo_ufps5-16 rtuo_ufpp_ufps5-17 rtuo_ufpp_ufps5-18 fpu_stress_ufps5-19 switcher_ufps6-0 rtk6-1 rtk6-2 rtup6-3 rtup6-4 rtup_ufpp6-5 rtup_ufpp6-6 rtus6-7 rtus6-8 rtus_ufps6-9 rtus_ufps6-10 rtuo6-11 rtuo6-12 rtuo_ufpp6-13 rtuo_ufpp6-14 rtuo_ufps6-15 rtuo_ufps6-16 rtuo_ufpp_ufps6-17 rtuo_ufpp_ufps6-18 fpu_stress_ufps6-19 switcher_ufps7-0 rtk7-1 rtk7-2 rtup7-3 rtup7-4 rtup_ufpp7-5 rtup_ufpp7-6 rtus7-7 rtus7-8 rtus_ufps7-9 rtus_ufps7-10 rtuo7-11 rtuo7-12 rtuo_ufpp7-13 rtuo_ufpp7-14 rtuo_ufps7-15 rtuo_ufps7-16 rtuo_ufpp_ufps7-17 rtuo_ufpp_ufps7-18 fpu_stress_ufps7-19 switcher_ufps8-0 rtk8-1 rtk8-2 rtup8-3 rtup8-4 rtup_ufpp8-5 rtup_ufpp8-6 rtus8-7 rtus8-8 rtus_ufps8-9 rtus_ufps8-10 rtuo8-11 rtuo8-12 rtuo_ufpp8-13 rtuo_ufpp8-14 rtuo_ufps8-15 rtuo_ufps8-16 rtuo_ufpp_ufps8-17 rtuo_ufpp_ufps8-18 fpu_stress_ufps8-19 switcher_ufps9-0 rtk9-1 rtk9-2 rtup9-3 rtup9-4 rtup_ufpp9-5 rtup_ufpp9-6 rtus9-7 rtus9-8 rtus_ufps9-9 rtus_ufps9-10 rtuo9-11 rtuo9-12 rtuo_ufpp9-13 rtuo_ufpp9-14 rtuo_ufps9-15 rtuo_ufps9-16 rtuo_ufpp_ufps9-17 rtuo_ufpp_ufps9-18 fpu_stress_ufps9-19 switcher_ufps10-0 rtk10-1 rtk10-2 rtup10-3 rtup10-4 rtup_ufpp10-5 rtup_ufpp10-6 rtus10-7 rtus10-8 rtus_ufps10-9 rtus_ufps10-10 rtuo10-11 rtuo10-12 rtuo_ufpp10-13 rtuo_ufpp10-14 rtuo_ufps10-15 rtuo_ufps10-16 rtuo_ufpp_ufps10-17 rtuo_ufpp_ufps10-18 fpu_stress_ufps10-19 switcher_ufps11-0 rtk11-1 rtk11-2 rtup11-3 rtup11-4 rtup_ufpp11-5 rtup_ufpp11-6 rtus11-7 rtus11-8 rtus_ufps11-9 rtus_ufps11-10 rtuo11-11 rtuo11-12 rtuo_ufpp11-13 rtuo_ufpp11-14 rtuo_ufps11-15 rtuo_ufps11-16 rtuo_ufpp_ufps11-17 rtuo_ufpp_ufps11-18 fpu_stress_ufps11-19 switcher_ufps12-0 rtk12-1 rtk12-2 rtup12-3 revl_attach(): Resource temporarily unavailable

Regards
Hannes

#1 lore raw
Quoted 14 lines · Hannes Diethelm

Hello

This is the reason for the patch for evl-test. I guess a beginners mistake / edge case but if you want to look into it:

If I don't set evl.oobcpus in my VM or just use to many OOB CPUs, the hectic test crashes and never exits. I have to use kill -9 to stop it.

evl_attach(): Resource temporarily unavailable: This is probably not handled correctly.

evl.oobcpus=0-15 -> Freeze
evl.oobcpus=1-15 -> Freeze
evl.oobcpus=2-15 -> Ok

Quoted 7 lines

/usr/libexec/evl/tests/hectic -T 5

rtus_ufps11-10 rtuo11-11 rtuo11-12 rtuo_ufpp11-13 rtuo_ufpp11-14 rtuo_ufps11-15 rtuo_ufps11-16 rtuo_ufpp_ufps11-17 rtuo_ufpp_ufps11-18 fpu_stress_ufps11-19 switcher_ufps12-0 rtk12-1 rtk12-2 rtup12-3 revl_attach(): Resource temporarily unavailable

Mm, ok. This one is indeed an old and obscure issue which has never been cared of. Increasing CONFIG_EVL_COREMEM_SIZE and CONFIG_EVL_NR_THREADS in order to account for the massive amount of threads involved in large CPU configurations should fix it.

At any rate, the test should unwind and exit cleanly when that fails, so there is still this issue to solve at some point.

Signature · 1 line
Philippe.
#2 lore raw

PATCH 1/1 test: hectic: sem_post task_init on failure

This fixes the issue where hectic doesn't exit on init failure when there are for example not enough resources due to to many available CPU's.

---
 tests/hectic.c | 32 ++++++++++++++++++++++----------
 1 file changed, 22 insertions(+), 10 deletions(-)
tests/hectic.c +22 -10
index af8f988..b07e074 100644
@@ -270,6 +270,7 @@ static void *sleeper_switcher(void *cookie)
 	CPU_SET(param->cpu->index, &cpu_set);
 	if (sched_setaffinity(0, sizeof(cpu_set), &cpu_set)) {
 		perror("sleeper: sched_setaffinity");
+		sem_post(&task_init);
 		clean_exit(EXIT_FAILURE);
 	}
 
@@ -379,6 +380,7 @@ static void *fpu_stress(void *cookie)
 	CPU_SET(param->cpu->index, &cpu_set);
 	if (sched_setaffinity(0, sizeof(cpu_set), &cpu_set)) {
 		perror("sleeper: sched_setaffinity");
+		sem_post(&task_init);
 		clean_exit(EXIT_FAILURE);
 	}
 
@@ -399,19 +401,14 @@ static void *fpu_stress(void *cookie)
 	return NULL;
 }
 
-static void attach_thread(struct task_params *param)
+static int attach_thread(struct task_params *param)
 {
 	char buffer[64];
-	int efd;
 
 	task_name(buffer, sizeof(buffer), param->cpu,param->swt.index);
 
 	/* Make it a public thread only for demo purpose. */
-	efd = evl_attach_self("/%s:%d", buffer, getpid());
-	if (efd < 0) {
-		perror("evl_attach()");
-		clean_exit(EXIT_FAILURE);
-	}
+	return evl_attach_self("/%s:%d", buffer, getpid());
 }
 
 static void *rtup(void *cookie)
@@ -423,12 +420,17 @@ static void *rtup(void *cookie)
 	cpu_set_t cpu_set;
 	unsigned i = 0;
 
-	attach_thread(param);
+	if (attach_thread(param) < 0) {
+		perror("evl_attach()");
+		sem_post(&task_init);
+		clean_exit(EXIT_FAILURE);
+	}
 
 	CPU_ZERO(&cpu_set);
 	CPU_SET(param->cpu->index, &cpu_set);
 	if (sched_setaffinity(0, sizeof(cpu_set), &cpu_set)) {
 		perror("rtup: sched_setaffinity");
+		sem_post(&task_init);
 		clean_exit(EXIT_FAILURE);
 	}
 
@@ -503,12 +505,17 @@ static void *rtus(void *cookie)
 	cpu_set_t cpu_set;
 	unsigned i = 0;
 
-	attach_thread(param);
+	if (attach_thread(param) < 0) {
+		perror("evl_attach()");
+		sem_post(&task_init);
+		clean_exit(EXIT_FAILURE);
+	}
 
 	CPU_ZERO(&cpu_set);
 	CPU_SET(param->cpu->index, &cpu_set);
 	if (sched_setaffinity(0, sizeof(cpu_set), &cpu_set)) {
 		perror("rtus: sched_setaffinity");
+		sem_post(&task_init);
 		clean_exit(EXIT_FAILURE);
 	}
 
@@ -600,12 +607,17 @@ static void *rtuo(void *cookie)
 	cpu_set_t cpu_set;
 	unsigned i = 0;
 
-	attach_thread(param);
+	if (attach_thread(param) < 0) {
+		perror("evl_attach()");
+		sem_post(&task_init);
+		clean_exit(EXIT_FAILURE);
+	}
 
 	CPU_ZERO(&cpu_set);
 	CPU_SET(param->cpu->index, &cpu_set);
 	if (sched_setaffinity(0, sizeof(cpu_set), &cpu_set)) {
 		perror("rtuo: sched_setaffinity");
+		sem_post(&task_init);
 		clean_exit(EXIT_FAILURE);
 	}
 
Signature · 1 line
2.47.3
#3 lore raw

Re: [PATCH] test: hectic: sem_post task_init on failure

Quoted 97 lines · Hannes Diethelm

This fixes the issue where hectic doesn't exit on init failure when there are for example not enough resources due to to many available CPU's.

---
 tests/hectic.c | 32 ++++++++++++++++++++++----------
 1 file changed, 22 insertions(+), 10 deletions(-)
tests/hectic.c +19 -10
index af8f988..b07e074 100644
@@ -270,6 +270,7 @@ static void *sleeper_switcher(void *cookie)
 	CPU_SET(param->cpu->index, &cpu_set);
 	if (sched_setaffinity(0, sizeof(cpu_set), &cpu_set)) {
 		perror("sleeper: sched_setaffinity");
+		sem_post(&task_init);
 		clean_exit(EXIT_FAILURE);
 	}
 
@@ -379,6 +380,7 @@ static void *fpu_stress(void *cookie)
 	CPU_SET(param->cpu->index, &cpu_set);
 	if (sched_setaffinity(0, sizeof(cpu_set), &cpu_set)) {
 		perror("sleeper: sched_setaffinity");
+		sem_post(&task_init);
 		clean_exit(EXIT_FAILURE);
 	}
 
@@ -399,19 +401,14 @@ static void *fpu_stress(void *cookie)
 	return NULL;
 }
 
-static void attach_thread(struct task_params *param)
+static int attach_thread(struct task_params *param)
 {
 	char buffer[64];
-	int efd;
 
 	task_name(buffer, sizeof(buffer), param->cpu,param->swt.index);
 
 	/* Make it a public thread only for demo purpose. */
-	efd = evl_attach_self("/%s:%d", buffer, getpid());
-	if (efd < 0) {
-		perror("evl_attach()");
-		clean_exit(EXIT_FAILURE);
-	}
+	return evl_attach_self("/%s:%d", buffer, getpid());
 }
 
 static void *rtup(void *cookie)
@@ -423,12 +420,17 @@ static void *rtup(void *cookie)
 	cpu_set_t cpu_set;
 	unsigned i = 0;
 
-	attach_thread(param);
+	if (attach_thread(param) < 0) {
+		perror("evl_attach()");
+		sem_post(&task_init);
+		clean_exit(EXIT_FAILURE);
+	}
 
 	CPU_ZERO(&cpu_set);
 	CPU_SET(param->cpu->index, &cpu_set);
 	if (sched_setaffinity(0, sizeof(cpu_set), &cpu_set)) {
 		perror("rtup: sched_setaffinity");
+		sem_post(&task_init);
 		clean_exit(EXIT_FAILURE);
 	}
 
@@ -503,12 +505,17 @@ static void *rtus(void *cookie)
 	cpu_set_t cpu_set;
 	unsigned i = 0;
 
-	attach_thread(param);
+	if (attach_thread(param) < 0) {
+		perror("evl_attach()");
+		sem_post(&task_init);
+		clean_exit(EXIT_FAILURE);
+	}
 
 	CPU_ZERO(&cpu_set);
 	CPU_SET(param->cpu->index, &cpu_set);
 	if (sched_setaffinity(0, sizeof(cpu_set), &cpu_set)) {
 		perror("rtus: sched_setaffinity");
+		sem_post(&task_init);
 		clean_exit(EXIT_FAILURE);
 	}
 
@@ -600,12 +607,17 @@ static void *rtuo(void *cookie)
 	cpu_set_t cpu_set;
 	unsigned i = 0;
 
-	attach_thread(param);
+	if (attach_thread(param) < 0) {
+		perror("evl_attach()");
+		sem_post(&task_init);

tests/hectic.c static void *rtuo(void *cookie) · line 622

Quoted 10 lines
tests/hectic.c +3 -0
+		clean_exit(EXIT_FAILURE);
+	}
 
 	CPU_ZERO(&cpu_set);
 	CPU_SET(param->cpu->index, &cpu_set);
 	if (sched_setaffinity(0, sizeof(cpu_set), &cpu_set)) {
 		perror("rtuo: sched_setaffinity");
+		sem_post(&task_init);
 		clean_exit(EXIT_FAILURE);
 	}

Merged, thanks.

Signature · 1 line
Philippe.
#4 lore raw
In reply to Philippe Gerum

Am 07.07.26 um 15:34 schrieb Philippe Gerum:

Quoted 24 lines
Quoted 14 lines · Hannes Diethelm

Hello

This is the reason for the patch for evl-test. I guess a beginners mistake / edge case but if you want to look into it:

If I don't set evl.oobcpus in my VM or just use to many OOB CPUs, the hectic test crashes and never exits. I have to use kill -9 to stop it.

evl_attach(): Resource temporarily unavailable: This is probably not handled correctly.

evl.oobcpus=0-15 -> Freeze
evl.oobcpus=1-15 -> Freeze
evl.oobcpus=2-15 -> Ok

Quoted 7 lines

/usr/libexec/evl/tests/hectic -T 5

rtus_ufps11-10 rtuo11-11 rtuo11-12 rtuo_ufpp11-13 rtuo_ufpp11-14 rtuo_ufps11-15 rtuo_ufps11-16 rtuo_ufpp_ufps11-17 rtuo_ufpp_ufps11-18 fpu_stress_ufps11-19 switcher_ufps12-0 rtk12-1 rtk12-2 rtup12-3 revl_attach(): Resource temporarily unavailable

Quoted 8 lines

Mm, ok. This one is indeed an old and obscure issue which has never been cared of. Increasing CONFIG_EVL_COREMEM_SIZE and CONFIG_EVL_NR_THREADS in order to account for the massive amount of threads involved in large CPU configurations should fix it.

At any rate, the test should unwind and exit cleanly when that fails, so there is still this issue to solve at some point.

The supplied patch should fix the issue with the forever running hectic test. It looks like the main thread waited forever for the task_init semaphore.

Probably not a fully clean way of doing it but it exits now with a many error messages in case there is not enough memory instead of running forever.