Skip to contentSkip to conversation

realtime/prio-preempt: take cpu isolation into consideration

PATCH Superseded Shizhao Chen 2 participants 6 messages Jul 10, 2024 – Jul 11, 2024

lore
Share & tools
lore
#0 lore raw

PATCH 1/1 realtime/prio-preempt: take cpu isolation into consideration

By default the test starts N-1 busy threads to monopolize N-1 available CPUs, where N is the number of all available CPUs. However, when CPU isolation is applied, this will lead to a hang scenario where all housekeeping CPUs are hogged as isolated CPUs do not share the N-1 busy threads.

---
 .../realtime/func/prio-preempt/prio-preempt.c  | 18 +++++++++++++++---
 1 file changed, 15 insertions(+), 3 deletions(-)
testcases/realtime/func/prio-preempt/prio-preempt.c +15 -3
index 9bd5e7ad8..79d2115a7 100644
@@ -59,6 +59,7 @@
  *
  *****************************************************************************/
 
+#define _GNU_SOURCE
 #include <stdio.h>
 #include <stdlib.h>
 #include <signal.h>
@@ -287,6 +288,17 @@ void *master_thread(void *arg)
 	return NULL;
 }
 
+int get_numcpus(void)
+{
+	cpu_set_t mask;
+	CPU_ZERO(&mask);
+
+	/* Get the number of cpus accessible to the current process */
+	sched_getaffinity(getpid(), sizeof(mask), &mask);
+
+	return CPU_COUNT(&mask);
+}
+
 int main(int argc, char *argv[])
 {
 	int pri_boost, numcpus;
@@ -295,10 +307,10 @@ int main(int argc, char *argv[])
 	pass_criteria = CHECK_LIMIT;
 	rt_init("hin:", parse_args, argc, argv);
 
-	numcpus = sysconf(_SC_NPROCESSORS_ONLN);
+	numcpus = get_numcpus();
 
-	/* Max no. of busy threads should always be less than/equal the no. of cpus
-	   Otherwise, the box will hang */
+	/* Max no. of busy threads should always be less than/equal the no. of
+	   housekeeping cpus. Otherwise, the box will hang */
 
 	if (rt_threads == -1 || rt_threads > numcpus) {
 		rt_threads = numcpus;
Signature · 5 lines
2.45.2


-- 
Mailing list info: https://lists.linux.it/listinfo/ltp
#1 lore raw

Hi Shizhao,

Quoted 29 lines · Shizhao Chen

By default the test starts N-1 busy threads to monopolize N-1 available CPUs,
where N is the number of all available CPUs. However, when CPU isolation is applied, this will lead to a hang scenario where all housekeeping CPUs are hogged as isolated CPUs do not share the N-1 busy threads.

---
 .../realtime/func/prio-preempt/prio-preempt.c  | 18 +++++++++++++++---
 1 file changed, 15 insertions(+), 3 deletions(-)
testcases/realtime/func/prio-preempt/prio-preempt.c +1 -0
index 9bd5e7ad8..79d2115a7 100644
@@ -59,6 +59,7 @@
  *
 
 *****************************************************************************/
 
+#define _GNU_SOURCE
 #include <stdio.h>
 #include <stdlib.h>

#include <signal.h>

testcases/realtime/func/prio-preempt/prio-preempt.c +0 -0
@@ -287,6 +288,17 @@ void *master_thread(void *arg)
        return NULL;
 }
 

testcases/realtime/func/prio-preempt/prio-preempt.c void *master_thread(void *arg) · line 300

Quoted 10 lines
testcases/realtime/func/prio-preempt/prio-preempt.c +10 -0
+int get_numcpus(void)
+{
+       cpu_set_t mask;
+       CPU_ZERO(&mask);
+
+       /* Get the number of cpus accessible to the current process */
+       sched_getaffinity(getpid(), sizeof(mask), &mask);
+
+       return CPU_COUNT(&mask);
+}

+

testcases/realtime/func/prio-preempt/prio-preempt.c int main(int argc, char *argv[]) · line 311

Quoted 10 lines
testcases/realtime/func/prio-preempt/prio-preempt.c +1 -1
 int main(int argc, char *argv[])
 {
        int pri_boost, numcpus;
@@ -295,10 +307,10 @@ int main(int argc, char *argv[])
        pass_criteria = CHECK_LIMIT;
        rt_init("hin:", parse_args, argc, argv);
 
-       numcpus = sysconf(_SC_NPROCESSORS_ONLN);
+       numcpus = get_numcpus();
 

Maybe we could make use of LTP lib function tst_ncpus_available directly? Seems the new get_numcpus() did a dup work on that.

Otherwise, looks good to me.

Quoted 13 lines
testcases/realtime/func/prio-preempt/prio-preempt.c +0 -1
-       /* Max no. of busy threads should always be less than/equal the

no. of cpus

testcases/realtime/func/prio-preempt/prio-preempt.c +1 -1
-          Otherwise, the box will hang */
+       /* Max no. of busy threads should always be less than/equal the

no. of
+ housekeeping cpus. Otherwise, the box will hang */

        if (rt_threads == -1 || rt_threads > numcpus) {
                rt_threads = numcpus;

--
2.45.2

Signature · 5 lines
Regards,
Li Wang

-- 
Mailing list info: https://lists.linux.it/listinfo/ltp
#2 lore raw
Quoted 53 lines · Li Wang

Hi Shizhao,

Quoted 30 lines · Shizhao Chen

By default the test starts N-1 busy threads to monopolize N-1 available CPUs,
where N is the number of all available CPUs. However, when CPU isolation is
applied, this will lead to a hang scenario where all housekeeping CPUs are hogged as isolated CPUs do not share the N-1 busy threads.

---
 .../realtime/func/prio-preempt/prio-preempt.c  | 18 +++++++++++++++---
 1 file changed, 15 insertions(+), 3 deletions(-)
testcases/realtime/func/prio-preempt/prio-preempt.c +1 -0
index 9bd5e7ad8..79d2115a7 100644
@@ -59,6 +59,7 @@
  *
 
 *****************************************************************************/
 
+#define _GNU_SOURCE
 #include <stdio.h>
 #include <stdlib.h>

#include <signal.h>

testcases/realtime/func/prio-preempt/prio-preempt.c +0 -0
@@ -287,6 +288,17 @@ void *master_thread(void *arg)
        return NULL;
 }
 

testcases/realtime/func/prio-preempt/prio-preempt.c void *master_thread(void *arg) · line 300

Quoted 10 lines
testcases/realtime/func/prio-preempt/prio-preempt.c +10 -0
+int get_numcpus(void)
+{
+       cpu_set_t mask;
+       CPU_ZERO(&mask);
+
+       /* Get the number of cpus accessible to the current process */
+       sched_getaffinity(getpid(), sizeof(mask), &mask);
+
+       return CPU_COUNT(&mask);
+}

+

Quoted 7 lines
testcases/realtime/func/prio-preempt/prio-preempt.c +0 -0
 int main(int argc, char *argv[])
 {
        int pri_boost, numcpus;
@@ -295,10 +307,10 @@ int main(int argc, char *argv[])
        pass_criteria = CHECK_LIMIT;
        rt_init("hin:", parse_args, argc, argv);
 
Quoted 7 lines

testcases/realtime/func/prio-preempt/prio-preempt.c int main(int argc, char *argv[]) · line 311

Quoted 3 lines
testcases/realtime/func/prio-preempt/prio-preempt.c +1 -1
-       numcpus = sysconf(_SC_NPROCESSORS_ONLN);
+       numcpus = get_numcpus();
 

Maybe we could make use of LTP lib function tst_ncpus_available directly? Seems the new get_numcpus() did a dup work on that.

Oops, seems the real-time tests does not include ltp library. So you patch make sense from this point, but maybe we need to refine the function to consider more situations.

e.g. uses a fixed-size cpu_set_t structure, which might not be sufficient on systems with a large number of CPUs.

Quoted 22 lines

Otherwise, looks good to me.

Quoted 7 lines
testcases/realtime/func/prio-preempt/prio-preempt.c +0 -1
-       /* Max no. of busy threads should always be less than/equal the

no. of cpus

testcases/realtime/func/prio-preempt/prio-preempt.c +1 -1
-          Otherwise, the box will hang */
+       /* Max no. of busy threads should always be less than/equal the

no. of
+ housekeeping cpus. Otherwise, the box will hang */

Quoted 6 lines
        if (rt_threads == -1 || rt_threads > numcpus) {
                rt_threads = numcpus;

--
2.45.2

--
Regards,
Li Wang

Signature · 5 lines
Regards,
Li Wang

-- 
Mailing list info: https://lists.linux.it/listinfo/ltp
#3 lore raw

PATCH v2 1/1 realtime/prio-preempt: take cpu isolation into consideration

By default the test starts N-1 busy threads to monopolize N-1 available CPUs, where N is the number of all available CPUs. However, when CPU isolation is applied, this will lead to a hang scenario where all housekeeping CPUs are hogged as isolated CPUs do not share the N-1 busy threads.

---
 .../realtime/func/prio-preempt/prio-preempt.c | 20 ++++++++++++++++---
 1 file changed, 17 insertions(+), 3 deletions(-)
testcases/realtime/func/prio-preempt/prio-preempt.c +17 -3
index 9bd5e7ad8..16abc1184 100644
@@ -59,6 +59,7 @@
  *
  *****************************************************************************/
 
+#define _GNU_SOURCE
 #include <stdio.h>
 #include <stdlib.h>
 #include <signal.h>
@@ -287,6 +288,19 @@ void *master_thread(void *arg)
 	return NULL;
 }
 
+int get_numcpus(void)
+{
+	long numcpus_conf = sysconf(_SC_NPROCESSORS_CONF);
+	size_t size = CPU_ALLOC_SIZE(numcpus_conf);
+	cpu_set_t *cpuset = CPU_ALLOC(numcpus_conf);
+
+	CPU_ZERO_S(size, cpuset);
+	/* Get the number of cpus accessible to the current process */
+	sched_getaffinity(0, size, cpuset);
+
+	return CPU_COUNT_S(size, cpuset);
+}
+
 int main(int argc, char *argv[])
 {
 	int pri_boost, numcpus;
@@ -295,10 +309,10 @@ int main(int argc, char *argv[])
 	pass_criteria = CHECK_LIMIT;
 	rt_init("hin:", parse_args, argc, argv);
 
-	numcpus = sysconf(_SC_NPROCESSORS_ONLN);
+	numcpus = get_numcpus();
 
-	/* Max no. of busy threads should always be less than/equal the no. of cpus
-	   Otherwise, the box will hang */
+	/* Max no. of busy threads should always be less than/equal the no. of
+	   housekeeping cpus. Otherwise, the box will hang */
 
 	if (rt_threads == -1 || rt_threads > numcpus) {
 		rt_threads = numcpus;
Signature · 5 lines
2.45.2


-- 
Mailing list info: https://lists.linux.it/listinfo/ltp
#4 lore raw

Hi Shizhao,

Patch merged, thanks!

Signature · 5 lines
Regards,
Li Wang

-- 
Mailing list info: https://lists.linux.it/listinfo/ltp
#5 lore raw
In reply to Li Wang

Thanks for the review! I just sent a v2 which is basically a minimized version of tst_ncpus_available.

Quoted 96 lines · Li Wang
Quoted 50 lines · Li Wang

Hi Shizhao,

Quoted 46 lines · Shizhao Chen

By default the test starts N-1 busy threads to monopolize N-1 available CPUs, where N is the number of all available CPUs. However, when CPU isolation is applied, this will lead to a hang scenario where all housekeeping CPUs are hogged as isolated CPUs do not share the N-1 busy threads.

---
 .../realtime/func/prio-preempt/prio-preempt.c  | 18 +++++++++++++++---
 1 file changed, 15 insertions(+), 3 deletions(-)
testcases/realtime/func/prio-preempt/prio-preempt.c +12 -0
index 9bd5e7ad8..79d2115a7 100644
@@ -59,6 +59,7 @@
  *
  *****************************************************************************/
 
+#define _GNU_SOURCE
 #include <stdio.h>
 #include <stdlib.h>
 #include <signal.h>
@@ -287,6 +288,17 @@ void *master_thread(void *arg)
        return NULL;
 }
 
+int get_numcpus(void)
+{
+       cpu_set_t mask;
+       CPU_ZERO(&mask);
+
+       /* Get the number of cpus accessible to the current process */
+       sched_getaffinity(getpid(), sizeof(mask), &mask);
+
+       return CPU_COUNT(&mask);
+}
 
+
 int main(int argc, char *argv[])
 {
        int pri_boost, numcpus;
testcases/realtime/func/prio-preempt/prio-preempt.c +0 -0
@@ -295,10 +307,10 @@ int main(int argc, char *argv[])
        pass_criteria = CHECK_LIMIT;
        rt_init("hin:", parse_args, argc, argv);
 
Quoted 6 lines

testcases/realtime/func/prio-preempt/prio-preempt.c int main(int argc, char *argv[]) · line 310

Quoted 2 lines
testcases/realtime/func/prio-preempt/prio-preempt.c +1 -1
-       numcpus = sysconf(_SC_NPROCESSORS_ONLN);
+       numcpus = get_numcpus();

Maybe we could make use of LTP lib function tst_ncpus_available directly? Seems the new get_numcpus() did a dup work on that.

Oops, seems the real-time tests does not include ltp library. So you patch make sense from this point, but maybe we need to refine the function to consider more situations.

e.g. uses a fixed-size cpu_set_t structure, which might not be sufficient on systems with a large number of CPUs.

Quoted 17 lines

Otherwise, looks good to me.

Quoted 11 lines
testcases/realtime/func/prio-preempt/prio-preempt.c +2 -3
-       /* Max no. of busy threads should always be less than/equal the no. of cpus
-          Otherwise, the box will hang */
+       /* Max no. of busy threads should always be less than/equal the no. of
+          housekeeping cpus. Otherwise, the box will hang */
 
        if (rt_threads == -1 || rt_threads > numcpus) {
                rt_threads = numcpus;
--

2.45.2

Quoted 3 lines

--
Regards,
Li Wang

--
Regards,
Li Wang

Signature · 1 line
Mailing list info: https://lists.linux.it/listinfo/ltp