kernel/irq: Add irqbalance01
A message that is not in the archive (sent off-list or not yet received): [email protected]
PATCH v2 1/1 kernel/irq: Add irqbalance01
Add first test specifically targeting interrupts and IRQ management.
This includes some comments inline because I think the parsing code is unavoidably confusing.
Note on the CPU mask parsing; there is already some code for parsing and manipulating bitmaps in the LTP. However it is absurdly complicated and we don't need actual bitmaps. In fact an array of bytes is more flexible.
---
V2:
* Read proc files in at most 1-page blocks
As Cyril suggested this looks like the best way. seq_file will only expand the internal buffer above a page if a record does not fit in one page. In this case a record is the per-cpus counts for an irq. However it won't expand the buffer for multiple records. It just returns whatver fits in the buffer.
* Read in CPU masks from the end of the buffer
If nr_cpus is not a multiple of 4 then one of the hex digits has some padding. We either need to pad irq_affinity as well (which creates complication) or ignore the padding. It's easier to ignore it if we start at the end.
* Print headers and some minor format changes to printing
* Add a few more parsing checks
* Remove freeing of buffers
runtest/irq | 1 + testcases/kernel/irq/.gitignore | 1 + testcases/kernel/irq/Makefile | 7 + testcases/kernel/irq/irqbalance01.c | 316 ++++++++++++++++++++++++++++ 4 files changed, 325 insertions(+) create mode 100644 runtest/irq create mode 100644 testcases/kernel/irq/.gitignore create mode 100644 testcases/kernel/irq/Makefile create mode 100644 testcases/kernel/irq/irqbalance01.c
runtest/irq new file +1 -0
@@ -0,0 +1 @@ +irqbalance01 irqbalance01
testcases/kernel/irq/.gitignore new file +1 -0
@@ -0,0 +1 @@ +irqbalance01
testcases/kernel/irq/Makefile new file +7 -0
@@ -0,0 +1,7 @@ +# SPDX-License-Identifier: GPL-2.0-or-later + +top_srcdir ?= ../../.. + +include $(top_srcdir)/include/mk/testcases.mk + +include $(top_srcdir)/include/mk/generic_leaf_target.mk
testcases/kernel/irq/irqbalance01.c new file +316 -0
316 lines not on this page · raw message
Signature · 1 line
2.31.1
PATCH v2 1/1 kernel/irq: Add irqbalance01
Hi Richie,
Changes to this version (it's actually v3) LGTM.
I suppose Cyril wants to recheck it, thus not merging it yet.
Thanks!
Kind regards,
Petr
testcases/kernel/irq/.gitignore line 1
Quoted 3 lines
+++ b/testcases/kernel/irq/.gitignore
testcases/kernel/irq/.gitignore new file +1
-0@@ -0,0 +1 @@ +irqbalance01
Very nit: we usually put '/' at the beginning to: /irqbalance01
(to apply files only in testcases/kernel/irq/, but that's irrelevant in most cases as there are no subdirectories). If we even bother, it can be changed during merge.
Hi!
testcases/kernel/irq/irqbalance01.c line 200
Quoted 12 lines
testcases/kernel/irq/irqbalance01.c new file +12
-0+/* Read the CPU affinity masks for each IRQ. The first CPU is in the +* right most (least significant) bit. See bitmap_string() in the kernel +* (%*pb) +*/ +for (row =0 ; row < nr_irqs; row++) { +sprintf (path,"/proc/irq/%u/smp_affinity" , irq_ids[row]); + buf =read_proc_file (path); + c = buf; + col =0 ; + +while (*c !='\0' ) + c++;
A minor nit, we alredy know the lenght, but we do not return it from the read_proc_file() we may as well change this so taht the read_proc_file returns both buffer pointer and the size. E.g. add size_t *len parameter to the read_proc_file() and set it if it's not NULL.
Other than that the parsing looks good now.
Quoted 10 lines
testcases/kernel/irq/irqbalance01.c new file +10
-0+static void evidence_of_change (void ) +{ +size_t row, col, changed =0 ; + +for (row =0 ; row < nr_irqs; row++) { +for (col =0 ; col < nr_cpus; col++) { +if (!irq_stats[row * nr_cpus + col]) +continue ; + +if (irq_affinity[row * nr_cpus + col] ==ALLOW )
testcases/kernel/irq/irqbalance01.c line 291
Quoted 10 lines
testcases/kernel/irq/irqbalance01.c new file +10
-0+continue ; + + changed++; + } + } + +tst_res (changed ?TPASS :TFAIL , +"Heuristic: Detected %zu irq-cpu pairs have been dissallowed" , + changed); +}
I'm still unusure about this part though.
The test fails on my workstation where the IRQs are nicely distributed by the hardware/BIOS.
Maybe we should check if IRQs are distributed somehow evenly and PASS the test if that is the case as well.
In my case there is 27 IRQs, 18 of them have non-zero counters and there is 12 CPUs. For active IRQs that gives 18/12 = 1.5 IRQ per CPU and there are 7 CPUs that handle 2 IRQs, 4 CPUs that handle 1 IRQ and 1 CPU that doesn't handle anything. I wonder if we can figure out some kind of heuristic, but I guess that in the end it wouldn't be worth the effort.
Signature · 5 lines
Cyril Hrubis [email protected] -- Mailing list info: https://lists.linux.it/listinfo/ltp
Hello Cyril,
Quoted 10 lines · Cyril Hrubis
Hi!
Quoted 9 lines
testcases/kernel/irq/irqbalance01.c new file +9
-0+/* Read the CPU affinity masks for each IRQ. The first CPU is in the +* right most (least significant) bit. See bitmap_string() in the kernel +* (%*pb) +*/ +for (row =0 ; row < nr_irqs; row++) { +sprintf (path,"/proc/irq/%u/smp_affinity" , irq_ids[row]); + buf =read_proc_file (path); + c = buf; + col =0 ;
Quoted 8 lines
testcases/kernel/irq/irqbalance01.c line 200
Quoted 3 lines
testcases/kernel/irq/irqbalance01.c new file +3
-0+ +while (*c !='\0' ) + c++;A minor nit, we alredy know the lenght, but we do not return it from the
read_proc_file()we may as well change this so taht theread_proc_filereturns both buffer pointer and the size. E.g. addsize_t*len parameter to theread_proc_file()and set it if it's not NULL.
+1
Quoted 26 lines
Other than that the parsing looks good now.
Quoted 10 lines
testcases/kernel/irq/irqbalance01.c new file +10
-0+static void evidence_of_change (void ) +{ +size_t row, col, changed =0 ; + +for (row =0 ; row < nr_irqs; row++) { +for (col =0 ; col < nr_cpus; col++) { +if (!irq_stats[row * nr_cpus + col]) +continue ; + +if (irq_affinity[row * nr_cpus + col] ==ALLOW )testcases/kernel/irq/irqbalance01.c line 291
Quoted 10 lines
testcases/kernel/irq/irqbalance01.c new file +10
-0+continue ; + + changed++; + } + } + +tst_res (changed ?TPASS :TFAIL , +"Heuristic: Detected %zu irq-cpu pairs have been dissallowed" , + changed); +}I'm still unusure about this part though.
Quoted 2 lines
The test fails on my workstation where the IRQs are nicely distributed by the hardware/BIOS.
You don't need irqbalance then :-p
The original bug this is intened to avoid was caused by the irqbalance deamon silently failing.
Quoted 3 lines
Maybe we should check if IRQs are distributed somehow evenly and PASS the test if that is the case as well.
My plan was to have a second heuristic based on the IRQ distribution. However the intention was to make detection more sensitve to bad setups. I think what you are suggesting is the opposite. So that users can blindly run the test and let it figure out if it is appropriate for the hardware.
Quoted 7 lines
In my case there is 27 IRQs, 18 of them have non-zero counters and there is 12 CPUs. For active IRQs that gives 18/12 = 1.5 IRQ per CPU and there are 7 CPUs that handle 2 IRQs, 4 CPUs that handle 1 IRQ and 1 CPU that doesn't handle anything. I wonder if we can figure out some kind of heuristic, but I guess that in the end it wouldn't be worth the effort.
I general I think it is best to write tests which figure out if they are in the right environment. This makes all our lives easier. The problem with this test in particular is the number of assumptions we have to make about what a good IRQ distirbution looks like.
IRQs are (usually) evenly distributed for "performance". However not all IRQs are equal, you might get better performance through an uneven distribution. Especially if you only care about the performance some tasks and not others. Also there are some reasons to isolate some CPUs from IRQs regardless of performance.
Having said that we can probably find a heuristic for what a very bad setup looks like i.e. (almost) all NIC IRQs on one CPU when there are 30+ CPUs and multiple NICs. That is one CPU is totally saturated with IRQs from different NICs. This is a bit speculative so I'd like some feedback from testing/community before putting effort into that.
Signature · 5 lines
Thank you, Richard. -- Mailing list info: https://lists.linux.it/listinfo/ltp