Skip to contentSkip to conversation

irqbalance: Mark in-kernel irqbalance as obsolete, set to N by default

RFC In review Auke Kok 3 participants 4 messages Jul 31, 2006 – Aug 4, 2006

Subsystems (2)
lore
Share & tools
lore

Auke Kok

author
#0 lore raw

RFC 1/1 irqbalance: Mark in-kernel irqbalance as obsolete, set to N by default

We've recently seen a number of user bug reports against e1000 that the in-kernel irqbalance code is detrimental to network latency. The algorithm keeps swapping irq's for NICs from cpu to cpu causing extremely high network latency (>1000ms). Another NIC driver (cxgb) already has severe warnings in their documentation file against using CONFIG_IRQBALANCE, but this is a general problem for all NIC drivers and other subsystems. This is especially so with cpufreq scaling where the system is slowed down and the migrations take much longer.

I suggest that the in-kernel irqbalance is phased out, by marking it OBSOLETE first and (perhaps) removing the code later. The userspace irqbalance daemon written by Arjan van de Ven does a wonderful job and should be used instead.

---

 Kconfig |   15 +++++++++++----
 1 file changed, 11 insertions(+), 4 deletions(-)

---

arch/i386/Kconfig +11 -4
index daa75ce..5a40cfe 100644
@@ -690,12 +690,19 @@ config EFI
 	kernel should continue to boot on existing non-EFI platforms.
 
 config IRQBALANCE
- 	bool "Enable kernel irq balancing"
+	bool "Enable kernel irq balancing (obsolete)"
 	depends on SMP && X86_IO_APIC
-	default y
+	default n
 	help
- 	  The default yes will allow the kernel to do irq load balancing.
-	  Saying no will keep the kernel from doing irq load balancing.
+	  The kernel irq balance will migrate interrupts between cpu's
+	  constantly, which may help reduce load in some cases. It is not
+	  beneficial for latency however, and a user-space daemon is available
+	  that does a much better job.
+
+	  The default no will keep the kernel from doing irq load balancing.
+	  Say yes will allow the kernel to do irq load balancing.
+
+	  If unsure, say N.
 
 # turning this on wastes a bunch of space.
 # Summit needs it only when NUMA is on
#1 lore raw
Quoted 4 lines · On Mon, 31 Jul 2006 10:35:26 -0700 Auke Kok <[email protected]> wrote:

We've recently seen a number of user bug reports against e1000 that the in-kernel irqbalance code is detrimental to network latency. The algorithm keeps swapping irq's for NICs from cpu to cpu causing extremely high network latency (>1000ms).

What kernel versions? Some IRQ balancer fixes went in shortly after 2.6.17.

It would be better if poss to fix the balancer rather than deprecating it.

author
#2 lore raw
Quoted 6 lines · Andrew Morton
Quoted 1 line · On Mon, 31 Jul 2006 10:35:26 -0700 Auke Kok <[email protected]> wrote:

We've recently seen a number of user bug reports against e1000 that the in-kernel irqbalance code is detrimental to network latency. The algorithm keeps swapping irq's for NICs from cpu to cpu causing extremely high network latency (>1000ms).

What kernel versions? Some IRQ balancer fixes went in shortly after 2.6.17.

user reports show 2.6.17.1 having the problem, I'm trying to get more details information, and will ask if 2.6.18rc3 or so does better for them.

Cheers,

Auke

#3 lore raw
In reply to Andrew Morton
Quoted 8 lines · Andrew Morton
Quoted 1 line · On Mon, 31 Jul 2006 10:35:26 -0700 Auke Kok <[email protected]> wrote:

We've recently seen a number of user bug reports against e1000 that the in-kernel irqbalance code is detrimental to network latency. The algorithm keeps swapping irq's for NICs from cpu to cpu causing extremely high network latency (>1000ms).

What kernel versions? Some IRQ balancer fixes went in shortly after 2.6.17.

It would be better if poss to fix the balancer rather than deprecating it.

to some degree the in kernel balancer cannot really make the level of decisions that a userspace balancer can make, at least not without making all kernel developers vomit ;) (for example the userspace balancer looks in /proc/interrupts and parses that to see which interrupts are used by networking versus which by storage etc, and has different balancing policies for those and other classes; the networking policy basically comes down to "pin the interrupt unless some higher networking interrupt really gets in the way")