Skip to contentSkip to conversation

drop note_interrupt() for per-CPU for proper scaling

PATCH In review John Hawkes 2 participants 2 messages May 25, 2005 – May 26, 2005 IRQ

lore
Share & tools
lore
#0 lore raw

PATCH 1/1 drop note_interrupt() for per-CPU for proper scaling

The "unhandled interrupts" catcher, note_interrupt(), increments a global desc->irq_count and grossly damages scaling of very large systems, e.g.,

Quoted 1 line

192p ia64 Altix, because of this highly contented cacheline, especially

for timer interrupts. 384p is severely crippled, and 512p is unuseable.

All calls to note_interrupt() can be disabled by booting with "noirqdebug", but this disables the useful interrupt checking for all interrupts.

I propose eliminating note_interrupt() for all per-CPU interrupts. This was the behavior of linux-2.6.10 and earlier, but in 2.6.11 a code restructuring added a call to note_interrupt() for per-CPU interrupts. Besides, note_interrupt() is a bit racy for concurrent CPU calls anyway, as the desc->irq_count++ increment isn't atomic (which, if done, would make scaling even worse).

---

kernel/irq/handle.c +0 -2
Index: linux/kernel/irq/handle.c
===================================================================
@@ -118,8 +118,6 @@
 		 */
 		desc->handler->ack(irq);
 		action_ret = handle_IRQ_event(irq, regs, desc->action);
-		if (!noirqdebug)
-			note_interrupt(irq, desc, action_ret);
 		desc->handler->end(irq);
 		return 1;
 	}
#1 lore raw
Quoted 8 lines · John Hawkes

The "unhandled interrupts" catcher, note_interrupt(), increments a global desc->irq_count and grossly damages scaling of very large systems, e.g.,

Quoted 1 line

192p ia64 Altix, because of this highly contented cacheline, especially

for timer interrupts. 384p is severely crippled, and 512p is unuseable.

All calls to note_interrupt() can be disabled by booting with "noirqdebug", but this disables the useful interrupt checking for all interrupts.

Quoted 6 lines

I propose eliminating note_interrupt() for all per-CPU interrupts. This was the behavior of linux-2.6.10 and earlier, but in 2.6.11 a code restructuring added a call to note_interrupt() for per-CPU interrupts. Besides, note_interrupt() is a bit racy for concurrent CPU calls anyway, as the desc->irq_count++ increment isn't atomic (which, if done, would make scaling even worse).

Would it be reasonable for note_interrupt() to operate on per-CPU or per-group-of-CPUs counters? Each CPU (or group) could either separately count and check, or, if accuracy is important (can't see why it would be), the per-CPU (or group) counters could be added to a global counter every 100 (or 1,000 or whatever) counts, and this global counter then checked against the limit.

Thanx, Paul

Quoted 28 lines

---

Index: linux/kernel/irq/handle.c
===================================================================

kernel/irq/handle.c +0 -3
@@ -118,8 +118,6 @@
 		 */
 		desc->handler->ack(irq);
 		action_ret = handle_IRQ_event(irq, regs, desc->action);
-		if (!noirqdebug)
-			note_interrupt(irq, desc, action_ret);
 		desc->handler->end(irq);
 		return 1;
 	}
 
 
-------------------------------------------------------

SF.Net email is sponsored by: GoToMeeting - the easiest way to collaborate online with coworkers and clients while avoiding the high cost of travel and communications. There is no equipment to buy and you can meet as often as you want. Try it free.http://ads.osdn.com/?ad_id=7402&alloc_id=16135&op=click
_______________________________________________
Lse-tech mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/lse-tech