Skip to contentSkip to conversation

i386 reduce spurious interrupt noise

PATCH In review Chris Wedgwood 5 participants 16 messages Sep 2, 2004 – Sep 11, 2004

lore
Share & tools
lore
#0 lore raw

PATCH 1/1 i386 reduce spurious interrupt noise

i386 hardware can (and does) see spurious interrupts from time to tome. Ideally I would like the printk removed completely but this is probably good enough for now.

===== arch/i386/kernel/apic.c 1.58 vs edited =====

arch/i386/kernel/apic.c +1 -1
@@ -1190,7 +1190,7 @@
 	   6: Received illegal vector
 	   7: Illegal register address
 	*/
-	printk (KERN_INFO "APIC error on CPU%d: %02lx(%02lx)\n",
+	printk (KERN_DEBUG "APIC error on CPU%d: %02lx(%02lx)\n",
 	        smp_processor_id(), v , v1);
 	irq_exit();
 }

===== arch/i386/kernel/i8259.c 1.36 vs edited =====

arch/i386/kernel/i8259.c +1 -1
@@ -226,7 +226,7 @@
 		 * lets ACK and report it. [once per IRQ]
 		 */
 		if (!(spurious_irq_mask & irqmask)) {
-			printk("spurious 8259A interrupt: IRQ%d.\n", irq);
+			printk(KERN_DEBUG "spurious 8259A interrupt: IRQ%d.\n", irq);
 			spurious_irq_mask |= irqmask;
 		}
 		atomic_inc(&irq_err_count);
#1 lore raw
Quoted 3 lines · Chris Wedgwood

i386 hardware can (and does) see spurious interrupts from time to tome. Ideally I would like the printk removed completely but this is probably good enough for now.

Please check printk_ratelimit().

-- wli

#2 lore raw
Quoted 1 line · William Lee Irwin III

Please check printk_ratelimit().

I don't want them displayed by default at *all* --- it wakes up the monitor on console machines and that's annoying.

You get about 1 or 2 a day --- rate limiting isn't useful, nor is reporting them IMO.

--cw

#3 lore raw
Quoted 1 line · William Lee Irwin III
Quoted 1 line

Please check printk_ratelimit().

Quoted 5 lines · Chris Wedgwood

I don't want them displayed by default at *all* --- it wakes up the monitor on console machines and that's annoying.
You get about 1 or 2 a day --- rate limiting isn't useful, nor is reporting them IMO.
--cw

That's okay. The reason why is that this is in response to an external stimulus which can, in principle, scream out of control, so even at KERN_DEBUG or other loglevels it's meaningful to rate limit it.

-- wli

#4 lore raw
Quoted 4 lines · William Lee Irwin III

That's okay. The reason why is that this is in response to an external stimulus which can, in principle, scream out of control, so even at KERN_DEBUG or other loglevels it's meaningful to rate limit it.

If you have enough to be a problem something is wrong and you're dead already. For such cases a more generic interrupt throttling approach is required.

--cw

#5 lore raw
In reply to Chris Wedgwood
Quoted 11 lines · Chris Wedgwood
Quoted 1 line · William Lee Irwin III

Please check printk_ratelimit().

I don't want them displayed by default at *all* --- it wakes up the monitor on console machines and that's annoying.

You get about 1 or 2 a day --- rate limiting isn't useful, nor is reporting them IMO.

Right, spurious interrupts aren't a big deal on i386. They happen now and then with some devices because some hardware timing tolerances are a little too tight. See for example sections 5.7.1.3/5.7.4 of the intel 850 chipset databook (http://developer.intel.com/design/chipsets/datashts/29068702.pdf) :

"6. Upon receiving the second internally generated INTA# pulse, the PIC returns the interrupt vector. If no interrupt request is present because the request was too short in duration, the PIC will return vector 7 from the master controller."

"In both the edge-triggered and level-triggered modes, the IRQ inputs must remain active until after the falling edge of the first internal INTA#. If the IRQ input goes inactive before this time, a default IRQ7 vector will be returned."

Quoted 9 lines

--cw
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/

#6 lore raw
Quoted 3 lines · Nathan Bryant

Right, spurious interrupts aren't a big deal on i386. They happen now and then with some devices because some hardware timing tolerances are a little too tight.

It also happens on a lot of hardware on the odd instance a non IRQ code path clears down an interrupt just as its being raised. IDE does it now and then for example.

#7 lore raw
Quoted 3 lines · Alan Cox

It also happens on a lot of hardware on the odd instance a non IRQ code path clears down an interrupt just as its being raised. IDE does it now and then for example.

So how about we just remove those printk statements completely then? I've never heard of a single need for them other than reporting things we don't really care about....

#8 lore raw
In reply to Chris Wedgwood
Quoted 10 lines · Chris Wedgwood

i386 hardware can (and does) see spurious interrupts from time to tome. Ideally I would like the printk removed completely but this is probably good enough for now.

===== arch/i386/kernel/apic.c 1.58 vs edited =====

arch/i386/kernel/apic.c line 1196

Quoted 8 lines
arch/i386/kernel/apic.c +1 -1
@@ -1190,7 +1190,7 @@
 	   6: Received illegal vector
 	   7: Illegal register address
 	*/
-	printk (KERN_INFO "APIC error on CPU%d: %02lx(%02lx)\n",
+	printk (KERN_DEBUG "APIC error on CPU%d: %02lx(%02lx)\n",
 	        smp_processor_id(), v , v1);
 	irq_exit();
 }

This should probably be KERN_ERR even. This is a serious condition -- if
you ever get such a message, then inter-APIC messages get corrupted and this affects system's stability. E.g. with a badly corrupted message you may get one or more of your processors halted if the matching destinations misinterpret the delivery mode as a result. You certainly want to know about these errors and perhaps get your hardware replaced (starting with the PSU as they've been repeatedly reported to be the causers).

arch/i386/kernel/i8259.c line 232

Quoted 12 lines

===== arch/i386/kernel/i8259.c 1.36 vs edited =====

arch/i386/kernel/i8259.c +1 -1
@@ -226,7 +226,7 @@
 		 * lets ACK and report it. [once per IRQ]
 		 */
 		if (!(spurious_irq_mask & irqmask)) {
-			printk("spurious 8259A interrupt: IRQ%d.\n", irq);
+			printk(KERN_DEBUG "spurious 8259A interrupt: IRQ%d.\n", irq);
 			spurious_irq_mask |= irqmask;
 		}
 		atomic_inc(&irq_err_count);

You may ever get a single message per system boot from this line. It
encourages to have a look at the ERR counter in /proc/interrupts to check for possible problems, though admittedly the suggestion isn't especially clear.

Maciej

#9 lore raw
Quoted 6 lines · Maciej W. Rozycki

arch/i386/kernel/apic.c line 1193

Quoted 2 lines
arch/i386/kernel/apic.c +1 -1
-	printk (KERN_INFO "APIC error on CPU%d: %02lx(%02lx)\n",
+	printk (KERN_DEBUG "APIC error on CPU%d: %02lx(%02lx)\n",

This should probably be KERN_ERR even. This is a serious condition -- if you ever get such a message, then inter-APIC messages get corrupted and this affects system's stability.

These messages are very common on many platforms, infrequent (once very few days to twice a day at most in my observations) and seemingly harmless.

I agree that if you get *many* of these certainly that would indicate there is a problem but I've not not heard a single instance of this and if that is the case we need to deal with it differently.

Quoted 4 lines

arch/i386/kernel/i8259.c line 229

Quoted 2 lines
arch/i386/kernel/i8259.c +1 -1
-			printk("spurious 8259A interrupt: IRQ%d.\n", irq);
+			printk(KERN_DEBUG "spurious 8259A interrupt: IRQ%d.\n", irq);

You may ever get a single message per system boot from this line.

Sometimes as boot, though often in my experience several minutes after boot.

Quoted 3 lines

It encourages to have a look at the ERR counter in /proc/interrupts to check for possible problems, though admittedly the suggestion isn't especially clear.

I think in *both* cases we want to detect a largish (more than 1 ever n seconds or so) number of these and then complain, not before and even then not excessively so that we printk our-selves to death.

I'm not inclined to offer such a patch right now as it feels like it's fixing a problem nobody has reported.

--cw

#10 lore raw
Quoted 9 lines · Chris Wedgwood
Quoted 6 lines · Maciej W. Rozycki

arch/i386/kernel/apic.c line 1193

Quoted 2 lines
arch/i386/kernel/apic.c +1 -1
-	printk (KERN_INFO "APIC error on CPU%d: %02lx(%02lx)\n",
+	printk (KERN_DEBUG "APIC error on CPU%d: %02lx(%02lx)\n",

This should probably be KERN_ERR even. This is a serious condition -- if you ever get such a message, then inter-APIC messages get corrupted and this affects system's stability.

Quoted 3 lines

These messages are very common on many platforms, infrequent (once very few days to twice a day at most in my observations) and seemingly harmless.

On a lot of 2.4 boxes they aren't harmless but thats 2.4 IPI messsage handling bugs. People sometimes assume an IPI is delivered once - but its not its delivered "at least once" and when you get a checksum error like on old dual celerons you get replays.

They also identify kernel bugs in some other bit combinations so they are useful there too. I'd say this should only go if we are sure 2.6.x handles IPI replay properly and we mask bits off to see if its real news or a retry.

Quoted 7 lines
Quoted 4 lines

arch/i386/kernel/i8259.c line 229

Quoted 2 lines
arch/i386/kernel/i8259.c +1 -1
-			printk("spurious 8259A interrupt: IRQ%d.\n", irq);
+			printk(KERN_DEBUG "spurious 8259A interrupt: IRQ%d.\n", irq);

You may ever get a single message per system boot from this line.

Sometimes as boot, though often in my experience several minutes after boot.

The IDE layer will generate these naturally as will any other code that happens to clear an IRQ causing event in non IRQ context. Eventually you clear it just as the IRQ is raised, and the pulse causes the error.

This should really go.

#11 lore raw
Quoted 4 lines · Alan Cox

On a lot of 2.4 boxes they aren't harmless but thats 2.4 IPI messsage handling bugs. People sometimes assume an IPI is delivered once - but its not its delivered "at least once" and when you get a checksum error like on old dual celerons you get replays.

it sounds like leaving it as KERN_DEBUG is the right thing to do then

Quoted 4 lines
Quoted 2 lines
Quoted 2 lines

arch/i386/kernel/i8259.c line 229

Quoted 2 lines
arch/i386/kernel/i8259.c +1 -1
-			printk("spurious 8259A interrupt: IRQ%d.\n", irq);
+			printk(KERN_DEBUG "spurious 8259A interrupt: IRQ%d.\n", irq);

This should really go.

do we want counters for this? what about the APIC case?

#12 lore raw
Quoted 6 lines · Chris Wedgwood
Quoted 4 lines
Quoted 2 lines
Quoted 2 lines

arch/i386/kernel/i8259.c line 229

Quoted 2 lines
arch/i386/kernel/i8259.c +1 -1
-			printk("spurious 8259A interrupt: IRQ%d.\n", irq);
+			printk(KERN_DEBUG "spurious 8259A interrupt: IRQ%d.\n", irq);

This should really go.

do we want counters for this? what about the APIC case?

I don't know enough about the APIC version to comment, just the PIC one.

#13 lore raw
In reply to Chris Wedgwood
Quoted 10 lines · Chris Wedgwood
Quoted 6 lines

arch/i386/kernel/apic.c line 1193

Quoted 2 lines
arch/i386/kernel/apic.c +1 -1
-	printk (KERN_INFO "APIC error on CPU%d: %02lx(%02lx)\n",
+	printk (KERN_DEBUG "APIC error on CPU%d: %02lx(%02lx)\n",

This should probably be KERN_ERR even. This is a serious condition -- if you ever get such a message, then inter-APIC messages get corrupted and this affects system's stability.

These messages are very common on many platforms, infrequent (once very few days to twice a day at most in my observations) and seemingly harmless.

These are just as harmless as single-bit RAM errors with ECC working.
In both cases you want the problem to be reported.

Quoted 3 lines

I agree that if you get *many* of these certainly that would indicate there is a problem but I've not not heard a single instance of this and if that is the case we need to deal with it differently.

Please search list archives for lots of such reports.

Quoted 7 lines
Quoted 4 lines

arch/i386/kernel/i8259.c line 229

Quoted 2 lines
arch/i386/kernel/i8259.c +1 -1
-			printk("spurious 8259A interrupt: IRQ%d.\n", irq);
+			printk(KERN_DEBUG "spurious 8259A interrupt: IRQ%d.\n", irq);

You may ever get a single message per system boot from this line.

Sometimes as boot, though often in my experience several minutes after boot.

And never again until you reboot. That's what I mean.

Quoted 7 lines
Quoted 3 lines

It encourages to have a look at the ERR counter in /proc/interrupts to check for possible problems, though admittedly the suggestion isn't especially clear.

I think in *both* cases we want to detect a largish (more than 1 ever n seconds or so) number of these and then complain, not before and even then not excessively so that we printk our-selves to death.

I agree for the latter case. I won't mind the message going away either.
For the former you only really want to rate-limit the report -- some people apparently want or need to run broken hardware and they'd probably appreciate limiting the output.

Maciej

#14 lore raw
Quoted 2 lines · Maciej W. Rozycki

These are just as harmless as single-bit RAM errors with ECC working.

Hence KERN_DEBUG

Quoted 3 lines

For the former you only really want to rate-limit the report -- some people apparently want or need to run broken hardware and they'd probably appreciate limiting the output.

A little more than rate-limit as I mentioned. I don't want the occasional surious APIC message waking up consoles that are asleep. This was the reason for the change.

--cw

#15 lore raw
Quoted 4 lines · Chris Wedgwood
Quoted 2 lines

These are just as harmless as single-bit RAM errors with ECC working.

Hence KERN_DEBUG

Both are serious hardware failures. KERN_DEBUG is for stuff that's
normally out of interest for most system operators.

Quoted 7 lines
Quoted 3 lines

For the former you only really want to rate-limit the report -- some people apparently want or need to run broken hardware and they'd probably appreciate limiting the output.

A little more than rate-limit as I mentioned. I don't want the occasional surious APIC message waking up consoles that are asleep. This was the reason for the change.

If that's the sole reason, then how about setting console_loglevel
appropriately for the systems you want the console to remain asleep? It's there exactly for a purpose like this. You can eliminate other messages you consider unimportant this way, too, without tweaking the log level of all of them.

Maciej