Skip to contentSkip to conversation

tickle nmi watchdog whilst doing serial writes.

PATCH Dave Jones 9 participants 19 messages May 13, 2005 – May 23, 2005

lore
Share & tools
lore

Dave Jones

author
#0 lore raw

This was fun. I inserted a music CD with some obnoxious copy-protection on it into the drive, and lots of SCSI errors went zipping over to the serial console. Unfortunatly, the box was also compiling a kernel, playing oggs, and doing a number of other things at the same time, so this happened..

NMI Watchdog detected LOCKUP on CPU2CPU 2

Modules linked in: loop usb_storage md5 ipv6 parport_pc lp parport autofs4 i2c_dev i2c_core rfcomm l2cap bluetooth sunrpc pcdPid: 3138, comm: gpm Not tainted 2.6.11-1.1290_FC4smp
RIP: 0010:[<ffffffff80273b8a>] <ffffffff80273b8a>{serial_in+106}
RSP: 0018:ffff81003afc3d50  EFLAGS: 00000002
RAX: 0000000000000020 RBX: 0000000000000020 RCX: 0000000000000000
RDX: 00000000000003fd RSI: 0000000000000005 RDI: ffffffff804dcd60
RBP: 00000000000024fc R08: 000000000000000a R09: 0000000000000033
R10: ffff81001beb7c20 R11: 0000000000000020 R12: ffffffff804dcd60
R13: ffffffff804ade76 R14: 000000000000002b R15: 000000000000002c
FS:  00002aaaaaac4920(0000) GS:ffffffff804fca00(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 000000008005003b
CR2: 00002aaaaabcb000 CR3: 000000003c0d0000 CR4: 00000000000006e0
Process gpm (pid: 3138, threadinfo ffff81003afc2000, task ffff81003eb63780)
Stack: ffffffff80275f2e 0000000000000000 ffffffff80448380 0000000000007d6b
       000000000000002c fffffffffffffbbf 0000000000000292 0000000000008000
       ffffffff80138e8c 0000000000007d97
Call Trace:<ffffffff80275f2e>{serial8250_console_write+270} <ffffffff80138e8c>{__call_console_drivers+76}
       <ffffffff8013914b>{release_console_sem+315} <ffffffff80260325>{con_open+149}
       <ffffffff80254e99>{tty_open+537} <ffffffff80192713>{chrdev_open+387}
       <ffffffff80188824>{dentry_open+260} <ffffffff80188994>{filp_open+68}
       <ffffffff80187b73>{get_unused_fd+227} <ffffffff80188a6c>{sys_open+76}
       <ffffffff8010ebc6>{tracesys+209}

Code: 0f b6 c0 c3 66 90 41 57 49 89 f7 41 56 41 be 00 01 00 00 41

console shuts up ...

drivers/serial/8250.c renamed from drivers/serial/8250.c~ +4 -1
@@ -40,6 +40,7 @@
 #include <linux/serial_core.h>
 #include <linux/serial.h>
 #include <linux/serial_8250.h>
+#include <linux/nmi.h>
 
 #include <asm/io.h>
 #include <asm/irq.h>
@@ -2099,8 +2100,10 @@ static inline void wait_for_xmitr(struct
 	if (up->port.flags & UPF_CONS_FLOW) {
 		tmout = 1000000;
 		while (--tmout &&
-		       ((serial_in(up, UART_MSR) & UART_MSR_CTS) == 0))
+		       ((serial_in(up, UART_MSR) & UART_MSR_CTS) == 0)) {
 			udelay(1);
+			touch_nmi_watchdog();
+		}
 	}
 }
 

We *could* tickle it less often, but given we're busy waiting anyway it probably doesnt make sense to not favour the more simple approach. Hmm, maybe we want a cpu_relax() in there too. opinions?

Dave

#1 lore raw
Quoted 12 lines · Dave Jones
drivers/serial/8250.c renamed from drivers/serial/8250.c~ +3 -1
 	if (up->port.flags & UPF_CONS_FLOW) {
 		tmout = 1000000;
 		while (--tmout &&
-		       ((serial_in(up, UART_MSR) & UART_MSR_CTS) == 0))
+		       ((serial_in(up, UART_MSR) & UART_MSR_CTS) == 0)) {
 			udelay(1);
+			touch_nmi_watchdog();
+		}
 	}
 }
Quoted 3 lines

We *could* tickle it less often, but given we're busy waiting anyway it probably doesnt make sense to not favour the more simple approach. Hmm, maybe we want a cpu_relax() in there too. opinions?

udelay() includes cpu_relax() already so that is futile.

However.. this is a hack. Do we really need to do busy waiting here for this long??

author
#2 lore raw
Quoted 13 lines · Arjan van de Ven
Quoted 12 lines · Dave Jones
drivers/serial/8250.c renamed from drivers/serial/8250.c~ +3 -1
 	if (up->port.flags & UPF_CONS_FLOW) {
 		tmout = 1000000;
 		while (--tmout &&
-		       ((serial_in(up, UART_MSR) & UART_MSR_CTS) == 0))
+		       ((serial_in(up, UART_MSR) & UART_MSR_CTS) == 0)) {
 			udelay(1);
+			touch_nmi_watchdog();
+		}
 	}
 }
Quoted 8 lines
Quoted 3 lines

We *could* tickle it less often, but given we're busy waiting anyway it probably doesnt make sense to not favour the more simple approach. Hmm, maybe we want a cpu_relax() in there too. opinions?

udelay() includes cpu_relax() already so that is futile.

However.. this is a hack. Do we really need to do busy waiting here for this long??

Ohhhhh no. I've fallen into this trap before.
I'm not looking any further into serial code than I have to :)

Russell / dwmw2 may have a more definitive answers as to why we have such a long wait here, but every time I learn something about the serial layer I end up regretting it, so mine is a drive-by patching only :-)

Dave

#3 lore raw
In reply to Arjan van de Ven
Quoted 13 lines · Arjan van de Ven
Quoted 12 lines · Dave Jones
drivers/serial/8250.c renamed from drivers/serial/8250.c~ +3 -1
 	if (up->port.flags & UPF_CONS_FLOW) {
 		tmout = 1000000;
 		while (--tmout &&
-		       ((serial_in(up, UART_MSR) & UART_MSR_CTS) == 0))
+		       ((serial_in(up, UART_MSR) & UART_MSR_CTS) == 0)) {
 			udelay(1);
+			touch_nmi_watchdog();
+		}
 	}
 }
Quoted 8 lines
Quoted 3 lines

We *could* tickle it less often, but given we're busy waiting anyway it probably doesnt make sense to not favour the more simple approach. Hmm, maybe we want a cpu_relax() in there too. opinions?

udelay() includes cpu_relax() already so that is futile.

However.. this is a hack. Do we really need to do busy waiting here for this long??

Of course you do. Think about CTS flow control where the other end is also busy and has to drain its tty buffers before it can allow the remote end to proceed. (And who says its another Linux box anyway?)

However, if people are using CTS flow control, they want reliable logging. Therefore, it's even questionable whether we should even time out (but we do to keep the system "running".)

Signature · 3 lines
Russell King
 Linux kernel    2.6 ARM Linux   - http://www.arm.linux.org.uk/
 maintainer of:  2.6 Serial core
#4 lore raw
In reply to Dave Jones
Quoted 8 lines · Dave Jones

This was fun. I inserted a music CD with some obnoxious copy-protection
on it into the drive, and lots of SCSI errors went zipping over to the serial console. Unfortunatly, the box was also compiling a kernel, playing oggs, and doing a number of other things at the same time, so this happened..

NMI Watchdog detected LOCKUP on CPU2CPU 2

OK.. But calling touch_nmi_watchdog() at 1MHz seems a bit excessive, and might perturb the finely-tuned timing in there.

How's about this?

drivers/serial/8250.c renamed from drivers/serial/8250.c~tickle-nmi-watchdog-whilst-doing-serial-writes +4 -2
@@ -40,6 +40,7 @@
 #include <linux/serial_core.h>
 #include <linux/serial.h>
 #include <linux/serial_8250.h>
+#include <linux/nmi.h>
 
 #include <asm/io.h>
 #include <asm/irq.h>
@@ -2098,9 +2099,10 @@ static inline void wait_for_xmitr(struct
 	/* Wait up to 1s for flow control if necessary */
 	if (up->port.flags & UPF_CONS_FLOW) {
 		tmout = 1000000;
-		while (--tmout &&
-		       ((serial_in(up, UART_MSR) & UART_MSR_CTS) == 0))
+		while (!(serial_in(up, UART_MSR) & UART_MSR_CTS) && --tmout)
 			udelay(1);
+		if (tmout < 1000000)
+			touch_nmi_watchdog();
 	}
 }
 

_

author
#5 lore raw
Quoted 8 lines · Andrew Morton
Quoted 7 lines · Dave Jones

This was fun. I inserted a music CD with some obnoxious copy-protection
on it into the drive, and lots of SCSI errors went zipping over to the serial console. Unfortunatly, the box was also compiling a kernel, playing oggs, and doing a number of other things at the same time, so this happened..

Quoted 6 lines
Quoted 1 line

NMI Watchdog detected LOCKUP on CPU2CPU 2

OK.. But calling touch_nmi_watchdog() at 1MHz seems a bit excessive, and might perturb the finely-tuned timing in there.

How's about this?

Umm.. Despite it being past my bedtime, I'm pretty sure I'm missing something here...

Review

Quoted 2 lines
(quoted diff) +1 -0
+		while (!(serial_in(up, UART_MSR) & UART_MSR_CTS) && --tmout)
 			udelay(1);

I don't see how this is any better than the current code. We're doing 1000000 udelays. Whilst we're doing that, the nmi watchdog goes bonkers.

Review

Quoted 2 lines
(quoted diff) +2 -0
+		if (tmout < 1000000)
+			touch_nmi_watchdog();

So by the time we do this, its already triggered.

How about..

drivers/serial/8250.c renamed from drivers/serial/8250.c~ +9 -1
@@ -40,6 +40,7 @@
 #include <linux/serial_core.h>
 #include <linux/serial.h>
 #include <linux/serial_8250.h>
+#include <linux/nmi.h>
 
 #include <asm/io.h>
 #include <asm/irq.h>
@@ -2099,8 +2100,15 @@ static inline void wait_for_xmitr(struct
 	if (up->port.flags & UPF_CONS_FLOW) {
 		tmout = 1000000;
 		while (--tmout &&
-		       ((serial_in(up, UART_MSR) & UART_MSR_CTS) == 0))
+		       ((serial_in(up, UART_MSR) & UART_MSR_CTS) == 0)) {
+			int cnt=0;
 			udelay(1);
+			cnt++;
+			if (cnt==100) {
+				touch_nmi_watchdog();
+				cnt=0;
+			}
+		}
 	}
 }
 

totally uncompiled & untested. I'm not even sure about that magic '100'. I'll let someone less sleep-deprived than myself choose a better number.

Dave

#6 lore raw
Quoted 27 lines · Dave Jones
Quoted 8 lines · Andrew Morton
Quoted 7 lines · Dave Jones

This was fun. I inserted a music CD with some obnoxious copy-protection
on it into the drive, and lots of SCSI errors went zipping over to the serial console. Unfortunatly, the box was also compiling a kernel, playing oggs, and doing a number of other things at the same time, so this happened..

Quoted 6 lines
Quoted 1 line

NMI Watchdog detected LOCKUP on CPU2CPU 2

OK.. But calling touch_nmi_watchdog() at 1MHz seems a bit excessive, and might perturb the finely-tuned timing in there.

How's about this?

Umm.. Despite it being past my bedtime, I'm pretty sure I'm missing something here...

Review

Quoted 2 lines
(quoted diff) +1 -0
+		while (!(serial_in(up, UART_MSR) & UART_MSR_CTS) && --tmout)
 			udelay(1);

I don't see how this is any better than the current code. We're doing 1000000 udelays. Whilst we're doing that, the nmi watchdog goes bonkers.

Quoted 4 lines

Review

Quoted 2 lines
(quoted diff) +2 -0
+		if (tmout < 1000000)
+			touch_nmi_watchdog();

So by the time we do this, its already triggered.

But the NMI watchdog won't expire after one second - normally it's set to fixe seconds.

Quoted 16 lines

How about..

drivers/serial/8250.c renamed from drivers/serial/8250.c~ +1 -0
@@ -40,6 +40,7 @@
 #include <linux/serial_core.h>
 #include <linux/serial.h>
 #include <linux/serial_8250.h>
+#include <linux/nmi.h>
 
 #include <asm/io.h>
 #include <asm/irq.h>
@@ -2099,8 +2100,15 @@ static inline void wait_for_xmitr(struct
 	if (up->port.flags & UPF_CONS_FLOW) {
 		tmout = 1000000;
 		while (--tmout &&

drivers/serial/8250.c static inline void wait_for_xmitr(struct · line 2111

Quoted 10 lines
drivers/serial/8250.c renamed from drivers/serial/8250.c~ +8 -1
-		       ((serial_in(up, UART_MSR) & UART_MSR_CTS) == 0))
+		       ((serial_in(up, UART_MSR) & UART_MSR_CTS) == 0)) {
+			int cnt=0;
 			udelay(1);
+			cnt++;
+			if (cnt==100) {
+				touch_nmi_watchdog();
+				cnt=0;
+			}
+		}

<obwhitespacewhine> spose so.

drivers/serial/8250.c renamed from drivers/serial/8250.c~tickle-nmi-watchdog-whilst-doing-serial-writes +5 -2
@@ -40,6 +40,7 @@
 #include <linux/serial_core.h>
 #include <linux/serial.h>
 #include <linux/serial_8250.h>
+#include <linux/nmi.h>
 
 #include <asm/io.h>
 #include <asm/irq.h>
@@ -2098,9 +2099,11 @@ static inline void wait_for_xmitr(struct
 	/* Wait up to 1s for flow control if necessary */
 	if (up->port.flags & UPF_CONS_FLOW) {
 		tmout = 1000000;
-		while (--tmout &&
-		       ((serial_in(up, UART_MSR) & UART_MSR_CTS) == 0))
+		while (!(serial_in(up, UART_MSR) & UART_MSR_CTS) && --tmout) {
 			udelay(1);
+			if ((tmout % 1000) == 0)
+				touch_nmi_watchdog();
+		}
 	}
 }
 

_

#7 lore raw

Hi,

Suspect this can be triggered using alt+sysreq+T on a busy system with a slow serial console. Might be a easy way to see if this patch fixes the issue?

Ed Tomlinson

Quoted 92 lines · Andrew Morton
Quoted 27 lines · Dave Jones
Quoted 8 lines · Andrew Morton
Quoted 7 lines · Dave Jones

This was fun. I inserted a music CD with some obnoxious copy-protection
on it into the drive, and lots of SCSI errors went zipping over to the serial console. Unfortunatly, the box was also compiling a kernel, playing oggs, and doing a number of other things at the same time, so this happened..

Quoted 6 lines
Quoted 1 line

NMI Watchdog detected LOCKUP on CPU2CPU 2

OK.. But calling touch_nmi_watchdog() at 1MHz seems a bit excessive, and might perturb the finely-tuned timing in there.

How's about this?

Umm.. Despite it being past my bedtime, I'm pretty sure I'm missing something here...

Review

Quoted 2 lines
(quoted diff) +1 -0
+		while (!(serial_in(up, UART_MSR) & UART_MSR_CTS) && --tmout)
 			udelay(1);

I don't see how this is any better than the current code. We're doing 1000000 udelays. Whilst we're doing that, the nmi watchdog goes bonkers.

Quoted 4 lines

Review

Quoted 2 lines
(quoted diff) +2 -0
+		if (tmout < 1000000)
+			touch_nmi_watchdog();

So by the time we do this, its already triggered.

But the NMI watchdog won't expire after one second - normally it's set to fixe seconds.

Quoted 16 lines

How about..

drivers/serial/8250.c renamed from drivers/serial/8250.c~ +1 -0
@@ -40,6 +40,7 @@
 #include <linux/serial_core.h>
 #include <linux/serial.h>
 #include <linux/serial_8250.h>
+#include <linux/nmi.h>
 
 #include <asm/io.h>
 #include <asm/irq.h>
@@ -2099,8 +2100,15 @@ static inline void wait_for_xmitr(struct
 	if (up->port.flags & UPF_CONS_FLOW) {
 		tmout = 1000000;
 		while (--tmout &&

drivers/serial/8250.c static inline void wait_for_xmitr(struct · line 2111

Quoted 10 lines
drivers/serial/8250.c renamed from drivers/serial/8250.c~ +8 -1
-		       ((serial_in(up, UART_MSR) & UART_MSR_CTS) == 0))
+		       ((serial_in(up, UART_MSR) & UART_MSR_CTS) == 0)) {
+			int cnt=0;
 			udelay(1);
+			cnt++;
+			if (cnt==100) {
+				touch_nmi_watchdog();
+				cnt=0;
+			}
+		}

<obwhitespacewhine> spose so.

drivers/serial/8250.c renamed from drivers/serial/8250.c~tickle-nmi-watchdog-whilst-doing-serial-writes +5 -2
@@ -40,6 +40,7 @@
 #include <linux/serial_core.h>
 #include <linux/serial.h>
 #include <linux/serial_8250.h>
+#include <linux/nmi.h>
 
 #include <asm/io.h>
 #include <asm/irq.h>
@@ -2098,9 +2099,11 @@ static inline void wait_for_xmitr(struct
 	/* Wait up to 1s for flow control if necessary */
 	if (up->port.flags & UPF_CONS_FLOW) {
 		tmout = 1000000;
-		while (--tmout &&
-		       ((serial_in(up, UART_MSR) & UART_MSR_CTS) == 0))
+		while (!(serial_in(up, UART_MSR) & UART_MSR_CTS) && --tmout) {
 			udelay(1);
+			if ((tmout % 1000) == 0)
+				touch_nmi_watchdog();
+		}
 	}
 }
 

_

-

#8 lore raw
Quoted 5 lines

Hi,

Suspect this can be triggered using alt+sysreq+T on a busy system with a slow serial console. Might be a easy way to see if this patch fixes the issue?

Sysrq-t already tickles the watchdog between the printing of each task so you'll have to remove the nmi tickling in show_state() if you want to use it as a test case.

But uhm, it should take at least 5 seconds of no-interrupts before the NMI watchdog decides the box is dead so this is kind of weird.

Quoted 26 lines
Quoted 26 lines

<obwhitespacewhine> spose so.

drivers/serial/8250.c renamed from drivers/serial/8250.c~tickle-nmi-watchdog-whilst-doing-serial-writes +5 -2
@@ -40,6 +40,7 @@
 #include <linux/serial_core.h>
 #include <linux/serial.h>
 #include <linux/serial_8250.h>
+#include <linux/nmi.h>
 
 #include <asm/io.h>
 #include <asm/irq.h>
@@ -2098,9 +2099,11 @@ static inline void wait_for_xmitr(struct
 	/* Wait up to 1s for flow control if necessary */
 	if (up->port.flags & UPF_CONS_FLOW) {
 		tmout = 1000000;
-		while (--tmout &&
-		       ((serial_in(up, UART_MSR) & UART_MSR_CTS) == 0))
+		while (!(serial_in(up, UART_MSR) & UART_MSR_CTS) && --tmout) {
 			udelay(1);
+			if ((tmout % 1000) == 0)
+				touch_nmi_watchdog();
+		}
 	}
 }
 
#10 lore raw
In reply to Ed Tomlinson

Hi,

The message from a+s+T on a serial console (x86_64) is:

BUG: soft lockup detected on cpu#0

Its not changed by this patch. Probably would be a good idea to correct this?

Ed

Quoted 99 lines · Ed Tomlinson

Suspect this can be triggered using alt+sysreq+T on a busy system with a slow serial console. Might be a easy way to see if this patch fixes the issue?

Ed Tomlinson

Quoted 92 lines · Andrew Morton
Quoted 27 lines · Dave Jones
Quoted 8 lines · Andrew Morton
Quoted 7 lines · Dave Jones

This was fun. I inserted a music CD with some obnoxious copy-protection
on it into the drive, and lots of SCSI errors went zipping over to the serial console. Unfortunatly, the box was also compiling a kernel, playing oggs, and doing a number of other things at the same time, so this happened..

Quoted 6 lines
Quoted 1 line

NMI Watchdog detected LOCKUP on CPU2CPU 2

OK.. But calling touch_nmi_watchdog() at 1MHz seems a bit excessive, and might perturb the finely-tuned timing in there.

How's about this?

Umm.. Despite it being past my bedtime, I'm pretty sure I'm missing something here...

Review

Quoted 2 lines
(quoted diff) +1 -0
+		while (!(serial_in(up, UART_MSR) & UART_MSR_CTS) && --tmout)
 			udelay(1);

I don't see how this is any better than the current code. We're doing 1000000 udelays. Whilst we're doing that, the nmi watchdog goes bonkers.

Quoted 4 lines

Review

Quoted 2 lines
(quoted diff) +2 -0
+		if (tmout < 1000000)
+			touch_nmi_watchdog();

So by the time we do this, its already triggered.

But the NMI watchdog won't expire after one second - normally it's set to fixe seconds.

Quoted 16 lines

How about..

drivers/serial/8250.c renamed from drivers/serial/8250.c~ +1 -0
@@ -40,6 +40,7 @@
 #include <linux/serial_core.h>
 #include <linux/serial.h>
 #include <linux/serial_8250.h>
+#include <linux/nmi.h>
 
 #include <asm/io.h>
 #include <asm/irq.h>
@@ -2099,8 +2100,15 @@ static inline void wait_for_xmitr(struct
 	if (up->port.flags & UPF_CONS_FLOW) {
 		tmout = 1000000;
 		while (--tmout &&

drivers/serial/8250.c static inline void wait_for_xmitr(struct · line 2111

Quoted 10 lines
drivers/serial/8250.c renamed from drivers/serial/8250.c~ +8 -1
-		       ((serial_in(up, UART_MSR) & UART_MSR_CTS) == 0))
+		       ((serial_in(up, UART_MSR) & UART_MSR_CTS) == 0)) {
+			int cnt=0;
 			udelay(1);
+			cnt++;
+			if (cnt==100) {
+				touch_nmi_watchdog();
+				cnt=0;
+			}
+		}

<obwhitespacewhine> spose so.

drivers/serial/8250.c renamed from drivers/serial/8250.c~tickle-nmi-watchdog-whilst-doing-serial-writes +5 -2
@@ -40,6 +40,7 @@
 #include <linux/serial_core.h>
 #include <linux/serial.h>
 #include <linux/serial_8250.h>
+#include <linux/nmi.h>
 
 #include <asm/io.h>
 #include <asm/irq.h>
@@ -2098,9 +2099,11 @@ static inline void wait_for_xmitr(struct
 	/* Wait up to 1s for flow control if necessary */
 	if (up->port.flags & UPF_CONS_FLOW) {
 		tmout = 1000000;
-		while (--tmout &&
-		       ((serial_in(up, UART_MSR) & UART_MSR_CTS) == 0))
+		while (!(serial_in(up, UART_MSR) & UART_MSR_CTS) && --tmout) {
 			udelay(1);
+			if ((tmout % 1000) == 0)
+				touch_nmi_watchdog();
+		}
 	}
 }
 

_

-

#11 lore raw
In reply to Dave Jones

drivers/serial/8250.c static inline void wait_for_xmitr(struct · line 2105

Quoted 11 lines · Dave Jones

#include <asm/io.h>
#include <asm/irq.h>

drivers/serial/8250.c renamed from drivers/serial/8250.c~ +2 -1
@@ -2099,8 +2100,10 @@ static inline void wait_for_xmitr(struct
 	if (up->port.flags & UPF_CONS_FLOW) {
 		tmout = 1000000;
 		while (--tmout &&
-		       ((serial_in(up, UART_MSR) & UART_MSR_CTS) == 0))
+		       ((serial_in(up, UART_MSR) & UART_MSR_CTS) == 0)) {
 			udelay(1);
+			touch_nmi_watchdog();

Note that touch_nmi_watchdog is not exported on i386 - Linus vetoed that some time ago. The real fix of course is to use schedule_timeout(), but that might break printk() with interrupts off :/

-Andi

#9 lore raw
In reply to Alexander Nyberg
Quoted 3 lines · Alexander Nyberg

But uhm, it should take at least 5 seconds of no-interrupts before the NMI watchdog decides the box is dead so this is kind of weird.

There is a bug somewhere that makes it sometimes expire faster. I found only one bug so far - touch_nmi_watchdog has a race (fixed now in -mm*). You can try if it still happens with the next -mm.

Somehow i doubt it is the race though because it should be too unlikely to trigger this. Could be some other bug too.

-Andi

#12 lore raw
In reply to Andi Kleen
Quoted 16 lines · Andi Kleen

drivers/serial/8250.c static inline void wait_for_xmitr(struct · line 2105

Quoted 11 lines · Dave Jones

#include <asm/io.h>
#include <asm/irq.h>

drivers/serial/8250.c renamed from drivers/serial/8250.c~ +2 -1
@@ -2099,8 +2100,10 @@ static inline void wait_for_xmitr(struct
 	if (up->port.flags & UPF_CONS_FLOW) {
 		tmout = 1000000;
 		while (--tmout &&
-		       ((serial_in(up, UART_MSR) & UART_MSR_CTS) == 0))
+		       ((serial_in(up, UART_MSR) & UART_MSR_CTS) == 0)) {
 			udelay(1);
+			touch_nmi_watchdog();

Note that touch_nmi_watchdog is not exported on i386 - Linus vetoed that some time ago. The real fix of course is to use schedule_timeout(), but that might break printk() with interrupts off :/

Not to mention printk() from atomic contexts and panic(). No, schedule_timeout() is _not_ a "real fix" but a kludge.

Signature · 3 lines
Russell King
 Linux kernel    2.6 ARM Linux   - http://www.arm.linux.org.uk/
 maintainer of:  2.6 Serial core
#13 lore raw
Quoted 18 lines · Russell King
Quoted 16 lines · Andi Kleen

drivers/serial/8250.c static inline void wait_for_xmitr(struct · line 2105

Quoted 11 lines · Dave Jones

#include <asm/io.h>
#include <asm/irq.h>

drivers/serial/8250.c renamed from drivers/serial/8250.c~ +2 -1
@@ -2099,8 +2100,10 @@ static inline void wait_for_xmitr(struct
 	if (up->port.flags & UPF_CONS_FLOW) {
 		tmout = 1000000;
 		while (--tmout &&
-		       ((serial_in(up, UART_MSR) & UART_MSR_CTS) == 0))
+		       ((serial_in(up, UART_MSR) & UART_MSR_CTS) == 0)) {
 			udelay(1);
+			touch_nmi_watchdog();

Note that touch_nmi_watchdog is not exported on i386 - Linus vetoed that some time ago. The real fix of course is to use schedule_timeout(), but that might break printk() with interrupts off :/

Quoted 2 lines

Not to mention printk() from atomic contexts and panic(). No, schedule_timeout() is _not_ a "real fix" but a kludge.

Then someone needs to convince Linus to export touch_nmi_watchdog again.

Or how about checking if interrupts are off here (iirc we have a generic function for that now) and then using
a smaller timeout and otherwise schedule_timeout() ?

-Andi

#14 lore raw
Quoted 6 lines · Andi Kleen

Then someone needs to convince Linus to export touch_nmi_watchdog again.

Or how about checking if interrupts are off here (iirc we have a generic function for that now) and then using
a smaller timeout and otherwise schedule_timeout() ?

The interrupt state doesn't tell us whether we can schedule. It tells us when we can't schedule, which is different from when we can. For example:

	spin_lock(foo_lock);
	...
	printk("blah blah blah\n");
	...
	spin_unlock(foo_lock);

This context is non-preemptable, but doesn't have IRQs disabled. The solution would be to keep a "spinlock depth" counter, but obviously that's not a possibility.

I would agree that the most correct thing to do would be to export touch_nmi_watchdog()... if only Linus would accept the arguments _for_ exporting it.

Signature · 3 lines
Russell King
 Linux kernel    2.6 ARM Linux   - http://www.arm.linux.org.uk/
 maintainer of:  2.6 Serial core
#15 lore raw
Quoted 11 lines · Russell King
Quoted 6 lines · Andi Kleen

Then someone needs to convince Linus to export touch_nmi_watchdog again.

Or how about checking if interrupts are off here (iirc we have a generic function for that now) and then using
a smaller timeout and otherwise schedule_timeout() ?

The interrupt state doesn't tell us whether we can schedule. It tells us when we can't schedule, which is different from when we can. For example:

Yes, you're right of course. Must have not been thinking clearly.

-Andi

#16 lore raw
In reply to Andi Kleen
Quoted 23 lines · Andi Kleen
Quoted 20 lines · Russell King
Quoted 17 lines · Andi Kleen

drivers/serial/8250.c static inline void wait_for_xmitr(struct · line 2105

Quoted 11 lines · Dave Jones

#include <asm/io.h>
#include <asm/irq.h>

drivers/serial/8250.c renamed from drivers/serial/8250.c~ +2 -1
@@ -2099,8 +2100,10 @@ static inline void wait_for_xmitr(struct
 	if (up->port.flags & UPF_CONS_FLOW) {
 		tmout = 1000000;
 		while (--tmout &&
-		       ((serial_in(up, UART_MSR) & UART_MSR_CTS) == 0))
+		       ((serial_in(up, UART_MSR) & UART_MSR_CTS) == 0)) {
 			udelay(1);
+			touch_nmi_watchdog();

Note that touch_nmi_watchdog is not exported on i386 - Linus vetoed that some time ago. The real fix of course is to use schedule_timeout(), but that might break printk() with interrupts off :/

Quoted 2 lines
Quoted 2 lines

Not to mention printk() from atomic contexts and panic(). No, schedule_timeout() is _not_ a "real fix" but a kludge.

Um... I would think the real fix is to set the UART up to generate the modem status interrupt and eliminate the pole loop. Why can't this be done? I, for one, don't want my cpu looping in the serial driver, even more so with the interrupt system off. This, in my mind, is a real bug in the serial driver and should be so handled.

Signature · 2 lines
George Anzinger   [email protected]
High-res-timers:  http://sourceforge.net/projects/high-res-timers/
#17 lore raw
Quoted 5 lines · George Anzinger

Um... I would think the real fix is to set the UART up to generate the modem status interrupt and eliminate the pole loop. Why can't this be done? I, for one, don't want my cpu looping in the serial driver, even more so with the interrupt system off. This, in my mind, is a real bug in the serial driver and should be so handled.

Because printk is *synchronous*. It never returns until it's written the entire message. There is no buffering.

Extra complexity, adding reliance on interrupts, etc all means that you reduce the probability that you'll get the panic or oops message out of the system.

Signature · 3 lines
Russell King
 Linux kernel    2.6 ARM Linux   - http://www.arm.linux.org.uk/
 maintainer of:  2.6 Serial core
#18 lore raw
Quoted 5 lines · Russell King
Quoted 1 line · George Anzinger

Um... I would think the real fix is to set the UART up to generate the modem status interrupt and eliminate the pole loop. Why can't this be done? I, for one, don't want my cpu looping in the serial driver, even more so with the interrupt system off. This, in my mind, is a real bug in the serial driver and should be so handled.

Quoted 7 lines

Because printk is *synchronous*. It never returns until it's written the entire message. There is no buffering.

Extra complexity, adding reliance on interrupts, etc all means that you reduce the probability that you'll get the panic or oops message out of the system.

This is a fine reason to loop in the serial code, I guess, but what's the reason for allowing the NMI oops? Having use of the serial console to catch an oops actually CAUSE an oops doesn't seem desirable, and is probably more likely than a hardlock in the serial driver.

Signature · 3 lines
   -bill davidsen ([email protected])
"The secret to procrastination is to put things off until the
 last possible moment - but no longer"  -me