Skip to contentSkip to conversation

2.6.16.18 kernel freezes while pppd is exiting

DISCUSSION Chuck Ebbert 3 participants 7 messages Jun 8, 2006 – Jun 12, 2006

Subsystems (2)
lore
Share & tools
lore
#0 lore raw

In-Reply-To: <[email protected]>

Quoted 8 lines · Paul Fulghum
Quoted 2 lines

Very infrequently I get kernel freezes while pppd is exiting. Today I finally got traces from the serial console.

Chuck:

What kind of device are you using with pppd?
(so I can identify which driver is feeding
the tty layer receive data)

Just a regular 16550A port at 57600 baud using the 8250 driver.

Quoted 2 lines

Are you setting the low_latency flag on that device? (setserial)

Not that I know of.

Signature · 1 line
Chuck
#1 lore raw
Quoted 5 lines · Chuck Ebbert
Quoted 3 lines

What kind of device are you using with pppd?
(so I can identify which driver is feeding
the tty layer receive data)

Quoted 8 lines

Just a regular 16550A port at 57600 baud using the 8250 driver.

Quoted 2 lines

Are you setting the low_latency flag on that device? (setserial)

Not that I know of.

OK, thanks.

I'm 99.9% sure I've identified the problem correctly, as it would effect the free list in precisely the way necessary to cause an infinite loop when trying to free the list.

My first suggested fix has problems, but there is a straight forward solution I'll try to post today (I've got a backlog of stuff to do here at work before I can get to that.)

Signature · 2 lines
Paul Fulghum
Microgate Systems, Ltd.
#2 lore raw
In reply to Chuck Ebbert

Chuck:

Here is a patch to serialize flush_to_ldisc with per device granularity. It should fix the corruption of the free list that is the probably cause of the freeze you are seeing. Test it to see if there are any ill effects (it works fine here on an AMD x2 SMP kernel). I realize that
the problem happens infrequently, so you won't be able to tell quickly if it fixed the freeze.

This is a necessary change regardless, but I'm confident it is the cause of your problem.

Just keep using it and keep me posted on the results.

Thanks,
Paul

include/linux/tty.h +1 -0
@@ -267,6 +267,7 @@ struct tty_struct {
 #define TTY_PTY_LOCK 		16	/* pty private */
 #define TTY_NO_WRITE_SPLIT 	17	/* Preserve write boundaries to driver */
 #define TTY_HUPPED 		18	/* Post driver->hangup() */
+#define TTY_FLUSHING 		19	/* Flushing tty buffers to line discipline */
 
 #define TTY_WRITE_FLUSH(tty) tty_write_flush((tty))
 
drivers/char/tty_io.c +3 -4
@@ -2785,10 +2785,8 @@ static void flush_to_ldisc(void *private
 	if (disc == NULL)	/*  !TTY_LDISC */
 		return;
 
-	if (test_bit(TTY_DONT_FLIP, &tty->flags)) {
-		/*
-		 * Do it after the next timer tick:
-		 */
+	if (test_bit(TTY_DONT_FLIP, &tty->flags) ||
+	    test_and_set_bit(TTY_FLUSHING, &tty->flags)) {
 		schedule_delayed_work(&tty->buf.work, 1);
 		goto out;
 	}
@@ -2810,6 +2808,7 @@ static void flush_to_ldisc(void *private
 		tty_buffer_free(tty, tbuf);
 	}
 	spin_unlock_irqrestore(&tty->buf.lock, flags);
+	clear_bit(TTY_FLUSHING, &tty->flags);
 out:
 	tty_ldisc_deref(disc);
 }
#3 lore raw

Ar Llu, 2006-06-12 am 10:07 -0500, ysgrifennodd Paul Fulghum:

drivers/char/tty_io.c static void flush_to_ldisc(void *private · line 2811

Quoted 8 lines

Chuck:

Here is a patch to serialize flush_to_ldisc
with per device granularity. It should fix the
corruption of the free list that is the probably

drivers/char/tty_io.c +1 -0
 	spin_unlock_irqrestore(&tty->buf.lock, flags);
+	clear_bit(TTY_FLUSHING, &tty->flags);

Shouldn't those two be reversed if you want to go with this path. How is this occuring anyway the flush_to_ldisc path should never re-enter.

#4 lore raw
Quoted 5 lines · Alan Cox

drivers/char/tty_io.c static void flush_to_ldisc(void *private · line 2811

Quoted 2 lines
	spin_unlock_irqrestore(&tty->buf.lock, flags);
drivers/char/tty_io.c +1 -0
+	clear_bit(TTY_FLUSHING, &tty->flags);

Shouldn't those two be reversed if you want to go with this path.

I don't see that is necessary.

Quoted 1 line

How is this occuring anyway the flush_to_ldisc path should never re-enter.

If a driver has low_latency set, flush_to_ldisc can be called from both scheduled work (due to hitting TTY_DONT_FLIP) and directly from an ISR. On an SMP system, they can run in parallel.

I don't know for sure that is the path being hit, it could be some other odd combination. But there is no inherent serialization
of flush_to_ldisc.

Signature · 2 lines
Paul Fulghum
Microgate Systems, Ltd.
#5 lore raw

Ar Llu, 2006-06-12 am 11:11 -0500, ysgrifennodd Paul Fulghum:

Quoted 4 lines

If a driver has low_latency set, flush_to_ldisc can be called from both scheduled work (due to hitting TTY_DONT_FLIP) and directly from an ISR. On an SMP system, they can run in parallel.

Ok, so yet another reason that the ugly TTY_DONT_FLIP hack should die forever.

Alan

#6 lore raw
Quoted 1 line · Alan Cox

... the ugly TTY_DONT_FLIP hack should die forever.

No tears here.

Signature · 2 lines
Paul Fulghum
Microgate Systems, Ltd.