Skip to contentSkip to conversation

2.6.16.18 kernel freezes while pppd is exiting

DISCUSSION Chuck Ebbert 2 participants 3 messages Jun 12, 2006

Subsystems (2)
lore
Share & tools
lore
#0 lore raw

In-Reply-To: <[email protected]>

Quoted 8 lines · Paul Fulghum

Here is a patch to serialize flush_to_ldisc with per device granularity. It should fix the corruption of the free list that is the probably cause of the freeze you are seeing. Test it to see if there are any ill effects (it works fine here on an AMD x2 SMP kernel). I realize that
the problem happens infrequently, so you won't be able to tell quickly if it fixed the freeze.

I already applied this, which should keep it from freezing and will print a warning, so I will apply your patch and then check the log every time pppd exits.

Check for infinite loops in tty_buffer_free_all(). Hangs have only been seen in the second loop, but check both anyway.

drivers/char/tty_io.c +12 -0
@@ -240,11 +240,23 @@ static int check_tty_count(struct tty_st
 static void tty_buffer_free_all(struct tty_struct *tty)
 {
 	struct tty_buffer *thead;
+	int free_loops;
+
+	free_loops = 0;
 	while((thead = tty->buf.head) != NULL) {
+		if (++free_loops > 9999) {
+			WARN_ON(1);
+			break;
+		}
 		tty->buf.head = thead->next;
 		kfree(thead);
 	}
+	free_loops = 0;
 	while((thead = tty->buf.free) != NULL) {
+		if (++free_loops > 9999) {
+			WARN_ON(1);
+			break;
+		}
 		tty->buf.free = thead->next;
 		kfree(thead);
 	}
Signature · 1 line
Chuck
#1 lore raw
Quoted 3 lines · Chuck Ebbert

I already applied this, which should keep it from freezing and will print a warning, so I will apply your patch and then check the log every time pppd exits.

Your debug output seems like a good idea for catching the mechanism of reentrance on flush_to_ldisc (if it is happening at all).

My reading of the workqueue code is that the work function can run in parallel on a multiple CPU system. The default workqueue has multiple, per CPU queues, and the pending bit for the work item is cleared before execution. This would allow the work item to be requeued for another CPU while the function is executing.
(If I'm wrong, I would be happy to be told why)

Anyways, if I can impose on you to use this version with debug output, it would help clarify this.

If the DONT_FLIP debug output happens too often and fills you syslog, comment out that part.

include/linux/tty.h +1 -0
@@ -267,6 +267,7 @@ struct tty_struct {
 #define TTY_PTY_LOCK 		16	/* pty private */
 #define TTY_NO_WRITE_SPLIT 	17	/* Preserve write boundaries to driver */
 #define TTY_HUPPED 		18	/* Post driver->hangup() */
+#define TTY_FLUSHING 		19	/* Flushing tty buffers to line discipline */
 
 #define TTY_WRITE_FLUSH(tty) tty_write_flush((tty))
 
drivers/char/tty_io.c +7 -3
@@ -2786,9 +2786,12 @@ static void flush_to_ldisc(void *private
 		return;
 
 	if (test_bit(TTY_DONT_FLIP, &tty->flags)) {
-		/*
-		 * Do it after the next timer tick:
-		 */
+		printk(KERN_ERR"flush_to_ldisc - TTY_DONT_FLIP set\n");
+		schedule_delayed_work(&tty->buf.work, 1);
+		goto out;
+	}
+	if (test_and_set_bit(TTY_FLUSHING, &tty->flags)) {
+		printk(KERN_ERR"flush_to_ldisc - TTY_FLUSHING set, low_latency=%d\n", tty->low_latency);
 		schedule_delayed_work(&tty->buf.work, 1);
 		goto out;
 	}
@@ -2810,6 +2813,7 @@ static void flush_to_ldisc(void *private
 		tty_buffer_free(tty, tbuf);
 	}
 	spin_unlock_irqrestore(&tty->buf.lock, flags);
+	clear_bit(TTY_FLUSHING, &tty->flags);
 out:
 	tty_ldisc_deref(disc);
 }
#2 lore raw
In reply to Chuck Ebbert

A little more clarification on the TTY_DONT_FLIP debug output... I don't expect it to trigger at all as the PPP line discipline does not use it (only n_tty).

The only chance it could trigger is after switching back to n_tty on termination of the PPP link.

Thanks,
Paul