Fix IXANY and restart after signal (e.g. ctrl-C) in n_tty line discipline

Here is a patch that fixes a couple of issues with ttys in the stopped state. See patch below for more details.

Note that this patch assumes that my previous patch from last month (called "tty-enable-the-echoing-of-c-in-the-n_tty-discipline.patch" in the -mm tree) has been applied, since part of this patch is adjacent to that code.

Happy holidays! Joe

Fix two N_TTY line discipline issues related to resuming a stopped TTY (typically done with ctrl-S):

1) Fix handling of character that resumes a stopped TTY (with IXANY)

With "stty ixany", the TTY line discipline would lose the first character after the stop, so typing, for example, "hi^Sthere" resulted in "hihere" (the 't' would cause the resume after ^S, but it would then be thrown away rather than processed as an input character). This was inconsistent with the behavior of other Unix systems.

2) Fix interrupt signal (e.g. ctrl-C) behavior in stopped TTYs

With "stty -ixany" (often the default), interrupt signals were ignored in a stopped TTY until the TTY was resumed with the start char (typically ctrl-Q), which was inconsistent with the behavior of other Unix systems.

---

diff -puN a/drivers/char/n_tty.c~tty-fix-ixany-and-restart-after-signal-in-the-n_tty-discipline a/drivers/char/n_tty.c
@@ -695,11 +695,8 @@ static inline void n_tty_receive_char(st
 		return;
 	}
 	
-	if (tty->stopped && !tty->flow_stopped &&
-	    I_IXON(tty) && I_IXANY(tty)) {
+	if (tty->stopped && !tty->flow_stopped && I_IXON(tty) && I_IXANY(tty))
 		start_tty(tty);
-		return;
-	}
 	
 	if (I_ISTRIP(tty))
 		c &= 0x7f;

@@ -769,6 +766,8 @@ static inline void n_tty_receive_char(st
 		signal = SIGTSTP;
 		if (c == SUSP_CHAR(tty)) {
 send_signal:
+			if (tty->stopped && !tty->flow_stopped && I_IXON(tty))
+				start_tty(tty);
 			/*
 			 * Echo character, and then send the signal.
 			 * Note that we do not use isig() here because we want

Here is a revised version of my patch, currently in the -mm kernel as "fix-ixany-and-restart-after-signal-eg-ctrl-c-in-n_tty-line-discipline". It fixes a couple of issues with ttys in the stopped state. See patch below for more details.

This patch should *replace* the old version of the patch. It is cleaner, and its behavior better matches that of other Unix platforms. It also avoids needless/redundant calls to start_tty() when ctrl-S or ctrl-Q are hit.

-Joe

Fix two N_TTY line discipline issues related to resuming a stopped TTY (typically done with ctrl-S):

1) Fix handling of character that resumes a stopped TTY (with IXANY)

With "stty ixany", the TTY line discipline would lose the first character after the stop, so typing, for example, "hi^Sthere" resulted in "hihere" (the 't' would cause the resume after ^S, but it would then be thrown away rather than processed as an input character). This was inconsistent with the behavior of other Unix systems.

2) Fix interrupt signal (e.g. ctrl-C) behavior in stopped TTYs

With "stty -ixany" (often the default), interrupt signals were ignored in a stopped TTY until the TTY was resumed with the start char (typically ctrl-Q), which was inconsistent with the behavior of other Unix systems.

---

diff -puN drivers/char/n_tty.c~fix-ixany-and-restart-after-signal-eg-ctrl-c-in-n_tty-line-discipline drivers/char/n_tty.c
@@ -695,17 +695,16 @@
 		return;
 	}
 	
-	if (tty->stopped && !tty->flow_stopped &&
-	    I_IXON(tty) && I_IXANY(tty)) {
-		start_tty(tty);
-		return;
-	}
-	
 	if (I_ISTRIP(tty))
 		c &= 0x7f;
 	if (I_IUCLC(tty) && L_IEXTEN(tty))
 		c=tolower(c);
 
+	if (tty->stopped && !tty->flow_stopped && I_IXON(tty) &&
+	    ((I_IXANY(tty) && c != START_CHAR(tty) && c != STOP_CHAR(tty)) ||
+	     c == INTR_CHAR(tty) || c == QUIT_CHAR(tty)))
+		start_tty(tty);
+
 	if (tty->closing) {
 		if (I_IXON(tty)) {
 			if (c == START_CHAR(tty))

What were the "couple of issues"?

I don't like replacements a lot. It forces one to re-review the whole thing from scratch. Which is why I almost always turn the replacement into an incremental patch, as below.

@@ -695,14 +695,16 @@ static inline void n_tty_receive_char(st
 		return;
 	}
 	
-	if (tty->stopped && !tty->flow_stopped && I_IXON(tty) && I_IXANY(tty))
-		start_tty(tty);
-	
 	if (I_ISTRIP(tty))
 		c &= 0x7f;
 	if (I_IUCLC(tty) && L_IEXTEN(tty))
 		c=tolower(c);
 
+	if (tty->stopped && !tty->flow_stopped && I_IXON(tty) &&
+	    ((I_IXANY(tty) && c != START_CHAR(tty) && c != STOP_CHAR(tty)) ||
+	     c == INTR_CHAR(tty) || c == QUIT_CHAR(tty)))
+		start_tty(tty);
+
 	if (tty->closing) {
 		if (I_IXON(tty)) {
 			if (c == START_CHAR(tty))

@@ -766,8 +768,6 @@ static inline void n_tty_receive_char(st
 		signal = SIGTSTP;
 		if (c == SUSP_CHAR(tty)) {
 send_signal:
-			if (tty->stopped && !tty->flow_stopped && I_IXON(tty))
-				start_tty(tty);
 			/*
 			 * Echo character, and then send the signal.
 			 * Note that we do not use isig() here because we want

_

They are the same 2 issues described in the original version of the patch and included in the text of this one too (i.e. the text at the start of this latest patch version is the same as before).

I see your point. Cool, thanks for making the update patch, and I'll submit them like that in the future.

-Thanks, Joe