Skip to contentSkip to conversation

PATCH: tty locking fixes take 1

PATCH Alan Cox 1 participant 1 message Sep 9, 2004

Subsystems (4)
lore
Share & tools
lore

Alan Cox

author
#0 lore raw

Much to my suprise with the bug DaveM fixed and a couple of others squashed this seems to work. It's the first step to fixing the ldisc mess. It should give us the guarantees that

ldisc functions don't get called after ldisc->close() ldisc functions don't get called before ->open() completes multiple line discipline changes don't occur at once line disciplines don't change mid I/O

This is still in part a prototype - the ldisc set can return EBUSY if you are randomly busy receiving bytes on another processor at the moment. The _wait ldisc deref code doesn't do proper sleeping and has debug traps. The TIOCSTI code needs some thought to deal with issues like TTY_DONT_FLIP. The termios locking is still totally missing.

It does however rationalise the setup/use/tear-down sequencing. I've only fixed the serial_core driver for driver side locking but that should be a good enough reference to fix the others.

Alan

Documentation/tty.txt +187 -0
diff -u --new-file --recursive --exclude-from /usr/src/exclude linux.vanilla-2.6.9rc1/Documentation/tty.txt linux-2.6.9rc1/Documentation/tty.txt
@@ -0,0 +1,187 @@
+
+			The Lockronomicon
+
+Your guide to the ancient and twisted locking policies of the tty layer and
+the warped logic behind them. Beware all ye who read on.
+
+
+
+Line Discipline
+---------------
+
+Line disciplines are registered with tty_register_ldisc() passing the
+discipline number and the ldisc structure. At the point of registration the 
+discipline must be ready to use and it is possible it will get used before
+the call returns success. If the call returns an error then it won't get
+called. Do not re-use ldisc numbers as they are part of the userspace ABI
+and writing over an existing ldisc will cause demons to eat your computer.
+After the return the ldisc data has been copied so you may free your own 
+copy of the structure. You must not re-register over the top of the line
+discipline even with the same data or your computer again will be eaten by
+demons.
+
+In order to remove a line discipline call tty_register_ldisc passing NULL.
+In ancient times this always worked. In modern times the function will
+return -EBUSY if the ldisc is currently in use. Since the ldisc referencing
+code manages the module counts this should not usually be a concern.
+
+Heed this warning: the reference count field of the registered copies of the
+tty_ldisc structure in the ldisc table counts the number of lines using this
+discipline. The reference count of the tty_ldisc structure within a tty 
+counts the number of active users of the ldisc at this instant. In effect it
+counts the number of threads of execution within an ldisc method (plus those
+about to enter and exit although this detail matters not).
+
+Line Discipline Methods
+-----------------------
+
+TTY side interfaces:
+
+close()		-	This is called on a terminal when the line
+			discipline is being unplugged. At the point of
+			execution no further users will enter the
+			ldisc code for this tty. Hopefully soon will always
+			be safe to sleep.
+
+open()		-	Called when the line discipline is attached to
+			the terminal. No other call into the line
+			discipline for this tty will occur until it
+			completes successfully. Hopefully soon will always
+			be safe to sleep.
+
+write()		-	A process is writing data from user space
+			through the line discipline. Multiple write calls
+			are serialized by the tty layer for the ldisc.
+
+flush_buffer()	-	May be called at any point between open and close.
+
+chars_in_buffer() -	Report the number of bytes in the buffer. 
+
+set_termios()	-	Called on termios structure changes. Semantics 
+			deeply mysterious right now.
+
+read()		-	Move data from the line discipline to the user.
+			Multiple read calls may occur in parallel and the
+			ldisc must deal with serialization issues. May 
+			sleep.
+
+poll()		-	Check the status for the poll/select calls. Multiple
+			poll calls may occur in parallel. May sleep.
+
+ioctl()		-	Called when an ioctl is handed to the tty layer
+			that might be for the ldisc. Multiple ioctl calls
+			may occur in parallel. May sleep. 
+
+Driver Side Interfaces:
+
+receive_buf()	-	Hand buffers of bytes from the driver to the ldisc
+			for processing. Semantics currently rather
+			mysterious 8(
+
+receive_room()	-	Can be called by the driver layer at any time when
+			the ldisc is opened. The ldisc must be able to
+			handle the reported amount of data at that instant.
+			Synchronization between active receive_buf and
+			receive_room calls is down to the driver not the
+			ldisc.
+
+write_wakeup()	-	May be called at any point between open and close.
+			The TTY_DO_WRITE_WAKEUP flag indicates if a call
+			is needed but always races versus calls. Thus the
+			ldisc must be careful about setting order and to
+			handle unexpected calls.
+
+
+Locking
+
+Callers to the line discipline functions from the tty layer are required to
+take line discipline locks. The same is true of calls from the driver side
+but not yet enforced.
+
+Three calls are now provided
+
+	ldisc = tty_ldisc_ref(tty);
+
+takes a handle to the line discipline in the tty and returns it. If no ldisc
+is currently attached or the ldisc is being closed and re-opened at this
+point then NULL is returned. While this handle is held the ldisc will not
+change or go away.
+
+	tty_ldisc_deref(ldisc)
+
+Returns the ldisc reference and allows the ldisc to be closed. Returning the
+reference takes away your right to call the ldisc functions until you take
+a new reference.
+
+	ldisc = tty_ldisc_ref_wait(tty);
+
+Performs the same function as tty_ldisc_ref except that it will wait for an
+ldisc change to complete and then return a reference to the new ldisc. 
+
+While these functions are slightly slower than the old code they should have
+minimal impact as most receive logic uses the flip buffers and they only
+need to take a reference when they push bits up through the driver.
+
+
+
+Driver Interface
+----------------
+
+open()		-	Called when a device is opened. May sleep
+
+close()		-	Called when a device is closed. At the point of
+			return from this call the driver must make no 
+			further ldisc calls of any kind. May sleep
+
+write()		-	Called to write bytes to the device. May not
+			sleep. May occur in parallel in special cases. 
+			Because this includes panic paths drivers generally
+			shouldn't try and do clever locking here.
+
+put_char()	-	Stuff a single character onto the queue. The
+			driver is guaranteed following up calls to
+			flush_chars.
+
+flush_chars()	-	Ask the kernel to write put_char queue
+
+write_room()	-	Return the number of characters tht can be stuffed
+			into the port buffers without overflow (or less).
+			The ldisc is responsible for being intelligent
+			about multi-threading of write_room/write calls
+
+ioctl()		-	Called when an ioctl may be for the driver
+
+set_termios()	-	Called on termios change, may get parallel calls,
+			may block for now (may change that)
+
+set_ldisc()	-	Notifier for discipline change. At the point this 
+			is done the discipline is not yet usable. Can now
+			sleep (I think)
+
+throttle()	-	Called by the ldisc to ask the driver to do flow
+			control.  Serialization including with unthrottle
+			is the job of the ldisc layer.
+
+unthrottle()	-	Called by the ldisc to ask the driver to stop flow
+			control.
+
+stop()		-	Ldisc notifier to the driver to stop output. As with
+			throttle the serializations with start() are down
+			to the ldisc layer.
+
+start()		-	Ldisc notifier to the driver to start output.
+
+hangup()	-	Ask the tty driver to cause a hangup initiated
+			from the host side. [Can sleep ??]
+
+break_ctl()	-	Send RS232 break. Can sleep. Can get called in
+			parallel, driver must serialize (for now), and
+			with write calls.
+
+wait_until_sent() -	Wait for characters to exit the hardware queue
+			of the driver. Can sleep
+
+send_xchar()	  -	Send XON/XOFF and if possible jump the queue with
+			it in order to get fast flow control responses.
+			Cannot sleep ??
+
drivers/char/tty_io.c +404 -80

793 lines not on this page · raw message

drivers/char/tty_ioctl.c +21 -8
diff -u --new-file --recursive --exclude-from /usr/src/exclude linux.vanilla-2.6.9rc1/drivers/char/tty_ioctl.c linux-2.6.9rc1/drivers/char/tty_ioctl.c
@@ -98,6 +98,7 @@
 {
 	int canon_change;
 	struct termios old_termios = *tty->termios;
+	struct tty_ldisc *ld;
 
 	local_irq_disable(); // FIXME: is this safe?
 	*tty->termios = *new_termios;
@@ -136,13 +137,16 @@
 	if (tty->driver->set_termios)
 		(*tty->driver->set_termios)(tty, &old_termios);
 
-	if (tty->ldisc.set_termios)
-		(*tty->ldisc.set_termios)(tty, &old_termios);
+	ld = tty_ldisc_ref(tty);
+	if (ld->set_termios)
+		(ld->set_termios)(tty, &old_termios);
+	tty_ldisc_deref(ld);
 }
 
 static int set_termios(struct tty_struct * tty, void __user *arg, int opt)
 {
 	struct termios tmp_termios;
+	struct tty_ldisc *ld;
 	int retval = tty_check_change(tty);
 
 	if (retval)
@@ -159,9 +163,13 @@
 			return -EFAULT;
 	}
 
-	if ((opt & TERMIOS_FLUSH) && tty->ldisc.flush_buffer)
-		tty->ldisc.flush_buffer(tty);
+	ld = tty_ldisc_ref(tty);
+	
+	if ((opt & TERMIOS_FLUSH) && ld->flush_buffer)
+		ld->flush_buffer(tty);
 
+	tty_ldisc_deref(ld);
+	
 	if (opt & TERMIOS_WAIT) {
 		tty_wait_until_sent(tty, 0);
 		if (signal_pending(current))
@@ -365,6 +373,7 @@
 	struct tty_struct * real_tty;
 	void __user *p = (void __user *)arg;
 	int retval;
+	struct tty_ldisc *ld;
 
 	if (tty->driver->type == TTY_DRIVER_TYPE_PTY &&
 	    tty->driver->subtype == PTY_TYPE_MASTER)
@@ -443,22 +452,26 @@
 			retval = tty_check_change(tty);
 			if (retval)
 				return retval;
+				
+			ld = tty_ldisc_ref(tty);
 			switch (arg) {
 			case TCIFLUSH:
-				if (tty->ldisc.flush_buffer)
-					tty->ldisc.flush_buffer(tty);
+				if (ld->flush_buffer)
+					ld->flush_buffer(tty);
 				break;
 			case TCIOFLUSH:
-				if (tty->ldisc.flush_buffer)
-					tty->ldisc.flush_buffer(tty);
+				if (ld->flush_buffer)
+					ld->flush_buffer(tty);
 				/* fall through */
 			case TCOFLUSH:
 				if (tty->driver->flush_buffer)
 					tty->driver->flush_buffer(tty);
 				break;
 			default:
+				tty_ldisc_deref(ld);
 				return -EINVAL;
 			}
+			tty_ldisc_deref(ld);
 			return 0;
 		case TIOCOUTQ:
 			return put_user(tty->driver->chars_in_buffer ?
fs/proc/proc_tty.c +6 -6
diff -u --new-file --recursive --exclude-from /usr/src/exclude linux.vanilla-2.6.9rc1/fs/proc/proc_tty.c linux-2.6.9rc1/fs/proc/proc_tty.c
@@ -15,9 +15,6 @@
 #include <linux/seq_file.h>
 #include <asm/bitops.h>
 
-extern struct tty_ldisc ldiscs[];
-
-
 static int tty_ldiscs_read_proc(char *page, char **start, off_t off,
 				int count, int *eof, void *data);
 
@@ -159,12 +156,15 @@
 	int	i;
 	int	len = 0;
 	off_t	begin = 0;
-
+	struct tty_ldisc *ld;
+	
 	for (i=0; i < NR_LDISCS; i++) {
-		if (!(ldiscs[i].flags & LDISC_FLAG_DEFINED))
+		ld = tty_ldisc_get(i);
+		if (ld == NULL)
 			continue;
 		len += sprintf(page+len, "%-10s %2d\n",
-			       ldiscs[i].name ? ldiscs[i].name : "???", i);
+			       ld->name ? ld->name : "???", i);
+		tty_ldisc_put(i);
 		if (len+begin > off+count)
 			break;
 		if (len+begin < off) {
include/linux/tty.h +21 -13
diff -u --new-file --recursive --exclude-from /usr/src/exclude linux.vanilla-2.6.9rc1/include/linux/tty.h linux-2.6.9rc1/include/linux/tty.h
@@ -306,19 +306,20 @@
  * tty->write.  Thus, you must use the inline functions set_bit() and
  * clear_bit() to make things atomic.
  */
-#define TTY_THROTTLED 0
-#define TTY_IO_ERROR 1
-#define TTY_OTHER_CLOSED 2
-#define TTY_EXCLUSIVE 3
-#define TTY_DEBUG 4
-#define TTY_DO_WRITE_WAKEUP 5
-#define TTY_PUSH 6
-#define TTY_CLOSING 7
-#define TTY_DONT_FLIP 8
-#define TTY_HW_COOK_OUT 14
-#define TTY_HW_COOK_IN 15
-#define TTY_PTY_LOCK 16
-#define TTY_NO_WRITE_SPLIT 17
+#define TTY_THROTTLED 		0	/* Call unthrottle() at threshold min */
+#define TTY_IO_ERROR 		1	/* Canse an I/O error (may be no ldisc too) */
+#define TTY_OTHER_CLOSED 	2	/* Other side (if any) has closed */
+#define TTY_EXCLUSIVE 		3	/* Exclusive open mode */
+#define TTY_DEBUG 		4	/* Debugging */
+#define TTY_DO_WRITE_WAKEUP 	5	/* Call write_wakeup after queuing new */
+#define TTY_PUSH 		6	/* n_tty private */
+#define TTY_CLOSING 		7	/* ->close() in progress */
+#define TTY_DONT_FLIP 		8	/* Defer buffer flip */
+#define TTY_LDISC 		9	/* Line discipline attached */
+#define TTY_HW_COOK_OUT 	14	/* Hardware can do output cooking */
+#define TTY_HW_COOK_IN 		15	/* Hardware can do input cooking */
+#define TTY_PTY_LOCK 		16	/* pty private */
+#define TTY_NO_WRITE_SPLIT 	17	/* Preserve write boundaries to driver */
 
 #define TTY_WRITE_FLUSH(tty) tty_write_flush((tty))
 
@@ -362,6 +363,13 @@
 extern int tty_get_baud_rate(struct tty_struct *tty);
 extern int tty_termios_baud_rate(struct termios *termios);
 
+extern struct tty_ldisc *tty_ldisc_ref(struct tty_struct *);
+extern void tty_ldisc_deref(struct tty_ldisc *);
+extern struct tty_ldisc *tty_ldisc_ref_wait(struct tty_struct *);
+
+extern struct tty_ldisc *tty_ldisc_get(int);
+extern void tty_ldisc_put(int);
+
 struct semaphore;
 extern struct semaphore tty_sem;
 
include/linux/tty_ldisc.h +2 -0
diff -u --new-file --recursive --exclude-from /usr/src/exclude linux.vanilla-2.6.9rc1/include/linux/tty_ldisc.h linux-2.6.9rc1/include/linux/tty_ldisc.h
@@ -132,6 +132,8 @@
 	void	(*write_wakeup)(struct tty_struct *);
 
 	struct  module *owner;
+	
+	int refcount;
 };
 
 #define TTY_LDISC_MAGIC	0x5403