Skip to contentSkip to conversation

PROBLEM: serial port driver grabs occupied port

PATCH Christoph Bartelmus 2 participants 2 messages Feb 2, 2002 – Feb 3, 2002 Char and misc drivers

lore
Share & tools
lore
#0 lore raw

Hi,

there's a problem with linux/drivers/char/serial.c that appears when using the LIRC serial port drivers (see http://www.lirc.org). Here's how to reproduce the problem:

1. Load the lirc_serial module (included in the LIRC package). This module uses request_region() for the used serial port's IO space.

2. Load the kernel serial module. In rs_init() the available serial ports are initialized. For the port already claimed by the lirc_serial module check_region() fails and state->type for this port stays PORT_UNKNOWN.

Now the problem:
3. Open the /dev/ttySx for the port that is occupied by the lirc_serial module. rs_open() does not check the state of the serial port and returns no error. Now you can use setserial to hijack the port... lirc_serial stops working.

The patch below fixes the problem for me (tested with 2.2.19 and 2.2.20). There's one problem with this patch though. Once state->type ist PORT_UNKNOWN for a port there's no way to reclaim the port with e.g. setserial unless you remove the serial port module and load it again.

Christoph

PS: I'm not subscribed, cc me if replying.

--- Schnipp ---

serial.c renamed from drivers/char/serial.c +17 -2
@@ -928,6 +928,11 @@
 	unsigned short ICP;
 #endif
 
+	if(state->type == PORT_UNKNOWN)
+	{
+		return -ENODEV;
+	}
+
 	page = get_free_page(GFP_KERNEL);
 	if (!page)
 		return -ENOMEM;
@@ -1738,7 +1743,10 @@
 	info->xmit_fifo_size = state->xmit_fifo_size =
 		new_serial.xmit_fifo_size;
 
-	release_region(state->port,8);
+	if(old_state.type != PORT_UNKNOWN)
+	{
+		release_region(state->port,8);
+	}
 	if (change_port || change_irq) {
 		/*
 		 * We need to shutdown the serial port at the old
@@ -1750,8 +1758,14 @@
 		info->hub6 = state->hub6 = new_serial.hub6;
 	}
 	if (state->type != PORT_UNKNOWN)
+	{
+		retval=check_region(state->port,8);
+		if(retval<0)
+		{
+			return retval;
+		}
 		request_region(state->port,8,"serial(set)");
-
+	}
 	
 check_and_exit:
 	if (!state->port || !state->type)
@@ -3586,3 +3600,4 @@
 	return kmem_start;
 }
 #endif
+

--- Schnipp ---

#1 lore raw
Quoted 5 lines · Christoph Bartelmus

Now the problem:
3. Open the /dev/ttySx for the port that is occupied by the lirc_serial module. rs_open() does not check the state of the serial port and returns no error. Now you can use setserial to hijack the port... lirc_serial stops working.

One of setserial's basic functions is to configure a ttyS device for a serial port that you know is there. In this patch, you are preventing any port that is in 'PORT_UNKNOWN' state from being changed, even if you want to change the IRQ and IO base address. This is a different, but nevertheless fundamental problem.

The real bug is that serial.c should allows you to change the port from 'PORT_UNKNOWN' even when it can't grab the resources for the port. Thanks for finding it - are you able to work on this more to come up with a better fix?

Signature · 2 lines
Russell King ([email protected])                The developer of ARM Linux
             http://www.arm.linux.org.uk/personal/aboutme.html