Skip to contentSkip to conversation

How to avoid serial port buffer overruns?

DISCUSSION Raphael Hertzog 7 participants 30 messages Aug 16, 2006 – Aug 23, 2006

lore
Share & tools
lore
#0 lore raw

(Please CC me when replying)

Hello,

While using Linux on low-end (semi-embedded) hardware (386 SX 40Mhz, 8Mb RAM), I discovered that Linux on that machine would suffer from serial port buffer overruns quite easily if I use a baudrate high enough (I start loosing bytes at >19200 bauds and I would like to make it reliable up to 115 kbauds). I check if overruns are happening with /proc/tty/driver/serial ("oe" field).

Back when I was using the 2.4 kernel, I reduced dramatically the frequency of overruns by using the "low latency" and "preemptible kernel" patch [1]. But it still happened sometimes at 115 kbauds if the system was a bit loaded (with disk I/O for example).

Now I switched to stock 2.6 and while the stock kernel improved in responsiveness, it still isn't enough by default (even with CONFIG_PREEMPT=y and CONFIG_HZ=1000). So I wanted to try the "rt" patch of Ingo Molnar and Thomas Gleixner, but the patched kernel doesn't boot (see bug report in a separate mail on this list).

Other things that I tried which didn't help (enough) are:

- tuning with hdparm
- make disk IRQ interruptible with hdparm -u 1 /dev/hda
- activating DMA is suggested but my disk is a "disk on module" and doesn't support DMA
- using irqtune (http://cae.best.vwh.net/irqtune/) to reprioritize interrupts
irqtune is old and it's very difficult to know if it still works reliably with recent kernel and as there's no way to "read" the interrupt priorities, I have no way to know if irqtune changed anything at all...

My questions are thus:
1/ Is there a way to patch the kernel to make it handle serial IRQ as
the highest priority? If yes, how? Where should I look at to create such a patch?
2/ How can I identify why the serial interrupts are delayed? Or, in other
words, how can I find the code blocking for too long the treatment of the serial IRQ?
I suppose that network IRQ and disk IRQ are responsible for that but I'm not sure and furthermore disk interrupts are supposed to be interruptible given the hdparm config that I use.
3/ What other suggestions do you have to avoid those serial buffer
overruns?

As usual, I'll gladly try out patches/ideas and will provide any required additional information that I didn't include in this mail.

Some infos on the hardware:
http://www.icop.com.tw/products_detail.asp?ProductID=205
More detailed spec of the CPU are here:
http://www.dmp.com.tw/tech/m6117d/

bash-2.05a# cat /proc/interrupts
CPU0

0:   88503366          XT-PIC  timer
2:          0          XT-PIC  cascade
3:         11          XT-PIC  serial
4:         12          XT-PIC  serial
5:       4958          XT-PIC  NE2000
14:      10771          XT-PIC  ide0
NMI:          0
ERR:          0

Regards,

[1] I documented that in a blog post last year:
http://www.ouaza.com/wordpress/2005/10/19/serial-overrun-on-linux/

Signature · 4 lines
Raphaël Hertzog

Premier livre français sur Debian GNU/Linux :
http://www.ouaza.com/livre/admin-debian/
#1 lore raw
Quoted 11 lines · Raphael Hertzog

While using Linux on low-end (semi-embedded) hardware (386 SX 40Mhz, 8Mb RAM), I discovered that Linux on that machine would suffer from serial port buffer overruns quite easily if I use a baudrate high enough (I start loosing bytes at >19200 bauds and I would like to make it reliable up to 115 kbauds). I check if overruns are happening with /proc/tty/driver/serial ("oe" field).

Back when I was using the 2.4 kernel, I reduced dramatically the frequency of overruns by using the "low latency" and "preemptible kernel" patch [1]. But it still happened sometimes at 115 kbauds if the system was a bit loaded (with disk I/O for example).

In my experience, the way to avoid overruns on a serial port was to use a buffered serial port UART (such as a 16550A for example). I remember my 486 wasn't reliable about 19200 or 38400 (depending on how busy the cpu was) when using an 8250. Using a 16550A based card and I could do 115200 without any issues since the UART had a 16 byte buffer to help out the system. Unless your 386 has an add in card for the serial port, it almost certainly has a very crappy UART and it would be very hard to make it work reliably at higher speeds.

--
Len Sorensen

#2 lore raw
Quoted 8 lines · Lennart Sorensen

In my experience, the way to avoid overruns on a serial port was to use a buffered serial port UART (such as a 16550A for example). I remember my 486 wasn't reliable about 19200 or 38400 (depending on how busy the cpu was) when using an 8250. Using a 16550A based card and I could do 115200 without any issues since the UART had a 16 byte buffer to help out the system. Unless your 386 has an add in card for the serial port, it almost certainly has a very crappy UART and it would be very hard to make it work reliably at higher speeds.

I forgot to mention the kind of UART in my mail but it is a 16650A (and configured as such and detected as such) and I have overruns nevertheless.

bash-2.05a# cat /proc/tty/driver/serial
serinfo:1.0 driver revision:

0: uart:16550A port:000003F8 irq:4 tx:31 rx:7 RTS|CTS|DTR|DSR
1: uart:16550A port:000002F8 irq:3 tx:24 rx:7 RTS|CTS|DTR|DSR

(here there's no overrun but I almost didn't use the serial port since last reboot)

Cheers,

Signature · 4 lines
Raphaël Hertzog

Premier livre français sur Debian GNU/Linux :
http://www.ouaza.com/livre/admin-debian/
#3 lore raw
In reply to Raphael Hertzog
Quoted 16 lines · Raphael Hertzog

(Please CC me when replying)

Hello,

While using Linux on low-end (semi-embedded) hardware (386 SX 40Mhz, 8Mb RAM), I discovered that Linux on that machine would suffer from serial port buffer overruns quite easily if I use a baudrate high enough (I start loosing bytes at >19200 bauds and I would like to make it reliable up to 115 kbauds). I check if overruns are happening with /proc/tty/driver/serial ("oe" field).

Back when I was using the 2.4 kernel, I reduced dramatically the frequency of overruns by using the "low latency" and "preemptible kernel" patch [1]. But it still happened sometimes at 115 kbauds if the system was a bit loaded (with disk I/O for example).

Quoted 5 lines

Now I switched to stock 2.6 and while the stock kernel improved in responsiveness, it still isn't enough by default (even with CONFIG_PREEMPT=y and CONFIG_HZ=1000). So I wanted to try the "rt" patch of Ingo Molnar and Thomas Gleixner, but the patched kernel doesn't boot (see bug report in a separate mail on this list).

Does the serial performance seem to have regressed from 2.4 to 2.6? I am chasing a similar issue with a serial MIDI card (supported by the bog standard 8250 serial driver) that drops notes under 2.6 but works with 2.4. I don't have details yet, but it sounds like a similar problem.

Lee

#4 lore raw
Quoted 8 lines · Lee Revell
Quoted 5 lines · Raphael Hertzog

Now I switched to stock 2.6 and while the stock kernel improved in responsiveness, it still isn't enough by default (even with CONFIG_PREEMPT=y and CONFIG_HZ=1000). So I wanted to try the "rt" patch of Ingo Molnar and Thomas Gleixner, but the patched kernel doesn't boot (see bug report in a separate mail on this list).

Quoted 4 lines

Does the serial performance seem to have regressed from 2.4 to 2.6? I am chasing a similar issue with a serial MIDI card (supported by the bog standard 8250 serial driver) that drops notes under 2.6 but works with 2.4. I don't have details yet, but it sounds like a similar problem.

What specific 2.6 kernels are each of you using?

Signature · 2 lines
Paul Fulghum
Microgate Systems, Ltd.
#5 lore raw
Quoted 9 lines · Paul Fulghum
Quoted 8 lines · Lee Revell
Quoted 5 lines · Raphael Hertzog

Now I switched to stock 2.6 and while the stock kernel improved in responsiveness, it still isn't enough by default (even with CONFIG_PREEMPT=y and CONFIG_HZ=1000). So I wanted to try the "rt" patch of Ingo Molnar and Thomas Gleixner, but the patched kernel doesn't boot (see bug report in a separate mail on this list).

Quoted 7 lines
Quoted 4 lines

Does the serial performance seem to have regressed from 2.4 to 2.6? I am chasing a similar issue with a serial MIDI card (supported by the bog standard 8250 serial driver) that drops notes under 2.6 but works with 2.4. I don't have details yet, but it sounds like a similar problem.

What specific 2.6 kernels are each of you using?

2.6.15 and 2.6.16. Here is the .config:

http://members.dca.net/rlrevell/serialbug-config

I don't know which 2.4 version they tested on that worked - just that the problem started when moving to 2.6.

Lee

#6 lore raw
Quoted 1 line · Lee Revell

2.6.15 and 2.6.16. Here is the .config:

Alan's rework of the receive tty buffering went into 2.6.16 and cured some problems, but clearly not yours. Some more adjustments are in 2.6.18-rc4, so that would be interesting to try for diagnosing this.

I was wondering if the problem was interrupt latency, the tty receive buffering, or something totally different. I don't know if your problem and Raphael's are caused by the same mechanism. I would still like to know which kernel versions he has tried.

Does the MIDI device using the standard N_TTY line discipline? Are you using the low_latency flag on the serial device? What type of UART has been tested (16550? other?) Are you seeing overruns or just lost data?

Thanks,
Paul

#7 lore raw
Quoted 4 lines · Paul Fulghum

Does the MIDI device using the standard N_TTY line discipline? Are you using the low_latency flag on the serial device? What type of UART has been tested (16550? other?) Are you seeing overruns or just lost data?

MIDI uses its own driver - sound/drivers/serial-u16550.c. My guess is there's something in the system starving interrupt servicing. Serial is very sensitive to that, and increases in other system latencies tends to have an adverse impact on serial.

Signature · 3 lines
Russell King
 Linux kernel    2.6 ARM Linux   - http://www.arm.linux.org.uk/
 maintainer of:  2.6 Serial core
#8 lore raw
Quoted 11 lines · Russell King
Quoted 4 lines · Paul Fulghum

Does the MIDI device using the standard N_TTY line discipline? Are you using the low_latency flag on the serial device? What type of UART has been tested (16550? other?) Are you seeing overruns or just lost data?

MIDI uses its own driver - sound/drivers/serial-u16550.c. My guess is there's something in the system starving interrupt servicing. Serial is very sensitive to that, and increases in other system latencies tends to have an adverse impact on serial.

Thanks.

Have you seen many other reports of serial working reliably in 2.4 but not in 2.6? Right now this is the only clue I have to go on...

Lee

#9 lore raw
Quoted 8 lines · Lee Revell
Quoted 4 lines · Russell King

MIDI uses its own driver - sound/drivers/serial-u16550.c. My guess is there's something in the system starving interrupt servicing. Serial is very sensitive to that, and increases in other system latencies tends to have an adverse impact on serial.

Have you seen many other reports of serial working reliably in 2.4 but not in 2.6? Right now this is the only clue I have to go on...

There have been one or two, but the above is basically as far as I've got. Unfortunately, I don't have any machines slow enough (or maybe with the right hardware) to exhibit the problem.

Signature · 3 lines
Russell King
 Linux kernel    2.6 ARM Linux   - http://www.arm.linux.org.uk/
 maintainer of:  2.6 Serial core
#10 lore raw
Quoted 7 lines · Russell King
Quoted 6 lines · Lee Revell
Quoted 4 lines · Russell King

MIDI uses its own driver - sound/drivers/serial-u16550.c. My guess is there's something in the system starving interrupt servicing. Serial is very sensitive to that, and increases in other system latencies tends to have an adverse impact on serial.

Quoted 7 lines
Quoted 2 lines

Have you seen many other reports of serial working reliably in 2.4 but not in 2.6? Right now this is the only clue I have to go on...

There have been one or two, but the above is basically as far as I've got. Unfortunately, I don't have any machines slow enough (or maybe with the right hardware) to exhibit the problem.

OK, thanks. FWIW here is the serial board we are using:

http://www.moschip.com/html/MCS9845.html

The hardware guy says "The mn9845cv, have in default 2 serial ports and one ISA bus, where we have connected the tl16c554, quad serial port."

Hopefully Ingo's latency tracer can tell me what is holding off interrupts.

Lee

#11 lore raw

R: How to avoid serial port buffer overruns?

Quoted 9 lines

OK, thanks. FWIW here is the serial board we are using:

http://www.moschip.com/html/MCS9845.html

The hardware guy says "The mn9845cv, have in default 2 serial ports and one ISA bus, where we have connected the tl16c554, quad serial port."

Hopefully Ingo's latency tracer can tell me what is holding off interrupts.

That may be an interrupt-sharing issue: quad port often use at most two irq lines and, FWIK, ISA irqs are edge-triggered, not level-triggered. Are you tring to use two MIDI ports at the same time?

Giampaolo

Quoted 9 lines

Lee

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/

#12 lore raw

R: How to avoid serial port buffer overruns?

In reply to Lee Revell
Quoted 5 lines · Russell King

OK, thanks. FWIW here is the serial board we are using:

Quoted 7 lines

http://www.moschip.com/html/MCS9845.html

The hardware guy says "The mn9845cv, have in default 2 serial ports and one ISA bus, where we have connected the tl16c554, quad serial port."

Hopefully Ingo's latency tracer can tell me what is holding off interrupts.

I beg your pardon: I'm not used that much to interrupts handling in Linux, but this piece of code from sound/drivers/serial-u16550.c in a linux-2.6.16:

	static irqreturn_t snd_uart16550_interrupt(int irq, void *dev_id, struct pt_regs *regs)
	{
	        snd_uart16550_t *uart;
	
	        uart = (snd_uart16550_t *) dev_id;
	        spin_lock(&uart->open_lock);
	        if (uart->filemode == SERIAL_MODE_NOT_OPENED) {
	                spin_unlock(&uart->open_lock);
	                return IRQ_NONE;
	        }
	        inb(uart->base + UART_IIR);             /* indicate to the UART that the interrupt has been serviced */
	        snd_uart16550_io_loop(uart);
	        spin_unlock(&uart->open_lock);
	        return IRQ_HANDLED;
	}

means to me that IRQ_HANDLED is returned even when the interrupt is not issued by the specific UART. This may lead to problems when two or more uarts share the same irq line and the irq line is edge-triggered instead of level-triggered, as is the case with ISA.

To my knowledge, IRQ_HANDLED should be returned when an interrupt had been served by that specific device handler. Returning a IRQ_HANDLED when the device didn't request for service, in the best case cuases interrupt latencies, in the worst (like in an ISA environment) impairs servicing requests from devices sharing the same IRQ line.

The byte returned from inb(uart->base + UART_IIR) can be used to detect if this is the requesting UART.

Am I wrong?

Regards,

Giampaolo

#13 lore raw

Re: R: How to avoid serial port buffer overruns?

Quoted 7 lines · Giampaolo Tomassoni
Quoted 7 lines · Russell King

OK, thanks. FWIW here is the serial board we are using:

http://www.moschip.com/html/MCS9845.html

Quoted 7 lines
Quoted 5 lines

The hardware guy says "The mn9845cv, have in default 2 serial ports and one ISA bus, where we have connected the tl16c554, quad serial port."

Hopefully Ingo's latency tracer can tell me what is holding off interrupts.

I beg your pardon: I'm not used that much to interrupts handling in Linux, but this piece of code from sound/drivers/serial-u16550.c in a linux-2.6.16:

OK, they are not using serial-u16550 but 8250_fourport for some reason:

# cat /proc/tty/driver/serial 
serinfo:1.0 driver revision:
0: uart:16550A port:000003F8 irq:4 tx:0 rx:0
1: uart:unknown port:000002B8 irq:5
2: uart:unknown port:000003E8 irq:4
3: uart:unknown port:000002E8 irq:3
4: uart:16550A port:0000DD00 irq:185 tx:234335 rx:47502 RTS|DTR
5: uart:16550A port:0000E300 irq:185 tx:249926 rx:27732 RTS|DTR
6: uart:16550A port:0000E400 irq:185 tx:120958 rx:0 RTS|DTR
7: uart:16550A port:0000D000 irq:185 tx:0 rx:0
8: uart:16550A port:0000D100 irq:185 tx:0 rx:0 RTS|DTR
9: uart:16550A port:0000D200 irq:185 tx:0 rx:123406 RTS|DTR

It looks like no overruns are reported, but I have to find out whether they have reproduced the bug since the last reboot.

#14 lore raw

Re: R: How to avoid serial port buffer overruns?

Quoted 11 lines · Lee Revell
Quoted 10 lines · Giampaolo Tomassoni
Quoted 10 lines · Russell King

OK, thanks. FWIW here is the serial board we are using:

http://www.moschip.com/html/MCS9845.html

The hardware guy says "The mn9845cv, have in default 2 serial ports and one ISA bus, where we have connected the tl16c554, quad serial port."

Quoted 6 lines
Quoted 4 lines
Quoted 2 lines

Hopefully Ingo's latency tracer can tell me what is holding off interrupts.

I beg your pardon: I'm not used that much to interrupts handling in Linux, but this piece of code from sound/drivers/serial-u16550.c in a linux-2.6.16:

OK, they are not using serial-u16550 but 8250_fourport for some reason:

Doesn't look like it. fourport cards have their ports at 0x1a0..0x1bf and 0x2a0..0x2bf, and have some special and non-standard features.

Quoted 15 lines
# cat /proc/tty/driver/serial 
serinfo:1.0 driver revision:
0: uart:16550A port:000003F8 irq:4 tx:0 rx:0
1: uart:unknown port:000002B8 irq:5
2: uart:unknown port:000003E8 irq:4
3: uart:unknown port:000002E8 irq:3
4: uart:16550A port:0000DD00 irq:185 tx:234335 rx:47502 RTS|DTR
5: uart:16550A port:0000E300 irq:185 tx:249926 rx:27732 RTS|DTR
6: uart:16550A port:0000E400 irq:185 tx:120958 rx:0 RTS|DTR
7: uart:16550A port:0000D000 irq:185 tx:0 rx:0
8: uart:16550A port:0000D100 irq:185 tx:0 rx:0 RTS|DTR
9: uart:16550A port:0000D200 irq:185 tx:0 rx:123406 RTS|DTR

It looks like no overruns are reported, but I have to find out whether they have reproduced the bug since the last reboot.

Signature · 3 lines
Russell King
 Linux kernel    2.6 ARM Linux   - http://www.arm.linux.org.uk/
 maintainer of:  2.6 Serial core
#15 lore raw

Re: R: How to avoid serial port buffer overruns?

Quoted 15 lines · Russell King
Quoted 14 lines · Lee Revell
Quoted 13 lines · Giampaolo Tomassoni
Quoted 12 lines · Russell King

OK, thanks. FWIW here is the serial board we are using:

http://www.moschip.com/html/MCS9845.html

The hardware guy says "The mn9845cv, have in default 2 serial ports and one ISA bus, where we have connected the tl16c554, quad serial port."

Hopefully Ingo's latency tracer can tell me what is holding off interrupts.

Quoted 7 lines
Quoted 3 lines
Quoted 1 line

I beg your pardon: I'm not used that much to interrupts handling in Linux, but this piece of code from sound/drivers/serial-u16550.c in a linux-2.6.16:

OK, they are not using serial-u16550 but 8250_fourport for some reason:

Doesn't look like it. fourport cards have their ports at 0x1a0..0x1bf and 0x2a0..0x2bf, and have some special and non-standard features.

Which driver is being used then?

# cat /proc/interrupts 
           CPU0       
  0:     801050    IO-APIC-edge  timer
  1:         42    IO-APIC-edge  i8042
  7:          0    IO-APIC-edge  parport0
  8:          4    IO-APIC-edge  rtc
  9:          0   IO-APIC-level  acpi
 12:      11022    IO-APIC-edge  i8042
 14:       2517    IO-APIC-edge  ide0
169:     536096   IO-APIC-level  ohci1394, ICE1712
185:       3908   IO-APIC-level  eth0, serial
193:       8610   IO-APIC-level  libata
201:        326   IO-APIC-level  uhci_hcd:usb1, uhci_hcd:usb2,
uhci_hcd:usb3, uhci_hcd:usb4, ehci_hcd:usb5
217:      84947   IO-APIC-level  nvidia
NMI:          0 
LOC:     800973 
ERR:          0
MIS:          0
# lsmod | egrep serial\|8250
8250_fourport           2048  0 [permanent]
8250_pnp                8704  0 
parport_serial          7680  0 
parport_pc             31984  1 parport_serial
8250_pci               19968  1 parport_serial
8250                   22704  12 8250_pnp,8250_pci
serial_core            19200  1 8250

Serial: 8250/16550 driver $Revision: 1.90 $ 10 ports, IRQ sharing enabled
serial8250: ttyS0 at I/O 0x3f8 (irq = 4) is a 16550A
pci_hotplug: PCI Hot Plug PCI Core version: 0.5
shpchp: Standard Hot Plug PCI Controller Driver version: 0.4 Linux agpgart interface v0.101 (c) Dave Jones
agpgart: Detected AGP bridge 0
agpgart: AGP aperture is 128M @ 0xe8000000
parport: PnPBIOS parport detected.
parport0: PC-style at 0x378, irq 7 [PCSPP,TRISTATE]
Real Time Clock Driver v1.12ac
input: PC Speaker as /class/input/input1
ACPI: PCI Interrupt 0000:00:0b.0[A] -> GSI 19 (level, low) -> IRQ 185 0000:00:0b.0: ttyS4 at I/O 0xdd00 (irq = 185) is a 16550A 0000:00:0b.0: ttyS5 at I/O 0xe300 (irq = 185) is a 16550A 0000:00:0b.0: ttyS6 at I/O 0xe400 (irq = 185) is a 16550A 0000:00:0b.0: ttyS7 at I/O 0xd000 (irq = 185) is a 16550A 0000:00:0b.0: ttyS8 at I/O 0xd100 (irq = 185) is a 16550A 0000:00:0b.0: ttyS9 at I/O 0xd200 (irq = 185) is a 16550A

Lee

#16 lore raw

Re: R: How to avoid serial port buffer overruns?

Quoted 46 lines · Lee Revell
Quoted 15 lines · Russell King
Quoted 14 lines · Lee Revell
Quoted 13 lines · Giampaolo Tomassoni
Quoted 12 lines · Russell King

OK, thanks. FWIW here is the serial board we are using:

http://www.moschip.com/html/MCS9845.html

The hardware guy says "The mn9845cv, have in default 2 serial ports and one ISA bus, where we have connected the tl16c554, quad serial port."

Hopefully Ingo's latency tracer can tell me what is holding off interrupts.

Quoted 7 lines
Quoted 3 lines
Quoted 1 line

I beg your pardon: I'm not used that much to interrupts handling in Linux, but this piece of code from sound/drivers/serial-u16550.c in a linux-2.6.16:

OK, they are not using serial-u16550 but 8250_fourport for some reason:

Doesn't look like it. fourport cards have their ports at 0x1a0..0x1bf and 0x2a0..0x2bf, and have some special and non-standard features.

Which driver is being used then?

# lsmod | egrep serial\|8250
8250_fourport           2048  0 [permanent]
8250_pnp                8704  0 
parport_serial          7680  0 
parport_pc             31984  1 parport_serial
8250_pci               19968  1 parport_serial
8250                   22704  12 8250_pnp,8250_pci
serial_core            19200  1 8250

Serial: 8250/16550 driver $Revision: 1.90 $ 10 ports, IRQ sharing enabled
serial8250: ttyS0 at I/O 0x3f8 (irq = 4) is a 16550A
pci_hotplug: PCI Hot Plug PCI Core version: 0.5
shpchp: Standard Hot Plug PCI Controller Driver version: 0.4 Linux agpgart interface v0.101 (c) Dave Jones
agpgart: Detected AGP bridge 0
agpgart: AGP aperture is 128M @ 0xe8000000
parport: PnPBIOS parport detected.
parport0: PC-style at 0x378, irq 7 [PCSPP,TRISTATE] Real Time Clock Driver v1.12ac

Quoted 8 lines

input: PC Speaker as /class/input/input1
ACPI: PCI Interrupt 0000:00:0b.0[A] -> GSI 19 (level, low) -> IRQ 185 0000:00:0b.0: ttyS4 at I/O 0xdd00 (irq = 185) is a 16550A 0000:00:0b.0: ttyS5 at I/O 0xe300 (irq = 185) is a 16550A 0000:00:0b.0: ttyS6 at I/O 0xe400 (irq = 185) is a 16550A 0000:00:0b.0: ttyS7 at I/O 0xd000 (irq = 185) is a 16550A 0000:00:0b.0: ttyS8 at I/O 0xd100 (irq = 185) is a 16550A 0000:00:0b.0: ttyS9 at I/O 0xd200 (irq = 185) is a 16550A

That "0000:00:0b.0" looks like a PCI device ID. If it were a fourport board, it would be "serial8250.3" according to the current enumeration in linux/serial_8250.h.

Also, another give away is that IRQ185 is being setup as a PCI interrupt immediately prior to the devices being registered.

And I doubt that an ISA board (which is what fourport is) would ever get such a high IRQ number.

Signature · 3 lines
Russell King
 Linux kernel    2.6 ARM Linux   - http://www.arm.linux.org.uk/
 maintainer of:  2.6 Serial core
#17 lore raw

Re: R: How to avoid serial port buffer overruns?

Quoted 10 lines · Russell King

That "0000:00:0b.0" looks like a PCI device ID. If it were a fourport board, it would be "serial8250.3" according to the current enumeration in linux/serial_8250.h.

Also, another give away is that IRQ185 is being setup as a PCI interrupt immediately prior to the devices being registered.

And I doubt that an ISA board (which is what fourport is) would ever get such a high IRQ number.

So you're saying that the standard 8250 driver is being used?

This is the board they are using:

http://www.moschip.com/html/MCS9845.html

Lee

#18 lore raw

Re: R: How to avoid serial port buffer overruns?

Quoted 5 lines · Lee Revell
Quoted 4 lines · Russell King

That "0000:00:0b.0" looks like a PCI device ID. If it were a fourport board, it would be "serial8250.3" according to the current enumeration in linux/serial_8250.h.

Quoted 8 lines
Quoted 6 lines

Also, another give away is that IRQ185 is being setup as a PCI interrupt immediately prior to the devices being registered.

And I doubt that an ISA board (which is what fourport is) would ever get such a high IRQ number.

So you're saying that the standard 8250 driver is being used?

Yes, which is also the case with 8250_fourport. 8250_fourport is just a probe module just like 8250_pnp or 8250_pci.

Quoted 1 line

http://www.moschip.com/html/MCS9845.html

That also clearly says its a PCI device. 8)

What problem are we talking about here again? Sorry, I've completely lost track and this particular thread of 26 messages is soo convoluted and too much to re-read.

Since you only appear to be the messenger, wouldn't it be far better to get the person with the problem to report and respond rather than sitting in the middle?

Signature · 3 lines
Russell King
 Linux kernel    2.6 ARM Linux   - http://www.arm.linux.org.uk/
 maintainer of:  2.6 Serial core
#19 lore raw

Re: R: How to avoid serial port buffer overruns?

Quoted 22 lines · Russell King
Quoted 5 lines · Lee Revell
Quoted 4 lines · Russell King

That "0000:00:0b.0" looks like a PCI device ID. If it were a fourport board, it would be "serial8250.3" according to the current enumeration in linux/serial_8250.h.

Quoted 8 lines
Quoted 6 lines

Also, another give away is that IRQ185 is being setup as a PCI interrupt immediately prior to the devices being registered.

And I doubt that an ISA board (which is what fourport is) would ever get such a high IRQ number.

So you're saying that the standard 8250 driver is being used?

Yes, which is also the case with 8250_fourport. 8250_fourport is just a probe module just like 8250_pnp or 8250_pci.

Quoted 1 line

http://www.moschip.com/html/MCS9845.html

That also clearly says its a PCI device. 8)

Quoted 7 lines

What problem are we talking about here again? Sorry, I've completely lost track and this particular thread of 26 messages is soo convoluted and too much to re-read.

Since you only appear to be the messenger, wouldn't it be far better to get the person with the problem to report and respond rather than sitting in the middle?

BTW, let's have _one_ email which describes what problem it is, the hardware which its being seen on, the kernel configuration, what modules are loaded, the lspci output and so forth. Let's not spread all the information over 20 emails.

I just can't work with dribbled small bits of inforamtion sparsely spread. Neither can I work when it takes days to get that information - I forget both the reported details and the thread of thought I was following to maybe work towards a conclusion.

(Though at present I'm rather devoid of ideas in this area.)

Signature · 3 lines
Russell King
 Linux kernel    2.6 ARM Linux   - http://www.arm.linux.org.uk/
 maintainer of:  2.6 Serial core
#20 lore raw

Re: R: How to avoid serial port buffer overruns?

In reply to Russell King
Quoted 4 lines · Russell King

What problem are we talking about here again? Sorry, I've completely lost track and this particular thread of 26 messages is soo convoluted and too much to re-read.

I thought I might have been seeing the same problem as the OP, but I've found it's a separate issue. I'll start a new thread.

The original poster in this thread was just wondering how to reduce the number of serial overruns at baud rates over 19200.

Quoted 3 lines

Since you only appear to be the messenger, wouldn't it be far better to get the person with the problem to report and respond rather than sitting in the middle?

Sorry, it took me a while but I have access to the machine with the problem now.

Lee

#21 lore raw
In reply to Russell King

Ar Iau, 2006-08-17 am 00:10 +0100, ysgrifennodd Russell King:

Quoted 1 line

MIDI uses its own driver - sound/drivers/serial-u16550.c. My guess

How peculiar

Quoted 3 lines

is there's something in the system starving interrupt servicing. Serial is very sensitive to that, and increases in other system latencies tends to have an adverse impact on serial.

I see no support for the 16650 specific bits in the driver, so that alone may be a problem ?

#22 lore raw
Quoted 11 lines · Alan Cox

Ar Iau, 2006-08-17 am 00:10 +0100, ysgrifennodd Russell King:

Quoted 1 line

MIDI uses its own driver - sound/drivers/serial-u16550.c. My guess

How peculiar

Quoted 3 lines

is there's something in the system starving interrupt servicing. Serial is very sensitive to that, and increases in other system latencies tends to have an adverse impact on serial.

I see no support for the 16650 specific bits in the driver, so that alone may be a problem ?

Huh? 16550 not 16650. Did you miss this?

        outb(UART_FCR_ENABLE_FIFO       /* Enable FIFO's (if available) */
             | UART_FCR_CLEAR_RCVR      /* Clear receiver FIFO */
             | UART_FCR_CLEAR_XMIT      /* Clear transmitter FIFO */
             | UART_FCR_TRIGGER_4       /* Set FIFO trigger at 4-bytes */
        /* NOTE: interrupt generated after T=(time)4-bytes
         * if less than UART_FCR_TRIGGER bytes received
         */
             ,uart->base + UART_FCR);   /* FIFO Control Register */
Signature · 3 lines
Russell King
 Linux kernel    2.6 ARM Linux   - http://www.arm.linux.org.uk/
 maintainer of:  2.6 Serial core
#23 lore raw

Ar Iau, 2006-08-17 am 10:28 +0100, ysgrifennodd Russell King:

Quoted 6 lines · Alan Cox
Quoted 5 lines

Ar Iau, 2006-08-17 am 00:10 +0100, ysgrifennodd Russell King:

Quoted 1 line

MIDI uses its own driver - sound/drivers/serial-u16550.c. My guess

How peculiar

Quoted 8 lines
Quoted 6 lines
Quoted 3 lines

is there's something in the system starving interrupt servicing. Serial is very sensitive to that, and increases in other system latencies tends to have an adverse impact on serial.

I see no support for the 16650 specific bits in the driver, so that alone may be a problem ?

Huh? 16550 not 16650. Did you miss this?

Raphael said "I forgot to mention the kind of UART in my mail but it is a 16650A"

#24 lore raw
Quoted 2 lines · Alan Cox

Raphael said "I forgot to mention the kind of UART in my mail but it is a 16650A"

There are two different applications reporting problems in this one thread.

Lee's friend is the only one using MIDI,
Raphael is having problems with regular
serial communications.

Both are receive problems that started
with the 2.6 series, but may or may not be
the same problem. Lee is reporting lost data and Raphael is reporting rx overruns.

I asked Lee to clarify which driver his friend is using (the MIDI 16550 or standard serial 8250). His original post stated "a serial MIDI card (supported by the bog standard 8250 serial driver)".

Signature · 2 lines
Paul Fulghum
Microgate Systems, Ltd.
#25 lore raw
In reply to Alan Cox
Quoted 11 lines · Alan Cox
Quoted 8 lines
Quoted 6 lines
Quoted 3 lines

is there's something in the system starving interrupt servicing. Serial is very sensitive to that, and increases in other system latencies tends to have an adverse impact on serial.

I see no support for the 16650 specific bits in the driver, so that alone may be a problem ?

Huh? 16550 not 16650. Did you miss this?

Raphael said "I forgot to mention the kind of UART in my mail but it is a 16650A"

That's a typo. :-/
Two lines further in the same mail there was:

bash-2.05a# cat /proc/tty/driver/serial
serinfo:1.0 driver revision:
0: uart:16550A port:000003F8 irq:4 tx:31 rx:7 RTS|CTS|DTR|DSR

        ^^^^^^

So I meant 16550A. Sorry.

Regards,

Signature · 4 lines
Raphaël Hertzog

Premier livre français sur Debian GNU/Linux :
http://www.ouaza.com/livre/admin-debian/
#26 lore raw
In reply to Paul Fulghum
Quoted 7 lines · Paul Fulghum
Quoted 1 line · Lee Revell

2.6.15 and 2.6.16. Here is the .config:

Alan's rework of the receive tty buffering went into 2.6.16 and cured some problems, but clearly not yours. Some more adjustments are in 2.6.18-rc4, so that would be interesting to try for diagnosing this.

I will try 2.6.18-rc4 and keep you informed.

Quoted 5 lines

I was wondering if the problem was interrupt latency, the tty receive buffering, or something totally different. I don't know if your problem and Raphael's are caused by the same mechanism. I would still like to know which kernel versions he has tried.

I tried 2.6.17.7.

But I'm really not sure that the 2.6 is a regression from 2.4, in fact I think it does better by default.

The stock 2.4.31 kernel I was using had serial overruns at 9600 bauds already. Once patched with the low latency/preemptive kernel patchs, it was way better and I had only overruns at 115200 bauds.

With the 2.6.17.7 kernel (configured with CONFIG_PREEMPT and CONFIG_HZ=1000), I'm seeing overruns starting at 38400 bauds. So compared to plain 2.4, it's better. However compared to the patched 2.4, it's worse.

(the figures are *very approximative* as the overruns haven't been detected with the same test conditions on 2.4 and on 2.6)

I have no result with the 2.6.17.7 patched with the real time patch of Ingo/Thomas since it currently doesn't work on my card (see my separate bugreport).

(those questions may have been directed to Lee but I'll respond in my case as well)

Quoted 1 line

Are you using the low_latency flag on the serial device?

I tried that option with the 2.4 kernel and it didn't improve the situation at all, and I haven't retried it with the 2.6 yet. But I will do.

Quoted 1 line

What type of UART has been tested (16550? other?)

I'm using only 16550 and I have no choice here, it's an "off the shelf" card.

Quoted 1 line

Are you seeing overruns or just lost data?

I'm seeing overruns.

Regards,

Signature · 4 lines
Raphaël Hertzog

Premier livre français sur Debian GNU/Linux :
http://www.ouaza.com/livre/admin-debian/
#27 lore raw
Quoted 5 lines · Raphael Hertzog

I tried 2.6.17.7.

But I'm really not sure that the 2.6 is a regression from 2.4, in fact I think it does better by default.

Quoted 8 lines

The stock 2.4.31 kernel I was using had serial overruns at 9600 bauds already. Once patched with the low latency/preemptive kernel patchs, it was way better and I had only overruns at 115200 bauds.

With the 2.6.17.7 kernel (configured with CONFIG_PREEMPT and CONFIG_HZ=1000), I'm seeing overruns starting at 38400 bauds. So compared to plain 2.4, it's better. However compared to the patched 2.4, it's worse.

This tells me your issue is not a problem with the serial or tty code, but rather a matter of IRQ latency. (Which you may have already known, but I was unclear on) I do not expect 2.6.18-rc4 to make a difference.

For fun, have you tried playing with the rx FIFO trigger level in the 16550A entry in drivers/serial/8250.c ? You could try replacing UART_FCR_R_TRIG_10 (8 char trigger) with UART_FCR_R_TRIG_01 (4 char trigger) or even UART_FCR_R_TRIG_00 (1 char trigger).
That creates more interrupts, but allows
more time to activate the ISR before overrun.

Lee's issue may still merit investigation into the serial/tty code.

Signature · 2 lines
Paul Fulghum
Microgate Systems, Ltd.
#28 lore raw
In reply to Raphael Hertzog
Quoted 5 lines · Raphael Hertzog

With the 2.6.17.7 kernel (configured with CONFIG_PREEMPT and CONFIG_HZ=1000), I'm seeing overruns starting at 38400 bauds. So compared to plain 2.4, it's better. However compared to the patched 2.4, it's worse.

(Sorry for hijacking your thread)

Have you tried it with HZ=100? HZ=1000 might just be too much for that board.

Lee

#29 lore raw

Hello,

I took the time to try some of your suggestions and I managed to get rid of the overruns.

Quoted 2 lines · Lee Revell

Have you tried it with HZ=100? HZ=1000 might just be too much for that board.

This indeed was a major problem and a bad choice of mine at the very beginning. It goes way better with HZ=100.

Quoted 7 lines · Paul Fulghul

For fun, have you tried playing with the rx FIFO trigger level in the 16550A entry in drivers/serial/8250.c ? You could try replacing UART_FCR_R_TRIG_10 (8 char trigger) with UART_FCR_R_TRIG_01 (4 char trigger) or even UART_FCR_R_TRIG_00 (1 char trigger).
That creates more interrupts, but allows
more time to activate the ISR before overrun.

I changed the rx FIFO trigger level to 1 byte (UART_FCR_R_TRIG_00) and it helped a lot as well. With this combination I completely resolved the problem of overruns at full speed (115200 bauds).

For the sake of comparison, I made a similar change to the 2.4.31 kernel I was using (ie a kernel with low latency/preemptible kernel patches).

It helped a lot as well: most of the time I wouldn't have overruns (before they were very frequent, like at least one overrun in 10k chars received). However from time to time I would suffer from a single big overrun (like 30 chars lost). And using heavily the disk on module will increase the likelihood to have a buffer overrun.

With the 2.6.17.7 kernel (CONFIG_HZ=100 and patched to trigger IRQ at 1 byte received), I have been completely unable to reproduce the buffer overruns whatever read/write operation I've been triggering during the data exchange.

So all in all, the 2.6 kernel behaves better than the 2.4 in this case.

Cheers,

Signature · 4 lines
Raphaël Hertzog

Premier livre français sur Debian GNU/Linux :
http://www.ouaza.com/livre/admin-debian/