Skip to contentSkip to conversation

PCI DMA Interrupt latency

DISCUSSION Burkhard Schölpen 2 participants 2 messages Jan 10, 2006

lore
Share & tools
lore
#0 lore raw

Hello,

I'm writing a driver for a custom pci card with an FPGA (Xilinx Spartan 2 (XC2S150-6) with PCI 32 LogiCore), which can act as a pci bus master. The device is designed to do DMA transfers with high bandwidth. One task is to send image data to a printer which already works quite well, but sometimes there are randomly occuring problems concerning the timing between two DMA transfers. The issue seems to be something like interrupt latency in hardware. Measuring some signals with an oscilloscope shows, that the delay from generating the interrupt, which signals a finished transfer, to the time when the interrupt register on the card is reset (i.e. the beginning of the ISR) sometimes increases to more than 500 microseconds, which is dimensions too high.

I already tried with other hardware deactivated, which could cause traffic on the pci bus or generate many interrupts (except hard disk). I also increased the priority of the IRQ used by the pci board (with a tool called irqtune) to the maximum possible value. Another consideration is, that another driver could lock all interrupts for too long (but for 500 us??).

As my experience on DMA stuff is not yet too great, I would be very glad if somebody could give me some advice to solve this problem. Below there is some further information about my environment and how I set up the DMA transfers.

Kind regards,
Burkhard Schölpen

Here is how I set up dma transfers from RAM to the pci device:

while (down_interruptible(my_device->write_semaphore)); my_device->dma_write_complete = 0; my_device->dma_direction = PCI_DMA_TODEVICE;

writel (cpu_to_le32 (virt_to_phys(dma_buffer)), MY_DMA_ADDR_REGISTER);
writel (cpu_to_le32 (my_device->dma_size/4), MY_DMA_COUNT_REGISTER); //triggers dma transfer

if (wait_event_interruptible(write_wait_queue, my_device->dma_write_complete)) {
//handle error...
}
//test, if MY_DMA_COUNT_REGISTER contains 0 up(my_device->write_semaphore);

Inside the Interrupt-handler I do the following:

my_device->dma_write_complete = 1;
wake_up_interruptible(&write_wait_queue);
return IRQ_HANDLED;

Here is some information about the PC:
- Gigabyte GA-8I945GMF mainboard with Pentium D processor
- custom pci board with Xilinx FPGA Spartan 2 (XC2S150-6) with PCI 32 LogiCore
- Debian Linux with 2.6.13.4 SMP kernel

______________________________________________________________
Verschicken Sie romantische, coole und witzige Bilder per SMS! Jetzt bei WEB.DE FreeMail: http://f.web.de/?mc=021193

#1 lore raw
Quoted 3 lines · Burkhard Schölpen

The issue seems to be something like interrupt latency in hardware. Measuring some signals with an oscilloscope shows, that the delay from generating the interrupt, which signals a finished transfer, to the time when the interrupt register on the card is reset (i.e. the beginning of the ISR) sometimes increases to more than 500 microseconds, which is dimensions too high.

... Another consideration is, that another driver could lock all interrupts for too long (but for 500 us??).

I also have implemented bus master devices based on the Spartan 2 + Xilinx PCI core and wrote the associated Linux driver.

My observations of interrupt latency using a similar setup to what you describe showed 50usec is common but rare events on
the order of milliseconds are possible. You are probably correct that the problem as a poorly behaved driver.

If you have complete control over every system your device is installed in, you can find and eliminate any device that causes high interrupt latency.

If you don't have that control, your hardware needs to tolerate interrupt latency of that magnitude.

You describe an implementation with a single DMA buffer. You could switch to multiple buffers (ring structure or 2 alternating buffers) with an interrupt triggered after each buffer is exhausted (I use a ring of buffers). The remaining buffers allow the transfer to continue while waiting for ISR execution.

Or, more crudely, trigger the interrupt when the single buffer reaches some low water mark and poll in the ISR for actual completion.

Signature · 2 lines
Paul Fulghum
Microgate Systems, Ltd.