Skip to contentSkip to conversation

PCIe device driver question

DISCUSSION Sanka Piyaratna 1 participant 1 message Jul 31, 2008

lore
Share & tools
lore
#0 lore raw

Hi Robert,

Thanks for that, I did not notice that in the spec. I am currently using 64 bit format unconditionally and this must be what is going wrong.

Thank you so much for that.

Sanka

----- Original Message ----
From: Robert Hancock <[email protected]>
To: Sanka Piyaratna <[email protected]>
Cc: [email protected]
Sent: Thursday, 31 July, 2008 6:14:54 PM
Subject: Re: PCIe device driver question

Quoted 2 lines · Sanka Piyaratna

I allocate memory in the user land using memalign function (typically I allocate about 500 MB) and pass this to the kernel space. In my device driver, I call get_user_pages() to lock down the memory and extract the relevant pages. A scatter-gather list is generated using these page addresses and hence derive the dma_addresses using page_to_phys() function. These addresses are programmed into a FIFO in the hardware device using a memory mapped register interface (PCI BAR based). Subsequently the hardware start filling up the pages and interrupt when a block of pages are complete. I notice the hardware hang (PCIe packets don't seem to get the acknowledgements from the root complex) when the DMA address is < 0x0000_0001_0000_0000. I have verified in the hardware that the PCIe packet is created with the correct address as programed by the device driver dma_address. If i can guard some how that the memory allocation is with in a certain area, I can stop the
problem from occuring. Are there any bridge functionality in the Intel architecture that may mask a certain region of memory?

How are you formatting the addresses in the TLP? The PCI Express spec says that for addresses below 4GB the 32-bit addressing format must be used (you can't use the 64-bit format unconditionally). It could be that is what is making the chipset unhappy.

Find a better answer, faster with the new Yahoo!7 Search. www.yahoo7.com.au/search