DMA transfers in 2.5.67
What do I need to do in a driver before doing DMA transfers to a PCI card? Using a driver that worked in 2.4 gives a throughput of only 10 MB/s in 2.5.67. Is there some magic initialization that I have missed?
Signature · 2 lines
Måns Rullgård [email protected]
Quoted 4 lines · Måns Rullgård
What do I need to do in a driver before doing DMA transfers to a PCI card? Using a driver that worked in 2.4 gives a throughput of only 10 MB/s in 2.5.67. Is there some magic initialization that I have missed?
Assuming your driver uses the new PCI api for DMA in 2.4/2.5 then there isnt really anything to watch. Is this on a box with > 800Mb of memory however ?
Quoted 8 lines · Alan Cox
Quoted 4 lines
What do I need to do in a driver before doing DMA transfers to a PCI card? Using a driver that worked in 2.4 gives a throughput of only 10 MB/s in 2.5.67. Is there some magic initialization that I have missed?
Assuming your driver uses the new PCI api for DMA in 2.4/2.5 then there isnt really anything to watch. Is this on a box with > 800Mb of memory however ?
It's an Alpha with 768 MB. Is it the pci_alloc_* functions you are referring to? I don't think they are used currently. How much memory can these allocate? I need chunks of up to 1 MB, not necessarily phycically continuous.
What do those functions do that normal memory allocation does not? Apart from setting up sg mappings, that is.
Signature · 2 lines
Måns Rullgård [email protected]
Quoted 7 lines · Måns Rullgård
It's an Alpha with 768 MB. Is it the pci_alloc_* functions you are referring to? I don't think they are used currently. How much memory can these allocate? I need chunks of up to 1 MB, not necessarily phycically continuous.
What do those functions do that normal memory allocation does not? Apart from setting up sg mappings, that is.
A normal memory allocation might not be visible from the device, however pci_map_sg() deals with such things. What I really meant was are you using the pci_ DMA functionality
Quoted 11 lines · Alan Cox
Quoted 7 lines
It's an Alpha with 768 MB. Is it the pci_alloc_* functions you are referring to? I don't think they are used currently. How much memory can these allocate? I need chunks of up to 1 MB, not necessarily phycically continuous.
What do those functions do that normal memory allocation does not? Apart from setting up sg mappings, that is.
A normal memory allocation might not be visible from the device, however
pci_map_sg()deals with such things. What I really meant was are you using thepci_DMA functionality
Those functions are not used at the moment, but I could change that. The question remains why DMA transfers are so slow. The memory is clearly visible from the bus.
Btw, I just noticed that hard disk throughput is much lower with 2.5 than 2.4. With 2.4.21-pre5 I get ~40 MB/s, but with 2.5.67 the speed drops to 25-30 MB/s. Everything according to hdparm. Is it possible that DMA is generally slow for some reason?
Signature · 2 lines
Måns Rullgård [email protected]
Quoted 4 lines · MЕns RullgЕrd
Btw, I just noticed that hard disk throughput is much lower with 2.5 than 2.4. With 2.4.21-pre5 I get ~40 MB/s, but with 2.5.67 the speed drops to 25-30 MB/s. Everything according to hdparm. Is it possible that DMA is generally slow for some reason?
Possible reason is that in 2.4 we've forced reasonable latency timer value for all PCI devices, while in 2.5 we haven't as yet.
Ivan.
Quoted 8 lines · Ivan Kokshaysky
Quoted 4 lines · Måns Rullgård
Btw, I just noticed that hard disk throughput is much lower with 2.5 than 2.4. With 2.4.21-pre5 I get ~40 MB/s, but with 2.5.67 the speed drops to 25-30 MB/s. Everything according to hdparm. Is it possible that DMA is generally slow for some reason?
Possible reason is that in 2.4 we've forced reasonable latency timer value for all PCI devices, while in 2.5 we haven't as yet.
Do you mean whatever causes this message (for a 3com NIC)?
PCI: Setting latency timer of device 00:05.0 to 64
Would that also explain why my hard disks are slow under 2.5? There is no corresponding message for the ide controller (htp374).
Signature · 2 lines
Måns Rullgård [email protected]
Quoted 8 lines · [email protected] (Måns Rullgård) writes:
Quoted 7 lines
Quoted 4 lines
Btw, I just noticed that hard disk throughput is much lower with 2.5 than 2.4. With 2.4.21-pre5 I get ~40 MB/s, but with 2.5.67 the speed drops to 25-30 MB/s. Everything according to hdparm. Is it possible that DMA is generally slow for some reason?
Possible reason is that in 2.4 we've forced reasonable latency timer value for all PCI devices, while in 2.5 we haven't as yet.
Quoted 6 lines
Do you mean whatever causes this message (for a 3com NIC)?
PCI: Setting latency timer of device 00:05.0 to 64
Would that also explain why my hard disks are slow under 2.5? There is no corresponding message for the ide controller (htp374).
I just checked the troublesome board. Here's what lspci has to say:
00:06.0 Display controller: 3DLabs GLINT R3 (rev 01)
Subsystem: 3DLabs: Unknown device 0121 Flags: bus master, 66Mhz, medium devsel, latency 0, IRQ 27 Memory at 0000000009020000 (32-bit, non-prefetchable) [size=128K] Memory at 000000000c000000 (32-bit, prefetchable) [size=64M] Memory at 0000000010000000 (32-bit, prefetchable) [size=64M] Expansion ROM at 0000000009060000 [disabled] [size=64K] Capabilities: <available only to root>
The latency 0 doesn't look too good. What should I do to change it?
Signature · 2 lines
Måns Rullgård [email protected]
Quoted 26 lines · [email protected] (Måns Rullgård) writes:
Quoted 8 lines
Quoted 7 lines
Quoted 4 lines
Btw, I just noticed that hard disk throughput is much lower with 2.5 than 2.4. With 2.4.21-pre5 I get ~40 MB/s, but with 2.5.67 the speed drops to 25-30 MB/s. Everything according to hdparm. Is it possible that DMA is generally slow for some reason?
Possible reason is that in 2.4 we've forced reasonable latency timer value for all PCI devices, while in 2.5 we haven't as yet.
Quoted 6 lines
Do you mean whatever causes this message (for a 3com NIC)?
PCI: Setting latency timer of device 00:05.0 to 64
Would that also explain why my hard disks are slow under 2.5? There is no corresponding message for the ide controller (htp374).
I just checked the troublesome board. Here's what lspci has to say:
00:06.0 Display controller: 3DLabs GLINT R3 (rev 01)
Subsystem: 3DLabs: Unknown device 0121 Flags: bus master, 66Mhz, medium devsel, latency 0, IRQ 27 Memory at 0000000009020000 (32-bit, non-prefetchable) [size=128K] Memory at 000000000c000000 (32-bit, prefetchable) [size=64M] Memory at 0000000010000000 (32-bit, prefetchable) [size=64M] Expansion ROM at 0000000009060000 [disabled] [size=64K] Capabilities: <available only to root>
Quoted 1 line
The latency 0 doesn't look too good. What should I do to change it?
I set the latency to 128 using setpci and now I get 66 MB/s. Other values give slower transfer rates. Is there some other setting that could improve it even more?
Signature · 2 lines
Måns Rullgård [email protected]
Quoted 3 lines · MЕns RullgЕrd
I set the latency to 128 using setpci and now I get 66 MB/s. Other values give slower transfer rates. Is there some other setting that could improve it even more?
Interesting. Did you set latency 128 for all devices or only for 3Dlabs card?
Ivan.
Quoted 6 lines · Ivan Kokshaysky
Quoted 3 lines
I set the latency to 128 using setpci and now I get 66 MB/s. Other values give slower transfer rates. Is there some other setting that could improve it even more?
Interesting. Did you set latency 128 for all devices or only for 3Dlabs card?
Only the 3Dlabs. The remaining cards are configured like this:
00:05.0 Ethernet controller: 3Com Corporation 3c905C-TX [Fast Etherlink] (rev 78)
Subsystem: 3Com Corporation 3C905C-TX Fast Etherlink for PC Management NIC
Flags: bus master, medium devsel, latency 64, IRQ 25
00:06.0 Display controller: 3DLabs GLINT R3 (rev 01)
Subsystem: 3DLabs: Unknown device 0121
Flags: bus master, 66Mhz, medium devsel, latency 128, IRQ 27
00:07.0 RAID bus controller: Triones Technologies, Inc.: Unknown device 0008 (rev 07)
Subsystem: Triones Technologies, Inc.: Unknown device 0001
Flags: bus master, 66Mhz, medium devsel, latency 120, IRQ 26
00:07.1 RAID bus controller: Triones Technologies, Inc.: Unknown device 0008 (rev 07)
Subsystem: Triones Technologies, Inc.: Unknown device 0001
Flags: bus master, 66Mhz, medium devsel, latency 120, IRQ 26
00:08.1 IDE interface: Contaq Microsystems 82c693 (prog-if 80 [Master])
Flags: bus master, medium devsel, latency 0
00:08.2 IDE interface: Contaq Microsystems 82c693 (prog-if 00 [])
Flags: bus master, medium devsel, latency 0
00:08.3 USB Controller: Contaq Microsystems 82c693 (prog-if 10 [OHCI])
Flags: bus master, medium devsel, latency 0
00:09.0 USB Controller: NEC Corporation USB (rev 41) (prog-if 10 [OHCI])
Subsystem: Adaptec: Unknown device 0035
Flags: bus master, medium devsel, latency 64, IRQ 24
00:09.1 USB Controller: NEC Corporation USB (rev 41) (prog-if 10 [OHCI])
Subsystem: Adaptec: Unknown device 0035
Flags: bus master, medium devsel, latency 64, IRQ 28
00:09.2 USB Controller: NEC Corporation: Unknown device 00e0 (rev 02) (prog-if 20)
Subsystem: Adaptec: Unknown device 00e0
Flags: bus master, medium devsel, latency 68, IRQ 32
As you can see, the good IDE controller (Triones/Highpoint hpt374) has a latency of 120. Disk transfer rates are still only half of what I get with a 2.4 kernel. What could be the cause of this?
Signature · 2 lines
Måns Rullgård [email protected]