Problem with NETIF_F_HIGHDMA
I was looking at a problem with a new system we are trying to get up and running. It has a 32-bit only PCI network device, but is a 64-bit (x86_64) system. Looking at the code for NETIF_F_HIGHDMA (which, when not set on a PCI network device, means that it cannot do 64-bit accesses) in net/core/dev.c, it seems wrong to me.
It is dependent on HIGHMEM, but HIGHMEM has nothing to do with 32/64 bit accesses. On 64-bit systems, HIGHMEM is not set, thus the network code will pass any address (including those >32bits) to the driver. Plus, highmem on 32-bit systems may very well be 32-bit accessible, possibly resulting in unecessary copies. AFAICT, the current code will only work with i386 and PAE and is sub-optimal.
If I am right, it is a little messy to fix, but I think doable. I propose the following:
* Create a new zone named ZONE_HIGHMEM32 for 32-bit HIGHMEM addresses.
* Modify 64-bit architectures (and i386 with HIGHMEM) to put the proper pages into the new zone.
* Add a "PageIn32Bits()" function/macro to check for this, and use it in illegal_highdma() in net/core/dev.c
* Allocate from ZONE_HIGHMEM32 if illegal_highdma() returns true.
I think this will solve the problem. I haven't looked at other parts of the kernel (IDE, SCSI, etc.) to see if they have similar problems.
Anyway, does the above change sound reasonable? Maybe there's an easier way? Maybe I've missed something?
Thanks,
-Corey
Quoted 6 lines · Corey Minyard
I was looking at a problem with a new system we are trying to get up and running. It has a 32-bit only PCI network device, but is a 64-bit (
x86_64) system. Looking at the code forNETIF_F_HIGHDMA(which, when not set on a PCI network device, means that it cannot do 64-bit accesses) innet/core/dev.c, it seems wrong to me.
Quoted 6 lines
It is dependent on HIGHMEM, but HIGHMEM has nothing to do with 32/64 bit accesses. On 64-bit systems, HIGHMEM is not set, thus the network code will pass any address (including those >32bits) to the driver. Plus, highmem on 32-bit systems may very well be 32-bit accessible, possibly resulting in unecessary copies. AFAICT, the current code will only work with i386 and PAE and is sub-optim
you use the PCI mapping api right? if you do that then there's no problem, after pci mapping the addresses will be in the lower address range perfectly fine....
Quoted 18 lines · Arjan van de Ven
Quoted 6 lines · Corey Minyard
I was looking at a problem with a new system we are trying to get up and running. It has a 32-bit only PCI network device, but is a 64-bit (
x86_64) system. Looking at the code forNETIF_F_HIGHDMA(which, when not set on a PCI network device, means that it cannot do 64-bit accesses) innet/core/dev.c, it seems wrong to me.Quoted 8 lines
It is dependent on HIGHMEM, but HIGHMEM has nothing to do with 32/64 bit accesses. On 64-bit systems, HIGHMEM is not set, thus the network code will pass any address (including those >32bits) to the driver. Plus, highmem on 32-bit systems may very well be 32-bit accessible, possibly resulting in unecessary copies. AFAICT, the current code will only work with i386 and PAE and is sub-optim
Quoted 5 lines
you use the PCI mapping api right? if you do that then there's no problem, after pci mapping the addresses will be in the lower address range perfectly fine....
Ah, cool, physical memory remapping. Then the problem lies elsewhere. Thanks.
-Corey
Quoted 7 lines · Corey Minyard
Quoted 5 lines
you use the PCI mapping api right? if you do that then there's no problem, after pci mapping the addresses will be in the lower address range perfectly fine....
Ah, cool, physical memory remapping. Then the problem lies elsewhere. Thanks.
iommu's are cool things ;)
Quoted 1 line · Arjan van de Ven
iommu's are cool things ;)
Say that again from your other email address :)
Signature · 1 line
dwmw2
Quoted 5 lines · * Corey Minyard ([email protected]) wrote:
I was looking at a problem with a new system we are trying to get up and running. It has a 32-bit only PCI network device, but is a 64-bit (
x86_64) system. Looking at the code forNETIF_F_HIGHDMA(which, when not set on a PCI network device, means that it cannot do 64-bit accesses) innet/core/dev.c, it seems wrong to me.
Why would NETIF_F_HIGHDMA if it's 32-bit only PCI network device? Typically net drivers do pci_set_dma_mask, and set features accordingly. Then pci_map_* should just work.
thanks,
-chris
Quoted 6 lines · Corey Minyard
I was looking at a problem with a new system we are trying to get up and running. It has a 32-bit only PCI network device, but is a 64-bit (
x86_64) system. Looking at the code forNETIF_F_HIGHDMA(which, when not set on a PCI network device, means that it cannot do 64-bit accesses) innet/core/dev.c, it seems wrong to me.
Quoted 6 lines
It is dependent on HIGHMEM, but HIGHMEM has nothing to do with 32/64 bit accesses. On 64-bit systems, HIGHMEM is not set, thus the network code will pass any address (including those >32bits) to the driver. Plus, highmem on 32-bit systems may very well be 32-bit accessible, possibly resulting in unecessary copies. AFAICT, the current code will only work with i386 and PAE and is sub-optimal.
x86-64 uses the PCI DMA API to handle this. NETIF_F_HIGHDMA is only a i386 specific hack, mostly to work around the fact that not all memory might be accessible from the CPU and thus break PIO drivers. This problem doesn't exist on 64bit.
-Andi