Skip to contentSkip to conversation

Problem with NETIF_F_HIGHDMA

DISCUSSION Corey Minyard 5 participants 7 messages Feb 22, 2006 – Feb 23, 2006

lore
Share & tools
lore
#0 lore raw

I was looking at a problem with a new system we are trying to get up and running. It has a 32-bit only PCI network device, but is a 64-bit (x86_64) system. Looking at the code for NETIF_F_HIGHDMA (which, when not set on a PCI network device, means that it cannot do 64-bit accesses) in net/core/dev.c, it seems wrong to me.

It is dependent on HIGHMEM, but HIGHMEM has nothing to do with 32/64 bit accesses. On 64-bit systems, HIGHMEM is not set, thus the network code will pass any address (including those >32bits) to the driver. Plus, highmem on 32-bit systems may very well be 32-bit accessible, possibly resulting in unecessary copies. AFAICT, the current code will only work with i386 and PAE and is sub-optimal.

If I am right, it is a little messy to fix, but I think doable. I propose the following:

* Create a new zone named ZONE_HIGHMEM32 for 32-bit HIGHMEM addresses.
* Modify 64-bit architectures (and i386 with HIGHMEM) to put the proper pages into the new zone.
* Add a "PageIn32Bits()" function/macro to check for this, and use it in illegal_highdma() in net/core/dev.c
* Allocate from ZONE_HIGHMEM32 if illegal_highdma() returns true.

I think this will solve the problem. I haven't looked at other parts of the kernel (IDE, SCSI, etc.) to see if they have similar problems.

Anyway, does the above change sound reasonable? Maybe there's an easier way? Maybe I've missed something?

Thanks,

-Corey

#1 lore raw
Quoted 6 lines · Corey Minyard

I was looking at a problem with a new system we are trying to get up and running. It has a 32-bit only PCI network device, but is a 64-bit (x86_64) system. Looking at the code for NETIF_F_HIGHDMA (which, when not set on a PCI network device, means that it cannot do 64-bit accesses) in net/core/dev.c, it seems wrong to me.

Quoted 6 lines

It is dependent on HIGHMEM, but HIGHMEM has nothing to do with 32/64 bit accesses. On 64-bit systems, HIGHMEM is not set, thus the network code will pass any address (including those >32bits) to the driver. Plus, highmem on 32-bit systems may very well be 32-bit accessible, possibly resulting in unecessary copies. AFAICT, the current code will only work with i386 and PAE and is sub-optim

you use the PCI mapping api right? if you do that then there's no problem, after pci mapping the addresses will be in the lower address range perfectly fine....

#2 lore raw
Quoted 18 lines · Arjan van de Ven
Quoted 6 lines · Corey Minyard

I was looking at a problem with a new system we are trying to get up and running. It has a 32-bit only PCI network device, but is a 64-bit (x86_64) system. Looking at the code for NETIF_F_HIGHDMA (which, when not set on a PCI network device, means that it cannot do 64-bit accesses) in net/core/dev.c, it seems wrong to me.

Quoted 8 lines

It is dependent on HIGHMEM, but HIGHMEM has nothing to do with 32/64 bit accesses. On 64-bit systems, HIGHMEM is not set, thus the network code will pass any address (including those >32bits) to the driver. Plus, highmem on 32-bit systems may very well be 32-bit accessible, possibly resulting in unecessary copies. AFAICT, the current code will only work with i386 and PAE and is sub-optim

Quoted 5 lines

you use the PCI mapping api right? if you do that then there's no problem, after pci mapping the addresses will be in the lower address range perfectly fine....

Ah, cool, physical memory remapping. Then the problem lies elsewhere. Thanks.

-Corey

#3 lore raw
Quoted 7 lines · Corey Minyard
Quoted 5 lines

you use the PCI mapping api right? if you do that then there's no problem, after pci mapping the addresses will be in the lower address range perfectly fine....

Ah, cool, physical memory remapping. Then the problem lies elsewhere. Thanks.

iommu's are cool things ;)

#4 lore raw
Quoted 1 line · Arjan van de Ven

iommu's are cool things ;)

Say that again from your other email address :)

Signature · 1 line
dwmw2
#5 lore raw
In reply to Corey Minyard
Quoted 5 lines · * Corey Minyard ([email protected]) wrote:

I was looking at a problem with a new system we are trying to get up and running. It has a 32-bit only PCI network device, but is a 64-bit (x86_64) system. Looking at the code for NETIF_F_HIGHDMA (which, when not set on a PCI network device, means that it cannot do 64-bit accesses) in net/core/dev.c, it seems wrong to me.

Why would NETIF_F_HIGHDMA if it's 32-bit only PCI network device? Typically net drivers do pci_set_dma_mask, and set features accordingly. Then pci_map_* should just work.

thanks,
-chris

#6 lore raw
In reply to Corey Minyard
Quoted 6 lines · Corey Minyard

I was looking at a problem with a new system we are trying to get up and running. It has a 32-bit only PCI network device, but is a 64-bit (x86_64) system. Looking at the code for NETIF_F_HIGHDMA (which, when not set on a PCI network device, means that it cannot do 64-bit accesses) in net/core/dev.c, it seems wrong to me.

Quoted 6 lines

It is dependent on HIGHMEM, but HIGHMEM has nothing to do with 32/64 bit accesses. On 64-bit systems, HIGHMEM is not set, thus the network code will pass any address (including those >32bits) to the driver. Plus, highmem on 32-bit systems may very well be 32-bit accessible, possibly resulting in unecessary copies. AFAICT, the current code will only work with i386 and PAE and is sub-optimal.

x86-64 uses the PCI DMA API to handle this. NETIF_F_HIGHDMA is only a i386 specific hack, mostly to work around the fact that not all memory might be accessible from the CPU and thus break PIO drivers. This problem doesn't exist on 64bit.

-Andi