BitKeeper tree for 2.4.x
I am maintaining a BitKeeper tree that tracks Marcelo's 2.4.x tree and the current ia64 patch. My intent is that we will still produce a plain patch against Marcelo's releases, just as David has always done. The BitKeeper tree merely provides visibility into the current state of what will become the next ia64 patch.
I try to keep the BitKeeper tree working, but I don't do much testing before updating it, so it should be regarded as unstable work-in-progress.
The BitKeeper tools, licensing agreement, and documentation are available here:
To get a local copy of the tree, use a BitKeeper command like this:
bk clone bk://lia64.bkbits.net/linux-ia64-2.4 linux-ia64-2.4
Subsequently, you can use "bk pull" in the linux-ia64-2.4/ directory to update your local repository.
The repository is hosted by BitMover, Inc., which provides a web interface with a browsable source tree and ChangeSet history here:
http://lia64.bkbits.net:8080/linux-ia64-2.4
Here are the interesting changes since the 2.4.19-ia64-020821 patch:
* Driver changes:
- bcm: dropped (you may be able to use tg3 instead).
- e1000: dropped 4.1.7, adopted upstream (currently 4.3.15-k1).
- e100: RX_ALIGN now upstream; only last_rx_time fix in ia64 patch.
- acenic: dropped ia64 patches; upstream has most or all.
- forte: dropped 1.45, adopted upstream (currently 1.51).
* SPCR/DBGP support removed (encumbered by Microsoft IP; use HCDP instead).
* McKinley A-step config doc removed (code was already gone).
* HP prototype code removed (Matthew Wilcox).
* include/asm-ia64/offsets.h removed.
* sim{eth,scsi,serial} (drivers for HP simulator) moved to arch/ia64/hp/sim/.
* drivers/media/{radio,video}/dummy.c removed.
* Support for write-coalescing mappings removed.
* Support for non-cached mappings of main memory removed.
* Support scatterlist page/offset in sba_iommu.
* AGP/DRM rework to make it more presentable.
- DRM: dropped obsolete #ifdef __alpha__ diffs.
- DRM: r128, radeon: made all 460GX checks run-time, not compile-time.
* ACPI CRS cleanup (Takayoshi Kochi).
* ACPI debug fixes (Takayoshi Kochi).
* Fix many warnings (Andreas Schwab).
* Fix I/O macros (inb, outb, etc) (Andreas Schwab, David Mosberger).
* If more than NR_CPUS found, ignore extras.
* Add generic RAID xor routines with prefetch (Matthew Wilcox).
* Discard *.text.exit and *.data.exit sections (Matthew Wilcox).
* Fix edge-triggered IRQ handling (David Mosberger).
* Alternate signal stack fixes (David Mosberger).
* VFS extended attribute syscall numbers (Andreas Gruenbacher).
* binfmt argv[1] preservation (David Mosberger).
* Preserve FP registers around firmware calls (John Marvin, David Mosberger).
* FPU load/save optimization (Fenghua Yu).
* Syscalls for Extended Attribute VFS infrastructure (Andreas Gruenbacher).
The following major pieces of the ia64 patch are unchanged since 2.4.19-ia64-020821:
* ACPI: CA version 20020517 (upstream has version 20011018).
* qla1280: 3.23 Beta (upstream has 3.00-Beta).
* qla2x00: 4.31.7b (not in upstream).
NOTE: I removed the mmap support for MAP_WRITECOMBINED and MAP_NONCACHED to avoid issues with memory attribute aliasing. The only user of these that I know about is XFree86, which still seems functional when we ignore the attributes it requests. I'm very interested in any problems caused by this change.
Bjorn
Quoted 11 lines · Steffen Persvold
Well, without
MAP_WRITECOMBINED, streaming PIO accesses to the PCI bus drops from 330MByte/sec to approx. 100MByte/s using the Dolphin SCI card.This is an rather significant drop which makes the IA64 platform kind of uninteresting for high performance clustering (the Dolphin SCI cards are used for shared memory message passing between nodes).
We can of course hack this in for our usage (in our kernel driver) if you are totally against having it in the kernel headers, but I think since the hardware allows different mapping types, the kernel should take advantage of it.
I'm really not opposed to having it in the kernel headers. The thing is, we were allowing WC mappings regardless of whether the hardware actually supported them. What we need is some mechanism for looking at the EFI memory map, which tells us what attributes are supported.
We've got a mechanism for looking up the supported attributes (efi_mem_attributes()); what I'd really like is for somebody to hook that up with the mmap infrastructure so that we only allow the supported attributes.
Bjorn
Quoted 6 lines · Bjorn Helgaas
NOTE: I removed the mmap support for
MAP_WRITECOMBINEDandMAP_NONCACHEDto avoid issues with memory attribute aliasing. The only user of these that I know about is XFree86, which still seems functional when we ignore the attributes it requests. I'm very interested in any problems caused by this change.
Well, without MAP_WRITECOMBINED, streaming PIO accesses to the PCI bus drops from 330MByte/sec to approx. 100MByte/s using the Dolphin SCI card.
This is an rather significant drop which makes the IA64 platform kind of uninteresting for high performance clustering (the Dolphin SCI cards are used for shared memory message passing between nodes).
We can of course hack this in for our usage (in our kernel driver) if you are totally against having it in the kernel headers, but I think since the hardware allows different mapping types, the kernel should take advantage of it.
Regards,
--
Steffen Persvold | Scali AS mailto:[email protected] | http://www.scali.com Tel: (+47) 2262 8950 | Olaf Helsets vei 6 Fax: (+47) 2262 8951 | N0621 Oslo, NORWAY
Quoted 4 lines · Bjorn Helgaas
We've got a mechanism for looking up the supported attributes (
efi_mem_attributes()); what I'd really like is for somebody to hook that up with the mmap infrastructure so that we only allow the supported attributes.
If MAP_WRITECOMBINED attribute isn't supported, is there any harm in returning memory without that attribute?
I'd expect some platform specific support to know when it's required vs a "performance hint". Forcing exact matches is one way to deal with it but pushes the burden to the "application" (may a device driver in this case) to find an exact match.
grant
Quoted 2 lines
Which IA64 hardware doesn't support
_PAGE_MA_WCin the PTE ? My IA64 docs doesn't mention anything about that.
The *processors* all support WC, no problem there. The problem is that the *chipset* may not. The EFI memory descriptor table (from GetMemoryMap()) tells us which attributes are supported for each region of address space.
According to the EFI shell "memmap" command, the i2000 (BigSur) supports WB or UC for memory, and UC for MMIO space. No mention of WC, so we have to assume it's not supported.
HP ZX1 machines report that they support WB for memory and UC for MMIO space. The ZX1 chipset is supposed to support WC for MMIO space, so the fact that EFI doesn't report that looks like a firmware defect.
Quoted 3 lines
Actually I think
_PAGE_MA_WCis only applicable to IO memory the same way as_PAGE_MA_UCis. Can't it be handled in the same way ? (you've already done the fix for UC, right ?)
WC could be used for memory as well as for MMIO space -- an example is for AGP, where drivers like to have main memory buffers mapped with WC, and the AGP engine can do non-coherent DMA from the buffers. That doesn't work on IA64 because we use large TLB pages to map all of main memory with the WB attribute, and there's no easy way to support WC mappings at the same time.
The mmap support for UC currently cheats a little bit. We don't look at the EFI tables; we just use UC whenever we mmap something that isn't main memory (this is in mmap_mem(), BTW).
Bjorn
Quoted 13 lines · Bjorn Helgaas
Quoted 11 lines · Steffen Persvold
Well, without
MAP_WRITECOMBINED, streaming PIO accesses to the PCI bus drops from 330MByte/sec to approx. 100MByte/s using the Dolphin SCI card.This is an rather significant drop which makes the IA64 platform kind of uninteresting for high performance clustering (the Dolphin SCI cards are used for shared memory message passing between nodes).
We can of course hack this in for our usage (in our kernel driver) if you are totally against having it in the kernel headers, but I think since the hardware allows different mapping types, the kernel should take advantage of it.
Quoted 5 lines
I'm really not opposed to having it in the kernel headers. The thing is, we were allowing WC mappings regardless of whether the hardware actually supported them. What we need is some mechanism for looking at the EFI memory map, which tells us what attributes are supported.
Which IA64 hardware doesn't support _PAGE_MA_WC in the PTE ? My IA64 docs doesn't mention anything about that.
Quoted 5 lines
We've got a mechanism for looking up the supported attributes (
efi_mem_attributes()); what I'd really like is for somebody to hook that up with the mmap infrastructure so that we only allow the supported attributes.
Actually I think _PAGE_MA_WC is only applicable to IO memory the same way as _PAGE_MA_UC is. Can't it be handled in the same way ? (you've already done the fix for UC, right ?)
Regards,
Signature · 4 lines
Steffen Persvold | Scali AS mailto:[email protected] | http://www.scali.com Tel: (+47) 2262 8950 | Olaf Helsets vei 6 Fax: (+47) 2262 8951 | N0621 Oslo, NORWAY
Quoted 8 lines · Bjorn Helgaas
Quoted 2 lines
Which IA64 hardware doesn't support
_PAGE_MA_WCin the PTE ? My IA64 docs doesn't mention anything about that.The *processors* all support WC, no problem there. The problem is that the *chipset* may not. The EFI memory descriptor table (from
GetMemoryMap()) tells us which attributes are supported for each region of address space.
Quoted 9 lines
According to the EFI shell "memmap" command, the i2000 (BigSur) supports WB or UC for memory, and UC for MMIO space. No mention of WC, so we have to assume it's not supported.
HP ZX1 machines report that they support WB for memory and UC for MMIO space. The ZX1 chipset is supposed to support WC for MMIO space, so the fact that EFI doesn't report that looks like a firmware defect.
So what does "supported" by the chipset actually mean ? I've tested the SCI cards in both BigSurs and ZX2000 and it works fine on IO memory (roughly 330 MByte on both platforms).
I was in under the impression that WC can be implemented in different sections of the system (processor->memorycontroller, memorycontroller->pci etc), and that the section we control with the WC attribute in the PTE is transactions on the processor->memorycontroller bus (as it is on the IA32 systems with MTRRs). Then the question of wether the memory contoller (chipset) is able to support large bursts of data on the processor->memorycontroller bus comes down to buffer space in the memory controller, and I can't really imagine that this isn't large enough (haven't checked though).
_But_ I agree that if the firmware has the ability to report this (and we can trust the information) we should take advantage of it (sort of the same thing which is done when someone tries to create a WC memory region on IA32, not all processors supports this).
Quoted 4 lines
Quoted 3 lines
Actually I think
_PAGE_MA_WCis only applicable to IO memory the same way as_PAGE_MA_UCis. Can't it be handled in the same way ? (you've already done the fix for UC, right ?)
Quoted 8 lines
WC could be used for memory as well as for MMIO space -- an example is for AGP, where drivers like to have main memory buffers mapped with WC, and the AGP engine can do non-coherent DMA from the buffers. That doesn't work on IA64 because we use large TLB pages to map all of main memory with the WB attribute, and there's no easy way to support WC mappings at the same time.
I can't really see why mapping main memory with WC (processor) should affect the DMA performance of the AGP card. The other way around (mapping the AGP card's memory with WC) makes sense though and I know a lot of drivers that does this.
Quoted 4 lines
The mmap support for UC currently cheats a little bit. We don't look at the EFI tables; we just use UC whenever we mmap something that isn't main memory (this is in
mmap_mem(), BTW).
Some time ago I submitted a patch do David to allow mapping IO pages with WC in kernel space (ioremap()). Back then I remember this discussion about multiple processes having different PTE attributes on the same pages, so I guess this stuff with PTE attributes should be cleaned up (for all archs really).
Regards,
--
Steffen Persvold | Scalable Linux Systems | Try out the world's best mailto:[email protected] | http://www.scali.com | performing MPI implementation: Tel: (+47) 2262 8950 | Olaf Helsets vei 6 | - ScaMPI 1.13.8 - Fax: (+47) 2262 8951 | N0621 Oslo, NORWAY | >320MBytes/s and <4uS latency
Quoted 3 lines
So what does "supported" by the chipset actually mean ? I've tested the SCI cards in both BigSurs and ZX2000 and it works fine on IO memory (roughly 330 MByte on both platforms).
One thing it means is that supported attributes have been considered in the hardware design, tested (hopefully), and if they don't work, the vendor should be interested. If we use unsupported attributes, we're exposing ourselves to the possibility of MCAs and data corruption.
Quoted 2 lines
I can't really see why mapping main memory with WC (processor) should affect the DMA performance of the AGP card.
If the AGP card reads main memory with non-coherent DMA, those transactions don't need to be snooped by the processor, and may not need to appear on the processor bus.
Bjorn
Quoted 9 lines · Bjorn Helgaas
Quoted 3 lines
So what does "supported" by the chipset actually mean ? I've tested the SCI cards in both BigSurs and ZX2000 and it works fine on IO memory (roughly 330 MByte on both platforms).
One thing it means is that supported attributes have been considered in the hardware design, tested (hopefully), and if they don't work, the vendor should be interested. If we use unsupported attributes, we're exposing ourselves to the possibility of MCAs and data corruption.
I agree, I actually have some bad experiences on IA32 platforms where data corruption happened with CPU WC enabled (in MTRRs). Sorry for the fuzz.
Quoted 7 lines
Quoted 2 lines
I can't really see why mapping main memory with WC (processor) should affect the DMA performance of the AGP card.
If the AGP card reads main memory with non-coherent DMA, those transactions don't need to be snooped by the processor, and may not need to appear on the processor bus.
Ok, and then I also see what you mean by "supported by the chipset" because this must be managed by the memory controller and therefore it must know what type of mappings are used by the processor. Kinda complicated though... I thought the IO system didn't use coherence protocols, and it was because of that we needed API functions such as pci_dma_sync_single() and pci_dma_sync_sg() to manually ensure coherence..
Regards,
--
Steffen Persvold | Scalable Linux Systems | Try out the world's best mailto:[email protected] | http://www.scali.com | performing MPI implementation: Tel: (+47) 2262 8950 | Olaf Helsets vei 6 | - ScaMPI 1.13.8 - Fax: (+47) 2262 8951 | N0621 Oslo, NORWAY | >320MBytes/s and <4uS latency
Quoted 3 lines · Steffen Persvold
Kinda complicated though... I thought the IO system didn't use coherence protocols, and it was because of that we needed API functions such as
pci_dma_sync_single()andpci_dma_sync_sg()to manually ensure coherence..
The original problem was some IOMMU's can not support a static translation for all physical memory - ie virt_to_bus() wouldn't work. "manual coherence" works because the design accomodates SW IOTLB ("bounce buffers") model.
Supported HP PARISC and IA64 IOMMU's participate in the coherency protocol.
grant
Quoted 5 lines · Grant Grundler
Quoted 3 lines · Steffen Persvold
Kinda complicated though... I thought the IO system didn't use coherence protocols, and it was because of that we needed API functions such as
pci_dma_sync_single()andpci_dma_sync_sg()to manually ensure coherence..
Quoted 7 lines
The original problem was some IOMMU's can not support a static translation for all physical memory - ie
virt_to_bus()wouldn't work. "manual coherence" works because the design accomodates SW IOTLB ("bounce buffers") model.Supported HP PARISC and IA64 IOMMU's participate in the coherency protocol.
True, but I still can't see how a PTE with the WC attribute (in the CPU MMU) can affect the coherency protocol the IOMMU uses on DMA read transfers. I thought the WC attribute just controlled how the CPU buffered data.
Anyway, although I agree that the availability of the WC feature should be checked with the information from the firware, I don't agree that this should be done now on the 2.4 series. Atleast don't remove WC (or UC) support in any 2.4 kernels before the validation routines are ready and implemented.
Regards,
Signature · 4 lines
Steffen Persvold | Scali AS mailto:[email protected] | http://www.scali.com Tel: (+47) 2262 8950 | Olaf Helsets vei 6 Fax: (+47) 2262 8951 | N0621 Oslo, NORWAY