PATCH: performance problems with swiotlb.c
Quoted 3 lines · Tony
This problem was found and this fix suggested by Dori Eldar here at Intel (I just critiqued it for a while and pointed out some corner cases that needed to be addressed).
Unfortionatly the idea of a software MMU is broken by design. The current DMA API does not allow for one in practice, and, thankfully, there's also no need for one. The linux kernel allows perfectly well for systems without IO MMU and will do the right thing at higher layers, where things CAN be done properly (for example, the software IOMMU cannot sleep to wait for memory and hence panic()'s the kernel in this case, while the higher layers often can either sleep (in the case of the blockdevice layer) or just drop the packet on near-OOM in the case of the network layer.
I'll mail a patch to implement this for ia64 to this list shortly; my current patch is against 2.4.9 and needs forward porting to 2.4.16....
Greetings,
Arjan van de Ven
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line · Arjan van de Ven
Arjan> Unfortionatly the idea of a software MMU is broken by Arjan> design. The current DMA API does not allow for one in Arjan> practice, and, thankfully, there's also no need for one. The Arjan> linux kernel allows perfectly well for systems without IO MMU Arjan> and will do the right thing at higher layers, where things Arjan> CAN be done properly (for example, the software IOMMU cannot Arjan> sleep to wait for memory and hence panic()'s the kernel in Arjan> this case, while the higher layers often can either sleep (in Arjan> the case of the blockdevice layer) or just drop the packet on Arjan> near-OOM in the case of the network layer.
But longer term, the question is whether it is better to support multiple bounce buffer schemes or whether the PCI DMA interface should be enhanced to better handle "out of memory" situations. In my opinion, it would be better to enhance the PCI DMA interface with a throttling/OOM-feedback mechanism.
Note that even with hardware I/O TLB, you can get into such situations. The only real difference is that the software I/O TLB consumes memory primarily as a functiion of the amount of data being buffered, whereas a hardware I/O TLB it's primarily limited by the number of *descriptors* needed.
--david
Quoted 13 lines · David Mosberger
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line · Arjan van de Ven
Arjan> Unfortionatly the idea of a software MMU is broken by Arjan> design. The current DMA API does not allow for one in Arjan> practice, and, thankfully, there's also no need for one. The Arjan> linux kernel allows perfectly well for systems without IO MMU Arjan> and will do the right thing at higher layers, where things Arjan> CAN be done properly (for example, the software IOMMU cannot Arjan> sleep to wait for memory and hence panic()'s the kernel in Arjan> this case, while the higher layers often can either sleep (in Arjan> the case of the blockdevice layer) or just drop the packet on Arjan> near-OOM in the case of the network layer.
Quoted 5 lines
But longer term, the question is whether it is better to support multiple bounce buffer schemes or whether the PCI DMA interface should be enhanced to better handle "out of memory" situations. In my opinion, it would be better to enhance the PCI DMA interface with a throttling/OOM-feedback mechanism.
Fully agreed and this appears to be the plan for 2.5
Quoted 5 lines
Note that even with hardware I/O TLB, you can get into such situations. The only real difference is that the software I/O TLB consumes memory primarily as a functiion of the amount of data being buffered, whereas a hardware I/O TLB it's primarily limited by the number of *descriptors* needed.
yes; I can imagine the 2.5 scenario be "try the hw io mmu and if not, report failure" at which point the higher layer does it's thing (either blocks for resources to become available or to drop the network packet). The "highmem" scenario would be a the 2.4 scenario, and the scenario where you don't have a real io mmu.
Greetings,
Arjan van de Ven