Intel vs AMD x86-64
Quoted 11 lines · David S. Miller
Quoted 4 lines · Linus Torvalds
In fact, I _think_ you could actually use the AGP bridge as a strange IOMMU. Of course, right now their AGP bridges are all 32-bit limited anyway, but the point being that they at least in theory would seem to have the capability to do this.
Ok, I see. In fact, I remember some vague notion that the AGP bridge on the Athlon's could technically be used as a full-on IOMMU, especially since it was all derived from Alpha PCI chipsets which did use things this way.
This is exactly the way it works. The AGP bridge is a replicated per-CPU thing, along with the memory. Good boards have a BIOS option marked "Linux only" that lets you choose an IO-MMU window size ranging from 32 MB to 2 GB. Direct your 32-bit PCI DMA into that window and you get an IO-MMU.
Memory can be interleaved across the CPUs or not, on 4 kB boundries. It's a BIOS option, though some insane code could be written to change the setting. Setting up only one of the IO-MMUs would be neat.
Quoted 1 line · Albert Cahalan
Setting up only one of the IO-MMUs would be neat.
AGPv3 standard mandates that you MUST keep all GARTs coherent, so this isn't possible. The amd64 GART driver goes to great lengths to make sure it does update the northbridges on every CPU whenever something changes.
Dave
Quoted 7 lines · Dave Jones
Quoted 1 line · Albert Cahalan
Setting up only one of the IO-MMUs would be neat.
AGPv3 standard mandates that you MUST keep all GARTs coherent, so this isn't possible. The amd64 GART driver goes to great lengths to make sure it does update the northbridges on every CPU whenever something changes.
Of course. That's the easy way; you won't need to worry about memory interleave or out-of-bounds prefetch if you keep everything coherent.
I'm just saying it would be neat, and potentially useful, to intentionally violate this. Of greatest interest would be the 2-way Opteron boards that only have RAM connected to the CPU closest to PCI. The sidecar CPU :-) could be ignored.
Quoted 8 lines · Albert Cahalan
Quoted 3 lines
so this isn't possible. The amd64 GART driver goes to great lengths to make sure it does update the northbridges on every CPU whenever something changes.
Of course. That's the easy way; you won't need to worry about memory interleave or out-of-bounds prefetch if you keep everything coherent.
Quoted 5 lines
I'm just saying it would be neat, and potentially useful, to intentionally violate this. Of greatest interest would be the 2-way Opteron boards that only have RAM connected to the CPU closest to PCI. The sidecar CPU :-) could be ignored.
Why on earth would you want to do that ?
It wouldn't buy you anything at all other than a world of pain.
Dave
Quoted 10 lines · Dave Jones
Quoted 8 lines · Albert Cahalan
Quoted 3 lines
so this isn't possible. The amd64 GART driver goes to great lengths to make sure it does update the northbridges on every CPU whenever something changes.
Of course. That's the easy way; you won't need to worry about memory interleave or out-of-bounds prefetch if you keep everything coherent.
Quoted 8 lines
Quoted 5 lines
I'm just saying it would be neat, and potentially useful, to intentionally violate this. Of greatest interest would be the 2-way Opteron boards that only have RAM connected to the CPU closest to PCI. The sidecar CPU :-) could be ignored.
Why on earth would you want to do that ?
It wouldn't buy you anything at all other than a world of pain.
faster IO-MMU operations
Quoted 8 lines · Albert Cahalan
Quoted 7 lines
Quoted 5 lines
I'm just saying it would be neat, and potentially useful, to intentionally violate this. Of greatest interest would be the 2-way Opteron boards that only have RAM connected to the CPU closest to PCI. The sidecar CPU :-) could be ignored.
Why on earth would you want to do that ?
It wouldn't buy you anything at all other than a world of pain.faster IO-MMU operations
huh ? You're talking about saving a handful of pci config space writes per every insertion/removal of entries in that GATT, which need a cpu cache flush anyway.
Dave