arm64: Enable PCI write-combine resources under sysfs
PATCH v2 1/1 arm64: Enable PCI write-combine resources under sysfs
This change exposes write-combine mappings under sysfs for prefetchable PCI resources on arm64.
Originally, the usage of "write combine" here was driven by the x86 definition of write combine. This definition is specific to x86 and does not generalize to other architectures. However, the usage of WC has mutated to "write combine" semantics, which is implemented differently on each arch.
Generally, prefetchable BARs are accepted to allow speculative accesses, write combining, and re-ordering-- from the PCI perspective, this means there are no read side effects. (This contradicts the PCI spec which allows prefetchable BARs to have read side effects, but this definition is ill-advised as it is impossible to meet.) On x86, prefetchable BARs are mapped as WC as originally defined (with some conditionals on arch features). On arm64, WC is taken to mean normal non-cacheable memory.
In practice, write combine semantics are used to minimize write operations. A common usage of this is minimizing PCI TLPs which can significantly improve performance with PCI devices. In order to provide the same benefits to userspace, we need to allow userspace to map prefetchable BARs with write combine semantics. The resourceX_wc mapping is used today by userspace programs and libraries.
While this model is flawed as "write combine" is very ill-defined, it is already used by multiple non-x86 archs to expose write combine semantics to user space. We enable this on arm64 to give userspace on arm64 an equivalent mechanism for utilizing write combining with PCI devices.
---
Changes in v2:
- Rewrote the commit message.
arch/arm64/include/asm/pci.h | 1 + 1 file changed, 1 insertion(+)
arch/arm64/include/asm/pci.h +1 -0
@@ -17,6 +17,7 @@#define pcibios_assign_all_busses () \ (pci_has_flag(PCI_REASSIGN_ALL_BUS)) +#define arch_can_pci_mmap_wc () 1#define ARCH_GENERIC_PCI_MMAP_RESOURCE 1extern int isa_dma_bridge_buggy;
Signature · 1 line
2.23.3
Quoted 38 lines · Clint Sbisa
This change exposes write-combine mappings under sysfs for prefetchable PCI resources on arm64.
Originally, the usage of "write combine" here was driven by the x86 definition of write combine. This definition is specific to x86 and does not generalize to other architectures. However, the usage of WC has mutated to "write combine" semantics, which is implemented differently on each arch.
Generally, prefetchable BARs are accepted to allow speculative accesses, write combining, and re-ordering-- from the PCI perspective, this means there are no read side effects. (This contradicts the PCI spec which allows prefetchable BARs to have read side effects, but this definition is ill-advised as it is impossible to meet.) On x86, prefetchable BARs are mapped as WC as originally defined (with some conditionals on arch features). On arm64, WC is taken to mean normal non-cacheable memory.
In practice, write combine semantics are used to minimize write operations. A common usage of this is minimizing PCI TLPs which can significantly improve performance with PCI devices. In order to provide the same benefits to userspace, we need to allow userspace to map prefetchable BARs with write combine semantics. The
resourceX_wcmapping is used today by userspace programs and libraries.While this model is flawed as "write combine" is very ill-defined, it is already used by multiple non-x86 archs to expose write combine semantics to user space. We enable this on arm64 to give userspace on arm64 an equivalent mechanism for utilizing write combining with PCI devices.
Cc Benjamin Herrenschmidt <[email protected]>Cc Bjorn Helgaas <[email protected]>Cc Catalin Marinas <[email protected]>Cc Jason Gunthorpe <[email protected]>Cc Lorenzo Pieralisi <[email protected]>Cc Will Deacon <[email protected]>Signed-off-by Clint Sbisa <[email protected]>
Quoted 6 lines
---
Changes in v2:
- Rewrote the commit message.arch/arm64/include/asm/pci.h | 1 + 1 file changed, 1 insertion(+)
It would be great if we can add a link to the thread (sorry I forgot to tell you) for future reference:
With that:
Quoted 15 lines
arch/arm64/include/asm/pci.h +1
-0@@ -17,6 +17,7 @@#define pcibios_assign_all_busses () \ (pci_has_flag(PCI_REASSIGN_ALL_BUS)) +#define arch_can_pci_mmap_wc () 1#define ARCH_GENERIC_PCI_MMAP_RESOURCE 1extern int isa_dma_bridge_buggy;Signature · 1 line
2.23.3
Quoted 31 lines · Clint Sbisa
This change exposes write-combine mappings under sysfs for prefetchable PCI resources on arm64.
Originally, the usage of "write combine" here was driven by the x86 definition of write combine. This definition is specific to x86 and does not generalize to other architectures. However, the usage of WC has mutated to "write combine" semantics, which is implemented differently on each arch.
Generally, prefetchable BARs are accepted to allow speculative accesses, write combining, and re-ordering-- from the PCI perspective, this means there are no read side effects. (This contradicts the PCI spec which allows prefetchable BARs to have read side effects, but this definition is ill-advised as it is impossible to meet.) On x86, prefetchable BARs are mapped as WC as originally defined (with some conditionals on arch features). On arm64, WC is taken to mean normal non-cacheable memory.
In practice, write combine semantics are used to minimize write operations. A common usage of this is minimizing PCI TLPs which can significantly improve performance with PCI devices. In order to provide the same benefits to userspace, we need to allow userspace to map prefetchable BARs with write combine semantics. The
resourceX_wcmapping is used today by userspace programs and libraries.While this model is flawed as "write combine" is very ill-defined, it is already used by multiple non-x86 archs to expose write combine semantics to user space. We enable this on arm64 to give userspace on arm64 an equivalent mechanism for utilizing write combining with PCI devices.
Quoted 7 lines
Cc Benjamin Herrenschmidt <[email protected]>Cc Bjorn Helgaas <[email protected]>Cc Catalin Marinas <[email protected]>Cc Jason Gunthorpe <[email protected]>Cc Lorenzo Pieralisi <[email protected]>Cc Will Deacon <[email protected]>Signed-off-by Clint Sbisa <[email protected]>
Ard Biesheuvel
Acked-byQuoted 41 lines · Catalin Marinas
Quoted 31 lines · Clint Sbisa
This change exposes write-combine mappings under sysfs for prefetchable PCI resources on arm64.
Originally, the usage of "write combine" here was driven by the x86 definition of write combine. This definition is specific to x86 and does not generalize to other architectures. However, the usage of WC has mutated to "write combine" semantics, which is implemented differently on each arch.
Generally, prefetchable BARs are accepted to allow speculative accesses, write combining, and re-ordering-- from the PCI perspective, this means there are no read side effects. (This contradicts the PCI spec which allows prefetchable BARs to have read side effects, but this definition is ill-advised as it is impossible to meet.) On x86, prefetchable BARs are mapped as WC as originally defined (with some conditionals on arch features). On arm64, WC is taken to mean normal non-cacheable memory.
In practice, write combine semantics are used to minimize write operations. A common usage of this is minimizing PCI TLPs which can significantly improve performance with PCI devices. In order to provide the same benefits to userspace, we need to allow userspace to map prefetchable BARs with write combine semantics. The
resourceX_wcmapping is used today by userspace programs and libraries.While this model is flawed as "write combine" is very ill-defined, it is already used by multiple non-x86 archs to expose write combine semantics to user space. We enable this on arm64 to give userspace on arm64 an equivalent mechanism for utilizing write combining with PCI devices.
Quoted 7 lines
Cc Benjamin Herrenschmidt <[email protected]>Cc Bjorn Helgaas <[email protected]>Cc Catalin Marinas <[email protected]>Cc Jason Gunthorpe <[email protected]>Cc Lorenzo Pieralisi <[email protected]>Cc Will Deacon <[email protected]>Signed-off-by Clint Sbisa <[email protected]>
Quoted 2 lines
Acked-by Catalin Marinas <[email protected]>
Quoted 10 lines · Clint Sbisa
This change exposes write-combine mappings under sysfs for prefetchable PCI resources on arm64.
Originally, the usage of "write combine" here was driven by the x86 definition of write combine. This definition is specific to x86 and does not generalize to other architectures. However, the usage of WC has mutated to "write combine" semantics, which is implemented differently on each arch.
[...]
Applied to arm64 (for-next/pci), thanks!
[1/1] arm64: Enable PCI write-combine resources under sysfs
https://git.kernel.org/arm64/c/5fd39dc22027
Cheers,
Signature · 5 lines
Will https://fixes.arm64.dev https://next.arm64.dev https://will.arm64.dev