PCI/ATS: Disable ATS when ACS TB is set upstream
PATCH v2 1/1 PCI/ATS: Disable ATS when ACS TB is set upstream
Enabling ATS allows an endpoint to cache translations and initiate DMA requests with Translated addresses, which an upstream bridge with ACS Translation Blocking (PCI_ACS_TB) set will block as ACS Violations. Because pci_enable_acs() preserves existing PCI_ACS_CTRL bits (e.g., from firmware, a kexec kernel, or "pci=config_acs=") without relaxing them, PCI_ACS_TB may remain set on upstream bridges.
Add pci_ats_check_acs() and call it from pci_dma_configure() after pci_enable_acs() to walk upstream bridges and clear dev->ats_cap if any bridge has PCI_ACS_TB enabled, leaving the endpoint to use Untranslated addresses.
Tested by booting a machine with an endpoint advertising the ATS Capability behind a Root Port with PCI_ACS_TB set; verified ATS is disabled and the system boots cleanly.
---
v2 (based on sashiko review):
- Move the upstream ACS TB check from pci_ats_supported() into pci_ats_check_acs() called after pci_enable_acs()
- Use pci_acs_enabled() instead of reading PCI_ACS_CTRL directly
drivers/pci/ats.c | 20 ++++++++++++++++++++ drivers/pci/pci-driver.c | 1 + drivers/pci/pci.h | 2 ++ 3 files changed, 23 insertions(+)
drivers/pci/ats.c +20 -0
@@ -31,6 +31,26 @@ void pci_ats_init(struct pci_dev *dev) dev->ats_cap = pos; } +void pci_ats_check_acs (struct pci_dev *dev) +{ +struct pci_dev *bridge; + +if (!dev->ats_cap || dev->is_virtfn || dev->ats_stu) +return ; + +/* Do not allow ATS if any upstream bridge blocks translation */ +for (bridge =pci_upstream_bridge (dev); bridge; + bridge =pci_upstream_bridge (bridge)) { +if ((bridge->acs_capabilities &PCI_ACS_TB ) && +pci_acs_enabled (bridge,PCI_ACS_TB )) { +pci_warn_once (dev,"ATS disabled: upstream bridge %s has ACS Translation Blocking enabled\n" , +pci_name (bridge)); + dev->ats_cap =0 ; +return ; + } + } +} +/** * pci_ats_supported - check if the device can use ATS * @dev: the PCI device
drivers/pci/pci-driver.c +1 -0
@@ -1713,6 +1713,7 @@ static int pci_dma_configure(struct device *dev)* quirks. */ pci_enable_acs (to_pci_dev (dev)); +pci_ats_check_acs (to_pci_dev (dev));pci_put_host_bridge_device (bridge);
drivers/pci/pci.h +2 -0
@@ -961,9 +961,11 @@ static inline void pcie_walk_rcec(struct pci_dev *rcec,#ifdef CONFIG_PCI_ATS /* Address Translation Service */ void pci_ats_init (struct pci_dev *dev); +void pci_ats_check_acs (struct pci_dev *dev);void pci_restore_ats_state (struct pci_dev *dev);#else static inline void pci_ats_init (struct pci_dev *d) { } +static inline void pci_ats_check_acs (struct pci_dev *d) { }static inline void pci_restore_ats_state (struct pci_dev *dev) { }#endif /* CONFIG_PCI_ATS */
Signature · 1 line
2.56.0.313.g2479322e68-goog
[email protected]
AI reviewSashiko has reviewed this patch and found no issues.
Quoted 12 lines
Enabling ATS allows an endpoint to cache translations and initiate DMA requests with Translated addresses, which an upstream bridge with ACS Translation Blocking (
PCI_ACS_TB) set will block as ACS Violations. Becausepci_enable_acs()preserves existingPCI_ACS_CTRLbits (e.g., from firmware, a kexec kernel, or "pci=config_acs=") without relaxing them,PCI_ACS_TBmay remain set on upstream bridges.Add
pci_ats_check_acs()and call it frompci_dma_configure()afterpci_enable_acs()to walk upstream bridges and cleardev->ats_capif any bridge hasPCI_ACS_TBenabled, leaving the endpoint to use Untranslated addresses.
Quoted 5 lines
Tested by booting a machine with an endpoint advertising the ATS Capability behind a Root Port with
PCI_ACS_TBset; verified ATS is disabled and the system boots cleanly.Signed-off-by Danny Gonzalez <[email protected]>
Sashiko has reviewed this patch and found no issues.
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=1
[+cc other folks interested in ATS]
Quoted 45 lines · Danny Gonzalez
Enabling ATS allows an endpoint to cache translations and initiate DMA requests with Translated addresses, which an upstream bridge with ACS Translation Blocking (
PCI_ACS_TB) set will block as ACS Violations. Becausepci_enable_acs()preserves existingPCI_ACS_CTRLbits (e.g., from firmware, a kexec kernel, or "pci=config_acs=") without relaxing them,PCI_ACS_TBmay remain set on upstream bridges.Add
pci_ats_check_acs()and call it frompci_dma_configure()afterpci_enable_acs()to walk upstream bridges and cleardev->ats_capif any bridge hasPCI_ACS_TBenabled, leaving the endpoint to use Untranslated addresses.Tested by booting a machine with an endpoint advertising the ATS Capability behind a Root Port with
PCI_ACS_TBset; verified ATS is disabled and the system boots cleanly.Signed-off-by Danny Gonzalez <[email protected]>---
v2 (based on sashiko review):
- Move the upstream ACS TB check frompci_ats_supported()intopci_ats_check_acs()called afterpci_enable_acs()
- Usepci_acs_enabled()instead of readingPCI_ACS_CTRLdirectlydrivers/pci/ats.c | 20 ++++++++++++++++++++ drivers/pci/pci-driver.c | 1 + drivers/pci/pci.h | 2 ++ 3 files changed, 23 insertions(+)drivers/pci/ats.c +8
-0@@ -31,6 +31,26 @@ void pci_ats_init(struct pci_dev *dev) dev->ats_cap = pos; } +void pci_ats_check_acs (struct pci_dev *dev) +{ +struct pci_dev *bridge; + +if (!dev->ats_cap || dev->is_virtfn || dev->ats_stu) +return ; + +/* Do not allow ATS if any upstream bridge blocks translation */
drivers/pci/ats.c void pci_ats_init(struct pci_dev *dev) · line 51
Quoted 10 lines
drivers/pci/ats.c +10
-0+for (bridge =pci_upstream_bridge (dev); bridge; + bridge =pci_upstream_bridge (bridge)) { +if ((bridge->acs_capabilities &PCI_ACS_TB ) && +pci_acs_enabled (bridge,PCI_ACS_TB )) { +pci_warn_once (dev,"ATS disabled: upstream bridge %s has ACS Translation Blocking enabled\n" , +pci_name (bridge)); + dev->ats_cap =0 ; +return ; + } + }
This works by making pci_ats_supported() return "false". Can we do this checking inside pci_ats_supported() instead? Then we wouldn't need a new function, and I think it would be easier to understand the control flow that way.
Quoted 35 lines
drivers/pci/ats.c +2
-0+} +/** * pci_ats_supported - check if the device can use ATS * @dev: the PCI device drivers/pci/pci-driver.c +1
-0@@ -1713,6 +1713,7 @@ static int pci_dma_configure(struct device *dev)* quirks. */ pci_enable_acs (to_pci_dev (dev)); +pci_ats_check_acs (to_pci_dev (dev));pci_put_host_bridge_device (bridge);drivers/pci/pci.h +2
-0@@ -961,9 +961,11 @@ static inline void pcie_walk_rcec(struct pci_dev *rcec,#ifdef CONFIG_PCI_ATS /* Address Translation Service */ void pci_ats_init (struct pci_dev *dev); +void pci_ats_check_acs (struct pci_dev *dev);void pci_restore_ats_state (struct pci_dev *dev);#else static inline void pci_ats_init (struct pci_dev *d) { } +static inline void pci_ats_check_acs (struct pci_dev *d) { }static inline void pci_restore_ats_state (struct pci_dev *dev) { }#endif /* CONFIG_PCI_ATS */ Signature · 1 line
2.56.0.313.g2479322e68-goog
Quoted 40 lines · Bjorn Helgaas
[+cc other folks interested in ATS]
Quoted 37 lines · Danny Gonzalez
Enabling ATS allows an endpoint to cache translations and initiate DMA requests with Translated addresses, which an upstream bridge with ACS Translation Blocking (
PCI_ACS_TB) set will block as ACS Violations. Becausepci_enable_acs()preserves existingPCI_ACS_CTRLbits (e.g., from firmware, a kexec kernel, or "pci=config_acs=") without relaxing them,PCI_ACS_TBmay remain set on upstream bridges.Add
pci_ats_check_acs()and call it frompci_dma_configure()afterpci_enable_acs()to walk upstream bridges and cleardev->ats_capif any bridge hasPCI_ACS_TBenabled, leaving the endpoint to use Untranslated addresses.Tested by booting a machine with an endpoint advertising the ATS Capability behind a Root Port with
PCI_ACS_TBset; verified ATS is disabled and the system boots cleanly.Signed-off-by Danny Gonzalez <[email protected]>---
v2 (based on sashiko review):
- Move the upstream ACS TB check frompci_ats_supported()intopci_ats_check_acs()called afterpci_enable_acs()
- Usepci_acs_enabled()instead of readingPCI_ACS_CTRLdirectlydrivers/pci/ats.c | 20 ++++++++++++++++++++ drivers/pci/pci-driver.c | 1 + drivers/pci/pci.h | 2 ++ 3 files changed, 23 insertions(+)drivers/pci/ats.c +0
-0@@ -31,6 +31,26 @@ void pci_ats_init(struct pci_dev *dev) dev->ats_cap = pos; }
Quoted 16 lines
Quoted 6 lines
drivers/pci/ats.c +6
-0+void pci_ats_check_acs (struct pci_dev *dev) +{ +struct pci_dev *bridge; + +if (!dev->ats_cap || dev->is_virtfn || dev->ats_stu) +return ;drivers/pci/ats.c void pci_ats_init(struct pci_dev *dev) · line 49
Quoted 10 lines
drivers/pci/ats.c +10
-0+ +/* Do not allow ATS if any upstream bridge blocks translation */ +for (bridge =pci_upstream_bridge (dev); bridge; + bridge =pci_upstream_bridge (bridge)) { +if ((bridge->acs_capabilities &PCI_ACS_TB ) && +pci_acs_enabled (bridge,PCI_ACS_TB )) { +pci_warn_once (dev,"ATS disabled: upstream bridge %s has ACS Translation Blocking enabled\n" , +pci_name (bridge)); + dev->ats_cap =0 ; +return ;
Isn't this pci_acs_path_enabled() ?
Quoted 4 lines
This works by making
pci_ats_supported()return "false". Can we do this checking insidepci_ats_supported()instead? Then we wouldn't need a new function, and I think it would be easier to understand the control flow that way.
Yeah, and I don't think this is great:
Quoted 6 lines
drivers/pci/ats.c static int pci_dma_configure(struct device *dev) · line 1717
Quoted 6 lines
drivers/pci/ats.c +1
-0@@ -1713,6 +1713,7 @@ static int pci_dma_configure(struct device *dev)* quirks. */ pci_enable_acs (to_pci_dev (dev)); +pci_ats_check_acs (to_pci_dev (dev));pci_put_host_bridge_device (bridge);
We shouldn't have to leak these details outside the PCI core..
The idea seems sound at least.
Jason