ACPI/IORT: Honor Root Complex PASID descriptors on SMMUv3
Subsystems (5)
cover ACPI/IORT: Honor Root Complex PASID descriptors on SMMUv3
Hi,
This series enhances Linux to read and honor the Root Complex (RC) PASID descriptors introduced by IORT Issue E.c (ARM DEN 0049E.c, January 2022, RC node revision 4):
- PASID Capabilities at byte offset 33, bits[4:0] = Max PASID Width.
- Flags at byte offset 36, bit 0 = "Root Complex supports PASID".
Both fields have been spec'd for over four years but are not consumed anywhere in the tree. Worse, include/acpi/actbl2.h only covers the PASID Capabilities field - the trailing "u8 reserved[]" flexible array stops just before the Flags field at offset 36, so even firmware that populates it cannot be read today. The existing ACPI_IORT_PRI_SUPPORTED, ACPI_IORT_PASID_FWD_SUPPORTED and ACPI_IORT_PASID_MAX_WIDTH macros are dead code, and ACPICA upstream has the same gap.
Today arm_smmu_enable_pasid() calls pci_enable_pasid() for any PCI master with a PASID capability, regardless of whether the upstream RC actually supports PASID and regardless of the RC's declared Max PASID Width. On a platform whose RC cannot forward PASID to the SMMU, this enables the PCIe PASID capability unnecessarily on the endpoint (the traffic then gets dropped upstream) and sets a non-zero master->ssid_bits, which complicates teardown ordering with ATS (see the comment block in arm_smmu_probe_device() for that ordering constraint).
The series fixes this in three bite-sized patches, modeled on the CANWBS support from commit 807404d66fcf ("ACPI/IORT: Support CANWBS memory access flag"):
Patch 1 (ACPICA): extend struct acpi_iort_root_complex to cover the
offset-36 Flags field and add the ACPI_IORT_RC_PASID_SUPPORTED mask. With #pragma pack(1) already in effect for actbl2.h, this lands the new field at the spec-mandated absolute offset 36.
Patch 2 (ACPI/IORT): add three static helpers that read the new RC
PASID fields (guarded by node->revision >= 4 where appropriate), set two new iommu_fwspec flags from iort_iommu_configure_id(), and expose two device-facing wrappers so IOMMU drivers can clamp endpoint PASID widths without walking the IORT themselves. Stubs for !CONFIG_ACPI_IORT live in include/linux/acpi_iort.h.
IOMMU_FWSPEC_PCI_RC_PASID (bit 2): RC declares PASID support
IOMMU_FWSPEC_PCI_RC_PASID_FWD (bit 3): RC forwards PASID on
translated transactions
Patch 3 (arm-smmu-v3): consume the new fwspec flag and the new
max-width helper in arm_smmu_enable_pasid(). If the RC explicitly declares no PASID support, skip pci_enable_pasid() silently. Otherwise clamp num_pasids by min(endpoint cap, 1 << RC width) and only proceed when the effective count is > 1.
Policy: all new gating is conditional on iort_pci_rc_pasid_max_width_known(), i.e. the presence of an IORT RC node at revision >= 4. Platforms booting with older IORT firmware (RC node revision 0..3) see no behavioural change and continue to enable PASID purely on the basis of the endpoint capability. This keeps the series safe for platforms that ship pre-E.c tables.
Out of scope intentionally:
- Wiring the new fwspec flags into Intel/AMD IOMMU drivers - IORT is ARM-only.
- Consuming IOMMU_FWSPEC_PCI_RC_PASID_FWD in arm-smmu-v3 beyond the existing CD/STE programming. The fwspec flag is plumbed now so future work (e.g. gating PASID forwarding hints in STE.S1DSS) has the information available.
- Exposing a "pasid-num-bits" software node property for PCI devices, parallel to what iort_named_component_init() already does for named components. The direct helper is simpler and avoids creating a software node per RC.
Thanks,
Vidya Sagar
Vidya Sagar (3):
ACPICA: IORT: Add Root Complex PASID Flags field ACPI/IORT: Plumb Root Complex PASID descriptors into iommu_fwspec iommu/arm-smmu-v3: Honor IORT Root Complex PASID descriptors
drivers/acpi/arm64/iort.c | 80 +++++++++++++++++++++ drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c | 26 ++++++- include/acpi/actbl2.h | 6 +- include/linux/acpi_iort.h | 6 ++ include/linux/iommu.h | 4 ++ 5 files changed, 118 insertions(+), 4 deletions(-)
Signature · 1 line
2.25.1
PATCH 1/3 ACPICA: IORT: Add Root Complex PASID Flags field
The IORT spec, Issue E.c (ARM DEN 0049E.c, January 2022), bumps the Root Complex Node to revision 4 and adds two PASID descriptors:
- PASID Capabilities at byte offset 33 (2 bytes), bits[4:0] of which report the Max PASID Width supported by the Root Complex.
- Flags at byte offset 36 (4 bytes), bit 0 of which reports whether the Root Complex itself supports PASID. This is distinct from the existing ATS Attribute bit 2 (at offset 24) that only reports whether the RC forwards PASID information on translated transactions.
The ACPICA struct in include/acpi/actbl2.h was updated for the E.c PASID Capabilities descriptor (offset 33) but stops short with a trailing 'u8 reserved[]' flexible array, so the new Flags field at offset 36 is unreachable and the existing ACPI_IORT_PASID_* definitions have no consumer.
Replace the trailing flexible array with a fixed 'u8 reserved[1]' followed by 'u32 flags' so the struct fully covers RC node revision 4, and add the ACPI_IORT_RC_PASID_SUPPORTED mask for bit 0 of the new field. With #pragma pack(1) in effect for actbl2.h, this lands the new field at the spec-mandated absolute offset 36.
--- include/acpi/actbl2.h | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-)
include/acpi/actbl2.h +5 -1
@@ -748,7 +748,8 @@ struct acpi_iort_root_complex {u32 pci_segment_number;u8 memory_address_limit;/* Memory address size limit */ u16 pasid_capabilities;/* PASID Capabilities */ -u8 reserved[];/* Reserved, must be zero */ +u8 reserved[1 ];/* Reserved, must be zero */ +u32 flags;/* Flags (IORT E.c, RC node revision >= 4) */ };/* Masks for ats_attribute field above */
@@ -760,6 +761,9 @@ struct acpi_iort_root_complex {/* Masks for pasid_capabilities field above */ #define ACPI_IORT_PASID_MAX_WIDTH (0x1F)/* Bits 0-4 */ +/* Masks for flags field above */ +#define ACPI_IORT_RC_PASID_SUPPORTED (1)/* The root complex PASID support */ +struct acpi_iort_smmu {u64 base_address;/* SMMU base address */ u64 span;/* Length of memory range */
Signature · 1 line
2.25.1
Re: [PATCH V1 1/3] ACPICA: IORT: Add Root Complex PASID Flags field
Quoted 23 lines · Vidya Sagar
The IORT spec, Issue
E.c(ARM DEN0049E.c, January 2022), bumps the Root Complex Node to revision 4 and adds two PASID descriptors:- PASID Capabilities at byte offset 33 (2 bytes), bits[4:0] of which report the Max PASID Width supported by the Root Complex.
- Flags at byte offset 36 (4 bytes), bit 0 of which reports whether the Root Complex itself supports PASID. This is distinct from the existing ATS Attribute bit 2 (at offset 24) that only reports whether the RC forwards PASID information on translated transactions.The ACPICA struct in
include/acpi/actbl2.hwas updated for theE.cPASID Capabilities descriptor (offset 33) but stops short with a trailing 'u8 reserved[]' flexible array, so the new Flags field at offset 36 is unreachable and the existing ACPI_IORT_PASID_* definitions have no consumer.Replace the trailing flexible array with a fixed 'u8 reserved[1]' followed by 'u32 flags' so the struct fully covers RC node revision 4, and add the
ACPI_IORT_RC_PASID_SUPPORTEDmask for bit 0 of the new field. With #pragma pack(1) in effect foractbl2.h, this lands the new field at the spec-mandated absolute offset 36.
Quoted 4 lines
Signed-off-by Vidya Sagar <[email protected]>--- include/acpi/actbl2.h | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-)
There is a whole process to change this file, I went through it once for canwbs..
https://lore.kernel.org/acpica-devel/[email protected]/
You have to make a pull request here:
https://github.com/acpica/acpica/pulls
And then use the
generate/linux/make-patches.sh -u <commit-id>
Script to make this patch for linux in the perscribed format.
I think, something like that
Jason
Hanjun Guo
maintainerRe: [PATCH V1 1/3] ACPICA: IORT: Add Root Complex PASID Flags field
Quoted 29 lines · Jason Gunthorpe
Quoted 23 lines · Vidya Sagar
The IORT spec, Issue
E.c(ARM DEN0049E.c, January 2022), bumps the Root Complex Node to revision 4 and adds two PASID descriptors:- PASID Capabilities at byte offset 33 (2 bytes), bits[4:0] of which report the Max PASID Width supported by the Root Complex.
- Flags at byte offset 36 (4 bytes), bit 0 of which reports whether the Root Complex itself supports PASID. This is distinct from the existing ATS Attribute bit 2 (at offset 24) that only reports whether the RC forwards PASID information on translated transactions.The ACPICA struct in
include/acpi/actbl2.hwas updated for theE.cPASID Capabilities descriptor (offset 33) but stops short with a trailing 'u8 reserved[]' flexible array, so the new Flags field at offset 36 is unreachable and the existing ACPI_IORT_PASID_* definitions have no consumer.Replace the trailing flexible array with a fixed 'u8 reserved[1]' followed by 'u32 flags' so the struct fully covers RC node revision 4, and add the
ACPI_IORT_RC_PASID_SUPPORTEDmask for bit 0 of the new field. With #pragma pack(1) in effect foractbl2.h, this lands the new field at the spec-mandated absolute offset 36.Quoted 4 lines
Signed-off-by Vidya Sagar <[email protected]>--- include/acpi/actbl2.h | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-)
Quoted 8 lines
There is a whole process to change this file, I went through it once for canwbs..
https://lore.kernel.org/acpica-devel/[email protected]/
You have to make a pull request here:
Yes, please get this patch merged in ACPICA first.
Thanks
Hanjun
PATCH 2/3 ACPI/IORT: Plumb Root Complex PASID descriptors into iommu_fwspec
In reply to Vidya SagarThe IORT spec, Issue E.c (ARM DEN 0049E.c, January 2022), gives software two ways to learn about Root Complex PASID capability:
- the Flags field at byte offset 36 (RC node revision >= 4), bit 0 of which says whether the Root Complex itself supports PASID, and
- bit 2 of the long-standing ATS Attribute field at offset 24, which says whether the Root Complex forwards PASID information on translated transactions to the SMMU.
Neither is consumed by Linux today. Modeled after the CANWBS support in commit 807404d66fcf ("ACPI/IORT: Support CANWBS memory access flag"), expose both bits via the existing iommu_fwspec mechanism so IOMMU drivers can consult them at device probe time:
- IOMMU_FWSPEC_PCI_RC_PASID (bit 2): RC declares PASID support
- IOMMU_FWSPEC_PCI_RC_PASID_FWD (bit 3): RC forwards PASID
Add three new static helpers in iort.c that read the corresponding fields, with iort_pci_rc_supports_pasid() and the new iort_pci_rc_pasid_max_width() helper guarded by node->revision >= 4 because their backing storage was only added in E.c. The PASID forwarding bit lives in the older ats_attribute field and needs no guard. Set the new fwspec flags from iort_iommu_configure_id().
In addition, expose two device-facing wrappers, iort_pci_rc_pasid_max_width_known() and _for_dev(), so IOMMU drivers can clamp endpoint PASID widths by what the Root Complex can actually carry without having to walk the IORT themselves. Provide stubs in include/linux/acpi_iort.h for !CONFIG_ACPI_IORT.
--- drivers/acpi/arm64/iort.c | 80 +++++++++++++++++++++++++++++++++++++++ include/linux/acpi_iort.h | 6 +++ include/linux/iommu.h | 4 ++ 3 files changed, 90 insertions(+)
drivers/acpi/arm64/iort.c +80 -0
@@ -1352,6 +1352,36 @@ static bool iort_pci_rc_supports_canwbs(struct acpi_iort_node *node)return memory_access->memory_flags &ACPI_IORT_MF_CANWBS ; } +static bool iort_pci_rc_supports_pasid (struct acpi_iort_node *node) +{ +struct acpi_iort_root_complex *pci_rc; + +if (node->revision <4 ) +return false; + + pci_rc = (struct acpi_iort_root_complex *)node->node_data; +return pci_rc->flags &ACPI_IORT_RC_PASID_SUPPORTED ; +} + +static bool iort_pci_rc_supports_pasid_fwd (struct acpi_iort_node *node) +{ +struct acpi_iort_root_complex *pci_rc; + + pci_rc = (struct acpi_iort_root_complex *)node->node_data; +return pci_rc->ats_attribute &ACPI_IORT_PASID_FWD_SUPPORTED ; +} + +static int iort_pci_rc_pasid_max_width (struct acpi_iort_node *node) +{ +struct acpi_iort_root_complex *pci_rc; + +if (node->revision <4 ) +return -ENODEV ; + + pci_rc = (struct acpi_iort_root_complex *)node->node_data; +return FIELD_GET (ACPI_IORT_PASID_MAX_WIDTH , pci_rc->pasid_capabilities); +} +static int iort_iommu_xlate (struct device *dev,struct acpi_iort_node *node,u32 streamid) {
@@ -1471,6 +1501,10 @@ int iort_iommu_configure_id(struct device *dev, const u32 *id_in) fwspec->flags |=IOMMU_FWSPEC_PCI_RC_ATS ;if (fwspec &&iort_pci_rc_supports_canwbs (node)) fwspec->flags |=IOMMU_FWSPEC_PCI_RC_CANWBS ; +if (fwspec &&iort_pci_rc_supports_pasid (node)) + fwspec->flags |=IOMMU_FWSPEC_PCI_RC_PASID ; +if (fwspec &&iort_pci_rc_supports_pasid_fwd (node)) + fwspec->flags |=IOMMU_FWSPEC_PCI_RC_PASID_FWD ; }else { node =iort_scan_node (ACPI_IORT_NODE_NAMED_COMPONENT , iort_match_node_callback, dev);
@@ -1556,6 +1590,52 @@ int iort_dma_get_ranges(struct device *dev, u64 *limit)return nc_dma_get_range (dev, limit); } +static struct acpi_iort_node *iort_pci_rc_node_for_dev (struct device *dev) +{ +struct pci_bus *pbus; + +if (!dev_is_pci (dev)) +return NULL ; + + pbus =to_pci_dev (dev)->bus; +return iort_scan_node (ACPI_IORT_NODE_PCI_ROOT_COMPLEX , + iort_match_node_callback, &pbus->dev); +} + +/** +* iort_pci_rc_pasid_max_width_known() - Whether IORT firmware describes the +* Root Complex PASID width for the given PCI device. +* @dev: PCI device to lookup +* +* Returns true iff a matching IORT Root Complex node exists and has revision +* >= 4 (IORT spec E.c), i.e. the PASID Capabilities descriptor is present. +*/ +bool iort_pci_rc_pasid_max_width_known (struct device *dev) +{ +struct acpi_iort_node *node =iort_pci_rc_node_for_dev (dev); + +return node && node->revision >=4 ; +} + +/** +* iort_pci_rc_pasid_max_width_for_dev() - Look up the Root Complex Max PASID +* Width for the given PCI device. +* @dev: PCI device to lookup +* +* Returns the Max PASID Width (bits[4:0] of PASID Capabilities) declared by +* the Root Complex node in IORT firmware, or a negative errno when the field +* is not present (RC node revision < 4) or the device is not PCI. +*/ +int iort_pci_rc_pasid_max_width_for_dev (struct device *dev) +{ +struct acpi_iort_node *node =iort_pci_rc_node_for_dev (dev); + +if (!node) +return -ENODEV ; + +return iort_pci_rc_pasid_max_width (node); +} +static void __init acpi_iort_register_irq (int hwirq,const char *name,int trigger,struct resource *res)
include/linux/acpi_iort.h +6 -0
@@ -45,6 +45,8 @@ void iort_put_rmr_sids(struct fwnode_handle *iommu_fwnode,int iort_dma_get_ranges (struct device *dev,u64 *limit);int iort_iommu_configure_id (struct device *dev,const u32 *id_in);void iort_iommu_get_resv_regions (struct device *dev,struct list_head *head); +bool iort_pci_rc_pasid_max_width_known (struct device *dev); +int iort_pci_rc_pasid_max_width_for_dev (struct device *dev);phys_addr_t acpi_iort_dma_get_max_cpu_address (void );#else static inline u32 iort_msi_map_id (struct device *dev,u32 id)
@@ -71,6 +73,10 @@ static inline int iort_iommu_configure_id(struct device *dev, const u32 *id_in)static inline void iort_iommu_get_resv_regions (struct device *dev,struct list_head *head) { } +static inline bool iort_pci_rc_pasid_max_width_known (struct device *dev) +{return false; } +static inline int iort_pci_rc_pasid_max_width_for_dev (struct device *dev) +{return -ENODEV ; }static inline phys_addr_t acpi_iort_dma_get_max_cpu_address (void ) {return PHYS_ADDR_MAX ; }
include/linux/iommu.h +4 -0
@@ -1118,6 +1118,10 @@ struct iommu_fwspec {#define IOMMU_FWSPEC_PCI_RC_ATS (1 << 0)/* CANWBS is supported */ #define IOMMU_FWSPEC_PCI_RC_CANWBS (1 << 1) +/* Root complex declares PASID support (IORT E.c Flags bit 0) */ +#define IOMMU_FWSPEC_PCI_RC_PASID (1 << 2) +/* Root complex forwards PASID on translated transactions (IORT ATS bit 2) */ +#define IOMMU_FWSPEC_PCI_RC_PASID_FWD (1 << 3)/* * An iommu attach handle represents a relationship between an iommu domain
Signature · 1 line
2.25.1
PATCH 3/3 iommu/arm-smmu-v3: Honor IORT Root Complex PASID descriptors
In reply to Vidya SagarThe SMMUv3 driver currently calls pci_enable_pasid() for any PCI master that exposes a PASID capability, regardless of whether the upstream Root Complex actually supports PASID and regardless of the RC's declared Max PASID Width. With IORT spec E.c (RC node revision
Quoted 1 line
= 4) firmware reports both, so we can do better:
- If the IORT Root Complex node says PASID is not supported (Flags bit 0 == 0 at byte offset 36), enabling PASID on the endpoint is futile - the RC will not forward the PASID prefix to the SMMU - so skip pci_enable_pasid() silently.
- If the IORT Root Complex node reports a Max PASID Width (bits[4:0] of PASID Capabilities at offset 33), clamp the endpoint's pci_max_pasids() result by 1 << width before computing the SMMU SSID width. This prevents master->ssid_bits from exceeding what the RC can actually carry.
Both behaviours are gated on iort_pci_rc_pasid_max_width_known(), i.e. RC node revision >= 4, so platforms with older IORT firmware see no behavioural change and continue to enable PASID purely on the basis of the endpoint capability.
Use the new IOMMU_FWSPEC_PCI_RC_PASID fwspec flag (set by IORT) for the support check, and call iort_pci_rc_pasid_max_width_for_dev() for the width clamp; both pieces are wired up in iort_iommu_configure_id() by the previous patch.
--- drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c | 26 ++++++++++++++++++--- 1 file changed, 23 insertions(+), 3 deletions(-)
drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c +23 -3
@@ -3071,16 +3071,28 @@ static void arm_smmu_enable_ats(struct arm_smmu_master *master)static int arm_smmu_enable_pasid (struct arm_smmu_master *master) { -int ret; -int features; -int num_pasids; +struct iommu_fwspec *fwspec =dev_iommu_fwspec_get (master->dev);struct pci_dev *pdev; +int features, num_pasids, ret, rc_width;if (!dev_is_pci (master->dev))return -ENODEV ; pdev =to_pci_dev (master->dev); +/* +* IORT E.c (RC node revision >= 4) reports whether the root +* complex actually supports PASID. If it does not, enabling +* PASID on the endpoint is futile - the RC will not forward +* the PASID prefix - so skip silently. Older firmware is +* treated as "unknown / assume supported" to preserve the +* pre-E.c behaviour. +*/ +if (fwspec && + !(fwspec->flags &IOMMU_FWSPEC_PCI_RC_PASID ) && +iort_pci_rc_pasid_max_width_known (master->dev)) +return 0 ; + features =pci_pasid_features (pdev);if (features <0 )return features;
@@ -3089,6 +3101,14 @@ static int arm_smmu_enable_pasid(struct arm_smmu_master *master)if (num_pasids <=0 )return num_pasids; +/* Clamp by what the root complex can carry, when known. */ + rc_width =iort_pci_rc_pasid_max_width_for_dev (master->dev); +if (rc_width >=0 ) + num_pasids =min_t (int, num_pasids,1 << rc_width); + +if (num_pasids <=1 ) +return 0 ; + ret =pci_enable_pasid (pdev, features);if (ret) {dev_err (&pdev->dev,"Failed to enable PASID\n" );
Signature · 1 line
2.25.1
Re: [PATCH V1 3/3] iommu/arm-smmu-v3: Honor IORT Root Complex PASID descriptors
Quoted 37 lines · Vidya Sagar
The SMMUv3 driver currently calls
pci_enable_pasid()for any PCI master that exposes a PASID capability, regardless of whether the upstream Root Complex actually supports PASID and regardless of the RC's declared Max PASID Width. With IORT specE.c(RC node revisionQuoted 1 line
= 4) firmware reports both, so we can do better:
- If the IORT Root Complex node says PASID is not supported (Flags bit 0 == 0 at byte offset 36), enabling PASID on the endpoint is futile - the RC will not forward the PASID prefix to the SMMU - so skip
pci_enable_pasid()silently.- If the IORT Root Complex node reports a Max PASID Width (bits[4:0] of PASID Capabilities at offset 33), clamp the endpoint's
pci_max_pasids()result by 1 << width before computing the SMMU SSID width. This preventsmaster->ssid_bitsfrom exceeding what the RC can actually carry.Both behaviours are gated on
iort_pci_rc_pasid_max_width_known(), i.e. RC node revision >= 4, so platforms with older IORT firmware see no behavioural change and continue to enable PASID purely on the basis of the endpoint capability.Use the new
IOMMU_FWSPEC_PCI_RC_PASIDfwspec flag (set by IORT) for the support check, and calliort_pci_rc_pasid_max_width_for_dev()for the width clamp; both pieces are wired up iniort_iommu_configure_id()by the previous patch.Signed-off-by Vidya Sagar <[email protected]>--- drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c | 26 ++++++++++++++++++--- 1 file changed, 23 insertions(+), 3 deletions(-)drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c +0
-0
drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c static void arm_smmu_enable_ats(struct arm_smmu_master *master) · line 3076
Quoted 9 lines
drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c +2
-3@@ -3071,16 +3071,28 @@ static void arm_smmu_enable_ats(struct arm_smmu_master *master)static int arm_smmu_enable_pasid (struct arm_smmu_master *master) { -int ret; -int features; -int num_pasids; +struct iommu_fwspec *fwspec =dev_iommu_fwspec_get (master->dev);struct pci_dev *pdev; +int features, num_pasids, ret, rc_width;
Don't reformat the code like this.
Otherwise the series broadly makes sense to me
Jason