Fix PF/VF dependency issue
v8 cover Fix PF/VF dependency issue
From: Bjorn Helgaas <[email protected]>
The current implementation of ATS, PASID, and PRI does not handle VF dependencies correctly. These are essentially Kuppuswamy's patches; I reordered and tweaked them slightly.
Please treat this as a proposal, not a done deal, and post a v9 with any changes needed.
Changes since v7:
* Moved PRI & PASID capability offset caching to last since they're just optimizations
* Cached offsets at first use instead of adding _init() functions
* Dropped complicated pci_prg_resp_pasid_required() #ifdefs by just searching for the capability again
* Dropped EA/VF validation since it only enforces spec language without fixing any known issues
Changes since v6:
* Removed locking interfaces used in v6.
* Made necessary changes to PRI/PASID enable/disable calls to allow register changes only for PF.
Changes since v5:
* Created new patches for PRI/PASID capability caching.
* Removed individual locks (pri_lock, pasid_lock) and added common resource lock to synchronize PRI/PASID updates between PF/VF.
* Addressed comments from Bjorn Helgaas.
Changes since v4:
* Defined empty functions for pci_pri_init() and pci_pasid_init() for cases where CONFIG_PCI_PRI and CONFIG_PCI_PASID are not enabled.
Changes since v3:
* Fixed critical path (lock context) in pci_restore_*_state functions.
Changes since v2:
* Added locking mechanism to synchronize accessing PF registers in VF.
* Removed spec compliance checks in patches.
* Addressed comments from Bjorn Helgaas.
Changes since v1:
* Added more details about the patches in commit log.
* Removed bulk spec check patch.
* Addressed comments from Bjorn Helgaas.
Kuppuswamy Sathyanarayanan (5):
PCI/ATS: Handle sharing of PF PRI Capability with all VFs PCI/ATS: Handle sharing of PF PASID Capability with all VFs PCI/ATS: Disable PF/VF ATS service independently PCI/ATS: Cache PRI Capability offset
PCI/ATS: Cache PASID Capability offset
drivers/pci/ats.c | 153 +++++++++++++++++++++++++++----------------- include/linux/pci.h | 3 +- 2 files changed, 96 insertions(+), 60 deletions(-)
Signature · 1 line
2.23.0.187.g17f5b7556c-goog
PATCH v8 1/5 PCI/ATS: Handle sharing of PF PRI Capability with all VFs
From: Kuppuswamy Sathyanarayanan <[email protected]>
Per PCIe r5.0, sec 9.3.7.11, VFs must not implement the PRI Capability. If the PF implements PRI, it is shared by the VFs. Since VFs don't have a PRI Capability, pci_enable_pri() always failed, which caused IOMMU setup to fail.
Update the PRI interfaces so for VFs they reflect the state of the PF PRI.
--- drivers/pci/ats.c | 24 ++++++++++++++++++++++++ 1 file changed, 24 insertions(+)
drivers/pci/ats.c +24 -0
@@ -182,6 +182,17 @@ int pci_enable_pri(struct pci_dev *pdev, u32 reqs)u32 max_requests;int pos; +/* +* VFs must not implement the PRI Capability. If their PF +* implements PRI, it is shared by the VFs, so if the PF PRI is +* enabled, it is also enabled for the VF. +*/ +if (pdev->is_virtfn) { +if (pci_physfn (pdev)->pri_enabled) +return 0 ; +return -EINVAL ; + } +if (WARN_ON (pdev->pri_enabled))return -EBUSY ;
@@ -218,6 +229,10 @@ void pci_disable_pri(struct pci_dev *pdev)u16 control;int pos; +/* VFs share the PF PRI */ +if (pdev->is_virtfn) +return ; +if (WARN_ON (!pdev->pri_enabled))return ;
@@ -243,6 +258,9 @@ void pci_restore_pri_state(struct pci_dev *pdev)u32 reqs = pdev->pri_reqs_alloc;int pos; +if (pdev->is_virtfn) +return ; +if (!pdev->pri_enabled)return ;
@@ -267,6 +285,9 @@ int pci_reset_pri(struct pci_dev *pdev)u16 control;int pos; +if (pdev->is_virtfn) +return 0 ; +if (WARN_ON (pdev->pri_enabled))return -EBUSY ;
@@ -412,6 +433,9 @@ int pci_prg_resp_pasid_required(struct pci_dev *pdev)u16 status;int pos; +if (pdev->is_virtfn) + pdev =pci_physfn (pdev); + pos =pci_find_ext_capability (pdev,PCI_EXT_CAP_ID_PRI );if (!pos)return 0 ;
Signature · 1 line
2.23.0.187.g17f5b7556c-goog
PATCH v8 2/5 PCI/ATS: Handle sharing of PF PASID Capability with all VFs
In reply to Bjorn HelgaasFrom: Kuppuswamy Sathyanarayanan <[email protected]>
Per PCIe r5.0, sec 9.3.7.14, if a PF implements the PASID Capability, the PF PASID configuration is shared by its VFs. VFs must not implement their own PASID Capability. Since VFs don't have a PASID Capability, pci_enable_pasid() always failed, which caused IOMMU setup to fail.
Update the PASID interfaces so for VFs they reflect the state of the PF PASID.
--- drivers/pci/ats.c | 23 +++++++++++++++++++++++ 1 file changed, 23 insertions(+)
drivers/pci/ats.c +23 -0
@@ -318,6 +318,16 @@ int pci_enable_pasid(struct pci_dev *pdev, int features)u16 control, supported;int pos; +/* +* VFs must not implement the PASID Capability, but if a PF +* supports PASID, its VFs share the PF PASID configuration. +*/ +if (pdev->is_virtfn) { +if (pci_physfn (pdev)->pasid_enabled) +return 0 ; +return -EINVAL ; + } +if (WARN_ON (pdev->pasid_enabled))return -EBUSY ;
@@ -355,6 +365,10 @@ void pci_disable_pasid(struct pci_dev *pdev)u16 control =0 ;int pos; +/* VFs share the PF PASID configuration */ +if (pdev->is_virtfn) +return ; +if (WARN_ON (!pdev->pasid_enabled))return ;
@@ -377,6 +391,9 @@ void pci_restore_pasid_state(struct pci_dev *pdev)u16 control;int pos; +if (pdev->is_virtfn) +return ; +if (!pdev->pasid_enabled)return ;
@@ -404,6 +421,9 @@ int pci_pasid_features(struct pci_dev *pdev)u16 supported;int pos; +if (pdev->is_virtfn) + pdev =pci_physfn (pdev); + pos =pci_find_ext_capability (pdev,PCI_EXT_CAP_ID_PASID );if (!pos)return -EINVAL ;
@@ -463,6 +483,9 @@ int pci_max_pasids(struct pci_dev *pdev)u16 supported;int pos; +if (pdev->is_virtfn) + pdev =pci_physfn (pdev); + pos =pci_find_ext_capability (pdev,PCI_EXT_CAP_ID_PASID );if (!pos)return -EINVAL ;
Signature · 1 line
2.23.0.187.g17f5b7556c-goog
PATCH v8 3/5 PCI/ATS: Disable PF/VF ATS service independently
In reply to Bjorn HelgaasFrom: Kuppuswamy Sathyanarayanan <[email protected]>
Previously we didn't disable the PF ATS until all associated VFs had disabled it. But per PCIe spec r5.0, sec 9.3.7.8, the ATS Capability in VFs and associated PFs may be enabled independently. Leaving ATS enabled in the PF unnecessarily may have power and performance impacts.
Remove this dependency logic in the ATS enable/disable code.
--- drivers/pci/ats.c | 11 ----------- include/linux/pci.h | 1 - 2 files changed, 12 deletions(-)
drivers/pci/ats.c +0 -11
@@ -60,8 +60,6 @@ int pci_enable_ats(struct pci_dev *dev, int ps) pdev =pci_physfn (dev);if (pdev->ats_stu != ps)return -EINVAL ; - -atomic_inc (&pdev->ats_ref_cnt);/* count enabled VFs */ }else { dev->ats_stu = ps; ctrl |=PCI_ATS_CTRL_STU (dev->ats_stu -PCI_ATS_MIN_STU );
@@ -79,20 +77,11 @@ EXPORT_SYMBOL_GPL(pci_enable_ats);*/ void pci_disable_ats (struct pci_dev *dev) { -struct pci_dev *pdev;u16 ctrl;if (WARN_ON (!dev->ats_enabled))return ; -if (atomic_read (&dev->ats_ref_cnt)) -return ;/* VFs still enabled */ - -if (dev->is_virtfn) { - pdev =pci_physfn (dev); -atomic_dec (&pdev->ats_ref_cnt); - } -pci_read_config_word (dev, dev->ats_cap +PCI_ATS_CTRL , &ctrl); ctrl &= ~PCI_ATS_CTRL_ENABLE ;pci_write_config_word (dev, dev->ats_cap +PCI_ATS_CTRL , ctrl);
include/linux/pci.h +0 -1
@@ -452,7 +452,6 @@ struct pci_dev { };u16 ats_cap;/* ATS Capability offset */ u8 ats_stu;/* ATS Smallest Translation Unit */ -atomic_t ats_ref_cnt;/* Number of VFs with ATS enabled */ #endif #ifdef CONFIG_PCI_PRI u32 pri_reqs_alloc;/* Number of PRI requests allocated */
Signature · 1 line
2.23.0.187.g17f5b7556c-goog
PATCH v8 4/5 PCI/ATS: Cache PRI Capability offset
In reply to Bjorn HelgaasFrom: Kuppuswamy Sathyanarayanan <[email protected]>
Previously each PRI interface searched for the PRI Capability. Cache the capability offset the first time we use it instead of searching each time.
[bhelgaas: commit log, reorder patch to later, save offset directly in pci_enable_pri() rather than adding pci_pri_init()]
--- drivers/pci/ats.c | 52 ++++++++++++++++++++++----------------------- include/linux/pci.h | 1 + 2 files changed, 27 insertions(+), 26 deletions(-)
drivers/pci/ats.c +26 -26
@@ -169,7 +169,7 @@ int pci_enable_pri(struct pci_dev *pdev, u32 reqs) {u16 control, status;u32 max_requests; -int pos; +int pri = pdev->pri_cap;/* * VFs must not implement the PRI Capability. If their PF
@@ -185,21 +185,24 @@ int pci_enable_pri(struct pci_dev *pdev, u32 reqs)if (WARN_ON (pdev->pri_enabled))return -EBUSY ; - pos =pci_find_ext_capability (pdev,PCI_EXT_CAP_ID_PRI ); -if (!pos) -return -EINVAL ; +if (!pri) { + pri =pci_find_ext_capability (pdev,PCI_EXT_CAP_ID_PRI ); +if (!pri) +return -EINVAL ; + pdev->pri_cap = pri; + } -pci_read_config_word (pdev, pos +PCI_PRI_STATUS , &status); +pci_read_config_word (pdev, pri +PCI_PRI_STATUS , &status);if (!(status &PCI_PRI_STATUS_STOPPED ))return -EBUSY ; -pci_read_config_dword (pdev, pos +PCI_PRI_MAX_REQ , &max_requests); +pci_read_config_dword (pdev, pri +PCI_PRI_MAX_REQ , &max_requests); reqs =min (max_requests, reqs); pdev->pri_reqs_alloc = reqs; -pci_write_config_dword (pdev, pos +PCI_PRI_ALLOC_REQ , reqs); +pci_write_config_dword (pdev, pri +PCI_PRI_ALLOC_REQ , reqs); control =PCI_PRI_CTRL_ENABLE ; -pci_write_config_word (pdev, pos +PCI_PRI_CTRL , control); +pci_write_config_word (pdev, pri +PCI_PRI_CTRL , control); pdev->pri_enabled =1 ;
@@ -216,7 +219,7 @@ EXPORT_SYMBOL_GPL(pci_enable_pri);void pci_disable_pri (struct pci_dev *pdev) {u16 control; -int pos; +int pri = pdev->pri_cap;/* VFs share the PF PRI */ if (pdev->is_virtfn)
@@ -225,13 +228,12 @@ void pci_disable_pri(struct pci_dev *pdev)if (WARN_ON (!pdev->pri_enabled))return ; - pos =pci_find_ext_capability (pdev,PCI_EXT_CAP_ID_PRI ); -if (!pos) +if (!pri)return ; -pci_read_config_word (pdev, pos +PCI_PRI_CTRL , &control); +pci_read_config_word (pdev, pri +PCI_PRI_CTRL , &control); control &= ~PCI_PRI_CTRL_ENABLE ; -pci_write_config_word (pdev, pos +PCI_PRI_CTRL , control); +pci_write_config_word (pdev, pri +PCI_PRI_CTRL , control); pdev->pri_enabled =0 ; }
@@ -245,7 +247,7 @@ void pci_restore_pri_state(struct pci_dev *pdev) {u16 control =PCI_PRI_CTRL_ENABLE ;u32 reqs = pdev->pri_reqs_alloc; -int pos; +int pri = pdev->pri_cap;if (pdev->is_virtfn)return ;
@@ -253,12 +255,11 @@ void pci_restore_pri_state(struct pci_dev *pdev)if (!pdev->pri_enabled)return ; - pos =pci_find_ext_capability (pdev,PCI_EXT_CAP_ID_PRI ); -if (!pos) +if (!pri)return ; -pci_write_config_dword (pdev, pos +PCI_PRI_ALLOC_REQ , reqs); -pci_write_config_word (pdev, pos +PCI_PRI_CTRL , control); +pci_write_config_dword (pdev, pri +PCI_PRI_ALLOC_REQ , reqs); +pci_write_config_word (pdev, pri +PCI_PRI_CTRL , control); }EXPORT_SYMBOL_GPL (pci_restore_pri_state);
@@ -272,7 +273,7 @@ EXPORT_SYMBOL_GPL(pci_restore_pri_state);int pci_reset_pri (struct pci_dev *pdev) {u16 control; -int pos; +int pri = pdev->pri_cap;if (pdev->is_virtfn)return 0 ;
@@ -280,12 +281,11 @@ int pci_reset_pri(struct pci_dev *pdev)if (WARN_ON (pdev->pri_enabled))return -EBUSY ; - pos =pci_find_ext_capability (pdev,PCI_EXT_CAP_ID_PRI ); -if (!pos) +if (!pri)return -EINVAL ; control =PCI_PRI_CTRL_RESET ; -pci_write_config_word (pdev, pos +PCI_PRI_CTRL , control); +pci_write_config_word (pdev, pri +PCI_PRI_CTRL , control);return 0 ; }
@@ -440,16 +440,16 @@ EXPORT_SYMBOL_GPL(pci_pasid_features);int pci_prg_resp_pasid_required (struct pci_dev *pdev) {u16 status; -int pos; +int pri;if (pdev->is_virtfn) pdev =pci_physfn (pdev); - pos =pci_find_ext_capability (pdev,PCI_EXT_CAP_ID_PRI ); -if (!pos) + pri =pci_find_ext_capability (pdev,PCI_EXT_CAP_ID_PRI ); +if (!pri)return 0 ; -pci_read_config_word (pdev, pos +PCI_PRI_STATUS , &status); +pci_read_config_word (pdev, pri +PCI_PRI_STATUS , &status);if (status &PCI_PRI_STATUS_PASID )return 1 ;
include/linux/pci.h +1 -0
@@ -454,6 +454,7 @@ struct pci_dev {u8 ats_stu;/* ATS Smallest Translation Unit */ #endif #ifdef CONFIG_PCI_PRI +u16 pri_cap;/* PRI Capability offset */ u32 pri_reqs_alloc;/* Number of PRI requests allocated */ #endif #ifdef CONFIG_PCI_PASID
Signature · 1 line
2.23.0.187.g17f5b7556c-goog
PATCH v8 5/5 PCI/ATS: Cache PASID Capability offset
In reply to Bjorn HelgaasFrom: Kuppuswamy Sathyanarayanan <[email protected]>
Previously each PASID interface searched for the PASID Capability. Cache the capability offset the first time we use it instead of searching each time.
[bhelgaas: commit log, reorder patch to later, save offset directly in pci_enable_pasid() rather than adding pci_pasid_init()]
--- drivers/pci/ats.c | 43 +++++++++++++++++++++---------------------- include/linux/pci.h | 1 + 2 files changed, 22 insertions(+), 22 deletions(-)
drivers/pci/ats.c +21 -22
@@ -305,7 +305,7 @@ EXPORT_SYMBOL_GPL(pci_reset_pri);int pci_enable_pasid (struct pci_dev *pdev,int features) {u16 control, supported; -int pos; +int pasid = pdev->pasid_cap;/* * VFs must not implement the PASID Capability, but if a PF
@@ -323,11 +323,14 @@ int pci_enable_pasid(struct pci_dev *pdev, int features)if (!pdev->eetlp_prefix_path)return -EINVAL ; - pos =pci_find_ext_capability (pdev,PCI_EXT_CAP_ID_PASID ); -if (!pos) -return -EINVAL ; +if (!pasid) { + pasid =pci_find_ext_capability (pdev,PCI_EXT_CAP_ID_PASID ); +if (!pasid) +return -EINVAL ; + pdev->pasid_cap = pasid; + } -pci_read_config_word (pdev, pos +PCI_PASID_CAP , &supported); +pci_read_config_word (pdev, pasid +PCI_PASID_CAP , &supported); supported &=PCI_PASID_CAP_EXEC |PCI_PASID_CAP_PRIV ;/* User wants to enable anything unsupported? */
@@ -337,7 +340,7 @@ int pci_enable_pasid(struct pci_dev *pdev, int features) control =PCI_PASID_CTRL_ENABLE | features; pdev->pasid_features = features; -pci_write_config_word (pdev, pos +PCI_PASID_CTRL , control); +pci_write_config_word (pdev, pasid +PCI_PASID_CTRL , control); pdev->pasid_enabled =1 ;
@@ -352,7 +355,7 @@ EXPORT_SYMBOL_GPL(pci_enable_pasid);void pci_disable_pasid (struct pci_dev *pdev) {u16 control =0 ; -int pos; +int pasid = pdev->pasid_cap;/* VFs share the PF PASID configuration */ if (pdev->is_virtfn)
@@ -361,11 +364,10 @@ void pci_disable_pasid(struct pci_dev *pdev)if (WARN_ON (!pdev->pasid_enabled))return ; - pos =pci_find_ext_capability (pdev,PCI_EXT_CAP_ID_PASID ); -if (!pos) +if (!pasid)return ; -pci_write_config_word (pdev, pos +PCI_PASID_CTRL , control); +pci_write_config_word (pdev, pasid +PCI_PASID_CTRL , control); pdev->pasid_enabled =0 ; }
@@ -378,7 +380,7 @@ EXPORT_SYMBOL_GPL(pci_disable_pasid);void pci_restore_pasid_state (struct pci_dev *pdev) {u16 control; -int pos; +int pasid = pdev->pasid_cap;if (pdev->is_virtfn)return ;
@@ -386,12 +388,11 @@ void pci_restore_pasid_state(struct pci_dev *pdev)if (!pdev->pasid_enabled)return ; - pos =pci_find_ext_capability (pdev,PCI_EXT_CAP_ID_PASID ); -if (!pos) +if (!pasid)return ; control =PCI_PASID_CTRL_ENABLE | pdev->pasid_features; -pci_write_config_word (pdev, pos +PCI_PASID_CTRL , control); +pci_write_config_word (pdev, pasid +PCI_PASID_CTRL , control); }EXPORT_SYMBOL_GPL (pci_restore_pasid_state);
@@ -408,16 +409,15 @@ EXPORT_SYMBOL_GPL(pci_restore_pasid_state);int pci_pasid_features (struct pci_dev *pdev) {u16 supported; -int pos; +int pasid = pdev->pasid_cap;if (pdev->is_virtfn) pdev =pci_physfn (pdev); - pos =pci_find_ext_capability (pdev,PCI_EXT_CAP_ID_PASID ); -if (!pos) +if (!pasid)return -EINVAL ; -pci_read_config_word (pdev, pos +PCI_PASID_CAP , &supported); +pci_read_config_word (pdev, pasid +PCI_PASID_CAP , &supported); supported &=PCI_PASID_CAP_EXEC |PCI_PASID_CAP_PRIV ;
@@ -470,16 +470,15 @@ EXPORT_SYMBOL_GPL(pci_prg_resp_pasid_required);int pci_max_pasids (struct pci_dev *pdev) {u16 supported; -int pos; +int pasid = pdev->pasid_cap;if (pdev->is_virtfn) pdev =pci_physfn (pdev); - pos =pci_find_ext_capability (pdev,PCI_EXT_CAP_ID_PASID ); -if (!pos) +if (!pasid)return -EINVAL ; -pci_read_config_word (pdev, pos +PCI_PASID_CAP , &supported); +pci_read_config_word (pdev, pasid +PCI_PASID_CAP , &supported); supported = (supported &PASID_NUMBER_MASK ) >>PASID_NUMBER_SHIFT ;
include/linux/pci.h +1 -0
@@ -458,6 +458,7 @@ struct pci_dev {u32 pri_reqs_alloc;/* Number of PRI requests allocated */ #endif #ifdef CONFIG_PCI_PASID +u16 pasid_cap;/* PASID Capability offset */ u16 pasid_features;#endif #ifdef CONFIG_PCI_P2PDMA
Signature · 1 line
2.23.0.187.g17f5b7556c-goog
Bjorn Helgaas
AppliedQuoted 56 lines · Bjorn Helgaas
From: Bjorn Helgaas <[email protected]>
The current implementation of ATS, PASID, and PRI does not handle VF dependencies correctly. These are essentially Kuppuswamy's patches; I reordered and tweaked them slightly.
Please treat this as a proposal, not a done deal, and post a v9 with any changes needed.
Changes since v7:
* Moved PRI & PASID capability offset caching to last since they're just optimizations
* Cached offsets at first use instead of adding_init()functions
* Dropped complicatedpci_prg_resp_pasid_required()#ifdefs by just searching for the capability again
* Dropped EA/VF validation since it only enforces spec language without fixing any known issuesChanges since v6:
* Removed locking interfaces used in v6.
* Made necessary changes to PRI/PASID enable/disable calls to allow register changes only for PF.Changes since v5:
* Created new patches for PRI/PASID capability caching.
* Removed individual locks (pri_lock,pasid_lock) and added common resource lock to synchronize PRI/PASID updates between PF/VF.
* Addressed comments from Bjorn Helgaas.Changes since v4:
* Defined empty functions forpci_pri_init()andpci_pasid_init()for cases whereCONFIG_PCI_PRIandCONFIG_PCI_PASIDare not enabled.Changes since v3:
* Fixed critical path (lock context) in pci_restore_*_state functions.Changes since v2:
* Added locking mechanism to synchronize accessing PF registers in VF.
* Removed spec compliance checks in patches.
* Addressed comments from Bjorn Helgaas.Changes since v1:
* Added more details about the patches in commit log.
* Removed bulk spec check patch.
* Addressed comments from Bjorn Helgaas.Kuppuswamy Sathyanarayanan (5):
PCI/ATS: Handle sharing of PF PRI Capability with all VFs PCI/ATS: Handle sharing of PF PASID Capability with all VFs PCI/ATS: Disable PF/VF ATS service independently PCI/ATS: Cache PRI Capability offset
PCI/ATS: Cache PASID Capability offset
Quoted 3 lines
drivers/pci/ats.c | 153 +++++++++++++++++++++++++++----------------- include/linux/pci.h | 3 +- 2 files changed, 96 insertions(+), 60 deletions(-)
Applied to pci/virtualization for v5.5.