Support for PRI and PASID capability
cover Support for PRI and PASID capability
Hi Jesse,
here is a patch-set that enables support for the PRI and PASID capabilities in the Linux PCI core. These capabilities of a PCIe device are tightly coupled with ATS, but not with IOV. So these patches start by moving the ATS code into its own file: ats.c.
Support for the other two capabilities is based on that change. Please review the patches and let me know of any objections you might have.
Thanks,
Joerg
Diffstat:
drivers/pci/Kconfig | 24 +++ drivers/pci/Makefile | 1 + drivers/pci/ats.c | 438 ++++++++++++++++++++++++++++++++++++++++++++++ drivers/pci/iov.c | 142 --------------- include/linux/pci-ats.h | 75 ++++++++ include/linux/pci_regs.h | 20 ++ 6 files changed, 558 insertions(+), 142 deletions(-)
PATCH 1/4 PCI: Move ATS implementation into own file
ATS does not depend on IOV support, so move the code into its own file. This file will also include support for the PRI and PASID capability later.
Also give ATS its own Kconfig variable to allow selecting it without IOV support.
--- drivers/pci/Kconfig | 4 + drivers/pci/Makefile | 1 + drivers/pci/ats.c | 155 +++++++++++++++++++++++++++++++++++++++++++++++ drivers/pci/iov.c | 142 ------------------------------------------- include/linux/pci-ats.h | 2 + 5 files changed, 162 insertions(+), 142 deletions(-) create mode 100644 drivers/pci/ats.c
drivers/pci/Kconfig +4 -0
@@ -71,9 +71,13 @@ config HT_IRQ If unsure say Y. +config PCI_ATS + bool + config PCI_IOV bool "PCI IOV support" depends on PCI + select PCI_ATS help I/O Virtualization is a PCI feature supported by some devices which allows them to create virtual devices which share their
drivers/pci/Makefile +1 -0
@@ -29,6 +29,7 @@ obj-$(CONFIG_PCI_MSI) += msi.o # Build the Hypertransport interrupt support obj-$(CONFIG_HT_IRQ) += htirq.o +obj-$(CONFIG_PCI_ATS) += ats.o obj-$(CONFIG_PCI_IOV) += iov.o #
drivers/pci/ats.c new file +155 -0
155 lines not on this page · raw message
drivers/pci/iov.c +0 -142
145 lines not on this page · raw message
include/linux/pci-ats.h +2 -0
@@ -1,6 +1,8 @@ #ifndef LINUX_PCI_ATS_H #define LINUX_PCI_ATS_H +#include <linux/pci.h> + /* Address Translation Service */ struct pci_ats { int pos; /* capability position */
Signature · 1 line
1.7.4.1
PATCH 2/4 PCI: Export ATS functions to modules
In reply to Joerg RoedelThis patch makes the ATS function usable for modules. This will be used by a modules implementing some advanced AMD IOMMU features.
--- drivers/pci/ats.c | 3 +++ 1 files changed, 3 insertions(+), 0 deletions(-)
drivers/pci/ats.c +3 -0
@@ -90,6 +90,7 @@ int pci_enable_ats(struct pci_dev *dev, int ps)return 0 ; } +EXPORT_SYMBOL_GPL (pci_enable_ats);/** * pci_disable_ats - disable the ATS capability
@@ -120,6 +121,7 @@ void pci_disable_ats(struct pci_dev *dev)if (!dev->is_physfn)ats_free_one (dev); } +EXPORT_SYMBOL_GPL (pci_disable_ats);/** * pci_ats_queue_depth - query the ATS Invalidate Queue Depth
@@ -153,3 +155,4 @@ int pci_ats_queue_depth(struct pci_dev *dev)return PCI_ATS_CAP_QDEP (cap) ?PCI_ATS_CAP_QDEP (cap) :PCI_ATS_MAX_QDEP ; } +EXPORT_SYMBOL_GPL (pci_ats_queue_depth);
Signature · 1 line
1.7.4.1
PATCH 3/4 PCI: Add implementation for PRI capability
In reply to Joerg RoedelImplement the necessary functions to handle PRI capabilities on PCIe devices. With PRI devices behind an IOMMU can signal page fault conditions to software and recover from such faults.
--- drivers/pci/Kconfig | 9 +++ drivers/pci/ats.c | 167 ++++++++++++++++++++++++++++++++++++++++++++++ include/linux/pci-ats.h | 42 ++++++++++++ include/linux/pci_regs.h | 12 +++ 4 files changed, 230 insertions(+), 0 deletions(-)
drivers/pci/Kconfig +9 -0
@@ -85,6 +85,15 @@ config PCI_IOV If unsure, say N. +config PCI_PRI + bool "PCI PRI support" + select PCI_ATS + help + PRI is the PCI Page Request Interface. It allows PCI devices that are + behind an IOMMU to recover from page faults. + + If unsure, say N. + config PCI_IOAPIC bool depends on PCI
drivers/pci/ats.c +167 -0
179 lines not on this page · raw message
include/linux/pci-ats.h +42 -0
52 lines not on this page · raw message
include/linux/pci_regs.h +12 -0
@@ -663,6 +663,18 @@#define PCI_ATS_CTRL_STU (x) ((x) & 0x1f)/* Smallest Translation Unit */ #define PCI_ATS_MIN_STU 12/* shift of minimum STU block */ +/* Page Request Interface */ +#define PCI_PRI_CAP 0x13/* PRI capability ID */ +#define PCI_PRI_CONTROL_OFF 0x04/* Offset of control register */ +#define PCI_PRI_STATUS_OFF 0x06/* Offset of status register */ +#define PCI_PRI_ENABLE 0x0001/* Enable mask */ +#define PCI_PRI_RESET 0x0002/* Reset bit mask */ +#define PCI_PRI_STATUS_RF 0x0001/* Request Failure */ +#define PCI_PRI_STATUS_UPRGI 0x0002/* Unexpected PRG index */ +#define PCI_PRI_STATUS_STOPPED 0x0100/* PRI Stopped */ +#define PCI_PRI_MAX_REQ_OFF 0x08/* Cap offset for max reqs supported */ +#define PCI_PRI_ALLOC_REQ_OFF 0x0c/* Cap offset for max reqs allowed */ +/* Single Root I/O Virtualization */ #define PCI_SRIOV_CAP 0x04/* SR-IOV Capabilities */ #define PCI_SRIOV_CAP_VFM 0x01/* VF Migration Capable */
Signature · 1 line
1.7.4.1
PATCH 4/4 PCI: Add support for PASID capability
In reply to Joerg RoedelDevices supporting the PASID capability can use multiple contexts on an IOMMU at the same time. This patch adds support to manage that capability.
--- drivers/pci/Kconfig | 11 +++++ drivers/pci/ats.c | 113 ++++++++++++++++++++++++++++++++++++++++++++++ include/linux/pci-ats.h | 31 +++++++++++++ include/linux/pci_regs.h | 8 +++ 4 files changed, 163 insertions(+), 0 deletions(-)
drivers/pci/Kconfig +11 -0
@@ -94,6 +94,17 @@ config PCI_PRI If unsure, say N. +config PCI_PASID + bool "PCI PASID support" + depends on PCI + select PCI_ATS + help + Devices supporting PASIDs can access multiple address spaces + on an IOMMU at the same time. This option compiles code for + that capability into the kernel. + + If unsure, say N. + config PCI_IOAPIC bool depends on PCI
drivers/pci/ats.c +113 -0
122 lines not on this page · raw message
include/linux/pci-ats.h +31 -0
@@ -93,4 +93,35 @@ static inline int pci_pri_status(struct pci_dev *pdev) }#endif /* CONFIG_PCI_PRI */ +#ifdef CONFIG_PCI_PASID + +extern int pci_enable_pasid (struct pci_dev *pdev,int features); +extern void pci_disable_pasid (struct pci_dev *pdev); +extern int pci_pasid_features (struct pci_dev *pdev); +extern int pci_max_pasids (struct pci_dev *pdev); + +#else /* CONFIG_PCI_PASID */ + +static inline int pci_enable_pasid (struct pci_dev *pdev,int features) +{ +return -EINVAL ; +} + +static inline void pci_disable_pasid (struct pci_dev *pdev) +{ +} + +static inline int pci_pasid_features (struct pci_dev *pdev) +{ +return -EINVAL ; +} + +static inline int pci_max_pasids (struct pci_dev *pdev) +{ +return -EINVAL ; +} + +#endif /* CONFIG_PCI_PASID */ + +#endif /* LINUX_PCI_ATS_H*/
include/linux/pci_regs.h +8 -0
@@ -675,6 +675,14 @@#define PCI_PRI_MAX_REQ_OFF 0x08/* Cap offset for max reqs supported */ #define PCI_PRI_ALLOC_REQ_OFF 0x0c/* Cap offset for max reqs allowed */ +/* PASID capability */ +#define PCI_PASID_CAP 0x1b/* PASID capability ID */ +#define PCI_PASID_CAP_OFF 0x04/* PASID feature register */ +#define PCI_PASID_CONTROL_OFF 0x06/* PASID control register */ +#define PCI_PASID_ENABLE 0x01/* Enable/Supported bit */ +#define PCI_PASID_EXEC 0x02/* Exec permissions Enable/Supported */ +#define PCI_PASID_PRIV 0x04/* Priviledge Mode Enable/Support */ +/* Single Root I/O Virtualization */ #define PCI_SRIOV_CAP 0x04/* SR-IOV Capabilities */ #define PCI_SRIOV_CAP_VFM 0x01/* VF Migration Capable */
Signature · 1 line
1.7.4.1
Bjorn Helgaas
maintainerRe: [PATCH 4/4] PCI: Add support for PASID capability
Quoted 3 lines · Joerg Roedel
Devices supporting the PASID capability can use multiple contexts on an IOMMU at the same time. This patch adds support to manage that capability.
These all look reasonable to me.
Is there anything else useful you could say in the PASID Kconfig help text? I'm guessing it stands for "PCI Address Space ID" or something, but I have no idea why I might want to enable it.
I didn't coordinate very well with Jesse. I'm hoping he'll be back before the 3.2 merge window, but I think we have a few patches queued up now, and it might be useful to have them in linux-next for a while before he returns. I'll look into that.
Bjorn
Quoted 227 lines
Signed-off-by Joerg Roedel <[email protected]>---
drivers/pci/Kconfig | 11 +++++
drivers/pci/ats.c | 113 ++++++++++++++++++++++++++++++++++++++++++++++include/linux/pci-ats.h| 31 +++++++++++++include/linux/pci_regs.h| 8 +++4 files changed, 163 insertions(+), 0 deletions(-)drivers/pci/Kconfig +0
-0@@ -94,6 +94,17 @@ config PCI_PRIIf unsure, say N.
drivers/pci/Kconfig +11
-0+config PCI_PASID + bool "PCI PASID support" + depends on PCI + select PCI_ATS + help + Devices supporting PASIDs can access multiple address spaces + on an IOMMU at the same time. This option compiles code for + that capability into the kernel. + + If unsure, say N. +config
PCI_IOAPIC
bool
depends on PCIdrivers/pci/ats.c +113
-0@@ -7,6 +7,7 @@* PCI Express I/O Virtualization (IOV) support. * Address Translation Service 1.0 * Page Request Interface added by Joerg Roedel <[email protected]> +* PASID support added by Joerg Roedel <[email protected]> */ #include <linux/pci-ats.h>@@ -323,3 +324,115 @@ int pci_pri_status(struct pci_dev *pdev) } EXPORT_SYMBOL_GPL(pci_pri_status); #endif /* CONFIG_PCI_PRI */ + +#ifdef CONFIG_PCI_PASID +/** +* pci_enable_pasid - Enable the PASID capability +* @pdev: PCI device structure +* @features: Features to enable +* +* Returns 0 on success, negative value on error. This function checks +* whether the features are actually supported by the device and returns +* an error if not. +*/ +int pci_enable_pasid (struct pci_dev *pdev,int features) +{ + u16 control, supported; + int pos; + + pos = pci_find_ext_capability(pdev, PCI_PASID_CAP); + if (!pos) + return -EINVAL; + + pci_read_config_word(pdev, pos + PCI_PASID_CONTROL_OFF, &control); + pci_read_config_word(pdev, pos + PCI_PASID_CAP_OFF, &supported); + + if (!(supported & PCI_PASID_ENABLE)) + return -EINVAL; + + supported &= PCI_PASID_EXEC | PCI_PASID_PRIV; + + /* User wants to enable anything unsupported? */ + if ((supported & features) != features) + return -EINVAL; + + control = PCI_PASID_ENABLE | features; + + pci_write_config_word(pdev, pos + PCI_PASID_CONTROL_OFF, control); + + return 0; +} +EXPORT_SYMBOL_GPL (pci_enable_pasid); + +/** +* pci_disable_pasid - Disable the PASID capability +* @pdev: PCI device structure +* +*/ +void pci_disable_pasid (struct pci_dev *pdev) +{ + u16 control = 0; + int pos; + + pos = pci_find_ext_capability(pdev, PCI_PASID_CAP); + if (!pos) + return; + + pci_write_config_word(pdev, pos + PCI_PASID_CONTROL_OFF, control); +} +EXPORT_SYMBOL_GPL (pci_disable_pasid); + +/** +* pci_pasid_features - Check which PASID features are supported +* @pdev: PCI device structure +* +* Returns a negative value when no PASI capability is present. +* Otherwise is returns a bitmask with supported features. Current +* features reported are: +* PCI_PASID_ENABLE - PASID capability can be enabled +* PCI_PASID_EXEC - Execute permission supported +* PCI_PASID_PRIV - Priviledged mode supported +*/ +int pci_pasid_features (struct pci_dev *pdev) +{ + u16 supported; + int pos; + + pos = pci_find_ext_capability(pdev, PCI_PASID_CAP); + if (!pos) + return -EINVAL; + + pci_read_config_word(pdev, pos + PCI_PASID_CAP_OFF, &supported); + + supported &= PCI_PASID_ENABLE | PCI_PASID_EXEC | PCI_PASID_PRIV; + + return supported; +} +EXPORT_SYMBOL_GPL (pci_pasid_features); + +#define PASID_NUMBER_SHIFT 8 +#define PASID_NUMBER_MASK (0x1f << PASID_NUMBER_SHIFT) +/** +* pci_max_pasid - Get maximum number of PASIDs supported by device +* @pdev: PCI device structure +* +* Returns negative value when PASID capability is not present. +* Otherwise it returns the numer of supported PASIDs. +*/ +int pci_max_pasids (struct pci_dev *pdev) +{ + u16 supported; + int pos; + + pos = pci_find_ext_capability(pdev, PCI_PASID_CAP); + if (!pos) + return -EINVAL; + + pci_read_config_word(pdev, pos + PCI_PASID_CAP_OFF, &supported); + + supported = (supported & PASID_NUMBER_MASK) >> PASID_NUMBER_SHIFT; + + return (1 << supported); +} +EXPORT_SYMBOL_GPL (pci_max_pasids); +#endif /* CONFIG_PCI_PASID */ include/linux/pci-ats.h +31
-0@@ -93,4 +93,35 @@ static inline int pci_pri_status(struct pci_dev *pdev) } #endif /* CONFIG_PCI_PRI */ +#ifdef CONFIG_PCI_PASID + +extern int pci_enable_pasid (struct pci_dev *pdev,int features); +extern void pci_disable_pasid (struct pci_dev *pdev); +extern int pci_pasid_features (struct pci_dev *pdev); +extern int pci_max_pasids (struct pci_dev *pdev); + +#else /* CONFIG_PCI_PASID */ + +static inline int pci_enable_pasid (struct pci_dev *pdev,int features) +{ + return -EINVAL; +} + +static inline void pci_disable_pasid (struct pci_dev *pdev) +{ +} + +static inline int pci_pasid_features (struct pci_dev *pdev) +{ + return -EINVAL; +} + +static inline int pci_max_pasids (struct pci_dev *pdev) +{ + return -EINVAL; +} + +#endif /* CONFIG_PCI_PASID */ + + #endif /* LINUX_PCI_ATS_H*/include/linux/pci_regs.h +8
-1@@ -675,6 +675,14 @@ #define PCI_PRI_MAX_REQ_OFF 0x08 /* Cap offset for max reqs supported */ #define PCI_PRI_ALLOC_REQ_OFF 0x0c /* Cap offset for max reqs allowed */ +/* PASID capability */ +#define PCI_PASID_CAP 0x1b/* PASID capability ID */ +#define PCI_PASID_CAP_OFF 0x04/* PASID feature register */ +#define PCI_PASID_CONTROL_OFF 0x06/* PASID control register */ +#define PCI_PASID_ENABLE 0x01/* Enable/Supported bit */ +#define PCI_PASID_EXEC 0x02/* Exec permissions Enable/Supported */ +#define PCI_PASID_PRIV 0x04/* Priviledge Mode Enable/Support */ + /* Single Root I/O Virtualization */ #define PCI_SRIOV_CAP 0x04 /* SR-IOV Capabilities */ #define PCI_SRIOV_CAP_VFM 0x01 /* VF Migration Capable */ --1.7.4.1
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH 4/4] PCI: Add support for PASID capability
Quoted 10 lines · Bjorn Helgaas
Quoted 3 lines · Joerg Roedel
Devices supporting the PASID capability can use multiple contexts on an IOMMU at the same time. This patch adds support to manage that capability.
These all look reasonable to me.
Is there anything else useful you could say in the PASID
Kconfighelp text? I'm guessing it stands for "PCI Address Space ID" or something, but I have no idea why I might want to enable it.
PASID stands for Process Address Space ID, it basically allows a PCI device to access multiple IO address spaces at the same time using an IOMMU. Users should enable it when they want to compile an IOMMU driver into the kernel that supports PASIDs for the translated devices. I'll add all that information to the Kconfig help. Thanks a lot for your feedback :)
Quoted 4 lines
I didn't coordinate very well with Jesse. I'm hoping he'll be back before the 3.2 merge window, but I think we have a few patches queued up now, and it might be useful to have them in linux-next for a while before he returns. I'll look into that.
Great, thanks again.
Regards,
Joerg
Re: [PATCH 4/4] PCI: Add support for PASID capability
Quoted 6 lines · Joerg Roedel
PASID stands for Process Address Space ID, it basically allows a PCI device to access multiple IO address spaces at the same time using an IOMMU. Users should enable it when they want to compile an IOMMU driver into the kernel that supports PASIDs for the translated devices. I'll add all that information to the
Kconfighelp. Thanks a lot for your feedback :)
Is this a case where the PCI device drivers in question should be doing a 'select' or 'depends on' if they support talking to an IOMMU using PASID?
Re: [PATCH 4/4] PCI: Add support for PASID capability
Quoted 11 lines · On Wed, Sep 28, 2011 at 03:18:15PM -0400, [email protected] wrote:
Quoted 6 lines · Joerg Roedel
PASID stands for Process Address Space ID, it basically allows a PCI device to access multiple IO address spaces at the same time using an IOMMU. Users should enable it when they want to compile an IOMMU driver into the kernel that supports PASIDs for the translated devices. I'll add all that information to the
Kconfighelp. Thanks a lot for your feedback :)Is this a case where the PCI device drivers in question should be doing a 'select' or 'depends on' if they support talking to an IOMMU using PASID?
Exactly. In fact, I plan to put the functionality which handles these capabilities and the fault-handling in a seperate module where all device drivers can use it in an easy way.
Joerg
Signature · 5 lines
AMD Operating System Research Center Advanced Micro Devices GmbH Einsteinring 24 85609 Dornach General Managers: Alberto Bozzo, Andrew Bowd Registration: Dornach, Landkr. Muenchen; Registerger. Muenchen, HRB Nr. 43632
Hi Jesse,
Quoted 7 lines · Joerg Roedel
here is a patch-set that enables support for the PRI and PASID capabilities in the Linux PCI core. These capabilities of a PCIe device are tightly coupled with ATS, but not with IOV. So these patches start by moving the ATS code into its own file:
ats.c.Support for the other two capabilities is based on that change. Please review the patches and let me know of any objections you might have.
Have you had a chance to look at these patches? Please let me know any questions or objections you might have.
Thanks,
Joerg
Bjorn Helgaas
maintainerQuoted 9 lines · Joerg Roedel
Hi Jesse,
Quoted 5 lines · Joerg Roedel
here is a patch-set that enables support for the PRI and PASID capabilities in the Linux PCI core. These capabilities of a PCIe device are tightly coupled with ATS, but not with IOV. So these patches start by moving the ATS code into its own file:
ats.c.
Quoted 5 lines
Quoted 2 lines
Support for the other two capabilities is based on that change. Please review the patches and let me know of any objections you might have.
Have you had a chance to look at these patches? Please let me know any questions or objections you might have.
Jesse's on vacation, but I'll take a look and give you my opinion (which won't necessarily coincide with Jesse's, but I guess it's a start :)).
Bjorn
Quoted 13 lines · Bjorn Helgaas
Quoted 12 lines · Joerg Roedel
Hi Jesse,
Quoted 7 lines · Joerg Roedel
here is a patch-set that enables support for the PRI and PASID capabilities in the Linux PCI core. These capabilities of a PCIe device are tightly coupled with ATS, but not with IOV. So these patches start by moving the ATS code into its own file:
ats.c.Support for the other two capabilities is based on that change. Please review the patches and let me know of any objections you might have.
Quoted 6 lines
Quoted 2 lines
Have you had a chance to look at these patches? Please let me know any questions or objections you might have.
Jesse's on vacation, but I'll take a look and give you my opinion (which won't necessarily coincide with Jesse's, but I guess it's a start :)).
Great. Thanks a lot :)
Joerg