AMD IOMMUv2 Performance Counter patches
Subsystems (4)
cover AMD IOMMUv2 Performance Counter patches
From: "Steven L. Kinney" <[email protected]>
These patches implement the AMD IOMMUv2.5 Performance Counter functionality via custom perf PMU and implement static counting for various IOMMU translations. The patches address three areas of functionality:
1) Add a PCI quirk for the enablement of IOMMUv2 EFR PC within a specific
AMD family/model in which BIOS has not enabled. Based on code implemented by Andreas Herrmann at AMD Dresden.
2) Extend the AMD IOMMU initialization to include IOMMUv2 PC enablement
and access to IOMMUv2 counter registers. For all AMD family/models that implement IOMMUv2.5 functionality.
3) Code the perf IOMMUv2 PMU to manage IOMMUv2 perf events, which call
the function(s) extending the AMD IOMMU core driver. For all AMD family/models that implement IOMMUv2.5 functionality.
The command-line, to invoke the iommuv2 PMU, is:
perf stat -e iommuv2/config=[data],config1=[data]/{u,r} [command]
For information regarding AMD IOMMU v2.5 PC configuration, see the public specification.
Steven L. Kinney (3):
AMD x86 quirks: Quirk for enabling IOMMUv2 PC feature AMD IOMMUv2 PC resource management hooks
AMD IOMMUv2 PC perf PMU implementation
arch/x86/kernel/cpu/Makefile | 1 + arch/x86/kernel/cpu/perf_event_amd_iommuv2.c | 429 ++++++++++++++++++++++++++ arch/x86/kernel/cpu/perf_event_amd_iommuv2.h | 42 +++ arch/x86/kernel/quirks.c | 17 + drivers/iommu/Kconfig | 10 + drivers/iommu/amd_iommu_init.c | 99 ++++++ drivers/iommu/amd_iommu_types.h | 12 + include/linux/pci_ids.h | 2 + 8 files changed, 612 insertions(+) create mode 100644 arch/x86/kernel/cpu/perf_event_amd_iommuv2.c create mode 100644 arch/x86/kernel/cpu/perf_event_amd_iommuv2.h
Signature · 1 line
1.7.9.5
PATCH 1/3 AMD x86 quirks: Quirk for enabling IOMMUv2 PC feature
From: "Steven L. Kinney" <[email protected]>
Original code base by Andreas Herrmann at AMD with minor changes.
--- arch/x86/kernel/quirks.c | 17 +++++++++++++++++ include/linux/pci_ids.h | 2 ++ 2 files changed, 19 insertions(+)
arch/x86/kernel/quirks.c +17 -0
@@ -567,3 +567,20 @@ DECLARE_PCI_FIXUP_FINAL(PCI_VENDOR_ID_AMD, PCI_DEVICE_ID_AMD_15H_NB_F5, quirk_amd_nb_node);#endif + +#if defined(CONFIG_PCI ) && defined(CONFIG_AMD_IOMMU_V2_PC ) + +static void amd_force_iommu_v2_pc (struct pci_dev *dev) +{ +u32 val; + +pci_read_config_dword (dev,0x70 , &val); + val |= (1 <<9 ); +pci_write_config_dword (dev,0x70 , val); +dev_printk (KERN_DEBUG , &dev->dev,"Enabled IOMMUv2 PC\n" ); +} + +DECLARE_PCI_FIXUP_FINAL (PCI_VENDOR_ID_AMD ,PCI_DEVICE_ID_AMD_15H_M10H_IOMMU , + amd_force_iommu_v2_pc); + +#endif
include/linux/pci_ids.h +2 -0
@@ -396,6 +396,8 @@/* AMD RD890 Chipset */ #define PCI_DEVICE_ID_RD890_IOMMU 0x5a23 +#define PCI_DEVICE_ID_AMD_15H_M10H_IOMMU 0x1419 +#define PCI_VENDOR_ID_ADL 0x1005#define PCI_DEVICE_ID_ADL_2301 0x2301
Signature · 1 line
1.7.9.5
Joerg Roedel
maintainerRe: [PATCH 1/3] AMD x86 quirks: Quirk for enabling IOMMUv2 PC feature
Quoted 3 lines · Steven L. Kinney
From: "Steven L. Kinney" <[email protected]>
Original code base by Andreas Herrmann at AMD with minor changes.
Why is it safe to enable this feature when it was disabled by BIOS? Was it disabled for no reason?
Joerg
RE: [PATCH 1/3] AMD x86 quirks: Quirk for enabling IOMMUv2 PC feature
Hi Joerg,
It was disabled for a reason that does not pertain to IOMMUv2 PC static counts (reading from the ICounter associated with the selected bank/counter). Testing with perf shows expected results.
Thanks,
Steve
-----Original Message-----
From: Joerg Roedel [mailto:[email protected]]
Sent: Monday, January 28, 2013 8:29 AM
To: Kinney, Steven
Cc: Thomas Gleixner; Ingo Molnar; H. Peter Anvin; [email protected]; Bjorn Helgaas; Greg Kroah-Hartman; Sebastian Andrzej Siewior; Myron Stowe; Hiroshi DOYU; Stephen Warren; Jiri Kosina; Kukjin Kim; [email protected]; [email protected]; Peter Zijlstra; Paul Mackerras; Arnaldo Carvalho de Melo; Thomas Renninger; Andi Kleen; Cyrill Gorcunov
Subject: Re: [PATCH 1/3] AMD x86 quirks: Quirk for enabling IOMMUv2 PC feature
Quoted 3 lines · Steven L. Kinney
From: "Steven L. Kinney" <[email protected]>
Original code base by Andreas Herrmann at AMD with minor changes.
Why is it safe to enable this feature when it was disabled by BIOS? Was it disabled for no reason?
Joerg
Joerg Roedel
maintainerRe: [PATCH 1/3] AMD x86 quirks: Quirk for enabling IOMMUv2 PC feature
Quoted 1 line · Steven
Testing with perf shows expected results.
Can you give me an impression on how the results look like when perf is used? Since the hardware is widely available yet I can't try this myself.
Joerg
PATCH 2/3 AMD IOMMUv2 PC resource management hooks
In reply to Steven L. KinneyFrom: "Steven L. Kinney" <[email protected]>
Add functionality to check the availability of the AMD IOMMUv2 Performance Counters and export this functionality to other core drivers, such as in this case, a perf IOMMUv2 PMU. This feature is not bound to any specific AMD family/model other than the presence of the IOMMUv2 with PC enabled.
The IOMMUv2 PC support static counting only at this time.
--- drivers/iommu/Kconfig | 10 ++++ drivers/iommu/amd_iommu_init.c | 99 +++++++++++++++++++++++++++++++++++++++ drivers/iommu/amd_iommu_types.h | 12 +++++ 3 files changed, 121 insertions(+)
drivers/iommu/Kconfig +10 -0
@@ -73,6 +73,16 @@ config AMD_IOMMU_V2 hardware. Select this option if you want to use devices that support the PCI PRI and PASID interface. +# AMD IOMMUv2 Performance Counter support +config AMD_IOMMU_V2_PC + bool "AMD IOMMUv2 Performance Counter (EXPERIMENTAL)" + depends on AMD_IOMMU_V2 + ---help--- + This option enables support for AMD IOMMUv2 Performance Counters. + Select this option if you want to enable IOMMUv2 Performance + Counter support. + If unsure, say N. + # Intel IOMMU support config DMAR_TABLE bool
drivers/iommu/amd_iommu_init.c +99 -0
114 lines not on this page · raw message
drivers/iommu/amd_iommu_types.h +12 -0
@@ -38,7 +38,12 @@#define RLOOKUP_TABLE_ENTRY_SIZE (sizeof(void *))/* Length of the MMIO region for the AMD IOMMU */ +#ifdef CONFIG_AMD_IOMMU_V2_PC +#define MMIO_REGION_LENGTH 0x80000 +#define IOMMU_V2_PC_REG_OFFSET 0x40000 +#else #define MMIO_REGION_LENGTH 0x4000 +#endif /* Capability offsets used by the driver */ #define MMIO_CAP_HDR_OFFSET 0x00
@@ -77,6 +82,7 @@#define MMIO_STATUS_OFFSET 0x2020#define MMIO_PPR_HEAD_OFFSET 0x2030#define MMIO_PPR_TAIL_OFFSET 0x2038 +#define MMIO_CNTR_CONF_OFFSET 0x4000/* Extended Feature Bits */
@@ -585,6 +591,12 @@ struct amd_iommu {/* The l2 indirect registers */ u32 stored_l2[0x83 ]; + +#ifdef CONFIG_AMD_IOMMU_V2_PC +/* The maximum PC banks and counters/bank (PCSup=1) */ +u8 max_banks; +u8 max_counters; +#endif };struct devid_map {
Signature · 1 line
1.7.9.5
Re: [PATCH 2/3] AMD IOMMUv2 PC resource management hooks
drivers/iommu/amd_iommu_init.c bool amd_iommu_v2_supported(void) · line 2116
Quoted 13 lines · Steven L. Kinney
drivers/iommu/amd_iommu_init.c +13
-0+u8 amd_iommu_v2_get_max_pc_banks (u16 devid) +{ +struct amd_iommu *iommu; + +/* locate the iommu governing the devid */ + iommu = amd_iommu_rlookup_table[devid]; + +if (iommu) +return iommu->max_banks; + +return -ENODEV ; +} +EXPORT_SYMBOL (amd_iommu_v2_get_max_pc_banks);
Hi Steven, may not it be better to return int here, instead of u8?
RE: [PATCH 2/3] AMD IOMMUv2 PC resource management hooks
Hi Cyrill,
Agree, will clean this up.
Thanks
-----Original Message-----
From: Cyrill Gorcunov [mailto:[email protected]] On Behalf Of Cyrill Gorcunov
Sent: Monday, January 21, 2013 3:32 PM
To: Kinney, Steven
Cc: Thomas Gleixner; Ingo Molnar; H. Peter Anvin; [email protected]; Joerg Roedel; Bjorn Helgaas; Greg Kroah-Hartman; Sebastian Andrzej Siewior; Myron Stowe; Hiroshi DOYU; Stephen Warren; Jiri Kosina; Kukjin Kim; [email protected]; iommu-cunTk1MwBs9QetFLy7KEm3xJsTq8ys+cHZ5vskTnxNA@public.gmane.org; Peter Zijlstra; Paul Mackerras; Arnaldo Carvalho de Melo; Thomas Renninger; Andi Kleen
Subject: Re: [PATCH 2/3] AMD IOMMUv2 PC resource management hooks
drivers/iommu/amd_iommu_init.c bool amd_iommu_v2_supported(void) · line 2116
Quoted 12 lines · Steven L. Kinney
drivers/iommu/amd_iommu_init.c +12
-0+u8 amd_iommu_v2_get_max_pc_banks (u16 devid) { +struct amd_iommu *iommu; + +/* locate the iommu governing the devid */ + iommu = amd_iommu_rlookup_table[devid]; + +if (iommu) +return iommu->max_banks; + +return -ENODEV ; +} +EXPORT_SYMBOL (amd_iommu_v2_get_max_pc_banks);
Hi Steven, may not it be better to return int here, instead of u8?
Re: [PATCH 2/3] AMD IOMMUv2 PC resource management hooks
Quoted 2 lines · Steven
Hi Cyrill,
Agree, will clean this up.
Yeah, but no rush. Lets wait for other opinions as well.
PATCH 3/3 AMD IOMMUv2 PC perf PMU implementation
In reply to Steven L. KinneyFrom: "Steven L. Kinney" <[email protected]>
Implement a perf PMU to handle IOMMUv2 PC perf events. This PMU will handle static counter perf events relative to the AMD IOMMUv2 Performance Counters.
To invoke the AMD IOMMUv2 PMU issue a perf tool command such as:
./perf stat -e iommuv2/config=<config-data>,config1=<config1-data>/u <command>
For example:
./perf stat -e iommuv2/config=08000000000000005,config1=0/u <command>
The resulting count will be how many IOMMUv2 totsal peripheral memory operations were performed during the command execution window.
--- arch/x86/kernel/cpu/Makefile | 1 + arch/x86/kernel/cpu/perf_event_amd_iommuv2.c | 429 ++++++++++++++++++++++++++ arch/x86/kernel/cpu/perf_event_amd_iommuv2.h | 42 +++ 3 files changed, 472 insertions(+) create mode 100644 arch/x86/kernel/cpu/perf_event_amd_iommuv2.c create mode 100644 arch/x86/kernel/cpu/perf_event_amd_iommuv2.h
arch/x86/kernel/cpu/Makefile +1 -0
@@ -32,6 +32,7 @@ obj-$(CONFIG_PERF_EVENTS) += perf_event.o ifdef CONFIG_PERF_EVENTS obj-$(CONFIG_CPU_SUP_AMD) += perf_event_amd.o +obj-$(CONFIG_AMD_IOMMU_V2_PC) += perf_event_amd_iommuv2.o obj-$(CONFIG_CPU_SUP_INTEL) += perf_event_p6.o perf_event_knc.o perf_event_p4.o obj-$(CONFIG_CPU_SUP_INTEL) += perf_event_intel_lbr.o perf_event_intel_ds.o perf_event_intel.o obj-$(CONFIG_CPU_SUP_INTEL) += perf_event_intel_uncore.o
arch/x86/kernel/cpu/perf_event_amd_iommuv2.c new file +429 -0
429 lines not on this page · raw message
arch/x86/kernel/cpu/perf_event_amd_iommuv2.h new file +42 -0
42 lines not on this page · raw message
Signature · 1 line
1.7.9.5
Quoted 16 lines · Steven L. Kinney
From: "Steven L. Kinney" <[email protected]>
These patches implement the AMD IOMMUv2.5 Performance Counter functionality via custom perf PMU and implement static counting for various IOMMU translations. The patches address three areas of functionality:
1) Add a PCI quirk for the enablement of IOMMUv2 EFR PC within a specific
AMD family/model in which BIOS has not enabled. Based on code implemented by Andreas Herrmann at AMD Dresden.
2) Extend the AMD IOMMU initialization to include IOMMUv2 PC enablement
and access to IOMMUv2 counter registers. For all AMD family/models that implement IOMMUv2.5 functionality.
3) Code the perf IOMMUv2 PMU to manage IOMMUv2 perf events, which call
the function(s) extending the AMD IOMMU core driver. For all AMD family/models that implement IOMMUv2.5 functionality.
Quoted 6 lines
The command-line, to invoke the iommuv2 PMU, is:
perf stat -e iommuv2/config=[data],config1=[data]/{u,r} [command]
For information regarding AMD IOMMU v2.5 PC configuration, see the public specification.
Is there a name for it? I see
http://support.amd.com/us/Processor_TechDocs/48882.pdf
but that looks to be for v2.0?
Quoted 26 lines
Steven L. Kinney (3):
AMD x86 quirks: Quirk for enabling IOMMUv2 PC feature AMD IOMMUv2 PC resource management hooks
AMD IOMMUv2 PC perf PMU implementationarch/x86/kernel/cpu/Makefile | 1 + arch/x86/kernel/cpu/perf_event_amd_iommuv2.c | 429 ++++++++++++++++++++++++++ arch/x86/kernel/cpu/perf_event_amd_iommuv2.h | 42 +++ arch/x86/kernel/quirks.c | 17 + drivers/iommu/Kconfig | 10 + drivers/iommu/amd_iommu_init.c | 99 ++++++ drivers/iommu/amd_iommu_types.h | 12 + include/linux/pci_ids.h | 2 + 8 files changed, 612 insertions(+) create mode 100644 arch/x86/kernel/cpu/perf_event_amd_iommuv2.c create mode 100644 arch/x86/kernel/cpu/perf_event_amd_iommuv2.hSignature · 7 lines
1.7.9.5 _______________________________________________ iommu mailing list iommu-cunTk1MwBs9QetFLy7KEm3xJsTq8ys+cHZ5vskTnxNA@public.gmane.org https://lists.linuxfoundation.org/mailman/listinfo/iommu
Hi Konrad,
Thanks for your response. When writing the driver, at AMD, I am provided an internal IOMMUv2.x specification that may not be out in the public as of yet; I am following up on this and will comment on the availability concerning this version. Also, I checked the v2.0 of the IOMMU Specification, specifically the following:
MMIO Offset [40-7F][0-F]08h IOMMU Counter Source Register - Table 64: Architectural Counter Input Group : This is the CSource and the format can be found in /sys/bus/event_sources/devices/iommuv2/forrmat/ MMIO Offset [40-7F][0-F]20h IOMMU DeviceID Match Register - For the following three registers, the config value defines the Match and config1 value defines the Mask; see the specification v2.0 for details; also see sys/...../format for details. MMIO Offset [40-7F][0-F]10h IOMMU PASID Match Register -
MMIO Offset [40-7F][0-F]18h IOMMU Domain Match Register -
The one caveat between the v2.0 and v2.5 concerns a slight expansion of the CSource register Input Group, but any event defined within v2.0 will execute properly regarding the driver patch(es) sent to the kernel community.
Please let me know, in any way, how I can better this driver code and I will make it so. Also, Will have an answer for you regarding AMD IOMMUv2.5 ASAP.
Thanks again for your comments, appreciated.
-----Original Message-----
From: Konrad Rzeszutek Wilk [mailto:[email protected]]
Sent: Tuesday, January 22, 2013 11:48 AM
To: Kinney, Steven
Cc: Thomas Gleixner; Ingo Molnar; H. Peter Anvin; [email protected]; Joerg Roedel; Kukjin Kim; Peter Zijlstra; Stephen Warren; Greg Kroah-Hartman; [email protected]; iommu-cunTk1MwBs9QetFLy7KEm3xJsTq8ys+cHZ5vskTnxNA@public.gmane.org; Cyrill Gorcunov; Sebastian Andrzej Siewior; Andi Kleen; Paul Mackerras; Arnaldo Carvalho de Melo; Jiri Kosina; Bjorn Helgaas; Myron Stowe; Thomas Renninger
Subject: Re: [PATCH 0/3] AMD IOMMUv2 Performance Counter patches
Quoted 16 lines · Steven L. Kinney
From: "Steven L. Kinney" <[email protected]>
These patches implement the AMD IOMMUv2.5 Performance Counter functionality via custom perf PMU and implement static counting for various IOMMU translations. The patches address three areas of functionality:
1) Add a PCI quirk for the enablement of IOMMUv2 EFR PC within a specific
AMD family/model in which BIOS has not enabled. Based on code implemented by Andreas Herrmann at AMD Dresden.
2) Extend the AMD IOMMU initialization to include IOMMUv2 PC enablement
and access to IOMMUv2 counter registers. For all AMD family/models that implement IOMMUv2.5 functionality.
3) Code the perf IOMMUv2 PMU to manage IOMMUv2 perf events, which call
the function(s) extending the AMD IOMMU core driver. For all AMD family/models that implement IOMMUv2.5 functionality.
Quoted 6 lines
The command-line, to invoke the iommuv2 PMU, is:
perf stat -e iommuv2/config=[data],config1=[data]/{u,r} [command]
For information regarding AMD IOMMU v2.5 PC configuration, see the public specification.
Is there a name for it? I see
http://support.amd.com/us/Processor_TechDocs/48882.pdf
but that looks to be for v2.0?
Quoted 26 lines
Steven L. Kinney (3):
AMD x86 quirks: Quirk for enabling IOMMUv2 PC feature AMD IOMMUv2 PC resource management hooks
AMD IOMMUv2 PC perf PMU implementationarch/x86/kernel/cpu/Makefile | 1 + arch/x86/kernel/cpu/perf_event_amd_iommuv2.c | 429 ++++++++++++++++++++++++++ arch/x86/kernel/cpu/perf_event_amd_iommuv2.h | 42 +++ arch/x86/kernel/quirks.c | 17 + drivers/iommu/Kconfig | 10 + drivers/iommu/amd_iommu_init.c | 99 ++++++ drivers/iommu/amd_iommu_types.h | 12 + include/linux/pci_ids.h | 2 + 8 files changed, 612 insertions(+) create mode 100644 arch/x86/kernel/cpu/perf_event_amd_iommuv2.c create mode 100644 arch/x86/kernel/cpu/perf_event_amd_iommuv2.h--
1.7.9.5_______________________________________________
iommu mailing list iommu-cunTk1MwBs9QetFLy7KEm3xJsTq8ys+cHZ5vskTnxNA@public.gmane.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu