Skip to contentSkip to conversation

Add Intel IOMMU debugfs support

PATCH Superseded Sohil Mehta 3 participants 13 messages Jan 10, 2018 – Jan 11, 2018 IOMMU

v6 of 8 Superseded Latest is v8 →

lore
Share & tools
lore
#0 lore raw

v6 cover Add Intel IOMMU debugfs support

Hi All,

This series aims to add debugfs support for Intel IOMMU. It exposes IOMMU registers, internal context and dumps individual table entries to help debug Intel IOMMUs.

The first patch does the ground work for the following patches by creating a new Kconfig option - INTEL_IOMMU_DEBUG. It also reorganizes some Intel IOMMU data structures. The next four patches add debugfs support for IOMMU context internals, register contents, PASID internals, and Interrupt remapping in that order. The information can be accessed in sysfs at '/sys/kernel/debug/intel_iommu/'.

Regards,
Sohil

Changes since v5:
- Change the order of includes to an alphabetical order
- Change seq_printf and seq_puts formatting

Changes since v4:
- Change to a SPDX license tag
- Fix seq_printf formatting and remove leading '\n's

Changes since v3:
- Remove an unused function parameter from some of the functions
- Fix checkpatch.pl warnings
- Remove error reporting for debugfs_create_file functions
- Fix unnecessary reprogramming of the context entries
- Simplify and merge the show context and extended context patch into one
- Remove redundant IOMMU null check under for_each_active_iommu
- Update the commit title to be consistent

Changes since v2:
- Added a macro for seq file operations based on recommendation by Andy Shevchenko. The marco can be moved to seq_file.h at a future point
- Changed the debugfs file names to more relevant ones
- Added information for MTRR registers in the regset file

Changes since v1:
- Fixed seq_printf formatting
- Handled the case when Interrupt remapping is not enabled

Gayatri Kammela (4):
iommu/vt-d: Add debugfs support for Intel IOMMU internals iommu/vt-d: Add debugfs support to show context internals iommu/vt-d: Add debugfs support to show register contents iommu/vt-d: Add debugfs support to show Pasid table contents

Sohil Mehta (1):
iommu/vt-d: Add debugfs support for Interrupt remapping

 drivers/iommu/Kconfig             |  10 ++
 drivers/iommu/Makefile            |   1 +
 drivers/iommu/intel-iommu-debug.c | 346 ++++++++++++++++++++++++++++++++++++++
 drivers/iommu/intel-iommu.c       |  35 +---
 drivers/iommu/intel-svm.c         |   8 -
 include/linux/intel-iommu.h       |  34 ++++
 include/linux/intel-svm.h         |  10 +-
 7 files changed, 407 insertions(+), 37 deletions(-)
 create mode 100644 drivers/iommu/intel-iommu-debug.c
Signature · 1 line
2.7.4
#1 lore raw

PATCH v6 1/5 iommu/vt-d: Add debugfs support for Intel IOMMU internals

From: Gayatri Kammela <[email protected]>

Enable Intel IOMMU debug to export Intel IOMMU internals in debugfs

---

v6: No change

v5: No change

v4: No change

v3: No change

v2: No change

 drivers/iommu/Kconfig       | 10 ++++++++++
 drivers/iommu/intel-iommu.c | 31 +++----------------------------
 include/linux/intel-iommu.h | 32 ++++++++++++++++++++++++++++++++
 include/linux/intel-svm.h   |  2 +-
 4 files changed, 46 insertions(+), 29 deletions(-)
drivers/iommu/Kconfig +10 -0
index f3a2134..d7588ee 100644
@@ -152,6 +152,16 @@ config INTEL_IOMMU
 	  and include PCI device scope covered by these DMA
 	  remapping devices.
 
+config INTEL_IOMMU_DEBUG
+	bool "Export Intel IOMMU internals in DebugFS"
+	depends on INTEL_IOMMU && DEBUG_FS
+	default n
+	help
+	  IOMMU internal states such as context, PASID tables can be seen via
+	  debugfs. Select this option if you want to export these internals.
+
+	  Say Y if you need this.
+
 config INTEL_IOMMU_SVM
 	bool "Support for Shared Virtual Memory with Intel IOMMU"
 	depends on INTEL_IOMMU && X86
drivers/iommu/intel-iommu.c +3 -28
index 4a2de34..e05b8e0 100644
@@ -183,16 +183,6 @@ static int rwbf_quirk;
 static int force_on = 0;
 int intel_iommu_tboot_noforce;
 
-/*
- * 0: Present
- * 1-11: Reserved
- * 12-63: Context Ptr (12 - (haw-1))
- * 64-127: Reserved
- */
-struct root_entry {
-	u64	lo;
-	u64	hi;
-};
 #define ROOT_ENTRY_NR (VTD_PAGE_SIZE/sizeof(struct root_entry))
 
 /*
@@ -218,21 +208,6 @@ static phys_addr_t root_entry_uctp(struct root_entry *re)
 
 	return re->hi & VTD_PAGE_MASK;
 }
-/*
- * low 64 bits:
- * 0: present
- * 1: fault processing disable
- * 2-3: translation type
- * 12-63: address space root
- * high 64 bits:
- * 0-2: address width
- * 3-6: aval
- * 8-23: domain id
- */
-struct context_entry {
-	u64 lo;
-	u64 hi;
-};
 
 static inline void context_clear_pasid_enable(struct context_entry *context)
 {
@@ -259,7 +234,7 @@ static inline bool __context_present(struct context_entry *context)
 	return (context->lo & 1);
 }
 
-static inline bool context_present(struct context_entry *context)
+bool context_present(struct context_entry *context)
 {
 	return context_pasid_enabled(context) ?
 	     __context_present(context) :
@@ -819,7 +794,7 @@ static void domain_update_iommu_cap(struct dmar_domain *domain)
 	domain->iommu_superpage = domain_update_iommu_superpage(NULL);
 }
 
-static inline struct context_entry *iommu_context_addr(struct intel_iommu *iommu,
+struct context_entry *iommu_context_addr(struct intel_iommu *iommu,
 						       u8 bus, u8 devfn, int alloc)
 {
 	struct root_entry *root = &iommu->root_entry[bus];
@@ -5208,7 +5183,7 @@ static void intel_iommu_put_resv_regions(struct device *dev,
 
 #ifdef CONFIG_INTEL_IOMMU_SVM
 #define MAX_NR_PASID_BITS (20)
-static inline unsigned long intel_iommu_get_pts(struct intel_iommu *iommu)
+unsigned long intel_iommu_get_pts(struct intel_iommu *iommu)
 {
 	/*
 	 * Convert ecap_pss to extend context entry pts encoding, also
include/linux/intel-iommu.h +32 -0
index f3274d9..b8591dc 100644
@@ -383,6 +383,33 @@ struct pasid_entry;
 struct pasid_state_entry;
 struct page_req_dsc;
 
+/*
+ * 0: Present
+ * 1-11: Reserved
+ * 12-63: Context Ptr (12 - (haw-1))
+ * 64-127: Reserved
+ */
+struct root_entry {
+	u64     lo;
+	u64     hi;
+};
+
+/*
+ * low 64 bits:
+ * 0: present
+ * 1: fault processing disable
+ * 2-3: translation type
+ * 12-63: address space root
+ * high 64 bits:
+ * 0-2: address width
+ * 3-6: aval
+ * 8-23: domain id
+ */
+struct context_entry {
+	u64 lo;
+	u64 hi;
+};
+
 struct intel_iommu {
 	void __iomem	*reg; /* Pointer to hardware regs, virtual addr */
 	u64 		reg_phys; /* physical address of hw register set */
@@ -488,8 +515,13 @@ struct intel_svm {
 
 extern int intel_iommu_enable_pasid(struct intel_iommu *iommu, struct intel_svm_dev *sdev);
 extern struct intel_iommu *intel_svm_device_to_iommu(struct device *dev);
+extern unsigned long intel_iommu_get_pts(struct intel_iommu *iommu);
 #endif
 
 extern const struct attribute_group *intel_iommu_groups[];
+extern void intel_iommu_debugfs_init(void);
+extern bool context_present(struct context_entry *context);
+extern struct context_entry *iommu_context_addr(struct intel_iommu *iommu,
+						u8 bus, u8 devfn, int alloc);
 
 #endif
include/linux/intel-svm.h +1 -1
index 99bc5b3..733eaf9 100644
@@ -130,7 +130,7 @@ static inline int intel_svm_unbind_mm(struct device *dev, int pasid)
 	BUG();
 }
 
-static int intel_svm_is_pasid_valid(struct device *dev, int pasid)
+static inline int intel_svm_is_pasid_valid(struct device *dev, int pasid)
 {
 	return -EINVAL;
 }
Signature · 1 line
2.7.4
#2 lore raw

Re: [PATCH v6 1/5] iommu/vt-d: Add debugfs support for Intel IOMMU internals

drivers/iommu/Kconfig config INTEL_IOMMU · line 158

Quoted 10 lines · Sohil Mehta

From: Gayatri Kammela <[email protected]>

Enable Intel IOMMU debug to export Intel IOMMU internals in debugfs

drivers/iommu/Kconfig +4 -0
+config INTEL_IOMMU_DEBUG
+	bool "Export Intel IOMMU internals in DebugFS"
+	depends on INTEL_IOMMU && DEBUG_FS
 
+	default n

I thought I commented on this...

Please, remove. "n" is default default, so no need to duplicate it.

drivers/iommu/Kconfig config INTEL_IOMMU · line 163

Quoted 7 lines
(quoted diff) +2 -0
+	help
+	  IOMMU internal states such as context, PASID tables can be

seen via

(quoted diff) +1 -0
+	  debugfs. Select this option if you want to export these

internals.

drivers/iommu/Kconfig +2 -0
+
+	  Say Y if you need this.

Also I lost a track how this option is being used here. It looks like this part is not related to the patch itself.

Signature · 2 lines
Andy Shevchenko <andriy.shevchenko-VuQAYsv1563Yd54FQh9/[email protected]>
Intel Finland Oy
#3 lore raw

PATCH v6 2/5 iommu/vt-d: Add debugfs support to show context internals

In reply to Sohil Mehta

From: Gayatri Kammela <[email protected]>

IOMMU internals states such as root and context can be exported to the userspace.

Example of such dump in Kabylake:

root@OTC-KBLH-01:~# cat
/sys/kernel/debug/intel_iommu/dmar_translation_struct

IOMMU dmar0: Extended Root Table Addr:402b9e800
Extended Root table entries:
Bus 0 L: 402150001 H: 0
Lower Context table entries for Bus: 0

[entry] DID :B :D .F    Low             High
[80]    0000:00:0a.00   40214fa05       102

Higher Context table entries for Bus: 0
[80] 0000:00:0a.00 40260000c 0

IOMMU dmar1: Root Table Addr:4558a3000
Root tbl entries:
Bus 0 L: 4558aa001 H: 0
Context table entries for Bus: 0

[entry] DID :B :D .F    Low             High
[160]   0000:00:14.00   4558a9001       102
[184]   0000:00:17.00   400eac001       302
[248]   0000:00:1f.00   4558af001       202
[251]   0000:00:1f.03   40070e001       502
[254]   0000:00:1f.06   4467c9001       602
 Root tbl entries:
Bus 1 L: 3fc8c2001 H: 0
 Context table entries for Bus: 1
[entry] DID :B :D .F    Low             High
[0]     0000:01:00.00   3fc8c3001       402

---

v6: Change the order of includes to an alphabetical order
Change seq_printf formatting

v5: Change to a SPDX license tag
Fix seq_printf formatting

v4: Remove the unused function parameter
Fix checkpatch.pl warnings
Remove error reporting for debugfs_create_file function
Fix unnecessary reprogramming of the context entries Simplify and merge the show context and extended context patch into one Remove redundant IOMMU null check under for_each_active_iommu

v3: Add a macro for seq file operations
Change the intel_iommu_ctx file name to dmar_translation_struct

v2: No change

 drivers/iommu/Makefile            |   1 +
 drivers/iommu/intel-iommu-debug.c | 139 ++++++++++++++++++++++++++++++++++++++
 drivers/iommu/intel-iommu.c       |   4 ++
 3 files changed, 144 insertions(+)
 create mode 100644 drivers/iommu/intel-iommu-debug.c
drivers/iommu/Makefile +1 -0
index 1fb6958..fdbaf46 100644
@@ -15,6 +15,7 @@ obj-$(CONFIG_ARM_SMMU) += arm-smmu.o
 obj-$(CONFIG_ARM_SMMU_V3) += arm-smmu-v3.o
 obj-$(CONFIG_DMAR_TABLE) += dmar.o
 obj-$(CONFIG_INTEL_IOMMU) += intel-iommu.o
+obj-$(CONFIG_INTEL_IOMMU_DEBUG) += intel-iommu-debug.o
 obj-$(CONFIG_INTEL_IOMMU_SVM) += intel-svm.o
 obj-$(CONFIG_IPMMU_VMSA) += ipmmu-vmsa.o
 obj-$(CONFIG_IRQ_REMAP) += intel_irq_remapping.o irq_remapping.o
drivers/iommu/intel-iommu-debug.c new file +139 -0

139 lines not on this page · raw message

drivers/iommu/intel-iommu.c +4 -0
index e05b8e0..351d4d9 100644
@@ -4806,6 +4806,10 @@ int __init intel_iommu_init(void)
 			  intel_iommu_cpu_dead);
 	intel_iommu_enabled = 1;
 
+#ifdef CONFIG_INTEL_IOMMU_DEBUG
+	intel_iommu_debugfs_init();
+#endif
+
 	return 0;
 
 out_free_reserved_range:
Signature · 6 lines
2.7.4

_______________________________________________
iommu mailing list
[email protected]
https://lists.linuxfoundation.org/mailman/listinfo/iommu
#4 lore raw

Re: [PATCH v6 2/5] iommu/vt-d: Add debugfs support to show context internals

Quoted 128 lines · Sohil Mehta

From: Gayatri Kammela <[email protected]>

IOMMU internals states such as root and context can be exported to the userspace.

Example of such dump in Kabylake:

root@OTC-KBLH-01:~# cat
/sys/kernel/debug/intel_iommu/dmar_translation_struct

IOMMU dmar0: Extended Root Table Addr:402b9e800
Extended Root table entries:
Bus 0 L: 402150001 H: 0
Lower Context table entries for Bus: 0

[entry] DID :B :D .F    Low             High
[80]    0000:00:0a.00   40214fa05       102

Higher Context table entries for Bus: 0
[80] 0000:00:0a.00 40260000c 0

IOMMU dmar1: Root Table Addr:4558a3000
Root tbl entries:
Bus 0 L: 4558aa001 H: 0
Context table entries for Bus: 0

[entry] DID :B :D .F    Low             High
[160]   0000:00:14.00   4558a9001       102
[184]   0000:00:17.00   400eac001       302
[248]   0000:00:1f.00   4558af001       202
[251]   0000:00:1f.03   40070e001       502
[254]   0000:00:1f.06   4467c9001       602
  Root tbl entries:
Bus 1 L: 3fc8c2001 H: 0
  Context table entries for Bus: 1
[entry] DID :B :D .F    Low             High
[0]     0000:01:00.00   3fc8c3001       402

---

v6: Change the order of includes to an alphabetical order
Change seq_printf formatting

v5: Change to a SPDX license tag
Fix seq_printf formatting

v4: Remove the unused function parameter
Fix checkpatch.pl warnings
Remove error reporting for debugfs_create_file function
Fix unnecessary reprogramming of the context entries Simplify and merge the show context and extended context patch into one Remove redundant IOMMU null check under for_each_active_iommu

v3: Add a macro for seq file operations
Change the intel_iommu_ctx file name to dmar_translation_struct

v2: No change

  drivers/iommu/Makefile            |   1 +
  drivers/iommu/intel-iommu-debug.c | 139 ++++++++++++++++++++++++++++++++++++++
  drivers/iommu/intel-iommu.c       |   4 ++
  3 files changed, 144 insertions(+)

create mode 100644 drivers/iommu/intel-iommu-debug.c

drivers/iommu/Makefile +1 -0
index 1fb6958..fdbaf46 100644
@@ -15,6 +15,7 @@ obj-$(CONFIG_ARM_SMMU) += arm-smmu.o
  obj-$(CONFIG_ARM_SMMU_V3) += arm-smmu-v3.o
  obj-$(CONFIG_DMAR_TABLE) += dmar.o
  obj-$(CONFIG_INTEL_IOMMU) += intel-iommu.o
+obj-$(CONFIG_INTEL_IOMMU_DEBUG) += intel-iommu-debug.o
  obj-$(CONFIG_INTEL_IOMMU_SVM) += intel-svm.o
  obj-$(CONFIG_IPMMU_VMSA) += ipmmu-vmsa.o
  obj-$(CONFIG_IRQ_REMAP) += intel_irq_remapping.o irq_remapping.o
drivers/iommu/intel-iommu-debug.c new file +44 -0
new file mode 100644
index 0000000..d5b0eea
@@ -0,0 +1,139 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Copyright © 2017 Intel Corporation.
+ *
+ * Authors: Gayatri Kammela <[email protected]>
+ *          Jacob Pan <[email protected]>
+ *
+ */
+
+#define pr_fmt(fmt)     "INTEL_IOMMU: " fmt
+#include <linux/debugfs.h>
+#include <linux/dmar.h>
+#include <linux/err.h>
+#include <linux/intel-iommu.h>
+#include <linux/intel-svm.h>
+#include <linux/io.h>
+#include <linux/iommu.h>
+#include <linux/pci.h>
+#include <linux/pm_runtime.h>
+#include <linux/slab.h>
+#include <linux/spinlock.h>
+#include <linux/uaccess.h>
+
+#include "irq_remapping.h"
+
+#define TOTAL_BUS_NR (256) /* full bus range 256 */
+#define DEFINE_SHOW_ATTRIBUTE(__name)					\
+static int __name ## _open(struct inode *inode, struct file *file)	\
+{									\
+	return single_open(file, __name ## _show, inode->i_private);	\
+}									\
+static const struct file_operations __name ## _fops =			\
+{									\
+	.open		= __name ## _open,				\
+	.read		= seq_read,					\
+	.llseek		= seq_lseek,					\
+	.release	= single_release,				\
+	.owner		= THIS_MODULE,					\
+}
+
+static void ctx_tbl_entry_show(struct seq_file *m, struct intel_iommu *iommu,
+			       int bus, bool ext, bool new_ext)
+{
+	struct context_entry *context;

drivers/iommu/intel-iommu-debug.c line 54

Quoted 10 lines
drivers/iommu/intel-iommu-debug.c new file +10 -0
+	int ctx;
+	unsigned long flags;
+
+	seq_printf(m, "%s Context table entries for Bus: %d\n",
+		   ext ? "Lower" : "", bus);
+	seq_puts(m, "[entry]\tDID :B :D .F\tLow\t\tHigh\n");
+
+	spin_lock_irqsave(&iommu->lock, flags);
+
+	/* Publish either context entries or extended contenxt entries */
                                                      ^^^^^^^^ typo

_______________________________________________
iommu mailing list
[email protected]
https://lists.linuxfoundation.org/mailman/listinfo/iommu

#5 lore raw

PATCH v6 3/5 iommu/vt-d: Add debugfs support to show register contents

In reply to Sohil Mehta

From: Gayatri Kammela <[email protected]>

Debugfs extension to dump all the register contents for each IOMMU device to the user space via debugfs.

example:

root@OTC-KBLH-01:~# cat /sys/kernel/debug/intel_iommu/iommu_regset

DMAR: dmar1: reg_base_addr fed90000

Name         Offset     Contents
VER          0x00       0x0000000000000010
CAP          0x08       0x01c0000c40660462
ECAP         0x10       0x0000019e2ff0505e
GCMD         0x18       0x0000000000000000
GSTS         0x1c       0x00000000c7000000
RTADDR       0x20       0x00000004558d6800
CCMD         0x28       0x0800000000000000
FSTS         0x34       0x0000000000000000
FECTL        0x38       0x0000000000000000
FEDATA       0x3c       0xfee0100c00004141
Cc Sohil Mehta <[email protected]>
Cc Fenghua Yu <[email protected]>
Cc Jacob Pan <jacob.jun.pan-VuQAYsv1563Yd54FQh9/[email protected]>
Cc Ashok Raj <[email protected]>

---

v6: No change

v5: No change

v4: Fix checkpatch.pl warnings
Remove error reporting for debugfs_create_file function Remove redundant IOMMU null check under for_each_active_iommu

v3: Use a macro for seq file operations
Change the intel_iommu_regset file name to iommu_regset Add information for MTRR registers

v2: Fix seq_printf formatting

 drivers/iommu/intel-iommu-debug.c | 78 +++++++++++++++++++++++++++++++++++++++
 include/linux/intel-iommu.h       |  2 +
 2 files changed, 80 insertions(+)
drivers/iommu/intel-iommu-debug.c +78 -0
index d5b0eea..9ee9a50 100644
@@ -125,6 +125,81 @@ static int dmar_translation_struct_show(struct seq_file *m, void *unused)
 
 DEFINE_SHOW_ATTRIBUTE(dmar_translation_struct);
 
+static int iommu_regset_show(struct seq_file *m, void *unused)
+{
+	struct dmar_drhd_unit *drhd;
+	struct intel_iommu *iommu;
+	unsigned long long base;
+	int i;
+	struct regset {
+		int offset;
+		char *regs;
+	};
+
+	static const struct regset regstr[] = {{DMAR_VER_REG, "VER"},
+					       {DMAR_CAP_REG, "CAP"},
+					       {DMAR_ECAP_REG, "ECAP"},
+					       {DMAR_GCMD_REG, "GCMD"},
+					       {DMAR_GSTS_REG, "GSTS"},
+					       {DMAR_RTADDR_REG, "RTADDR"},
+					       {DMAR_CCMD_REG, "CCMD"},
+					       {DMAR_FSTS_REG, "FSTS"},
+					       {DMAR_FECTL_REG, "FECTL"},
+					       {DMAR_FEDATA_REG, "FEDATA"},
+					       {DMAR_FEADDR_REG, "FEADDR"},
+					       {DMAR_FEUADDR_REG, "FEUADDR"},
+					       {DMAR_AFLOG_REG, "AFLOG"},
+					       {DMAR_PMEN_REG, "PMEN"},
+					       {DMAR_PLMBASE_REG, "PLMBASE"},
+					       {DMAR_PLMLIMIT_REG, "PLMLIMIT"},
+					       {DMAR_PHMBASE_REG, "PHMBASE"},
+					       {DMAR_PHMLIMIT_REG,  "PHMLIMIT"},
+					       {DMAR_IQH_REG, "IQH"},
+					       {DMAR_IQT_REG, "IQT"},
+					       {DMAR_IQ_SHIFT, "IQ"},
+					       {DMAR_IQA_REG, "IQA"},
+					       {DMAR_ICS_REG, "ICS"},
+					       {DMAR_IRTA_REG, "IRTA"},
+					       {DMAR_PQH_REG, "PQH"},
+					       {DMAR_PQT_REG, "PQT"},
+					       {DMAR_PQA_REG, "PQA"},
+					       {DMAR_PRS_REG, "PRS"},
+					       {DMAR_PECTL_REG, "PECTL"},
+					       {DMAR_PEDATA_REG, "PEDATA"},
+					       {DMAR_PEADDR_REG, "PEADDR"},
+					       {DMAR_PEUADDR_REG, "PEUADDR"},
+					       {DMAR_MTRRCAP_REG, "MTRRCAP"},
+					       {DMAR_MTRRDEF_REG, "MTRRDEF"} };
+
+	rcu_read_lock();
+	for_each_active_iommu(iommu, drhd) {
+		if (!drhd->reg_base_addr) {
+			seq_puts(m, "IOMMU: Invalid base address\n");
+			rcu_read_unlock();
+			return -EINVAL;
+		}
+
+		base = drhd->reg_base_addr;
+		seq_printf(m, "\nDMAR: %s: reg_base_addr %llx\n", iommu->name,
+			   base);
+		seq_puts(m, "Name\t\t\tOffset\t\tContents\n");
+		/*
+		 * Publish the contents of the 64-bit hardware registers
+		 * by adding the offset to the pointer(virtual addr)
+		 */
+		for (i = 0 ; i < ARRAY_SIZE(regstr); i++) {
+			seq_printf(m, "%-8s\t\t0x%02x\t\t0x%016lx\n",
+				   regstr[i].regs, regstr[i].offset,
+				   readq(iommu->reg + regstr[i].offset));
+		}
+	}
+	rcu_read_unlock();
+
+	return 0;
+}
+
+DEFINE_SHOW_ATTRIBUTE(iommu_regset);
+
 void __init intel_iommu_debugfs_init(void)
 {
 	struct dentry *iommu_debug_root;
@@ -136,4 +211,7 @@ void __init intel_iommu_debugfs_init(void)
 
 	debugfs_create_file("dmar_translation_struct", 0444, iommu_debug_root,
 			    NULL, &dmar_translation_struct_fops);
+
+	debugfs_create_file("iommu_regset", 0444, iommu_debug_root, NULL,
+			    &iommu_regset_fops);
 }
include/linux/intel-iommu.h +2 -0
index b8591dc..9d46250 100644
@@ -71,6 +71,8 @@
 #define	DMAR_PEDATA_REG	0xe4	/* Page request event interrupt data register */
 #define	DMAR_PEADDR_REG	0xe8	/* Page request event interrupt addr register */
 #define	DMAR_PEUADDR_REG 0xec	/* Page request event Upper address register */
+#define	DMAR_MTRRCAP_REG 0x100	/* Memory type range register capability register */
+#define	DMAR_MTRRDEF_REG 0x108	/* Memory type range register default type register */
 
 #define OFFSET_STRIDE		(9)
 
Signature · 1 line
2.7.4
#6 lore raw

Re: [PATCH v6 3/5] iommu/vt-d: Add debugfs support to show register contents

Quoted 21 lines · Sohil Mehta

Debugfs extension to dump all the register contents for each IOMMU device to the user space via debugfs.

example:

root@OTC-KBLH-01:~# cat /sys/kernel/debug/intel_iommu/iommu_regset

DMAR: dmar1: reg_base_addr fed90000

Name         Offset     Contents
VER          0x00       0x0000000000000010
CAP          0x08       0x01c0000c40660462
ECAP         0x10       0x0000019e2ff0505e
GCMD         0x18       0x0000000000000000
GSTS         0x1c       0x00000000c7000000
RTADDR       0x20       0x00000004558d6800
CCMD         0x28       0x0800000000000000
FSTS         0x34       0x0000000000000000
FECTL        0x38       0x0000000000000000
FEDATA       0x3c       0xfee0100c00004141
(quoted diff) +1 -0
+		seq_printf(m, "\nDMAR: %s: reg_base_addr %llx\n",

iommu->name,

drivers/iommu/intel-iommu-debug.c static int dmar_translation_struct_show(struct seq_file *m, void *unused) · line 184

Quoted 1 line
drivers/iommu/intel-iommu-debug.c +1 -0
+			   base);

Can we get rid of leading '\n'?

Signature · 2 lines
Andy Shevchenko <andriy.shevchenko-VuQAYsv1563Yd54FQh9/[email protected]>
Intel Finland Oy
#7 lore raw

PATCH v6 4/5 iommu/vt-d: Add debugfs support to show Pasid table contents

In reply to Sohil Mehta

From: Gayatri Kammela <[email protected]>

Debugfs extension to dump the internals such as pasid table entries for each IOMMU to the userspace.

Example of such dump in Kabylake:

root@OTC-KBLH-01:~# cat
/sys/kernel/debug/intel_iommu/dmar_translation_struct

IOMMU dmar0: Extended Root Table Addr:402b9e800
Extended Root table entries:
Bus 0 L: 402150001 H: 0
Lower Context table entries for Bus: 0

[entry] DID :B :D .F    Low             High
[80]    0000:00:0a.00   40214fa05       102

Higher Context table entries for Bus: 0
[80] 0000:00:0a.00 40260000c 0
Pasid Table Addr : ffff8e2d42600000
Pasid table entries for domain 0:
[Entry] Contents
[0] 12c409801

Cc Sohil Mehta <[email protected]>
Cc Fenghua Yu <[email protected]>
Cc Jacob Pan <jacob.jun.pan-VuQAYsv1563Yd54FQh9/[email protected]>
Cc Ashok Raj <[email protected]>

---

v6: No change

v5: No change

v4: Remove the unused function parameter
Fix checkpatch.pl warnings

v3: No change

v2: Fix seq_printf formatting

 drivers/iommu/intel-iommu-debug.c | 31 +++++++++++++++++++++++++++++++
 drivers/iommu/intel-svm.c         |  8 --------
 include/linux/intel-svm.h         |  8 ++++++++
 3 files changed, 39 insertions(+), 8 deletions(-)
drivers/iommu/intel-iommu-debug.c +31 -0
index 9ee9a50..d25f7c99 100644
@@ -38,6 +38,36 @@ static const struct file_operations __name ## _fops =			\
 	.owner		= THIS_MODULE,					\
 }
 
+#ifdef CONFIG_INTEL_IOMMU_SVM
+static void pasid_tbl_entry_show(struct seq_file *m, struct intel_iommu *iommu)
+{
+	int pasid_size = 0, i;
+
+	if (ecap_pasid(iommu->ecap)) {
+		pasid_size = intel_iommu_get_pts(iommu);
+		seq_printf(m, "Pasid Table Addr : %p\n", iommu->pasid_table);
+
+		if (iommu->pasid_table) {
+			seq_printf(m, "Pasid table entries for domain %d:\n",
+				   iommu->segment);
+			seq_puts(m, "[Entry]\t\tContents\n");
+
+			/* Publish the pasid table entries here */
+			for (i = 0; i < pasid_size; i++) {
+				if (!iommu->pasid_table[i].val)
+					continue;
+				seq_printf(m, "[%d]\t\t%04llx\n", i,
+					   iommu->pasid_table[i].val);
+			}
+		}
+	}
+}
+#else /* CONFIG_INTEL_IOMMU_SVM */
+static void pasid_tbl_entry_show(struct seq_file *m, struct intel_iommu *iommu)
+{
+}
+#endif /* CONFIG_INTEL_IOMMU_SVM */
+
 static void ctx_tbl_entry_show(struct seq_file *m, struct intel_iommu *iommu,
 			       int bus, bool ext, bool new_ext)
 {
@@ -71,6 +101,7 @@ static void ctx_tbl_entry_show(struct seq_file *m, struct intel_iommu *iommu,
 			}
 		}
 	}
+	pasid_tbl_entry_show(m, iommu);
 out:
 	spin_unlock_irqrestore(&iommu->lock, flags);
 }
drivers/iommu/intel-svm.c +0 -8
index ed1cf7c..c646724 100644
@@ -28,14 +28,6 @@
 
 static irqreturn_t prq_event_thread(int irq, void *d);
 
-struct pasid_entry {
-	u64 val;
-};
-
-struct pasid_state_entry {
-	u64 val;
-};
-
 int intel_svm_alloc_pasid_tables(struct intel_iommu *iommu)
 {
 	struct page *pages;
include/linux/intel-svm.h +8 -0
index 733eaf9..a8abad6 100644
@@ -18,6 +18,14 @@
 
 struct device;
 
+struct pasid_entry {
+	u64 val;
+};
+
+struct pasid_state_entry {
+	u64 val;
+};
+
 struct svm_dev_ops {
 	void (*fault_cb)(struct device *dev, int pasid, u64 address,
 			 u32 private, int rwxp, int response);
Signature · 1 line
2.7.4
#8 lore raw

Re: [PATCH v6 4/5] iommu/vt-d: Add debugfs support to show Pasid table contents

Quoted 26 lines · Sohil Mehta

Debugfs extension to dump the internals such as pasid table entries for
each IOMMU to the userspace.

Example of such dump in Kabylake:

root@OTC-KBLH-01:~# cat
/sys/kernel/debug/intel_iommu/dmar_translation_struct

IOMMU dmar0: Extended Root Table Addr:402b9e800
Extended Root table entries:
Bus 0 L: 402150001 H: 0
Lower Context table entries for Bus: 0

[entry] DID :B :D .F    Low             High
[80]    0000:00:0a.00   40214fa05       102

Higher Context table entries for Bus: 0
[80] 0000:00:0a.00 40260000c 0
Pasid Table Addr : ffff8e2d42600000
Pasid table entries for domain 0:
[Entry] Contents
[0] 12c409801

(quoted diff) +2 -0
+#ifdef CONFIG_INTEL_IOMMU_SVM
+static void pasid_tbl_entry_show(struct seq_file *m, struct

intel_iommu *iommu)

drivers/iommu/intel-iommu-debug.c static const struct file_operations __name ## _fops = \ · line 46

Quoted 5 lines
drivers/iommu/intel-iommu-debug.c +4 -0
+{
+	int pasid_size = 0, i;
+
 
+	if (ecap_pasid(iommu->ecap)) {

A nit:
if you use negative condition you can reduce indentation level below.

if (!...)
return;

pasid_size = ...;
...

Review

Quoted 11 lines
drivers/iommu/intel-iommu-debug.c +6 -0
+		pasid_size = intel_iommu_get_pts(iommu);
+
 
+	}
+}
+#else /* CONFIG_INTEL_IOMMU_SVM */
 
+static void pasid_tbl_entry_show(struct seq_file *m, struct

intel_iommu *iommu)

(quoted diff) +2 -0
+{
+}

I think is not a big deal if you put all in one line like static ... {}

Quoted 1 line

+#endif /* CONFIG_INTEL_IOMMU_SVM */

Signature · 2 lines
Andy Shevchenko <[email protected]>
Intel Finland Oy
#9 lore raw

PATCH v6 5/5 iommu/vt-d: Add debugfs support for Interrupt remapping

In reply to Sohil Mehta

Debugfs extension for Intel IOMMU to dump Interrupt remapping table entries for Interrupt remapping and Interrupt posting.

The file /sys/kernel/debug/intel_iommu/ir_translation_struct provides detailed information, such as Index, Source Id, Destination Id, Vector and the raw values for entries with the present bit set, in the format shown.

Remapped Interrupt supported on IOMMU: dmar5
IR table address:ffff93e09d54c310
-----------------------------------------------------------

 Index  SID  Dest_ID  Vct Raw_value_high   Raw_value_low
 1      3a00 00000600 2c  0000000000043a00 00000600002c0009
 111    4301 00000900 a2  0000000000044301 0000090000a20009

Posted Interrupt supported on IOMMU: dmar5
IR table address:ffff93e09d54c310
--------------------------------------------------------------------

 Index  SID  PDA_high PDA_low  Vct Raw_value_high   Raw_value_low
 4      4300 00000010 40c7c880 41  0000001000044300 40c7c88000418001
 5      4300 00000010 40c7c880 51  0000001000044300 40c7c88000518001
Cc Gayatri Kammela <[email protected]>
Cc Jacob Pan <jacob.jun.pan-VuQAYsv1563Yd54FQh9/[email protected]>
Cc Fenghua Yu <[email protected]>
Cc Ashok Raj <[email protected]>

---

v6: Change a couple of seq_puts to seq_putc

v5: Fix seq_puts formatting and remove leading '\n's

v4: Remove the unused function parameter
Fix checkpatch.pl warnings
Remove error reporting for debugfs_create_file function Remove redundant IOMMU null check under for_each_active_iommu

v3: Use a macro for seq file operations
Change the intel_iommu_interrupt_remap file name to ir_translation_struct

v2: Handle the case when IR is not enabled. Fix seq_printf formatting

 drivers/iommu/intel-iommu-debug.c | 98 +++++++++++++++++++++++++++++++++++++++
 1 file changed, 98 insertions(+)
drivers/iommu/intel-iommu-debug.c +98 -0
index d25f7c99..b714c6f 100644
@@ -4,6 +4,7 @@
  *
  * Authors: Gayatri Kammela <[email protected]>
  *          Jacob Pan <jacob.jun.pan-VuQAYsv1563Yd54FQh9/[email protected]>
+ *          Sohil Mehta <[email protected]>
  *
  */
 
@@ -231,6 +232,98 @@ static int iommu_regset_show(struct seq_file *m, void *unused)
 
 DEFINE_SHOW_ATTRIBUTE(iommu_regset);
 
+#ifdef CONFIG_IRQ_REMAP
+static void ir_tbl_remap_entry_show(struct seq_file *m,
+				    struct intel_iommu *iommu)
+{
+	int idx;
+	struct irte *ri_entry;
+
+	/* Print the header only once */
+	seq_puts(m, " Index  SID  Dest_ID  Vct Raw_value_high   Raw_value_low\n");
+
+	for (idx = 0; idx < INTR_REMAP_TABLE_ENTRIES; idx++) {
+		ri_entry = &iommu->ir_table->base[idx];
+		if (!ri_entry->present || ri_entry->p_pst)
+			continue;
+		seq_printf(m, " %d\t%04x %08x %02x  %016llx %016llx\n", idx,
+			   ri_entry->sid, ri_entry->dest_id, ri_entry->vector,
+			   ri_entry->high, ri_entry->low);
+	}
+}
+
+static void ir_tbl_posted_entry_show(struct seq_file *m,
+				     struct intel_iommu *iommu)
+{
+	int idx;
+	struct irte *pi_entry;
+
+	/* Print the header only once */
+	seq_puts(m, " Index  SID  PDA_high PDA_low  Vct Raw_value_high   Raw_value_low\n");
+
+	for (idx = 0; idx < INTR_REMAP_TABLE_ENTRIES; idx++) {
+		pi_entry = &iommu->ir_table->base[idx];
+		if (!pi_entry->present || !pi_entry->p_pst)
+			continue;
+		seq_printf(m, " %d\t%04x %08x %08x %02x  %016llx %016llx\n",
+			   idx, pi_entry->sid, pi_entry->pda_h,
+			   (pi_entry->pda_l)<<6, pi_entry->vector,
+			   pi_entry->high, pi_entry->low);
+	}
+}
+
+/*
+ * For active IOMMUs go through the Interrupt remapping
+ * table and print valid entries in a table format for
+ * Remapped and Posted Interrupts.
+ */
+static int ir_translation_struct_show(struct seq_file *m, void *unused)
+{
+	struct dmar_drhd_unit *drhd;
+	struct intel_iommu *iommu;
+
+	rcu_read_lock();
+	for_each_active_iommu(iommu, drhd) {
+		if (!ecap_ir_support(iommu->ecap))
+			continue;
+
+		seq_printf(m, "Remapped Interrupt supported on IOMMU: %s\n"
+			   " IR table address:%p\n", iommu->name,
+			   iommu->ir_table);
+		seq_puts(m, "-----------------------------------------------------------\n");
+
+		if (iommu->ir_table)
+			ir_tbl_remap_entry_show(m, iommu);
+		else
+			seq_puts(m, "Interrupt Remapping is not enabled\n");
+		seq_putc(m, '\n');
+	}
+
+	seq_puts(m, "****\t****\t****\t****\t****\t****\t****\t****\n");
+
+	for_each_active_iommu(iommu, drhd) {
+		if (!cap_pi_support(iommu->cap))
+			continue;
+
+		seq_printf(m, "Posted Interrupt supported on IOMMU: %s\n"
+			   " IR table address:%p\n", iommu->name,
+			   iommu->ir_table);
+		seq_puts(m, "--------------------------------------------------------------------\n");
+
+		if (iommu->ir_table)
+			ir_tbl_posted_entry_show(m, iommu);
+		else
+			seq_puts(m, "Interrupt Remapping is not enabled\n");
+		seq_putc(m, '\n');
+	}
+	rcu_read_unlock();
+
+	return 0;
+}
+
+DEFINE_SHOW_ATTRIBUTE(ir_translation_struct);
+#endif
+
 void __init intel_iommu_debugfs_init(void)
 {
 	struct dentry *iommu_debug_root;
@@ -245,4 +338,9 @@ void __init intel_iommu_debugfs_init(void)
 
 	debugfs_create_file("iommu_regset", 0444, iommu_debug_root, NULL,
 			    &iommu_regset_fops);
+
+#ifdef CONFIG_IRQ_REMAP
+	debugfs_create_file("ir_translation_struct", 0444, iommu_debug_root,
+			    NULL, &ir_translation_struct_fops);
+#endif
 }
Signature · 1 line
2.7.4
#10 lore raw

Re: [PATCH v6 5/5] iommu/vt-d: Add debugfs support for Interrupt remapping

Quoted 26 lines · Sohil Mehta

Debugfs extension for Intel IOMMU to dump Interrupt remapping table entries for Interrupt remapping and Interrupt posting.

The file /sys/kernel/debug/intel_iommu/ir_translation_struct provides detailed information, such as Index, Source Id, Destination Id, Vector and the raw values for entries with the present bit set, in the format shown.

Remapped Interrupt supported on IOMMU: dmar5
IR table address:ffff93e09d54c310
-----------------------------------------------------------

 Index  SID  Dest_ID  Vct Raw_value_high   Raw_value_low
 1      3a00 00000600 2c  0000000000043a00 00000600002c0009
 111    4301 00000900 a2  0000000000044301 0000090000a20009

Posted Interrupt supported on IOMMU: dmar5
IR table address:ffff93e09d54c310
--------------------------------------------------------------------

 Index  SID  PDA_high PDA_low  Vct Raw_value_high   Raw_value_low
 4      4300 00000010 40c7c880 41  0000001000044300 40c7c88000418001
 5      4300 00000010 40c7c880 51  0000001000044300 40c7c88000518001
(quoted diff) +1 -0
+		seq_printf(m, "Remapped Interrupt supported on IOMMU:

%s\n"

Review

Quoted 2 lines
(quoted diff) +2 -0
+			   " IR table address:%p\n", iommu->name,
+			   iommu->ir_table);

A nit: If you indent second part of string literal to follow the indentation of first one, it would be slightly better for my opinion.

Review

Quoted 4 lines
(quoted diff) +1 -0
+		seq_printf(m, "Posted Interrupt supported on IOMMU:

%s\n"

(quoted diff) +2 -0
+			   " IR table address:%p\n", iommu->name,
+			   iommu->ir_table);

Ditto.

Signature · 2 lines
Andy Shevchenko <andriy.shevchenko-VuQAYsv1563Yd54FQh9/[email protected]>
Intel Finland Oy
#11 lore raw
In reply to Sohil Mehta
Quoted 8 lines · Sohil Mehta

Hi All,

This series aims to add debugfs support for Intel IOMMU. It exposes IOMMU
registers, internal context and dumps individual table entries to help debug
Intel IOMMUs.

Quoted 11 lines

The first patch does the ground work for the following patches by creating a
new Kconfig option - INTEL_IOMMU_DEBUG. It also reorganizes some Intel IOMMU
data structures. The next four patches add debugfs support for IOMMU context
internals, register contents, PASID internals, and Interrupt remapping in
that order. The information can be accessed in sysfs at '/sys/kernel/debug/intel_iommu/'.

Thanks for an update.
I found small issues, otherwise looks pretty much good.

P.S. During re-indendation take care of the lines that can be combined to one line.

Quoted 52 lines

Regards,
Sohil

Changes since v5:
- Change the order of includes to an alphabetical order
- Change seq_printf and seq_puts formatting

Changes since v4:
- Change to a SPDX license tag
- Fix seq_printf formatting and remove leading '\n's

Changes since v3:
- Remove an unused function parameter from some of the functions
- Fix checkpatch.pl warnings
- Remove error reporting for debugfs_create_file functions
- Fix unnecessary reprogramming of the context entries
- Simplify and merge the show context and extended context patch into
one
- Remove redundant IOMMU null check under for_each_active_iommu
- Update the commit title to be consistent

Changes since v2:
- Added a macro for seq file operations based on recommendation by
Andy
Shevchenko. The marco can be moved to seq_file.h at a future point
- Changed the debugfs file names to more relevant ones
- Added information for MTRR registers in the regset file

Changes since v1:
- Fixed seq_printf formatting
- Handled the case when Interrupt remapping is not enabled

Gayatri Kammela (4):
iommu/vt-d: Add debugfs support for Intel IOMMU internals iommu/vt-d: Add debugfs support to show context internals iommu/vt-d: Add debugfs support to show register contents iommu/vt-d: Add debugfs support to show Pasid table contents

Sohil Mehta (1):
iommu/vt-d: Add debugfs support for Interrupt remapping

 drivers/iommu/Kconfig             |  10 ++
 drivers/iommu/Makefile            |   1 +
 drivers/iommu/intel-iommu-debug.c | 346
++++++++++++++++++++++++++++++++++++++
 drivers/iommu/intel-iommu.c       |  35 +---
 drivers/iommu/intel-svm.c         |   8 -
 include/linux/intel-iommu.h       |  34 ++++
 include/linux/intel-svm.h         |  10 +-
 7 files changed, 407 insertions(+), 37 deletions(-)
 create mode 100644 drivers/iommu/intel-iommu-debug.c
Signature · 2 lines
Andy Shevchenko <[email protected]>
Intel Finland Oy
#12 lore raw
Quoted 25 lines · Andy Shevchenko
Quoted 10 lines · Sohil Mehta

Hi All,

This series aims to add debugfs support for Intel IOMMU. It exposes IOMMU
registers, internal context and dumps individual table entries to help
debug
Intel IOMMUs.

Quoted 14 lines

The first patch does the ground work for the following patches by creating a
new Kconfig option - INTEL_IOMMU_DEBUG. It also reorganizes some Intel
IOMMU
data structures. The next four patches add debugfs support for IOMMU
context
internals, register contents, PASID internals, and Interrupt remapping
in
that order. The information can be accessed in sysfs at '/sys/kernel/debug/intel_iommu/'.

Quoted 7 lines

Thanks for an update.
I found small issues, otherwise looks pretty much good.

P.S. During re-indendation take care of the lines that can be combined
to one line.

Thanks Andy for the review. Will fix the issues that you pointed and send out the updated series.

Sohil

Quoted 44 lines
Quoted 44 lines

Regards,
Sohil

Changes since v5:
 - Change the order of includes to an alphabetical order
 - Change seq_printf and seq_puts formatting

Changes since v4:
 - Change to a SPDX license tag
 - Fix seq_printf formatting and remove leading '\n's

Changes since v3:
 - Remove an unused function parameter from some of the functions
 - Fix checkpatch.pl warnings
 - Remove error reporting for debugfs_create_file functions
 - Fix unnecessary reprogramming of the context entries
 - Simplify and merge the show context and extended context patch into
one
 - Remove redundant IOMMU null check under for_each_active_iommu
 - Update the commit title to be consistent

Changes since v2:
 - Added a macro for seq file operations based on recommendation by Andy
   Shevchenko. The marco can be moved to seq_file.h at a future point
 - Changed the debugfs file names to more relevant ones
 - Added information for MTRR registers in the regset file

Changes since v1:
 - Fixed seq_printf formatting
 - Handled the case when Interrupt remapping is not enabled

Gayatri Kammela (4):
  iommu/vt-d: Add debugfs support for Intel IOMMU internals iommu/vt-d: Add debugfs support to show context internals iommu/vt-d: Add debugfs support to show register contents iommu/vt-d: Add debugfs support to show Pasid table contents

Sohil Mehta (1):
  iommu/vt-d: Add debugfs support for Interrupt remapping

Quoted 11 lines
Quoted 11 lines

 drivers/iommu/Kconfig             |  10 ++
 drivers/iommu/Makefile            |   1 +
 drivers/iommu/intel-iommu-debug.c | 346
++++++++++++++++++++++++++++++++++++++
 drivers/iommu/intel-iommu.c       |  35 +---
 drivers/iommu/intel-svm.c         |   8 -
 include/linux/intel-iommu.h       |  34 ++++
 include/linux/intel-svm.h         |  10 +-

 7 files changed, 407 insertions(+), 37 deletions(-)

 create mode 100644 drivers/iommu/intel-iommu-debug.c

_______________________________________________
iommu mailing list
[email protected]
https://lists.linuxfoundation.org/mailman/listinfo/iommu