Skip to contentSkip to conversation

Add Intel IOMMU debugfs support

PATCH Applied Sohil Mehta 2 participants 8 messages Sep 12, 2018 – Sep 25, 2018

Subsystems (2)
lore
Share & tools
lore
#0 lore raw

v8 cover Add Intel IOMMU debugfs support

Hi All,

This series aims to add debugfs support for Intel IOMMU. It exposes IOMMU registers, internal context and dumps individual table entries to help debug Intel IOMMUs.

Patches 1-3 do the ground work for the following patches by reorganizing some Intel IOMMU data structures, creating a new config option - INTEL_IOMMU_DEBUGFS and adding an Intel IOMMU debugfs directory. The following patches add debugfs support for IOMMU resigter contents, Interrupt remapping and context internals in that order. The information can be accessed in sysfs at '/sys/kernel/debug/iommu/intel/'.

This series is based on the new VT-d 3.0 specification [1]. The scalable mode and PASID table debugfs patches are not part of this series. They would be submitted later once the IOMMU driver patches for the same are accepted and merged upstream.

Regards,
Sohil

[1] https://software.intel.com/sites/default/files/managed/c5/15/vt-directed-io-spec.pdf

Changes since v7:
- Move to the common iommu directory approach suggested by Joerg
- Remove ECS debugfs support that has been deprecated with the new VT-d spec
- Change filename to intel-iommu-debugfs.c
- Remove the pasid table debugfs patch
- Add MTRR and Virtual command register support
- Use locks to protect register and table access
- Improve seq_printf formatting
- Simplify and split the patches differently

Changes since v6:
- Split patch 1/5 and 2/5 differently
- Simplify and improve code formatting
- Use macro for register set definitions
- Fix compiler warning for readq
- Add Co-Developed-by tag to commit messages

Changes since v5:
- Change the order of includes to an alphabetical order
- Change seq_printf and seq_puts formatting

Changes since v4:
- Change to a SPDX license tag
- Fix seq_printf formatting and remove leading '\n's

Changes since v3:
- Remove an unused function parameter from some of the functions
- Fix checkpatch.pl warnings
- Remove error reporting for debugfs_create_file functions
- Fix unnecessary reprogramming of the context entries
- Simplify and merge the show context and extended context patch into one
- Remove redundant IOMMU null check under for_each_active_iommu
- Update the commit title to be consistent

Changes since v2:
- Added a macro for seq file operations based on recommendation by Andy Shevchenko. The marco can be moved to seq_file.h at a future point
- Changed the debugfs file names to more relevant ones
- Added information for MTRR registers in the regset file

Changes since v1:
- Fixed seq_printf formatting
- Handled the case when Interrupt remapping is not enabled

Gayatri Kammela (1):
iommu/vt-d: Add debugfs support to show register contents

Sohil Mehta (5):
iommu/vt-d: Relocate struct/function declarations to its header files iommu/vt-d: Update register definitions to VT-d 3.0 specification iommu/vt-d: Enable base Intel IOMMU debugfs support iommu/vt-d: Add debugfs support for Interrupt remapping iommu/vt-d: Add debugfs support to show context internals

 arch/x86/include/asm/irq_remapping.h |   2 +
 drivers/iommu/Kconfig                |  13 ++
 drivers/iommu/Makefile               |   1 +
 drivers/iommu/intel-iommu-debugfs.c  | 314 +++++++++++++++++++++++++++
 drivers/iommu/intel-iommu.c          |  32 +--
 drivers/iommu/intel_irq_remapping.c  |   2 +-
 include/linux/intel-iommu.h          |  72 ++++++
 7 files changed, 407 insertions(+), 29 deletions(-)
 create mode 100644 drivers/iommu/intel-iommu-debugfs.c
Signature · 1 line
2.18.0
#1 lore raw

PATCH v8 1/6 iommu/vt-d: Relocate struct/function declarations to its header files

To reuse the static functions and the struct declarations, move them to corresponding header files and export the needed functions.

---

v8: Remove pasid table related changes
Add Interrupt remapping related changes
Remove redundant extern type from functions

v7: Split patch 1/5 and 2/5 differently
Update the commit message

v6: No change

v5: No change

v4: No change

v3: No change

v2: No change

 arch/x86/include/asm/irq_remapping.h |  2 ++
 drivers/iommu/intel-iommu.c          | 31 +++-------------------------
 drivers/iommu/intel_irq_remapping.c  |  2 +-
 include/linux/intel-iommu.h          | 30 +++++++++++++++++++++++++++
 4 files changed, 36 insertions(+), 29 deletions(-)
arch/x86/include/asm/irq_remapping.h +2 -0
index 5f26962eff42..67ed72f31cc2 100644
@@ -45,6 +45,8 @@ struct vcpu_data {
 
 #ifdef CONFIG_IRQ_REMAP
 
+extern raw_spinlock_t irq_2_ir_lock;
+
 extern bool irq_remapping_cap(enum irq_remap_cap cap);
 extern void set_irq_remapping_broken(void);
 extern int irq_remapping_prepare(void);
drivers/iommu/intel-iommu.c +3 -28
index 5f3f10cf9d9d..b5868c757995 100644
@@ -185,16 +185,6 @@ static int rwbf_quirk;
 static int force_on = 0;
 int intel_iommu_tboot_noforce;
 
-/*
- * 0: Present
- * 1-11: Reserved
- * 12-63: Context Ptr (12 - (haw-1))
- * 64-127: Reserved
- */
-struct root_entry {
-	u64	lo;
-	u64	hi;
-};
 #define ROOT_ENTRY_NR (VTD_PAGE_SIZE/sizeof(struct root_entry))
 
 /*
@@ -220,21 +210,6 @@ static phys_addr_t root_entry_uctp(struct root_entry *re)
 
 	return re->hi & VTD_PAGE_MASK;
 }
-/*
- * low 64 bits:
- * 0: present
- * 1: fault processing disable
- * 2-3: translation type
- * 12-63: address space root
- * high 64 bits:
- * 0-2: address width
- * 3-6: aval
- * 8-23: domain id
- */
-struct context_entry {
-	u64 lo;
-	u64 hi;
-};
 
 static inline void context_clear_pasid_enable(struct context_entry *context)
 {
@@ -261,7 +236,7 @@ static inline bool __context_present(struct context_entry *context)
 	return (context->lo & 1);
 }
 
-static inline bool context_present(struct context_entry *context)
+bool context_present(struct context_entry *context)
 {
 	return context_pasid_enabled(context) ?
 	     __context_present(context) :
@@ -788,8 +763,8 @@ static void domain_update_iommu_cap(struct dmar_domain *domain)
 	domain->iommu_superpage = domain_update_iommu_superpage(NULL);
 }
 
-static inline struct context_entry *iommu_context_addr(struct intel_iommu *iommu,
-						       u8 bus, u8 devfn, int alloc)
+struct context_entry *iommu_context_addr(struct intel_iommu *iommu, u8 bus,
+					 u8 devfn, int alloc)
 {
 	struct root_entry *root = &iommu->root_entry[bus];
 	struct context_entry *context;
drivers/iommu/intel_irq_remapping.c +1 -1
index 967450bd421a..c2d6c11431de 100644
@@ -76,7 +76,7 @@ static struct hpet_scope ir_hpet[MAX_HPET_TBS];
  * in single-threaded environment with interrupt disabled, so no need to tabke
  * the dmar_global_lock.
  */
-static DEFINE_RAW_SPINLOCK(irq_2_ir_lock);
+DEFINE_RAW_SPINLOCK(irq_2_ir_lock);
 static const struct irq_domain_ops intel_ir_domain_ops;
 
 static void iommu_disable_irq_remapping(struct intel_iommu *iommu);
include/linux/intel-iommu.h +30 -0
index 28004d74ae04..b7cf32e8ae1f 100644
@@ -389,6 +389,33 @@ struct pasid_entry;
 struct pasid_state_entry;
 struct page_req_dsc;
 
+/*
+ * 0: Present
+ * 1-11: Reserved
+ * 12-63: Context Ptr (12 - (haw-1))
+ * 64-127: Reserved
+ */
+struct root_entry {
+	u64     lo;
+	u64     hi;
+};
+
+/*
+ * low 64 bits:
+ * 0: present
+ * 1: fault processing disable
+ * 2-3: translation type
+ * 12-63: address space root
+ * high 64 bits:
+ * 0-2: address width
+ * 3-6: aval
+ * 8-23: domain id
+ */
+struct context_entry {
+	u64 lo;
+	u64 hi;
+};
+
 struct dmar_domain {
 	int	nid;			/* node id */
 
@@ -559,5 +586,8 @@ extern struct intel_iommu *intel_svm_device_to_iommu(struct device *dev);
 #endif
 
 extern const struct attribute_group *intel_iommu_groups[];
+bool context_present(struct context_entry *context);
+struct context_entry *iommu_context_addr(struct intel_iommu *iommu, u8 bus,
+					 u8 devfn, int alloc);
 
 #endif
Signature · 1 line
2.18.0
#2 lore raw

PATCH v8 2/6 iommu/vt-d: Update register definitions to VT-d 3.0 specification

In reply to Sohil Mehta

Add new register definitions added in the VT-d 3.0 specification. Also include registers that were missing previously.

Cc Lu Baolu <[email protected]>
Cc Fenghua Yu <[email protected]>
Cc Jacob Pan <[email protected]>
Cc Ashok Raj <[email protected]>
Cc Gayatri Kammela <[email protected]>

---

v8: Simplify and split the patches differently

 include/linux/intel-iommu.h | 36 ++++++++++++++++++++++++++++++++++++
 1 file changed, 36 insertions(+)
include/linux/intel-iommu.h +36 -0
index b7cf32e8ae1f..3bdb9aa198af 100644
@@ -72,6 +72,42 @@
 #define	DMAR_PEDATA_REG	0xe4	/* Page request event interrupt data register */
 #define	DMAR_PEADDR_REG	0xe8	/* Page request event interrupt addr register */
 #define	DMAR_PEUADDR_REG 0xec	/* Page request event Upper address register */
+#define DMAR_MTRRCAP_REG 0x100	/* MTRR capability register */
+#define DMAR_MTRRDEF_REG 0x108	/* MTRR default type register */
+#define DMAR_MTRR_FIX64K_00000_REG 0x120 /* MTRR Fixed range registers */
+#define DMAR_MTRR_FIX16K_80000_REG 0x128
+#define DMAR_MTRR_FIX16K_A0000_REG 0x130
+#define DMAR_MTRR_FIX4K_C0000_REG 0x138
+#define DMAR_MTRR_FIX4K_C8000_REG 0x140
+#define DMAR_MTRR_FIX4K_D0000_REG 0x148
+#define DMAR_MTRR_FIX4K_D8000_REG 0x150
+#define DMAR_MTRR_FIX4K_E0000_REG 0x158
+#define DMAR_MTRR_FIX4K_E8000_REG 0x160
+#define DMAR_MTRR_FIX4K_F0000_REG 0x168
+#define DMAR_MTRR_FIX4K_F8000_REG 0x170
+#define DMAR_MTRR_PHYSBASE0_REG 0x180 /* MTRR Variable range registers */
+#define DMAR_MTRR_PHYSMASK0_REG 0x188
+#define DMAR_MTRR_PHYSBASE1_REG 0x190
+#define DMAR_MTRR_PHYSMASK1_REG 0x198
+#define DMAR_MTRR_PHYSBASE2_REG 0x1a0
+#define DMAR_MTRR_PHYSMASK2_REG 0x1a8
+#define DMAR_MTRR_PHYSBASE3_REG 0x1b0
+#define DMAR_MTRR_PHYSMASK3_REG 0x1b8
+#define DMAR_MTRR_PHYSBASE4_REG 0x1c0
+#define DMAR_MTRR_PHYSMASK4_REG 0x1c8
+#define DMAR_MTRR_PHYSBASE5_REG 0x1d0
+#define DMAR_MTRR_PHYSMASK5_REG 0x1d8
+#define DMAR_MTRR_PHYSBASE6_REG 0x1e0
+#define DMAR_MTRR_PHYSMASK6_REG 0x1e8
+#define DMAR_MTRR_PHYSBASE7_REG 0x1f0
+#define DMAR_MTRR_PHYSMASK7_REG 0x1f8
+#define DMAR_MTRR_PHYSBASE8_REG 0x200
+#define DMAR_MTRR_PHYSMASK8_REG 0x208
+#define DMAR_MTRR_PHYSBASE9_REG 0x210
+#define DMAR_MTRR_PHYSMASK9_REG 0x218
+#define DMAR_VCCAP_REG		0xe00 /* Virtual command capability register */
+#define DMAR_VCMD_REG		0xe10 /* Virtual command register */
+#define DMAR_VCRSP_REG		0xe20 /* Virtual command response register */
 
 #define OFFSET_STRIDE		(9)
 
Signature · 1 line
2.18.0
#3 lore raw

PATCH v8 3/6 iommu/vt-d: Enable base Intel IOMMU debugfs support

In reply to Sohil Mehta

Add a new config option CONFIG_INTEL_IOMMU_DEBUGFS and do the base enabling for Intel IOMMU debugfs.

---

v8: Move to the common iommu debugfs directory
Simplify and split the patches differently
Change filename to intel-iommu-debugfs.c

 drivers/iommu/Kconfig               | 13 +++++++++++++
 drivers/iommu/Makefile              |  1 +
 drivers/iommu/intel-iommu-debugfs.c | 20 ++++++++++++++++++++
 drivers/iommu/intel-iommu.c         |  1 +
 include/linux/intel-iommu.h         |  6 ++++++
 5 files changed, 41 insertions(+)
 create mode 100644 drivers/iommu/intel-iommu-debugfs.c
drivers/iommu/Kconfig +13 -0
index c60395b7470f..51ba19c8847b 100644
@@ -186,6 +186,19 @@ config INTEL_IOMMU
 	  and include PCI device scope covered by these DMA
 	  remapping devices.
 
+config INTEL_IOMMU_DEBUGFS
+	bool "Export Intel IOMMU internals in Debugfs"
+	depends on INTEL_IOMMU && IOMMU_DEBUGFS
+	help
+	  !!!WARNING!!!
+
+	  DO NOT ENABLE THIS OPTION UNLESS YOU REALLY KNOW WHAT YOU ARE DOING!!!
+
+	  Expose Intel IOMMU internals in Debugfs.
+
+	  This option is -NOT- intended for production environments, and should
+	  only be enabled for debugging Intel IOMMU.
+
 config INTEL_IOMMU_SVM
 	bool "Support for Shared Virtual Memory with Intel IOMMU"
 	depends on INTEL_IOMMU && X86
drivers/iommu/Makefile +1 -0
index ab5eba6edf82..a158a68c8ea8 100644
@@ -17,6 +17,7 @@ obj-$(CONFIG_ARM_SMMU) += arm-smmu.o
 obj-$(CONFIG_ARM_SMMU_V3) += arm-smmu-v3.o
 obj-$(CONFIG_DMAR_TABLE) += dmar.o
 obj-$(CONFIG_INTEL_IOMMU) += intel-iommu.o intel-pasid.o
+obj-$(CONFIG_INTEL_IOMMU_DEBUGFS) += intel-iommu-debugfs.o
 obj-$(CONFIG_INTEL_IOMMU_SVM) += intel-svm.o
 obj-$(CONFIG_IPMMU_VMSA) += ipmmu-vmsa.o
 obj-$(CONFIG_IRQ_REMAP) += intel_irq_remapping.o irq_remapping.o
drivers/iommu/intel-iommu-debugfs.c new file +20 -0
new file mode 100644
index 000000000000..33e0a6c12d92
@@ -0,0 +1,20 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Copyright © 2018 Intel Corporation.
+ *
+ * Authors: Gayatri Kammela <[email protected]>
+ *	    Sohil Mehta <[email protected]>
+ *	    Jacob Pan <[email protected]>
+ */
+
+#include <linux/debugfs.h>
+#include <linux/dmar.h>
+#include <linux/intel-iommu.h>
+#include <linux/pci.h>
+
+#include <asm/irq_remapping.h>
+
+void __init intel_iommu_debugfs_init(void)
+{
+	debugfs_create_dir("intel", iommu_debugfs_dir);
+}
drivers/iommu/intel-iommu.c +1 -0
index b5868c757995..b9cf7c8c1616 100644
@@ -4837,6 +4837,7 @@ int __init intel_iommu_init(void)
 	cpuhp_setup_state(CPUHP_IOMMU_INTEL_DEAD, "iommu/intel:dead", NULL,
 			  intel_iommu_cpu_dead);
 	intel_iommu_enabled = 1;
+	intel_iommu_debugfs_init();
 
 	return 0;
 
include/linux/intel-iommu.h +6 -0
index 3bdb9aa198af..b0ae25837361 100644
@@ -621,6 +621,12 @@ extern int intel_iommu_enable_pasid(struct intel_iommu *iommu, struct intel_svm_
 extern struct intel_iommu *intel_svm_device_to_iommu(struct device *dev);
 #endif
 
+#ifdef CONFIG_INTEL_IOMMU_DEBUGFS
+void intel_iommu_debugfs_init(void);
+#else
+static inline void intel_iommu_debugfs_init(void) {}
+#endif /* CONFIG_INTEL_IOMMU_DEBUGFS */
+
 extern const struct attribute_group *intel_iommu_groups[];
 bool context_present(struct context_entry *context);
 struct context_entry *iommu_context_addr(struct intel_iommu *iommu, u8 bus,
Signature · 1 line
2.18.0
#4 lore raw

PATCH v8 4/6 iommu/vt-d: Add debugfs support to show register contents

In reply to Sohil Mehta

From: Gayatri Kammela <[email protected]>

Add Intel IOMMU debugfs extension to dump all the register contents for each IOMMU device.

Example:

root@OTC-KBLH-01:~# cat /sys/kernel/debug/iommu/intel/iommu_regset
IOMMU: dmar0 Register Base Address: fed90000
Name            Offset          Contents
VER             0x00            0x0000000000000010
CAP             0x08            0x01c0000c40660462
ECAP            0x10            0x0000000000f0101a
GCMD            0x18            0x0000000000000000
GSTS            0x1c            0x00000000c7000000
RTADDR          0x20            0x00000004071d3800
CCMD            0x28            0x0800000000000000
FSTS            0x34            0x0000000000000000
FECTL           0x38            0x0000000000000000
FEDATA          0x3c            0xfee0100400004021

---

v8: Add MTRR and Virtual command register support
Use the iommu register lock to protect register access

v7: Use macro for register set definitions
Fix compiler warning for readq with 32bit architecture Remove leading '\n'

v6: No change

v5: No change

v4: Fix checkpatch.pl warnings
Remove error reporting for debugfs_create_file function Remove redundant IOMMU null check under for_each_active_iommu

v3: Use a macro for seq file operations
Change the intel_iommu_regset file name to iommu_regset Add information for MTRR registers

v2: Fix seq_printf formatting

 drivers/iommu/intel-iommu-debugfs.c | 123 +++++++++++++++++++++++++++-
 1 file changed, 122 insertions(+), 1 deletion(-)
drivers/iommu/intel-iommu-debugfs.c +122 -1
index 33e0a6c12d92..57af2f774312 100644
@@ -14,7 +14,128 @@
 
 #include <asm/irq_remapping.h>
 
+struct iommu_regset {
+	int offset;
+	const char *regs;
+};
+
+#define IOMMU_REGSET_ENTRY(_reg_)					\
+	{ DMAR_##_reg_##_REG, __stringify(_reg_) }
+static const struct iommu_regset iommu_regs[] = {
+	IOMMU_REGSET_ENTRY(VER),
+	IOMMU_REGSET_ENTRY(CAP),
+	IOMMU_REGSET_ENTRY(ECAP),
+	IOMMU_REGSET_ENTRY(GCMD),
+	IOMMU_REGSET_ENTRY(GSTS),
+	IOMMU_REGSET_ENTRY(RTADDR),
+	IOMMU_REGSET_ENTRY(CCMD),
+	IOMMU_REGSET_ENTRY(FSTS),
+	IOMMU_REGSET_ENTRY(FECTL),
+	IOMMU_REGSET_ENTRY(FEDATA),
+	IOMMU_REGSET_ENTRY(FEADDR),
+	IOMMU_REGSET_ENTRY(FEUADDR),
+	IOMMU_REGSET_ENTRY(AFLOG),
+	IOMMU_REGSET_ENTRY(PMEN),
+	IOMMU_REGSET_ENTRY(PLMBASE),
+	IOMMU_REGSET_ENTRY(PLMLIMIT),
+	IOMMU_REGSET_ENTRY(PHMBASE),
+	IOMMU_REGSET_ENTRY(PHMLIMIT),
+	IOMMU_REGSET_ENTRY(IQH),
+	IOMMU_REGSET_ENTRY(IQT),
+	IOMMU_REGSET_ENTRY(IQA),
+	IOMMU_REGSET_ENTRY(ICS),
+	IOMMU_REGSET_ENTRY(IRTA),
+	IOMMU_REGSET_ENTRY(PQH),
+	IOMMU_REGSET_ENTRY(PQT),
+	IOMMU_REGSET_ENTRY(PQA),
+	IOMMU_REGSET_ENTRY(PRS),
+	IOMMU_REGSET_ENTRY(PECTL),
+	IOMMU_REGSET_ENTRY(PEDATA),
+	IOMMU_REGSET_ENTRY(PEADDR),
+	IOMMU_REGSET_ENTRY(PEUADDR),
+	IOMMU_REGSET_ENTRY(MTRRCAP),
+	IOMMU_REGSET_ENTRY(MTRRDEF),
+	IOMMU_REGSET_ENTRY(MTRR_FIX64K_00000),
+	IOMMU_REGSET_ENTRY(MTRR_FIX16K_80000),
+	IOMMU_REGSET_ENTRY(MTRR_FIX16K_A0000),
+	IOMMU_REGSET_ENTRY(MTRR_FIX4K_C0000),
+	IOMMU_REGSET_ENTRY(MTRR_FIX4K_C8000),
+	IOMMU_REGSET_ENTRY(MTRR_FIX4K_D0000),
+	IOMMU_REGSET_ENTRY(MTRR_FIX4K_D8000),
+	IOMMU_REGSET_ENTRY(MTRR_FIX4K_E0000),
+	IOMMU_REGSET_ENTRY(MTRR_FIX4K_E8000),
+	IOMMU_REGSET_ENTRY(MTRR_FIX4K_F0000),
+	IOMMU_REGSET_ENTRY(MTRR_FIX4K_F8000),
+	IOMMU_REGSET_ENTRY(MTRR_PHYSBASE0),
+	IOMMU_REGSET_ENTRY(MTRR_PHYSMASK0),
+	IOMMU_REGSET_ENTRY(MTRR_PHYSBASE1),
+	IOMMU_REGSET_ENTRY(MTRR_PHYSMASK1),
+	IOMMU_REGSET_ENTRY(MTRR_PHYSBASE2),
+	IOMMU_REGSET_ENTRY(MTRR_PHYSMASK2),
+	IOMMU_REGSET_ENTRY(MTRR_PHYSBASE3),
+	IOMMU_REGSET_ENTRY(MTRR_PHYSMASK3),
+	IOMMU_REGSET_ENTRY(MTRR_PHYSBASE4),
+	IOMMU_REGSET_ENTRY(MTRR_PHYSMASK4),
+	IOMMU_REGSET_ENTRY(MTRR_PHYSBASE5),
+	IOMMU_REGSET_ENTRY(MTRR_PHYSMASK5),
+	IOMMU_REGSET_ENTRY(MTRR_PHYSBASE6),
+	IOMMU_REGSET_ENTRY(MTRR_PHYSMASK6),
+	IOMMU_REGSET_ENTRY(MTRR_PHYSBASE7),
+	IOMMU_REGSET_ENTRY(MTRR_PHYSMASK7),
+	IOMMU_REGSET_ENTRY(MTRR_PHYSBASE8),
+	IOMMU_REGSET_ENTRY(MTRR_PHYSMASK8),
+	IOMMU_REGSET_ENTRY(MTRR_PHYSBASE9),
+	IOMMU_REGSET_ENTRY(MTRR_PHYSMASK9),
+	IOMMU_REGSET_ENTRY(VCCAP),
+	IOMMU_REGSET_ENTRY(VCMD),
+	IOMMU_REGSET_ENTRY(VCRSP),
+};
+
+static int iommu_regset_show(struct seq_file *m, void *unused)
+{
+	struct dmar_drhd_unit *drhd;
+	struct intel_iommu *iommu;
+	unsigned long flag;
+	int i, ret = 0;
+	u64 value;
+
+	rcu_read_lock();
+	for_each_active_iommu(iommu, drhd) {
+		if (!drhd->reg_base_addr) {
+			seq_puts(m, "IOMMU: Invalid base address\n");
+			ret = -EINVAL;
+			goto out;
+		}
+
+		seq_printf(m, "IOMMU: %s Register Base Address: %llx\n",
+			   iommu->name, drhd->reg_base_addr);
+		seq_puts(m, "Name\t\t\tOffset\t\tContents\n");
+		/*
+		 * Publish the contents of the 64-bit hardware registers
+		 * by adding the offset to the pointer (virtual address).
+		 */
+		raw_spin_lock_irqsave(&iommu->register_lock, flag);
+		for (i = 0 ; i < ARRAY_SIZE(iommu_regs); i++) {
+			value = dmar_readq(iommu->reg + iommu_regs[i].offset);
+			seq_printf(m, "%-16s\t0x%02x\t\t0x%016llx\n",
+				   iommu_regs[i].regs, iommu_regs[i].offset,
+				   value);
+		}
+		raw_spin_unlock_irqrestore(&iommu->register_lock, flag);
+		seq_putc(m, '\n');
+	}
+out:
+	rcu_read_unlock();
+
+	return ret;
+}
+DEFINE_SHOW_ATTRIBUTE(iommu_regset);
+
 void __init intel_iommu_debugfs_init(void)
 {
-	debugfs_create_dir("intel", iommu_debugfs_dir);
+	struct dentry *intel_iommu_debug = debugfs_create_dir("intel",
+						iommu_debugfs_dir);
+
+	debugfs_create_file("iommu_regset", 0444, intel_iommu_debug, NULL,
+			    &iommu_regset_fops);
 }
Signature · 1 line
2.18.0
#5 lore raw

PATCH v8 5/6 iommu/vt-d: Add debugfs support for Interrupt remapping

In reply to Sohil Mehta

Debugfs extension for Intel IOMMU to dump Interrupt remapping table entries for Interrupt remapping and Interrupt posting.

The file /sys/kernel/debug/iommu/intel/ir_translation_struct provides detailed information, such as Index, Source Id, Destination Id, Vector and the IRTE values for entries with the present bit set, in the format shown.

Remapped Interrupt supported on IOMMU: dmar1
IR table address:85e500000

 Entry SrcID   DstID    Vct IRTE_high           IRTE_low
 24    01:00.0 00000001 24  0000000000040100    000000010024000d
 25    01:00.0 00000004 22  0000000000040100    000000040022000d

Posted Interrupt supported on IOMMU: dmar5
IR table address:85ec00000

 Entry SrcID   PDA_high PDA_low  Vct IRTE_high          IRTE_low
 4     43:00.0 0000000f ff765980 41  0000000f00044300   ff76598000418001
 5     43:00.0 0000000f ff765980 51  0000000f00044300   ff76598000518001

---

v8: Improve IR table formatting
Use irq lock to proctect IRTE access

v7: Print the IR table physical base address
Simplify IR table formatting

v6: Change a couple of seq_puts to seq_putc

v5: Fix seq_puts formatting and remove leading '\n's

v4: Remove the unused function parameter
Fix checkpatch.pl warnings
Remove error reporting for debugfs_create_file function Remove redundant IOMMU null check under for_each_active_iommu

v3: Use a macro for seq file operations
Change the intel_iommu_interrupt_remap file name to ir_translation_struct

v2: Handle the case when IR is not enabled. Fix seq_printf formatting

 drivers/iommu/intel-iommu-debugfs.c | 108 ++++++++++++++++++++++++++++
 1 file changed, 108 insertions(+)
drivers/iommu/intel-iommu-debugfs.c +108 -0
index 57af2f774312..2becbd78620f 100644
@@ -131,6 +131,110 @@ static int iommu_regset_show(struct seq_file *m, void *unused)
 }
 DEFINE_SHOW_ATTRIBUTE(iommu_regset);
 
+#ifdef CONFIG_IRQ_REMAP
+static void ir_tbl_remap_entry_show(struct seq_file *m,
+				    struct intel_iommu *iommu)
+{
+	struct irte *ri_entry;
+	unsigned long flags;
+	int idx;
+
+	seq_puts(m, " Entry SrcID   DstID    Vct IRTE_high\t\tIRTE_low\n");
+
+	raw_spin_lock_irqsave(&irq_2_ir_lock, flags);
+	for (idx = 0; idx < INTR_REMAP_TABLE_ENTRIES; idx++) {
+		ri_entry = &iommu->ir_table->base[idx];
+		if (!ri_entry->present || ri_entry->p_pst)
+			continue;
+
+		seq_printf(m, " %-5d %02x:%02x.%01x %08x %02x  %016llx\t%016llx\n",
+			   idx, PCI_BUS_NUM(ri_entry->sid),
+			   PCI_SLOT(ri_entry->sid), PCI_FUNC(ri_entry->sid),
+			   ri_entry->dest_id, ri_entry->vector,
+			   ri_entry->high, ri_entry->low);
+	}
+	raw_spin_unlock_irqrestore(&irq_2_ir_lock, flags);
+}
+
+static void ir_tbl_posted_entry_show(struct seq_file *m,
+				     struct intel_iommu *iommu)
+{
+	struct irte *pi_entry;
+	unsigned long flags;
+	int idx;
+
+	seq_puts(m, " Entry SrcID   PDA_high PDA_low  Vct IRTE_high\t\tIRTE_low\n");
+
+	raw_spin_lock_irqsave(&irq_2_ir_lock, flags);
+	for (idx = 0; idx < INTR_REMAP_TABLE_ENTRIES; idx++) {
+		pi_entry = &iommu->ir_table->base[idx];
+		if (!pi_entry->present || !pi_entry->p_pst)
+			continue;
+
+		seq_printf(m, " %-5d %02x:%02x.%01x %08x %08x %02x  %016llx\t%016llx\n",
+			   idx, PCI_BUS_NUM(pi_entry->sid),
+			   PCI_SLOT(pi_entry->sid), PCI_FUNC(pi_entry->sid),
+			   pi_entry->pda_h, pi_entry->pda_l << 6,
+			   pi_entry->vector, pi_entry->high,
+			   pi_entry->low);
+	}
+	raw_spin_unlock_irqrestore(&irq_2_ir_lock, flags);
+}
+
+/*
+ * For active IOMMUs go through the Interrupt remapping
+ * table and print valid entries in a table format for
+ * Remapped and Posted Interrupts.
+ */
+static int ir_translation_struct_show(struct seq_file *m, void *unused)
+{
+	struct dmar_drhd_unit *drhd;
+	struct intel_iommu *iommu;
+	u64 irta;
+
+	rcu_read_lock();
+	for_each_active_iommu(iommu, drhd) {
+		if (!ecap_ir_support(iommu->ecap))
+			continue;
+
+		seq_printf(m, "Remapped Interrupt supported on IOMMU: %s\n",
+			   iommu->name);
+
+		if (iommu->ir_table) {
+			irta = virt_to_phys(iommu->ir_table->base);
+			seq_printf(m, " IR table address:%llx\n", irta);
+			ir_tbl_remap_entry_show(m, iommu);
+		} else {
+			seq_puts(m, "Interrupt Remapping is not enabled\n");
+		}
+		seq_putc(m, '\n');
+	}
+
+	seq_puts(m, "****\n\n");
+
+	for_each_active_iommu(iommu, drhd) {
+		if (!cap_pi_support(iommu->cap))
+			continue;
+
+		seq_printf(m, "Posted Interrupt supported on IOMMU: %s\n",
+			   iommu->name);
+
+		if (iommu->ir_table) {
+			irta = virt_to_phys(iommu->ir_table->base);
+			seq_printf(m, " IR table address:%llx\n", irta);
+			ir_tbl_posted_entry_show(m, iommu);
+		} else {
+			seq_puts(m, "Interrupt Remapping is not enabled\n");
+		}
+		seq_putc(m, '\n');
+	}
+	rcu_read_unlock();
+
+	return 0;
+}
+DEFINE_SHOW_ATTRIBUTE(ir_translation_struct);
+#endif
+
 void __init intel_iommu_debugfs_init(void)
 {
 	struct dentry *intel_iommu_debug = debugfs_create_dir("intel",
@@ -138,4 +242,8 @@ void __init intel_iommu_debugfs_init(void)
 
 	debugfs_create_file("iommu_regset", 0444, intel_iommu_debug, NULL,
 			    &iommu_regset_fops);
+#ifdef CONFIG_IRQ_REMAP
+	debugfs_create_file("ir_translation_struct", 0444, intel_iommu_debug,
+			    NULL, &ir_translation_struct_fops);
+#endif
 }
Signature · 1 line
2.18.0
#6 lore raw

PATCH v8 6/6 iommu/vt-d: Add debugfs support to show context internals

In reply to Sohil Mehta

Export Intel IOMMU DMA remapping internal states, such as root and context table entries in debugfs.

Example of such dump in Kabylake:
/sys/kernel/debug/iommu/intel/dmar_translation_struct

IOMMU dmar0: Root Table Address:45699c000
Root Table Entries:
Bus: 0 H: 0 L: 45699f001
Context Table Entries for Bus: 0
Entry B:D.F High Low
16 00:02.0 102 45699e005

IOMMU dmar1: Root Table Address:45699d000
Root Table Entries:
Bus: 0 H: 0 L: 45644f001
Context Table Entries for Bus: 0

  Entry B:D.F   High    Low
  160   00:14.0 102     45644e001
  184   00:17.0 302     456310001
  248   00:1f.0 202     4564bf001

---

v8: Remove extended context support since it is deprecated in the new spec
Use iommu lock to protect context table access
Simplify and split the patches differently

v7: Split patch 1/5 and 2/5 differently
Update commit message and copyright year
Fix typo in a comment
Simplify code

v6: Change the order of includes to an alphabetical order
Change seq_printf formatting

v5: Change to a SPDX license tag
Fix seq_printf formatting

v4: Remove the unused function parameter
Fix checkpatch.pl warnings
Remove error reporting for debugfs_create_file function
Fix unnecessary reprogramming of the context entries Simplify and merge the show context and extended context patch into one Remove redundant IOMMU null check under for_each_active_iommu

v3: Add a macro for seq file operations
Change the intel_iommu_ctx file name to dmar_translation_struct

v2: No change

 drivers/iommu/intel-iommu-debugfs.c | 65 +++++++++++++++++++++++++++++
 1 file changed, 65 insertions(+)
drivers/iommu/intel-iommu-debugfs.c +65 -0
index 2becbd78620f..7fabf9b1c2dc 100644
@@ -131,6 +131,69 @@ static int iommu_regset_show(struct seq_file *m, void *unused)
 }
 DEFINE_SHOW_ATTRIBUTE(iommu_regset);
 
+static void ctx_tbl_entry_show(struct seq_file *m, struct intel_iommu *iommu,
+			       int bus)
+{
+	struct context_entry *context;
+	int devfn;
+
+	seq_printf(m, " Context Table Entries for Bus: %d\n", bus);
+	seq_puts(m, "  Entry\tB:D.F\tHigh\tLow\n");
+
+	for (devfn = 0; devfn < 256; devfn++) {
+		context = iommu_context_addr(iommu, bus, devfn, 0);
+		if (!context)
+			return;
+
+		if (!context_present(context))
+			continue;
+
+		seq_printf(m, "  %-5d\t%02x:%02x.%x\t%-6llx\t%llx\n", devfn,
+			   bus, PCI_SLOT(devfn), PCI_FUNC(devfn),
+			   context[0].hi, context[0].lo);
+	}
+}
+
+static void root_tbl_entry_show(struct seq_file *m, struct intel_iommu *iommu)
+{
+	unsigned long flags;
+	int bus;
+
+	spin_lock_irqsave(&iommu->lock, flags);
+	seq_printf(m, "IOMMU %s: Root Table Address:%llx\n", iommu->name,
+		   (u64)virt_to_phys(iommu->root_entry));
+	seq_puts(m, "Root Table Entries:\n");
+
+	for (bus = 0; bus < 256; bus++) {
+		if (!(iommu->root_entry[bus].lo & 1))
+			continue;
+
+		seq_printf(m, " Bus: %d H: %llx L: %llx\n", bus,
+			   iommu->root_entry[bus].hi,
+			   iommu->root_entry[bus].lo);
+
+		ctx_tbl_entry_show(m, iommu, bus);
+		seq_putc(m, '\n');
+	}
+	spin_unlock_irqrestore(&iommu->lock, flags);
+}
+
+static int dmar_translation_struct_show(struct seq_file *m, void *unused)
+{
+	struct dmar_drhd_unit *drhd;
+	struct intel_iommu *iommu;
+
+	rcu_read_lock();
+	for_each_active_iommu(iommu, drhd) {
+		root_tbl_entry_show(m, iommu);
+		seq_putc(m, '\n');
+	}
+	rcu_read_unlock();
+
+	return 0;
+}
+DEFINE_SHOW_ATTRIBUTE(dmar_translation_struct);
+
 #ifdef CONFIG_IRQ_REMAP
 static void ir_tbl_remap_entry_show(struct seq_file *m,
 				    struct intel_iommu *iommu)
@@ -242,6 +305,8 @@ void __init intel_iommu_debugfs_init(void)
 
 	debugfs_create_file("iommu_regset", 0444, intel_iommu_debug, NULL,
 			    &iommu_regset_fops);
+	debugfs_create_file("dmar_translation_struct", 0444, intel_iommu_debug,
+			    NULL, &dmar_translation_struct_fops);
 #ifdef CONFIG_IRQ_REMAP
 	debugfs_create_file("ir_translation_struct", 0444, intel_iommu_debug,
 			    NULL, &ir_translation_struct_fops);
Signature · 1 line
2.18.0
maintainer Applied
#7 lore raw
In reply to Sohil Mehta
Quoted 7 lines · Sohil Mehta

Sohil Mehta (5):
iommu/vt-d: Relocate struct/function declarations to its header files iommu/vt-d: Update register definitions to VT-d 3.0 specification iommu/vt-d: Enable base Intel IOMMU debugfs support iommu/vt-d: Add debugfs support for Interrupt remapping iommu/vt-d: Add debugfs support to show context internals

Quoted 9 lines
 arch/x86/include/asm/irq_remapping.h |   2 +
 drivers/iommu/Kconfig                |  13 ++
 drivers/iommu/Makefile               |   1 +
 drivers/iommu/intel-iommu-debugfs.c  | 314 +++++++++++++++++++++++++++
 drivers/iommu/intel-iommu.c          |  32 +--
 drivers/iommu/intel_irq_remapping.c  |   2 +-
 include/linux/intel-iommu.h          |  72 ++++++
 7 files changed, 407 insertions(+), 29 deletions(-)
 create mode 100644 drivers/iommu/intel-iommu-debugfs.c

Applied, thanks.