Skip to contentSkip to conversation

amd-iommu fixes

GIT PULL New Joerg Roedel 2 participants 8 messages Jun 16, 2009 x86

lore
Share & tools
lore
#0 lore raw

Hi Ingo,

The following changes since commit 92db1e6af747faa129e236d68386af26a0efc12b:
Ingo Molnar (1):
Merge branch 'amd-iommu/fixes' of git://git.kernel.org/.../joro/linux-2.6-iommu into core/iommu

are available in the git repository at:

git://git.kernel.org/pub/scm/linux/kernel/git/joro/linux-2.6-iommu.git amd-iommu/fixes

Chris Wright (3):
amd-iommu: flush domain tlb when attaching a new device
amd-iommu: disable cmd buffer and evt logging before reprogramming iommu
amd-iommu: resume cleanup

Joerg Roedel (3):
amd-iommu: disable IOMMU hardware on shutdown
x86: disable IOMMUs on kernel crash
amd-iommu: set event buffer head and tail to 0 manually

 arch/x86/include/asm/amd_iommu.h |    2 ++
 arch/x86/kernel/amd_iommu.c      |   16 ++++++++++++++++
 arch/x86/kernel/amd_iommu_init.c |   26 +++++++++++++++++++-------
 arch/x86/kernel/crash.c          |    6 ++++++
 arch/x86/kernel/pci-dma.c        |    2 ++
 5 files changed, 45 insertions(+), 7 deletions(-)

These patches mostly fixes issues in the AMD IOMMU driver with kexec and kdump. Another fix is to disable the IOMMU hardware on shutdown and crash to get out of the way for the BIOS. This fixed power-off on my test machine. Please pull.

Joerg

#1 lore raw

PATCH 1/6 amd-iommu: disable IOMMU hardware on shutdown

When the IOMMU stays enabled the BIOS may not be able to finish the machine shutdown properly. So disable the hardware on shutdown.

---
 arch/x86/include/asm/amd_iommu.h |    2 ++
 arch/x86/kernel/amd_iommu_init.c |    5 +++++
 arch/x86/kernel/pci-dma.c        |    2 ++
 3 files changed, 9 insertions(+), 0 deletions(-)
arch/x86/include/asm/amd_iommu.h +2 -0
index 262e028..bdf96f1 100644
@@ -29,9 +29,11 @@ extern void amd_iommu_detect(void);
 extern irqreturn_t amd_iommu_int_handler(int irq, void *data);
 extern void amd_iommu_flush_all_domains(void);
 extern void amd_iommu_flush_all_devices(void);
+extern void amd_iommu_shutdown(void);
 #else
 static inline int amd_iommu_init(void) { return -ENODEV; }
 static inline void amd_iommu_detect(void) { }
+static inline void amd_iommu_shutdown(void) { }
 #endif
 
 #endif /* _ASM_X86_AMD_IOMMU_H */
arch/x86/kernel/amd_iommu_init.c +5 -0
index 238989e..575ca46 100644
@@ -1273,6 +1273,11 @@ free:
 	goto out;
 }
 
+void amd_iommu_shutdown(void)
+{
+	disable_iommus();
+}
+
 /****************************************************************************
  *
  * Early detect code. This code runs at IOMMU detection time in the DMA
arch/x86/kernel/pci-dma.c +2 -0
index 745579b..328592f 100644
@@ -290,6 +290,8 @@ static int __init pci_iommu_init(void)
 void pci_iommu_shutdown(void)
 {
 	gart_iommu_shutdown();
+
+	amd_iommu_shutdown();
 }
 /* Must execute after PCI subsystem */
 fs_initcall(pci_iommu_init);
Signature · 1 line
1.6.3.1
#2 lore raw

PATCH 2/6 x86: disable IOMMUs on kernel crash

In reply to Joerg Roedel

If the IOMMUs are still enabled when the kexec kernel boots access to the disk is not possible. This is bad for tools like kdump or anything else which wants to use PCI devices.

---
 arch/x86/kernel/crash.c |    6 ++++++
 1 files changed, 6 insertions(+), 0 deletions(-)
arch/x86/kernel/crash.c +6 -0
index ff95824..5e409dc 100644
@@ -27,6 +27,7 @@
 #include <asm/cpu.h>
 #include <asm/reboot.h>
 #include <asm/virtext.h>
+#include <asm/iommu.h>
 
 
 #if defined(CONFIG_SMP) && defined(CONFIG_X86_LOCAL_APIC)
@@ -103,5 +104,10 @@ void native_machine_crash_shutdown(struct pt_regs *regs)
 #ifdef CONFIG_HPET_TIMER
 	hpet_disable();
 #endif
+
+#ifdef CONFIG_X86_64
+	pci_iommu_shutdown();
+#endif
+
 	crash_save_cpu(regs, safe_smp_processor_id());
 }
Signature · 1 line
1.6.3.1
#3 lore raw

PATCH 3/6 amd-iommu: flush domain tlb when attaching a new device

In reply to Joerg Roedel

From: Chris Wright <[email protected]>

When kexec'ing to a new kernel (for example, when crashing and launching a kdump session), the AMD IOMMU may have cached translations. The kexec'd kernel, during initialization, will invalidate the IOMMU device table entries, but not the domain translations. These stale entries can cause a device's DMA to fail, makes it rough to write a dump to disk when the disk controller can't DMA ;-)

---
 arch/x86/kernel/amd_iommu.c |   16 ++++++++++++++++
 1 files changed, 16 insertions(+), 0 deletions(-)
arch/x86/kernel/amd_iommu.c +16 -0
index 1c60554..9372f04 100644
@@ -434,6 +434,16 @@ static void iommu_flush_tlb(struct amd_iommu *iommu, u16 domid)
 	iommu_queue_inv_iommu_pages(iommu, address, domid, 0, 1);
 }
 
+/* Flush the whole IO/TLB for a given protection domain - including PDE */
+static void iommu_flush_tlb_pde(struct amd_iommu *iommu, u16 domid)
+{
+       u64 address = CMD_INV_IOMMU_ALL_PAGES_ADDRESS;
+
+       INC_STATS_COUNTER(domain_flush_single);
+
+       iommu_queue_inv_iommu_pages(iommu, address, domid, 1, 1);
+}
+
 /*
  * This function is used to flush the IO/TLB for a given protection domain
  * on every IOMMU in the system
@@ -1078,7 +1088,13 @@ static void attach_device(struct amd_iommu *iommu,
 	amd_iommu_pd_table[devid] = domain;
 	write_unlock_irqrestore(&amd_iommu_devtable_lock, flags);
 
+       /*
+        * We might boot into a crash-kernel here. The crashed kernel
+        * left the caches in the IOMMU dirty. So we have to flush
+        * here to evict all dirty stuff.
+        */
 	iommu_queue_inv_dev_entry(iommu, devid);
+	iommu_flush_tlb_pde(iommu, domain->id);
 }
 
 /*
Signature · 1 line
1.6.3.1
#4 lore raw

PATCH 4/6 amd-iommu: disable cmd buffer and evt logging before reprogramming iommu

In reply to Joerg Roedel

From: Chris Wright <[email protected]>

The IOMMU spec states that IOMMU behavior may be undefined when the IOMMU registers are rewritten while command or event buffer is enabled. Disable them in IOMMU disable path.

---
 arch/x86/kernel/amd_iommu_init.c |    9 +++++++++
 1 files changed, 9 insertions(+), 0 deletions(-)
arch/x86/kernel/amd_iommu_init.c +9 -0
index 575ca46..48a79b9 100644
@@ -260,6 +260,14 @@ static void iommu_enable(struct amd_iommu *iommu)
 
 static void iommu_disable(struct amd_iommu *iommu)
 {
+	/* Disable command buffer */
+	iommu_feature_disable(iommu, CONTROL_CMDBUF_EN);
+
+	/* Disable event logging and event interrupts */
+	iommu_feature_disable(iommu, CONTROL_EVT_INT_EN);
+	iommu_feature_disable(iommu, CONTROL_EVT_LOG_EN);
+
+	/* Disable IOMMU hardware itself */
 	iommu_feature_disable(iommu, CONTROL_IOMMU_EN);
 }
 
@@ -1042,6 +1050,7 @@ static void enable_iommus(void)
 	struct amd_iommu *iommu;
 
 	for_each_iommu(iommu) {
+		iommu_disable(iommu);
 		iommu_set_device_table(iommu);
 		iommu_enable_command_buffer(iommu);
 		iommu_enable_event_buffer(iommu);
Signature · 1 line
1.6.3.1
#5 lore raw

PATCH 5/6 amd-iommu: set event buffer head and tail to 0 manually

In reply to Joerg Roedel

These registers may contain values from previous kernels. So reset them to known values before enable the event buffer again.

---
 arch/x86/kernel/amd_iommu_init.c |    4 ++++
 1 files changed, 4 insertions(+), 0 deletions(-)
arch/x86/kernel/amd_iommu_init.c +4 -0
index 48a79b9..068a356 100644
@@ -486,6 +486,10 @@ static void iommu_enable_event_buffer(struct amd_iommu *iommu)
 	memcpy_toio(iommu->mmio_base + MMIO_EVT_BUF_OFFSET,
 		    &entry, sizeof(entry));
 
+	/* set head and tail to zero manually */
+	writel(0x00, iommu->mmio_base + MMIO_EVT_HEAD_OFFSET);
+	writel(0x00, iommu->mmio_base + MMIO_EVT_TAIL_OFFSET);
+
 	iommu_feature_enable(iommu, CONTROL_EVT_LOG_EN);
 }
 
Signature · 1 line
1.6.3.1
#6 lore raw

PATCH 6/6 amd-iommu: resume cleanup

In reply to Joerg Roedel

From: Chris Wright <[email protected]>

Now that enable_iommus() will call iommu_disable() for each iommu, the call to disable_iommus() during resume is redundant. Also, the order for an invalidation is to invalidate device table entries first, then domain translations.

---
 arch/x86/kernel/amd_iommu_init.c |    8 +-------
 1 files changed, 1 insertions(+), 7 deletions(-)
arch/x86/kernel/amd_iommu_init.c +1 -7
index 068a356..10b2acc 100644
@@ -1079,12 +1079,6 @@ static void disable_iommus(void)
 
 static int amd_iommu_resume(struct sys_device *dev)
 {
-	/*
-	 * Disable IOMMUs before reprogramming the hardware registers.
-	 * IOMMU is still enabled from the resume kernel.
-	 */
-	disable_iommus();
-
 	/* re-load the hardware */
 	enable_iommus();
 
@@ -1092,8 +1086,8 @@ static int amd_iommu_resume(struct sys_device *dev)
 	 * we have to flush after the IOMMUs are enabled because a
 	 * disabled IOMMU will never execute the commands we send
 	 */
-	amd_iommu_flush_all_domains();
 	amd_iommu_flush_all_devices();
+	amd_iommu_flush_all_domains();
 
 	return 0;
 }
Signature · 1 line
1.6.3.1
#7 lore raw
In reply to Joerg Roedel
Quoted 27 lines · Joerg Roedel

Hi Ingo,

The following changes since commit 92db1e6af747faa129e236d68386af26a0efc12b:
Ingo Molnar (1):
Merge branch 'amd-iommu/fixes' of git://git.kernel.org/.../joro/linux-2.6-iommu into core/iommu

are available in the git repository at:

git://git.kernel.org/pub/scm/linux/kernel/git/joro/linux-2.6-iommu.git amd-iommu/fixes

Chris Wright (3):
amd-iommu: flush domain tlb when attaching a new device
amd-iommu: disable cmd buffer and evt logging before reprogramming iommu
amd-iommu: resume cleanup

Joerg Roedel (3):
amd-iommu: disable IOMMU hardware on shutdown
x86: disable IOMMUs on kernel crash
amd-iommu: set event buffer head and tail to 0 manually

 arch/x86/include/asm/amd_iommu.h |    2 ++
 arch/x86/kernel/amd_iommu.c      |   16 ++++++++++++++++
 arch/x86/kernel/amd_iommu_init.c |   26 +++++++++++++++++++-------
 arch/x86/kernel/crash.c          |    6 ++++++
 arch/x86/kernel/pci-dma.c        |    2 ++
 5 files changed, 45 insertions(+), 7 deletions(-)
Quoted 4 lines

These patches mostly fixes issues in the AMD IOMMU driver with kexec and kdump. Another fix is to disable the IOMMU hardware on shutdown and crash to get out of the way for the BIOS. This fixed power-off on my test machine. Please pull.

Pulled, thanks Joerg!

Ingo