Skip to contentSkip to conversation

Revert "iommu/amd: Treat per-device exclusion ranges as r/w unity-mapped regions"

PATCH In review Baoquan He 3 participants 4 messages Sep 23, 2020 – Sep 24, 2020

Subsystems (2)
lore
Share & tools
lore

Baoquan He

author
#0 lore raw

PATCH 1/1 Revert "iommu/amd: Treat per-device exclusion ranges as r/w unity-mapped regions"

A regression failure of kdump kernel boot was reported on a HPE system. Bisect points at commit 387caf0b759ac43 ("iommu/amd: Treat per-device exclusion ranges as r/w unity-mapped regions") as criminal. Reverting it fix the failure.

With the commit, kdump kernel will always print below error message, then naturally AMD iommu can't function normally during kdump kernel bootup.

~~~~~~~~~
AMD-Vi: [Firmware Bug]: IVRS invalid checksum

Why commit 387caf0b759ac43 causing it haven't been made clear.

From the commit log, a discussion thread link is pasted. In that discussion thread, Adrian told the fix is for a system with already broken BIOS, and Joerg suggested two options. Finally option 2) is taken. Maybe option 1) should be the right approach?

1) Bail out and disable the IOMMU as the BIOS screwed up
2) Treat per-device exclusion ranges just as r/w unity-mapped regions.

https://lists.linuxfoundation.org/pipermail/iommu/2019-November/040117.html

---
 drivers/iommu/amd/init.c | 21 +++++++++++++--------
 1 file changed, 13 insertions(+), 8 deletions(-)
drivers/iommu/amd/init.c +13 -8
index 9aa1eae26634..bbe7ceae5949 100644
@@ -1109,17 +1109,22 @@ static int __init add_early_maps(void)
  */
 static void __init set_device_exclusion_range(u16 devid, struct ivmd_header *m)
 {
+	struct amd_iommu *iommu = amd_iommu_rlookup_table[devid];
+
 	if (!(m->flags & IVMD_FLAG_EXCL_RANGE))
 		return;
 
-	/*
-	 * Treat per-device exclusion ranges as r/w unity-mapped regions
-	 * since some buggy BIOSes might lead to the overwritten exclusion
-	 * range (exclusion_start and exclusion_length members). This
-	 * happens when there are multiple exclusion ranges (IVMD entries)
-	 * defined in ACPI table.
-	 */
-	m->flags = (IVMD_FLAG_IW | IVMD_FLAG_IR | IVMD_FLAG_UNITY_MAP);
+	if (iommu) {
+		/*
+		 * We only can configure exclusion ranges per IOMMU, not
+		 * per device. But we can enable the exclusion range per
+		 * device. This is done here
+		 */
+		set_dev_entry_bit(devid, DEV_ENTRY_EX);
+		iommu->exclusion_start = m->range_start;
+		iommu->exclusion_length = m->range_length;
+	}
+
 }
 
 /*
Signature · 1 line
2.17.2
author
#1 lore raw

Forgot CC-ing Jerry, add him.

Quoted 5 lines · Baoquan He

A regression failure of kdump kernel boot was reported on a HPE system. Bisect points at commit 387caf0b759ac43 ("iommu/amd: Treat per-device exclusion ranges as r/w unity-mapped regions") as criminal. Reverting it fix the failure.

Quoted 7 lines

With the commit, kdump kernel will always print below error message, then naturally AMD iommu can't function normally during kdump kernel bootup.

~~~~~~~~~
AMD-Vi: [Firmware Bug]: IVRS invalid checksum

Why commit 387caf0b759ac43 causing it haven't been made clear.

Hi Joerg, Adrian

We only have one machine which can reproduce the issue, it's a gen10-01 of HPE. If any log or info are needed, please let me know, I can attach here.

Thanks
Baoquan

Quoted 59 lines

From the commit log, a discussion thread link is pasted. In that discussion thread, Adrian told the fix is for a system with already broken BIOS, and Joerg suggested two options. Finally option 2) is taken. Maybe option 1) should be the right approach?

1) Bail out and disable the IOMMU as the BIOS screwed up
2) Treat per-device exclusion ranges just as r/w unity-mapped regions.

https://lists.linuxfoundation.org/pipermail/iommu/2019-November/040117.html

---
 drivers/iommu/amd/init.c | 21 +++++++++++++--------
 1 file changed, 13 insertions(+), 8 deletions(-)
drivers/iommu/amd/init.c +13 -8
index 9aa1eae26634..bbe7ceae5949 100644
@@ -1109,17 +1109,22 @@ static int __init add_early_maps(void)
  */
 static void __init set_device_exclusion_range(u16 devid, struct ivmd_header *m)
 {
+	struct amd_iommu *iommu = amd_iommu_rlookup_table[devid];
+
 	if (!(m->flags & IVMD_FLAG_EXCL_RANGE))
 		return;
 
-	/*
-	 * Treat per-device exclusion ranges as r/w unity-mapped regions
-	 * since some buggy BIOSes might lead to the overwritten exclusion
-	 * range (exclusion_start and exclusion_length members). This
-	 * happens when there are multiple exclusion ranges (IVMD entries)
-	 * defined in ACPI table.
-	 */
-	m->flags = (IVMD_FLAG_IW | IVMD_FLAG_IR | IVMD_FLAG_UNITY_MAP);
+	if (iommu) {
+		/*
+		 * We only can configure exclusion ranges per IOMMU, not
+		 * per device. But we can enable the exclusion range per
+		 * device. This is done here
+		 */
+		set_dev_entry_bit(devid, DEV_ENTRY_EX);
+		iommu->exclusion_start = m->range_start;
+		iommu->exclusion_length = m->range_length;
+	}
+
 }
 
 /*
Signature · 6 lines
2.17.2

_______________________________________________
iommu mailing list
[email protected]
https://lists.linuxfoundation.org/mailman/listinfo/iommu
#2 lore raw

Hi Baoquan,

Quoted 24 lines

-----Original Message-----
From: Baoquan He <[email protected]>
Sent: Wednesday, September 23, 2020 10:33 AM
To: [email protected]; Adrian Huang12 <[email protected]>
Cc: [email protected]; [email protected]; [email protected]
Subject: [External] Re: [PATCH] Revert "iommu/amd: Treat per-device exclusion ranges as r/w unity-mapped regions"

Forgot CC-ing Jerry, add him.

Quoted 7 lines · Baoquan He

A regression failure of kdump kernel boot was reported on a HPE system. Bisect points at commit 387caf0b759ac43 ("iommu/amd: Treat per-device exclusion ranges as r/w unity-mapped regions") as criminal. Reverting it fix the failure.

With the commit, kdump kernel will always print below error message, then naturally AMD iommu can't function normally during kdump kernel

bootup.

Quoted 4 lines

~~~~~~~~~
AMD-Vi: [Firmware Bug]: IVRS invalid checksum

Quoted 6 lines
Quoted 1 line

Why commit 387caf0b759ac43 causing it haven't been made clear.

Hi Joerg, Adrian

We only have one machine which can reproduce the issue, it's a gen10-01 of HPE. If any log or info are needed, please let me know, I can attach here.

Could you please provide the following info?
1. The booting log for both system kernel and kdump kernel by appending the kernel parameter 'amd_iommu_dump'
2. ACPI table (# acpidump > acpi-table) -> Send out the file 'acpi-table'.

-- Adrian

maintainer
#3 lore raw
In reply to Baoquan He
Quoted 5 lines · Baoquan He

A regression failure of kdump kernel boot was reported on a HPE system. Bisect points at commit 387caf0b759ac43 ("iommu/amd: Treat per-device exclusion ranges as r/w unity-mapped regions") as criminal. Reverting it fix the failure.

Quoted 7 lines

With the commit, kdump kernel will always print below error message, then naturally AMD iommu can't function normally during kdump kernel bootup.

~~~~~~~~~
AMD-Vi: [Firmware Bug]: IVRS invalid checksum

Why commit 387caf0b759ac43 causing it haven't been made clear.

I think this should be debugged further, in future IOMMUs the exclusion range feature will not be available anymore (mmio-fields get re-used for SNP). So starting to use them again is not going to work.

Regards,

Joerg