Skip to contentSkip to conversation

powerpc/powernv/pci-ioda: fix kdump with non-power-of-2 crashkernel=

PATCH Applied Nishanth Aravamudan 3 participants 5 messages Sep 4, 2015 – Sep 8, 2015 PowerPC

lore
Share & tools
lore
#0 lore raw

PATCH v2 1/1 powerpc/powernv/pci-ioda: fix kdump with non-power-of-2 crashkernel=

The 32-bit TCE table initialization relies on the DMA window having a size equal to a power of 2 (and checks for it explicitly). But crashkernel= has no constraint that requires a power-of-2 be specified. This causes the kdump kernel to fail to boot as none of the PCI devices (including the disk controller) are successfully initialized.

After this change, the PCI devices successfully set up the 32-bit TCE table and kdump succeeds.

Fixes aca6913f5551 ("powerpc/powernv/ioda2: Introduce helpers to allocate TCE pages")

---

Michael, I kept this as a follow-on patch to my previous one. If you'd rather I made a v3 of that patch with the two fixes combined, I can resend. Also, I fixed up the context on my end to be u64, but not sure if that will match your tree (next doesn't have my prior patch applied yet, that I can see).

arch/powerpc/platforms/powernv/pci-ioda.c +7 -1
index f1c74c28e564..d5e635f2c3aa 100644
@@ -2078,12 +2078,18 @@ static long pnv_pci_ioda2_setup_default_config(struct pnv_ioda_pe *pe)
 	struct iommu_table *tbl = NULL;
 	long rc;
 	/*
+	 * crashkernel= specifies the kdump kernel's maximum memory at
+	 * some offset and there is no guaranteed the result is a power
+	 * of 2, which will cause errors later.
+	 */
+	const u64 max_memory = __rounddown_pow_of_two(memory_hotplug_max());
+	/*
 	 * In memory constrained environments, e.g. kdump kernel, the
 	 * DMA window can be larger than available memory, which will
 	 * cause errors later.
 	 */
 	const u64 window_size =
-		min((u64)pe->table_group.tce32_size, memory_hotplug_max());
+		min((u64)pe->table_group.tce32_size, max_memory);
 
 	rc = pnv_pci_ioda2_create_table(&pe->table_group, 0,
 			iommu_page_shift_4k,
Tested-by on v2 1/1
#1 lore raw

----- Original Message -----

Quoted 15 lines

From: "Nishanth Aravamudan" <[email protected]>
To: "Michael Ellerman" <[email protected]>
Cc: "Hari Bathini" <[email protected]>, "Gavin Shan" <[email protected]>, "Alexey Kardashevskiy" <[email protected]>, "Ben Herrenschmidt" <[email protected]>, "Paul Mackerras" <[email protected]>, "David Gibson" <[email protected]>, "Wei Yang" <[email protected]>, [email protected], "Jan Stancek" <[email protected]>
Sent: Friday, 4 September, 2015 8:22:52 PM
Subject: [PATCH v2] powerpc/powernv/pci-ioda: fix kdump with non-power-of-2 crashkernel=

The 32-bit TCE table initialization relies on the DMA window having a size equal to a power of 2 (and checks for it explicitly). But crashkernel= has no constraint that requires a power-of-2 be specified. This causes the kdump kernel to fail to boot as none of the PCI devices (including the disk controller) are successfully initialized.

Quoted 7 lines

After this change, the PCI devices successfully set up the 32-bit TCE table and kdump succeeds.

Fixes aca6913f5551 ("powerpc/powernv/ioda2: Introduce helpers to allocate

TCE pages")

I can confirm, that this patch along with
http://patchwork.ozlabs.org/patch/513229/
applied on top of 4.2 is fixing kdump for me on Power S812L. (ppc64le bare metal, crashkernel=1024M)

Regards,
Jan

Quoted 37 lines

---

Michael, I kept this as a follow-on patch to my previous one. If you'd rather I made a v3 of that patch with the two fixes combined, I can resend. Also, I fixed up the context on my end to be u64, but not sure if that will match your tree (next doesn't have my prior patch applied yet, that I can see).

arch/powerpc/platforms/powernv/pci-ioda.c +7 -1
index f1c74c28e564..d5e635f2c3aa 100644
@@ -2078,12 +2078,18 @@ static long pnv_pci_ioda2_setup_default_config(struct
 pnv_ioda_pe *pe)
 	struct iommu_table *tbl = NULL;
 	long rc;
 	/*
+	 * crashkernel= specifies the kdump kernel's maximum memory at
+	 * some offset and there is no guaranteed the result is a power
+	 * of 2, which will cause errors later.
+	 */
+	const u64 max_memory = __rounddown_pow_of_two(memory_hotplug_max());
+	/*
 	 * In memory constrained environments, e.g. kdump kernel, the
 	 * DMA window can be larger than available memory, which will
 	 * cause errors later.
 	 */
 	const u64 window_size =
-		min((u64)pe->table_group.tce32_size, memory_hotplug_max());
+		min((u64)pe->table_group.tce32_size, max_memory);
 
 	rc = pnv_pci_ioda2_create_table(&pe->table_group, 0,

iommu_page_shift_4k,

#2 lore raw
In reply to Nishanth Aravamudan
Quoted 9 lines · Nishanth Aravamudan

The 32-bit TCE table initialization relies on the DMA window having a size equal to a power of 2 (and checks for it explicitly). But crashkernel= has no constraint that requires a power-of-2 be specified. This causes the kdump kernel to fail to boot as none of the PCI devices (including the disk controller) are successfully initialized.

After this change, the PCI devices successfully set up the 32-bit TCE table and kdump succeeds.

Quoted 7 lines
Fixes aca6913f5551 ("powerpc/powernv/ioda2: Introduce helpers to allocate TCE pages")

Michael, I kept this as a follow-on patch to my previous one. If you'd rather I made a v3 of that patch with the two fixes combined, I can resend.

No that's fine. I guess they could have been a single fix, but it's not a big deal.

Quoted 3 lines

Also, I fixed up the context on my end to be u64, but not sure if that will match your tree (next doesn't have my prior patch applied yet, that I can see).

next isn't open yet, because we're still in the merge window, ie. rc1 hasn't come out yet.

This is a fix so it'll go to my fixes branch. Whether I send that to Linus before or after rc1 depends on how urgent the fixes people send me are. Sounds like you'd like these two to go in asap?

cheers

#3 lore raw
Quoted 14 lines · Michael Ellerman
Quoted 13 lines · Nishanth Aravamudan

The 32-bit TCE table initialization relies on the DMA window having a size equal to a power of 2 (and checks for it explicitly). But crashkernel= has no constraint that requires a power-of-2 be specified. This causes the kdump kernel to fail to boot as none of the PCI devices (including the disk controller) are successfully initialized.

After this change, the PCI devices successfully set up the 32-bit TCE table and kdump succeeds.

Fixes aca6913f5551 ("powerpc/powernv/ioda2: Introduce helpers to allocate TCE pages")
Quoted 6 lines
Quoted 3 lines

Michael, I kept this as a follow-on patch to my previous one. If you'd rather I made a v3 of that patch with the two fixes combined, I can resend.

No that's fine. I guess they could have been a single fix, but it's not a big deal.

Ok, thanks for understanding.

Quoted 6 lines
Quoted 3 lines

Also, I fixed up the context on my end to be u64, but not sure if that will match your tree (next doesn't have my prior patch applied yet, that I can see).

next isn't open yet, because we're still in the merge window, ie. rc1 hasn't come out yet.

Ah got it. I did mean your next branch, not linux-next itself, fwiw.

Quoted 3 lines

This is a fix so it'll go to my fixes branch. Whether I send that to Linus before or after rc1 depends on how urgent the fixes people send me are. Sounds like you'd like these two to go in asap?

If at all possible, yes please.

-Nish

reviewer Applied
#4 lore raw
In reply to Nishanth Aravamudan
Quoted 6 lines · Nishanth Aravamudan

The 32-bit TCE table initialization relies on the DMA window having a size equal to a power of 2 (and checks for it explicitly). But crashkernel= has no constraint that requires a power-of-2 be specified. This causes the kdump kernel to fail to boot as none of the PCI devices (including the disk controller) are successfully initialized.

Quoted 7 lines

After this change, the PCI devices successfully set up the 32-bit TCE table and kdump succeeds.

Fixes aca6913f5551 ("powerpc/powernv/ioda2: Introduce helpers to allocate TCE pages")

Applied to powerpc fixes, thanks.

https://git.kernel.org/powerpc/c/fa14486979b3a47307bcdb10

cheers