powerpc/powernv/npu: Fix oops in pnv_try_setup_npu_table_group()
Subsystems (2)
PATCH 1/1 powerpc/powernv/npu: Fix oops in pnv_try_setup_npu_table_group()
With a recent change around IOMMU group, a system with an opencapi adapter is no longer booting and we get a kernel oops:
BUG: Kernel NULL pointer dereference at 0x00000028 Faulting instruction address: 0xc0000000000aa38c Oops: Kernel access of bad area, sig: 7 [#1] LE SMP NR_CPUS=2048 NUMA PowerNV Modules linked in: CPU: 5 PID: 1 Comm: swapper/4 Not tainted 5.0.0-rc1-fxb-00001-g3bd6e94bec12 NIP: c0000000000aa38c LR: c0000000000a6608 CTR: c000000000097480 REGS: c000000005783700 TRAP: 0300 Not tainted (5.0.0-rc1-fxb-00001-g3bd6 MSR: 9000000002009033 <SF,HV,VEC,EE,ME,IR,DR,RI,LE> CR: 28000228 XER: 20 CFAR: c0000000000a6604 DAR: 0000000000000028 DSISR: 00080000 IRQMASK: 0
GPR00: c0000000000a6608 c000000005783990 c000000001036100 c0000007bf761860
GPR04: 0000000000000000 c000000005783834 0000000000000000 0000000000000000
GPR08: 69626d2c6e707500 0000000000000000 0000000000000000 9000000002001003
GPR12: 0000000000000000 c0000007bfff8300 c000000000010450 0000000000000000
GPR16: c000000000ced938 0000000000000100 c000000000ced948 00000000000a0000
GPR20: 00000000000bfffe c000000000ced9a8 0000000000000200 c000000000ced978
GPR24: 00000000006080c0 c000000716d09828 c00000002e6fd000 0000000000000000
GPR28: c0000007bf4aff68 c0000007bf8d0080 c000000000f23938 c0000007bf761860 NIP [c0000000000aa38c] pnv_try_setup_npu_table_group+0x1c/0x1a0 LR [c0000000000a6608] pnv_pci_ioda_fixup+0x1f8/0x660
Call Trace: [c000000005783990] [c0000000000aa3d0] pnv_try_setup_npu_table_group+0x60/0x [c0000000057839d0] [c0000000000a661c] pnv_pci_ioda_fixup+0x20c/0x660 [c000000005783ab0] [c000000000e1d4c0] pcibios_resource_survey+0x2c8/0x31c [c000000005783b90] [c000000000e1caf4] pcibios_init+0xb0/0xe4 [c000000005783c10] [c000000000010054] do_one_initcall+0x64/0x264 [c000000005783ce0] [c000000000e1132c] kernel_init_freeable+0x36c/0x468 [c000000005783db0] [c000000000010474] kernel_init+0x2c/0x148 [c000000005783e20] [c00000000000b794] ret_from_kernel_thread+0x5c/0x68
An opencapi device is using a device PE, so the current code breaks because pe->pbus is not defined.
More generally, there's no need to define an IOMMU group for opencapi, as the device sends real addresses directly (admittedly, the virtualization story is yet to be written). So let's fix it by skipping the IOMMU group setup for opencapi PHBs.
--- arch/powerpc/platforms/powernv/pci-ioda.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-)
arch/powerpc/platforms/powernv/pci-ioda.c +2 -1
@@ -2681,7 +2681,8 @@ static void pnv_pci_ioda_setup_iommu_api(void)list_for_each_entry (hose, &hose_list, list_node) { phb = hose->private_data; -if (phb->type ==PNV_PHB_NPU_NVLINK ) +if (phb->type ==PNV_PHB_NPU_NVLINK || + phb->type ==PNV_PHB_NPU_OCAPI )continue ;list_for_each_entry (pe, &phb->ioda.pe_list, list) {
Signature · 1 line
2.19.1
Quoted 33 lines · On Wed, 9 Jan 2019 16:13:42 +0100 Frederic Barrat <[email protected]>…
With a recent change around IOMMU group, a system with an opencapi adapter is no longer booting and we get a kernel oops:
BUG: Kernel NULL pointer dereference at 0x00000028 Faulting instruction address: 0xc0000000000aa38c Oops: Kernel access of bad area, sig: 7 [#1] LE SMP NR_CPUS=2048 NUMA PowerNV Modules linked in: CPU: 5 PID: 1 Comm: swapper/4 Not tainted 5.0.0-rc1-fxb-00001-g3bd6e94bec12 NIP: c0000000000aa38c LR: c0000000000a6608 CTR: c000000000097480 REGS: c000000005783700 TRAP: 0300 Not tainted (5.0.0-rc1-fxb-00001-g3bd6 MSR: 9000000002009033 <SF,HV,VEC,EE,ME,IR,DR,RI,LE> CR: 28000228 XER: 20 CFAR: c0000000000a6604 DAR: 0000000000000028 DSISR: 00080000 IRQMASK: 0GPR00:
c0000000000a6608c000000005783990c000000001036100c0000007bf761860
GPR04: 0000000000000000c0000000057838340000000000000000 0000000000000000
GPR08:69626d2c6e7075000000000000000000 0000000000000000 9000000002001003
GPR12: 0000000000000000c0000007bfff8300c0000000000104500000000000000000
GPR16:c000000000ced9380000000000000100c000000000ced94800000000000a0000
GPR20:00000000000bfffec000000000ced9a80000000000000200c000000000ced978
GPR24:00000000006080c0c000000716d09828c00000002e6fd0000000000000000000
GPR28:c0000007bf4aff68c0000007bf8d0080c000000000f23938c0000007bf761860NIP [c0000000000aa38c]pnv_try_setup_npu_table_group+0x1c/0x1a0LR [c0000000000a6608]pnv_pci_ioda_fixup+0x1f8/0x660Call Trace: [c000000005783990] [c0000000000aa3d0] pnv_try_setup_npu_table_group+0x60/0x [c0000000057839d0] [c0000000000a661c] pnv_pci_ioda_fixup+0x20c/0x660 [c000000005783ab0] [c000000000e1d4c0] pcibios_resource_survey+0x2c8/0x31c [c000000005783b90] [c000000000e1caf4] pcibios_init+0xb0/0xe4 [c000000005783c10] [c000000000010054] do_one_initcall+0x64/0x264 [c000000005783ce0] [c000000000e1132c] kernel_init_freeable+0x36c/0x468 [c000000005783db0] [c000000000010474] kernel_init+0x2c/0x148 [c000000005783e20] [c00000000000b794] ret_from_kernel_thread+0x5c/0x68
Quoted 6 lines
An opencapi device is using a device PE, so the current code breaks because
pe->pbusis not defined.More generally, there's no need to define an IOMMU group for opencapi, as the device sends real addresses directly (admittedly, the virtualization story is yet to be written). So let's fix it by
Current plan is to go for mediated VFIO. The real HW stays under the control of the host ocxl driver, and we still don't need an IOMMU group.
Quoted 5 lines
skipping the IOMMU group setup for opencapi PHBs.
Fixes 0bd971676e68 ("powerpc/powernv/npu: Add compound IOMMU groups")Signed-off-by Frederic Barrat <[email protected]>---
and
Quoted 17 lines
arch/powerpc/platforms/powernv/pci-ioda.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-)arch/powerpc/platforms/powernv/pci-ioda.c +2
-1@@ -2681,7 +2681,8 @@ static void pnv_pci_ioda_setup_iommu_api(void)list_for_each_entry (hose, &hose_list, list_node) { phb = hose->private_data; -if (phb->type ==PNV_PHB_NPU_NVLINK ) +if (phb->type ==PNV_PHB_NPU_NVLINK || + phb->type ==PNV_PHB_NPU_OCAPI )continue ;list_for_each_entry (pe, &phb->ioda.pe_list, list) {
Le 09/01/2019 à 17:25, Greg Kurz a écrit :
Quoted 52 lines
Quoted 33 lines · On Wed, 9 Jan 2019 16:13:42 +0100 Frederic Barrat <[email protected]>…
With a recent change around IOMMU group, a system with an opencapi adapter is no longer booting and we get a kernel oops:
BUG: Kernel NULL pointer dereference at 0x00000028 Faulting instruction address: 0xc0000000000aa38c Oops: Kernel access of bad area, sig: 7 [#1] LE SMP NR_CPUS=2048 NUMA PowerNV Modules linked in: CPU: 5 PID: 1 Comm: swapper/4 Not tainted 5.0.0-rc1-fxb-00001-g3bd6e94bec12 NIP: c0000000000aa38c LR: c0000000000a6608 CTR: c000000000097480 REGS: c000000005783700 TRAP: 0300 Not tainted (5.0.0-rc1-fxb-00001-g3bd6 MSR: 9000000002009033 <SF,HV,VEC,EE,ME,IR,DR,RI,LE> CR: 28000228 XER: 20 CFAR: c0000000000a6604 DAR: 0000000000000028 DSISR: 00080000 IRQMASK: 0GPR00:
c0000000000a6608c000000005783990c000000001036100c0000007bf761860
GPR04: 0000000000000000c0000000057838340000000000000000 0000000000000000
GPR08:69626d2c6e7075000000000000000000 0000000000000000 9000000002001003
GPR12: 0000000000000000c0000007bfff8300c0000000000104500000000000000000
GPR16:c000000000ced9380000000000000100c000000000ced94800000000000a0000
GPR20:00000000000bfffec000000000ced9a80000000000000200c000000000ced978
GPR24:00000000006080c0c000000716d09828c00000002e6fd0000000000000000000
GPR28:c0000007bf4aff68c0000007bf8d0080c000000000f23938c0000007bf761860NIP [c0000000000aa38c]pnv_try_setup_npu_table_group+0x1c/0x1a0LR [c0000000000a6608]pnv_pci_ioda_fixup+0x1f8/0x660Call Trace: [c000000005783990] [c0000000000aa3d0] pnv_try_setup_npu_table_group+0x60/0x [c0000000057839d0] [c0000000000a661c] pnv_pci_ioda_fixup+0x20c/0x660 [c000000005783ab0] [c000000000e1d4c0] pcibios_resource_survey+0x2c8/0x31c [c000000005783b90] [c000000000e1caf4] pcibios_init+0xb0/0xe4 [c000000005783c10] [c000000000010054] do_one_initcall+0x64/0x264 [c000000005783ce0] [c000000000e1132c] kernel_init_freeable+0x36c/0x468 [c000000005783db0] [c000000000010474] kernel_init+0x2c/0x148 [c000000005783e20] [c00000000000b794] ret_from_kernel_thread+0x5c/0x68Quoted 6 lines
An opencapi device is using a device PE, so the current code breaks because
pe->pbusis not defined.More generally, there's no need to define an IOMMU group for opencapi, as the device sends real addresses directly (admittedly, the virtualization story is yet to be written). So let's fix it by
Current plan is to go for mediated VFIO. The real HW stays under the control of the host ocxl driver, and we still don't need an IOMMU group.
Quoted 5 lines
skipping the IOMMU group setup for opencapi PHBs.
Fixes 0bd971676e68 ("powerpc/powernv/npu: Add compound IOMMU groups")Signed-off-by Frederic Barrat <[email protected]>---
Quoted 5 lines
Reviewed-by Greg Kurz <[email protected]>and
Cc [email protected] # v4.20
Thanks for the review! But why did you add stable? that problem is only seen on 5.0-rc1, isn't it?
Fred
Quoted 18 lines
Quoted 17 lines
arch/powerpc/platforms/powernv/pci-ioda.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-)arch/powerpc/platforms/powernv/pci-ioda.c +2
-1@@ -2681,7 +2681,8 @@ static void pnv_pci_ioda_setup_iommu_api(void)list_for_each_entry (hose, &hose_list, list_node) { phb = hose->private_data; -if (phb->type ==PNV_PHB_NPU_NVLINK ) +if (phb->type ==PNV_PHB_NPU_NVLINK || + phb->type ==PNV_PHB_NPU_OCAPI )continue ;list_for_each_entry (pe, &phb->ioda.pe_list, list) {
Quoted 53 lines · On Wed, 9 Jan 2019 17:45:53 +0100 Frederic Barrat <[email protected]> wrote:
Le 09/01/2019 à 17:25, Greg Kurz a écrit :
Quoted 52 lines
Quoted 33 lines · On Wed, 9 Jan 2019 16:13:42 +0100 Frederic Barrat <[email protected]>…
With a recent change around IOMMU group, a system with an opencapi adapter is no longer booting and we get a kernel oops:
BUG: Kernel NULL pointer dereference at 0x00000028 Faulting instruction address: 0xc0000000000aa38c Oops: Kernel access of bad area, sig: 7 [#1] LE SMP NR_CPUS=2048 NUMA PowerNV Modules linked in: CPU: 5 PID: 1 Comm: swapper/4 Not tainted 5.0.0-rc1-fxb-00001-g3bd6e94bec12 NIP: c0000000000aa38c LR: c0000000000a6608 CTR: c000000000097480 REGS: c000000005783700 TRAP: 0300 Not tainted (5.0.0-rc1-fxb-00001-g3bd6 MSR: 9000000002009033 <SF,HV,VEC,EE,ME,IR,DR,RI,LE> CR: 28000228 XER: 20 CFAR: c0000000000a6604 DAR: 0000000000000028 DSISR: 00080000 IRQMASK: 0GPR00:
c0000000000a6608c000000005783990c000000001036100c0000007bf761860
GPR04: 0000000000000000c0000000057838340000000000000000 0000000000000000
GPR08:69626d2c6e7075000000000000000000 0000000000000000 9000000002001003
GPR12: 0000000000000000c0000007bfff8300c0000000000104500000000000000000
GPR16:c000000000ced9380000000000000100c000000000ced94800000000000a0000
GPR20:00000000000bfffec000000000ced9a80000000000000200c000000000ced978
GPR24:00000000006080c0c000000716d09828c00000002e6fd0000000000000000000
GPR28:c0000007bf4aff68c0000007bf8d0080c000000000f23938c0000007bf761860NIP [c0000000000aa38c]pnv_try_setup_npu_table_group+0x1c/0x1a0LR [c0000000000a6608]pnv_pci_ioda_fixup+0x1f8/0x660Call Trace: [c000000005783990] [c0000000000aa3d0] pnv_try_setup_npu_table_group+0x60/0x [c0000000057839d0] [c0000000000a661c] pnv_pci_ioda_fixup+0x20c/0x660 [c000000005783ab0] [c000000000e1d4c0] pcibios_resource_survey+0x2c8/0x31c [c000000005783b90] [c000000000e1caf4] pcibios_init+0xb0/0xe4 [c000000005783c10] [c000000000010054] do_one_initcall+0x64/0x264 [c000000005783ce0] [c000000000e1132c] kernel_init_freeable+0x36c/0x468 [c000000005783db0] [c000000000010474] kernel_init+0x2c/0x148 [c000000005783e20] [c00000000000b794] ret_from_kernel_thread+0x5c/0x68Quoted 6 lines
An opencapi device is using a device PE, so the current code breaks because
pe->pbusis not defined.More generally, there's no need to define an IOMMU group for opencapi, as the device sends real addresses directly (admittedly, the virtualization story is yet to be written). So let's fix it by
Current plan is to go for mediated VFIO. The real HW stays under the control of the host ocxl driver, and we still don't need an IOMMU group.
Quoted 5 lines
skipping the IOMMU group setup for opencapi PHBs.
Fixes 0bd971676e68 ("powerpc/powernv/npu: Add compound IOMMU groups")Signed-off-by Frederic Barrat <[email protected]>---
Quoted 9 lines
Quoted 5 lines
Reviewed-by Greg Kurz <[email protected]>and
Cc [email protected] # v4.20Thanks for the review! But why did you add stable? that problem is only seen on 5.0-rc1, isn't it?
Based on the fact that 0bd971676e68 was committed in 4.20... but I haven't tested :)
Quoted 22 lines
Fred
Quoted 18 lines
Quoted 17 lines
arch/powerpc/platforms/powernv/pci-ioda.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-)arch/powerpc/platforms/powernv/pci-ioda.c +2
-1@@ -2681,7 +2681,8 @@ static void pnv_pci_ioda_setup_iommu_api(void)list_for_each_entry (hose, &hose_list, list_node) { phb = hose->private_data; -if (phb->type ==PNV_PHB_NPU_NVLINK ) +if (phb->type ==PNV_PHB_NPU_NVLINK || + phb->type ==PNV_PHB_NPU_OCAPI )continue ;list_for_each_entry (pe, &phb->ioda.pe_list, list) {
Michael Ellerman
reviewerQuoted 60 lines · Greg Kurz
Quoted 57 lines · On Wed, 9 Jan 2019 17:45:53 +0100 Frederic Barrat <[email protected]> wrote:
Le 09/01/2019 à 17:25, Greg Kurz a écrit :
Quoted 56 lines
Quoted 33 lines · On Wed, 9 Jan 2019 16:13:42 +0100 Frederic Barrat <[email protected]>…
With a recent change around IOMMU group, a system with an opencapi adapter is no longer booting and we get a kernel oops:
BUG: Kernel NULL pointer dereference at 0x00000028 Faulting instruction address: 0xc0000000000aa38c Oops: Kernel access of bad area, sig: 7 [#1] LE SMP NR_CPUS=2048 NUMA PowerNV Modules linked in: CPU: 5 PID: 1 Comm: swapper/4 Not tainted 5.0.0-rc1-fxb-00001-g3bd6e94bec12 NIP: c0000000000aa38c LR: c0000000000a6608 CTR: c000000000097480 REGS: c000000005783700 TRAP: 0300 Not tainted (5.0.0-rc1-fxb-00001-g3bd6 MSR: 9000000002009033 <SF,HV,VEC,EE,ME,IR,DR,RI,LE> CR: 28000228 XER: 20 CFAR: c0000000000a6604 DAR: 0000000000000028 DSISR: 00080000 IRQMASK: 0GPR00:
c0000000000a6608c000000005783990c000000001036100c0000007bf761860
GPR04: 0000000000000000c0000000057838340000000000000000 0000000000000000
GPR08:69626d2c6e7075000000000000000000 0000000000000000 9000000002001003
GPR12: 0000000000000000c0000007bfff8300c0000000000104500000000000000000
GPR16:c000000000ced9380000000000000100c000000000ced94800000000000a0000
GPR20:00000000000bfffec000000000ced9a80000000000000200c000000000ced978
GPR24:00000000006080c0c000000716d09828c00000002e6fd0000000000000000000
GPR28:c0000007bf4aff68c0000007bf8d0080c000000000f23938c0000007bf761860NIP [c0000000000aa38c]pnv_try_setup_npu_table_group+0x1c/0x1a0LR [c0000000000a6608]pnv_pci_ioda_fixup+0x1f8/0x660Call Trace: [c000000005783990] [c0000000000aa3d0] pnv_try_setup_npu_table_group+0x60/0x [c0000000057839d0] [c0000000000a661c] pnv_pci_ioda_fixup+0x20c/0x660 [c000000005783ab0] [c000000000e1d4c0] pcibios_resource_survey+0x2c8/0x31c [c000000005783b90] [c000000000e1caf4] pcibios_init+0xb0/0xe4 [c000000005783c10] [c000000000010054] do_one_initcall+0x64/0x264 [c000000005783ce0] [c000000000e1132c] kernel_init_freeable+0x36c/0x468 [c000000005783db0] [c000000000010474] kernel_init+0x2c/0x148 [c000000005783e20] [c00000000000b794] ret_from_kernel_thread+0x5c/0x68Quoted 6 lines
An opencapi device is using a device PE, so the current code breaks because
pe->pbusis not defined.More generally, there's no need to define an IOMMU group for opencapi, as the device sends real addresses directly (admittedly, the virtualization story is yet to be written). So let's fix it by
Current plan is to go for mediated VFIO. The real HW stays under the control of the host ocxl driver, and we still don't need an IOMMU group.
Quoted 5 lines
skipping the IOMMU group setup for opencapi PHBs.
Fixes 0bd971676e68 ("powerpc/powernv/npu: Add compound IOMMU groups")Signed-off-by Frederic Barrat <[email protected]>---
Reviewed-by Greg Kurz <[email protected]>and
Quoted 7 lines
Quoted 4 lines
Quoted 1 line
Cc [email protected] # v4.20Thanks for the review! But why did you add stable? that problem is only seen on 5.0-rc1, isn't it?
Based on the fact that
0bd971676e68was committed in 4.20... but I haven't tested :)
It was committed to a branch based off 4.20-rc2, but it wasn't merged into the 4.20 release.
$ git describe --match "v[0-9]*" --contains 0bd971676e68 v5.0-rc1~137^2~15
So it doesn't need to go to stable.
cheers
Quoted 69 lines · On Thu, 10 Jan 2019 23:25:11 +1100 Michael Ellerman <[email protected]> wrote:
Quoted 60 lines · Greg Kurz
Quoted 57 lines · On Wed, 9 Jan 2019 17:45:53 +0100 Frederic Barrat <[email protected]> wrote:
Le 09/01/2019 à 17:25, Greg Kurz a écrit :
Quoted 56 lines
Quoted 33 lines · On Wed, 9 Jan 2019 16:13:42 +0100 Frederic Barrat <[email protected]>…
With a recent change around IOMMU group, a system with an opencapi adapter is no longer booting and we get a kernel oops:
BUG: Kernel NULL pointer dereference at 0x00000028 Faulting instruction address: 0xc0000000000aa38c Oops: Kernel access of bad area, sig: 7 [#1] LE SMP NR_CPUS=2048 NUMA PowerNV Modules linked in: CPU: 5 PID: 1 Comm: swapper/4 Not tainted 5.0.0-rc1-fxb-00001-g3bd6e94bec12 NIP: c0000000000aa38c LR: c0000000000a6608 CTR: c000000000097480 REGS: c000000005783700 TRAP: 0300 Not tainted (5.0.0-rc1-fxb-00001-g3bd6 MSR: 9000000002009033 <SF,HV,VEC,EE,ME,IR,DR,RI,LE> CR: 28000228 XER: 20 CFAR: c0000000000a6604 DAR: 0000000000000028 DSISR: 00080000 IRQMASK: 0GPR00:
c0000000000a6608c000000005783990c000000001036100c0000007bf761860
GPR04: 0000000000000000c0000000057838340000000000000000 0000000000000000
GPR08:69626d2c6e7075000000000000000000 0000000000000000 9000000002001003
GPR12: 0000000000000000c0000007bfff8300c0000000000104500000000000000000
GPR16:c000000000ced9380000000000000100c000000000ced94800000000000a0000
GPR20:00000000000bfffec000000000ced9a80000000000000200c000000000ced978
GPR24:00000000006080c0c000000716d09828c00000002e6fd0000000000000000000
GPR28:c0000007bf4aff68c0000007bf8d0080c000000000f23938c0000007bf761860NIP [c0000000000aa38c]pnv_try_setup_npu_table_group+0x1c/0x1a0LR [c0000000000a6608]pnv_pci_ioda_fixup+0x1f8/0x660Call Trace: [c000000005783990] [c0000000000aa3d0] pnv_try_setup_npu_table_group+0x60/0x [c0000000057839d0] [c0000000000a661c] pnv_pci_ioda_fixup+0x20c/0x660 [c000000005783ab0] [c000000000e1d4c0] pcibios_resource_survey+0x2c8/0x31c [c000000005783b90] [c000000000e1caf4] pcibios_init+0xb0/0xe4 [c000000005783c10] [c000000000010054] do_one_initcall+0x64/0x264 [c000000005783ce0] [c000000000e1132c] kernel_init_freeable+0x36c/0x468 [c000000005783db0] [c000000000010474] kernel_init+0x2c/0x148 [c000000005783e20] [c00000000000b794] ret_from_kernel_thread+0x5c/0x68Quoted 6 lines
An opencapi device is using a device PE, so the current code breaks because
pe->pbusis not defined.More generally, there's no need to define an IOMMU group for opencapi, as the device sends real addresses directly (admittedly, the virtualization story is yet to be written). So let's fix it by
Current plan is to go for mediated VFIO. The real HW stays under the control of the host ocxl driver, and we still don't need an IOMMU group.
Quoted 5 lines
skipping the IOMMU group setup for opencapi PHBs.
Fixes 0bd971676e68 ("powerpc/powernv/npu: Add compound IOMMU groups")Signed-off-by Frederic Barrat <[email protected]>---
Reviewed-by Greg Kurz <[email protected]>and
Quoted 7 lines
Quoted 4 lines
Quoted 1 line
Cc [email protected] # v4.20Thanks for the review! But why did you add stable? that problem is only seen on 5.0-rc1, isn't it?
Based on the fact that
0bd971676e68was committed in 4.20... but I haven't tested :)
Quoted 8 lines
It was committed to a branch based off 4.20-rc2, but it wasn't merged into the 4.20 release.
$ git describe --match "v[0-9]*" --contains 0bd971676e68 v5.0-rc1~137^2~15So it doesn't need to go to stable.
Yeah I realized that afterwards, sorry for the noise and Happy New Year :)
Quoted 1 line
cheers
Le 10/01/2019 à 13:25, Michael Ellerman a écrit :
Quoted 69 lines
Quoted 60 lines · Greg Kurz
Quoted 57 lines · On Wed, 9 Jan 2019 17:45:53 +0100 Frederic Barrat <[email protected]> wrote:
Le 09/01/2019 à 17:25, Greg Kurz a écrit :
Quoted 56 lines
Quoted 33 lines · On Wed, 9 Jan 2019 16:13:42 +0100 Frederic Barrat <[email protected]>…
With a recent change around IOMMU group, a system with an opencapi adapter is no longer booting and we get a kernel oops:
BUG: Kernel NULL pointer dereference at 0x00000028 Faulting instruction address: 0xc0000000000aa38c Oops: Kernel access of bad area, sig: 7 [#1] LE SMP NR_CPUS=2048 NUMA PowerNV Modules linked in: CPU: 5 PID: 1 Comm: swapper/4 Not tainted 5.0.0-rc1-fxb-00001-g3bd6e94bec12 NIP: c0000000000aa38c LR: c0000000000a6608 CTR: c000000000097480 REGS: c000000005783700 TRAP: 0300 Not tainted (5.0.0-rc1-fxb-00001-g3bd6 MSR: 9000000002009033 <SF,HV,VEC,EE,ME,IR,DR,RI,LE> CR: 28000228 XER: 20 CFAR: c0000000000a6604 DAR: 0000000000000028 DSISR: 00080000 IRQMASK: 0GPR00:
c0000000000a6608c000000005783990c000000001036100c0000007bf761860
GPR04: 0000000000000000c0000000057838340000000000000000 0000000000000000
GPR08:69626d2c6e7075000000000000000000 0000000000000000 9000000002001003
GPR12: 0000000000000000c0000007bfff8300c0000000000104500000000000000000
GPR16:c000000000ced9380000000000000100c000000000ced94800000000000a0000
GPR20:00000000000bfffec000000000ced9a80000000000000200c000000000ced978
GPR24:00000000006080c0c000000716d09828c00000002e6fd0000000000000000000
GPR28:c0000007bf4aff68c0000007bf8d0080c000000000f23938c0000007bf761860NIP [c0000000000aa38c]pnv_try_setup_npu_table_group+0x1c/0x1a0LR [c0000000000a6608]pnv_pci_ioda_fixup+0x1f8/0x660Call Trace: [c000000005783990] [c0000000000aa3d0] pnv_try_setup_npu_table_group+0x60/0x [c0000000057839d0] [c0000000000a661c] pnv_pci_ioda_fixup+0x20c/0x660 [c000000005783ab0] [c000000000e1d4c0] pcibios_resource_survey+0x2c8/0x31c [c000000005783b90] [c000000000e1caf4] pcibios_init+0xb0/0xe4 [c000000005783c10] [c000000000010054] do_one_initcall+0x64/0x264 [c000000005783ce0] [c000000000e1132c] kernel_init_freeable+0x36c/0x468 [c000000005783db0] [c000000000010474] kernel_init+0x2c/0x148 [c000000005783e20] [c00000000000b794] ret_from_kernel_thread+0x5c/0x68Quoted 6 lines
An opencapi device is using a device PE, so the current code breaks because
pe->pbusis not defined.More generally, there's no need to define an IOMMU group for opencapi, as the device sends real addresses directly (admittedly, the virtualization story is yet to be written). So let's fix it by
Current plan is to go for mediated VFIO. The real HW stays under the control of the host ocxl driver, and we still don't need an IOMMU group.
Quoted 5 lines
skipping the IOMMU group setup for opencapi PHBs.
Fixes 0bd971676e68 ("powerpc/powernv/npu: Add compound IOMMU groups")Signed-off-by Frederic Barrat <[email protected]>---
Reviewed-by Greg Kurz <[email protected]>and
Quoted 7 lines
Quoted 4 lines
Quoted 1 line
Cc [email protected] # v4.20Thanks for the review! But why did you add stable? that problem is only seen on 5.0-rc1, isn't it?
Based on the fact that
0bd971676e68was committed in 4.20... but I haven't tested :)
Quoted 7 lines
It was committed to a branch based off 4.20-rc2, but it wasn't merged into the 4.20 release.
$ git describe --match "v[0-9]*" --contains 0bd971676e68 v5.0-rc1~137^2~15So it doesn't need to go to stable.
Which makes me wonder if Greg (KH) was really talking about that original patch and whether something worthwhile was dropped from stable by mistake?
Fred
Quoted 73 lines · Frederic Barrat
Le 10/01/2019 à 13:25, Michael Ellerman a écrit :
Quoted 70 lines
Quoted 58 lines · Greg Kurz
Quoted 55 lines · On Wed, 9 Jan 2019 17:45:53 +0100 Frederic Barrat <[email protected]> wrote:
Le 09/01/2019 à 17:25, Greg Kurz a écrit :
Quoted 54 lines
Quoted 33 lines · On Wed, 9 Jan 2019 16:13:42 +0100 Frederic Barrat <[email protected]>…
With a recent change around IOMMU group, a system with an opencapi adapter is no longer booting and we get a kernel oops:
BUG: Kernel NULL pointer dereference at 0x00000028 Faulting instruction address: 0xc0000000000aa38c Oops: Kernel access of bad area, sig: 7 [#1] LE SMP NR_CPUS=2048 NUMA PowerNV Modules linked in: CPU: 5 PID: 1 Comm: swapper/4 Not tainted 5.0.0-rc1-fxb-00001-g3bd6e94bec12 NIP: c0000000000aa38c LR: c0000000000a6608 CTR: c000000000097480 REGS: c000000005783700 TRAP: 0300 Not tainted (5.0.0-rc1-fxb-00001-g3bd6 MSR: 9000000002009033 <SF,HV,VEC,EE,ME,IR,DR,RI,LE> CR: 28000228 XER: 20 CFAR: c0000000000a6604 DAR: 0000000000000028 DSISR: 00080000 IRQMASK: 0GPR00:
c0000000000a6608c000000005783990c000000001036100c0000007bf761860
GPR04: 0000000000000000c0000000057838340000000000000000 0000000000000000
GPR08:69626d2c6e7075000000000000000000 0000000000000000 9000000002001003
GPR12: 0000000000000000c0000007bfff8300c0000000000104500000000000000000
GPR16:c000000000ced9380000000000000100c000000000ced94800000000000a0000
GPR20:00000000000bfffec000000000ced9a80000000000000200c000000000ced978
GPR24:00000000006080c0c000000716d09828c00000002e6fd0000000000000000000
GPR28:c0000007bf4aff68c0000007bf8d0080c000000000f23938c0000007bf761860NIP [c0000000000aa38c]pnv_try_setup_npu_table_group+0x1c/0x1a0LR [c0000000000a6608]pnv_pci_ioda_fixup+0x1f8/0x660Call Trace: [c000000005783990] [c0000000000aa3d0] pnv_try_setup_npu_table_group+0x60/0x [c0000000057839d0] [c0000000000a661c] pnv_pci_ioda_fixup+0x20c/0x660 [c000000005783ab0] [c000000000e1d4c0] pcibios_resource_survey+0x2c8/0x31c [c000000005783b90] [c000000000e1caf4] pcibios_init+0xb0/0xe4 [c000000005783c10] [c000000000010054] do_one_initcall+0x64/0x264 [c000000005783ce0] [c000000000e1132c] kernel_init_freeable+0x36c/0x468 [c000000005783db0] [c000000000010474] kernel_init+0x2c/0x148 [c000000005783e20] [c00000000000b794] ret_from_kernel_thread+0x5c/0x68Quoted 6 lines
An opencapi device is using a device PE, so the current code breaks because
pe->pbusis not defined.More generally, there's no need to define an IOMMU group for opencapi, as the device sends real addresses directly (admittedly, the virtualization story is yet to be written). So let's fix it by
Current plan is to go for mediated VFIO. The real HW stays under the control of the host ocxl driver, and we still don't need an IOMMU group.
Quoted 5 lines
skipping the IOMMU group setup for opencapi PHBs.
Fixes 0bd971676e68 ("powerpc/powernv/npu: Add compound IOMMU groups")Signed-off-by Frederic Barrat <[email protected]>---
Reviewed-by Greg Kurz <[email protected]>and
Quoted 7 lines
Quoted 4 lines
Quoted 1 line
Cc [email protected] # v4.20Thanks for the review! But why did you add stable? that problem is only seen on 5.0-rc1, isn't it?
Based on the fact that
0bd971676e68was committed in 4.20... but I haven't tested :)It was committed to a branch based off 4.20-rc2, but it wasn't merged into the 4.20 release.
Quoted 7 lines
Quoted 4 lines
$ git describe --match "v[0-9]*" --contains 0bd971676e68 v5.0-rc1~137^2~15So it doesn't need to go to stable.
Which makes me wonder if Greg (KH) was really talking about that original patch and whether something worthwhile was dropped from stable by mistake?
Totally different thread, sorry for the noise, my fault...
greg k-h
Quoted 55 lines · Frederic Barrat
Le 09/01/2019 à 17:25, Greg Kurz a écrit :
Quoted 52 lines
Quoted 33 lines · On Wed, 9 Jan 2019 16:13:42 +0100 Frederic Barrat <[email protected]>…
With a recent change around IOMMU group, a system with an opencapi adapter is no longer booting and we get a kernel oops:
BUG: Kernel NULL pointer dereference at 0x00000028 Faulting instruction address: 0xc0000000000aa38c Oops: Kernel access of bad area, sig: 7 [#1] LE SMP NR_CPUS=2048 NUMA PowerNV Modules linked in: CPU: 5 PID: 1 Comm: swapper/4 Not tainted 5.0.0-rc1-fxb-00001-g3bd6e94bec12 NIP: c0000000000aa38c LR: c0000000000a6608 CTR: c000000000097480 REGS: c000000005783700 TRAP: 0300 Not tainted (5.0.0-rc1-fxb-00001-g3bd6 MSR: 9000000002009033 <SF,HV,VEC,EE,ME,IR,DR,RI,LE> CR: 28000228 XER: 20 CFAR: c0000000000a6604 DAR: 0000000000000028 DSISR: 00080000 IRQMASK: 0GPR00:
c0000000000a6608c000000005783990c000000001036100c0000007bf761860
GPR04: 0000000000000000c0000000057838340000000000000000 0000000000000000
GPR08:69626d2c6e7075000000000000000000 0000000000000000 9000000002001003
GPR12: 0000000000000000c0000007bfff8300c0000000000104500000000000000000
GPR16:c000000000ced9380000000000000100c000000000ced94800000000000a0000
GPR20:00000000000bfffec000000000ced9a80000000000000200c000000000ced978
GPR24:00000000006080c0c000000716d09828c00000002e6fd0000000000000000000
GPR28:c0000007bf4aff68c0000007bf8d0080c000000000f23938c0000007bf761860NIP [c0000000000aa38c]pnv_try_setup_npu_table_group+0x1c/0x1a0LR [c0000000000a6608]pnv_pci_ioda_fixup+0x1f8/0x660Call Trace: [c000000005783990] [c0000000000aa3d0] pnv_try_setup_npu_table_group+0x60/0x [c0000000057839d0] [c0000000000a661c] pnv_pci_ioda_fixup+0x20c/0x660 [c000000005783ab0] [c000000000e1d4c0] pcibios_resource_survey+0x2c8/0x31c [c000000005783b90] [c000000000e1caf4] pcibios_init+0xb0/0xe4 [c000000005783c10] [c000000000010054] do_one_initcall+0x64/0x264 [c000000005783ce0] [c000000000e1132c] kernel_init_freeable+0x36c/0x468 [c000000005783db0] [c000000000010474] kernel_init+0x2c/0x148 [c000000005783e20] [c00000000000b794] ret_from_kernel_thread+0x5c/0x68Quoted 6 lines
An opencapi device is using a device PE, so the current code breaks because
pe->pbusis not defined.More generally, there's no need to define an IOMMU group for opencapi, as the device sends real addresses directly (admittedly, the virtualization story is yet to be written). So let's fix it by
Current plan is to go for mediated VFIO. The real HW stays under the control of the host ocxl driver, and we still don't need an IOMMU group.
Quoted 5 lines
skipping the IOMMU group setup for opencapi PHBs.
Fixes 0bd971676e68 ("powerpc/powernv/npu: Add compound IOMMU groups")Signed-off-by Frederic Barrat <[email protected]>---
Quoted 8 lines
Quoted 5 lines
Reviewed-by Greg Kurz <[email protected]>and
Cc [email protected] # v4.20Thanks for the review! But why did you add stable? that problem is only seen on 5.0-rc1, isn't it?
No, this is fixing a patch that got backported to stable.
Well, attempted to be backported, I dropped it because of the problem :)
thanks,
greg k-h
Quoted 36 lines · Frederic Barrat
With a recent change around IOMMU group, a system with an opencapi adapter is no longer booting and we get a kernel oops:
BUG: Kernel NULL pointer dereference at 0x00000028 Faulting instruction address: 0xc0000000000aa38c Oops: Kernel access of bad area, sig: 7 [#1] LE SMP NR_CPUS=2048 NUMA PowerNV Modules linked in: CPU: 5 PID: 1 Comm: swapper/4 Not tainted 5.0.0-rc1-fxb-00001-g3bd6e94bec12 NIP: c0000000000aa38c LR: c0000000000a6608 CTR: c000000000097480 REGS: c000000005783700 TRAP: 0300 Not tainted (5.0.0-rc1-fxb-00001-g3bd6 MSR: 9000000002009033 <SF,HV,VEC,EE,ME,IR,DR,RI,LE> CR: 28000228 XER: 20 CFAR: c0000000000a6604 DAR: 0000000000000028 DSISR: 00080000 IRQMASK: 0GPR00:
c0000000000a6608c000000005783990c000000001036100c0000007bf761860
GPR04: 0000000000000000c0000000057838340000000000000000 0000000000000000
GPR08:69626d2c6e7075000000000000000000 0000000000000000 9000000002001003
GPR12: 0000000000000000c0000007bfff8300c0000000000104500000000000000000
GPR16:c000000000ced9380000000000000100c000000000ced94800000000000a0000
GPR20:00000000000bfffec000000000ced9a80000000000000200c000000000ced978
GPR24:00000000006080c0c000000716d09828c00000002e6fd0000000000000000000
GPR28:c0000007bf4aff68c0000007bf8d0080c000000000f23938c0000007bf761860NIP [c0000000000aa38c]pnv_try_setup_npu_table_group+0x1c/0x1a0LR [c0000000000a6608]pnv_pci_ioda_fixup+0x1f8/0x660Call Trace: [c000000005783990] [c0000000000aa3d0] pnv_try_setup_npu_table_group+0x60/0x [c0000000057839d0] [c0000000000a661c] pnv_pci_ioda_fixup+0x20c/0x660 [c000000005783ab0] [c000000000e1d4c0] pcibios_resource_survey+0x2c8/0x31c [c000000005783b90] [c000000000e1caf4] pcibios_init+0xb0/0xe4 [c000000005783c10] [c000000000010054] do_one_initcall+0x64/0x264 [c000000005783ce0] [c000000000e1132c] kernel_init_freeable+0x36c/0x468 [c000000005783db0] [c000000000010474] kernel_init+0x2c/0x148 [c000000005783e20] [c00000000000b794] ret_from_kernel_thread+0x5c/0x68An opencapi device is using a device PE, so the current code breaks because
pe->pbusis not defined.
Quoted 7 lines
More generally, there's no need to define an IOMMU group for opencapi, as the device sends real addresses directly (admittedly, the virtualization story is yet to be written). So let's fix it by skipping the IOMMU group setup for opencapi PHBs.
Fixes 0bd971676e68 ("powerpc/powernv/npu: Add compound IOMMU groups")Signed-off-by Frederic Barrat <[email protected]>
Quoted 19 lines
--- arch/powerpc/platforms/powernv/pci-ioda.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-)arch/powerpc/platforms/powernv/pci-ioda.c +2
-1@@ -2681,7 +2681,8 @@ static void pnv_pci_ioda_setup_iommu_api(void)list_for_each_entry (hose, &hose_list, list_node) { phb = hose->private_data; -if (phb->type ==PNV_PHB_NPU_NVLINK ) +if (phb->type ==PNV_PHB_NPU_NVLINK || + phb->type ==PNV_PHB_NPU_OCAPI )continue ;list_for_each_entry (pe, &phb->ioda.pe_list, list) {
Signature · 2 lines
Andrew Donnellan OzLabs, ADL Canberra [email protected] IBM Australia Limited
Michael Ellerman
AppliedQuoted 41 lines · Frederic Barrat
With a recent change around IOMMU group, a system with an opencapi adapter is no longer booting and we get a kernel oops:
BUG: Kernel NULL pointer dereference at 0x00000028 Faulting instruction address: 0xc0000000000aa38c Oops: Kernel access of bad area, sig: 7 [#1] LE SMP NR_CPUS=2048 NUMA PowerNV Modules linked in: CPU: 5 PID: 1 Comm: swapper/4 Not tainted 5.0.0-rc1-fxb-00001-g3bd6e94bec12 NIP: c0000000000aa38c LR: c0000000000a6608 CTR: c000000000097480 REGS: c000000005783700 TRAP: 0300 Not tainted (5.0.0-rc1-fxb-00001-g3bd6 MSR: 9000000002009033 <SF,HV,VEC,EE,ME,IR,DR,RI,LE> CR: 28000228 XER: 20 CFAR: c0000000000a6604 DAR: 0000000000000028 DSISR: 00080000 IRQMASK: 0GPR00:
c0000000000a6608c000000005783990c000000001036100c0000007bf761860
GPR04: 0000000000000000c0000000057838340000000000000000 0000000000000000
GPR08:69626d2c6e7075000000000000000000 0000000000000000 9000000002001003
GPR12: 0000000000000000c0000007bfff8300c0000000000104500000000000000000
GPR16:c000000000ced9380000000000000100c000000000ced94800000000000a0000
GPR20:00000000000bfffec000000000ced9a80000000000000200c000000000ced978
GPR24:00000000006080c0c000000716d09828c00000002e6fd0000000000000000000
GPR28:c0000007bf4aff68c0000007bf8d0080c000000000f23938c0000007bf761860NIP [c0000000000aa38c]pnv_try_setup_npu_table_group+0x1c/0x1a0LR [c0000000000a6608]pnv_pci_ioda_fixup+0x1f8/0x660Call Trace: [c000000005783990] [c0000000000aa3d0] pnv_try_setup_npu_table_group+0x60/0x [c0000000057839d0] [c0000000000a661c] pnv_pci_ioda_fixup+0x20c/0x660 [c000000005783ab0] [c000000000e1d4c0] pcibios_resource_survey+0x2c8/0x31c [c000000005783b90] [c000000000e1caf4] pcibios_init+0xb0/0xe4 [c000000005783c10] [c000000000010054] do_one_initcall+0x64/0x264 [c000000005783ce0] [c000000000e1132c] kernel_init_freeable+0x36c/0x468 [c000000005783db0] [c000000000010474] kernel_init+0x2c/0x148 [c000000005783e20] [c00000000000b794] ret_from_kernel_thread+0x5c/0x68An opencapi device is using a device PE, so the current code breaks because
pe->pbusis not defined.More generally, there's no need to define an IOMMU group for opencapi, as the device sends real addresses directly (admittedly, the virtualization story is yet to be written). So let's fix it by skipping the IOMMU group setup for opencapi PHBs.
Quoted 4 lines
Fixes 0bd971676e68 ("powerpc/powernv/npu: Add compound IOMMU groups")Signed-off-by Frederic Barrat <[email protected]>Reviewed-by Greg Kurz <[email protected]>Reviewed-by Andrew Donnellan <[email protected]>
Applied to powerpc fixes, thanks.
https://git.kernel.org/powerpc/c/6bca515917515b66b7e1dfc1d1d3b7bd
cheers