Unhandled level 2 translation fault on A72 board.

I met this problem when running the hackbench test on A72 chip board:

sh[4779]: unhandled level 2 translation fault (11) at 0x7f96be0c80, esr 0x83000006 pgd = ffffffc01a1f0000
[7f96be0c80] *pgd=0000000084a20003, *pud=0000000084a20003, *pmd=0000000000000000

CPU: 1 PID: 4779 Comm: sh Tainted: G O 4.1.15+ #21 
Hardware name: Hisilicon PhosphorHi1382 EVB (DT) 
task: ffffffc0163cc500 ti: ffffffc083abc000 task.ti: ffffffc083abc000 
PC is at 0x7f96be0c80 
LR is at 0x7fb2684eb4 
pc : [<0000007f96be0c80>] lr : [<0000007fb2684eb4>] pstate: 60000000 

sh[4963]: unhandled level 2 translation fault (11) at 0x00000000, esr 0x92000006 pgd = ffffffc0180c6000
[00000000] *pgd=0000000015157003, *pud=0000000015157003, *pmd=0000000000000000

CPU: 0 PID: 4963 Comm: sh Tainted: G O 4.1.15+ #21 
Hardware name: Hisilicon PhosphorHi1382 EVB (DT) 
task: ffffffc0163cb980 ti: ffffffc0840c8000 task.ti: ffffffc0840c8000 
PC is at 0x42c0c8 
LR is at 0x42c03c 
pc : [<000000000042c0c8>] lr : [<000000000042c03c>] pstate: 80000000 

if I run the benchmark only on the core which is in the same cluster, it looks fine and no error happened, but if I enable the core which in the different cluster, it will happened.

I remember that I met the same problem on the A57 and fix it by enable the [bit6] of the CPUECTLR_EL1 and enable MN, But this time, I enable the same setting and looks no effort, I have no idea about this problem, does A57 and A72 has so big difference on TLB?

Quoted 6 lines · Ding Tianhong

I met this problem when running the hackbench test on A72 chip board:

sh[4779]: unhandled level 2 translation fault (11) at 0x7f96be0c80, esr 0x83000006 pgd = ffffffc01a1f0000
[7f96be0c80] *pgd=0000000084a20003, *pud=0000000084a20003, *pmd=0000000000000000

Quoted 6 lines

CPU: 1 PID: 4779 Comm: sh Tainted: G O 4.1.15+ #21 
Hardware name: Hisilicon PhosphorHi1382 EVB (DT) 
task: ffffffc0163cc500 ti: ffffffc083abc000 task.ti: ffffffc083abc000 
PC is at 0x7f96be0c80 
LR is at 0x7fb2684eb4 
pc : [<0000007f96be0c80>] lr : [<0000007fb2684eb4>] pstate: 60000000 

So here it's user space trying to execute from 0x7f96be0c80 (instruction abort).

Quoted 10 lines

sh[4963]: unhandled level 2 translation fault (11) at 0x00000000, esr 0x92000006 pgd = ffffffc0180c6000
[00000000] *pgd=0000000015157003, *pud=0000000015157003, *pmd=0000000000000000

CPU: 0 PID: 4963 Comm: sh Tainted: G O 4.1.15+ #21 
Hardware name: Hisilicon PhosphorHi1382 EVB (DT) 
task: ffffffc0163cb980 ti: ffffffc0840c8000 task.ti: ffffffc0840c8000 
PC is at 0x42c0c8 
LR is at 0x42c03c 
pc : [<000000000042c0c8>] lr : [<000000000042c03c>] pstate: 80000000 

And here you have a null pointer dereference.

Quoted 8 lines

if I run the benchmark only on the core which is in the same cluster, it looks fine and no error happened, but if I enable the core which in the different cluster, it will happened.

I remember that I met the same problem on the A57 and fix it by enable the [bit6] of the CPUECTLR_EL1 and enable MN, But this time, I enable the same setting and looks no effort, I have no idea about this problem, does A57 and A72 has so big difference on TLB?

I can't tell for sure it's a TLB issue. The kernel page table dump shows *pmd being 0, so the fault is correctly called "level 2 translation fault". It also seems that there is no vma at this address, hence the kernel reports it as unhandled. It looks like data corruption which could be caused by cache or TLB incoherence. Just make sure the interconnect linking the two clusters is configured correctly by _firmware_ before Linux starts.

Quoted 6 lines · Catalin Marinas

Quoted 5 lines · Ding Tianhong

I met this problem when running the hackbench test on A72 chip board:

sh[4779]: unhandled level 2 translation fault (11) at 0x7f96be0c80, esr 0x83000006 pgd = ffffffc01a1f0000
[7f96be0c80] *pgd=0000000084a20003, *pud=0000000084a20003, *pmd=0000000000000000

Quoted 7 lines

I can't tell for sure it's a TLB issue. The kernel page table dump shows *pmd being 0, so the fault is correctly called "level 2 translation fault". It also seems that there is no vma at this address, hence the kernel reports it as unhandled. It looks like data corruption which could be caused by cache or TLB incoherence. Just make sure the interconnect linking the two clusters is configured correctly by _firmware_ before Linux starts.

Thanks for the apply, I have try to apply this patch to test:

--- arch/arm64/kernel/process.c | 9 +++++++++

1 file changed, 9 insertions(+)

arch/arm64/kernel/process.c +9 -0

index 6391485..d7d8439 100644
@@ -283,6 +283,13 @@ static void tls_thread_switch(struct task_struct *next)
 : : "r" (tpidr), "r" (tpidrro));
 }
+static void tlb_flush_thread(struct task_struct *prev)
+{
+/* Flush the prev task's TLB entries */
+if (prev->mm)
+flush_tlb_mm(prev->mm);
+}
+
 /*
  * Thread switching.
  */

@@ -296,6 +303,8 @@ struct task_struct *__switch_to(struct task_struct *prev,
 hw_breakpoint_thread_switch(next);
 contextidr_thread_switch(next);
+tlb_flush_thread(prev);
+
 /*
 * Complete any pending TLB or cache maintenance on this CPU in case
 * the thread migrates to a different CPU.
 

The hackbench would work fine after this patch, so I guess that the old thread tlb may not be invalidate as soon as possible, but I don't know why, everything is fine on A57, Does I miss something?

Quoted 7 lines · Ding Tianhong

Quoted 6 lines · Catalin Marinas

Quoted 5 lines · Ding Tianhong

I met this problem when running the hackbench test on A72 chip board:

sh[4779]: unhandled level 2 translation fault (11) at 0x7f96be0c80, esr 0x83000006 pgd = ffffffc01a1f0000
[7f96be0c80] *pgd=0000000084a20003, *pud=0000000084a20003, *pmd=0000000000000000

[...]

Quoted 39 lines

Quoted 7 lines

I can't tell for sure it's a TLB issue. The kernel page table dump shows *pmd being 0, so the fault is correctly called "level 2 translation fault". It also seems that there is no vma at this address, hence the kernel reports it as unhandled. It looks like data corruption which could be caused by cache or TLB incoherence. Just make sure the interconnect linking the two clusters is configured correctly by _firmware_ before Linux starts.

Thanks for the apply, I have try to apply this patch to test:

--- arch/arm64/kernel/process.c | 9 +++++++++

1 file changed, 9 insertions(+)

arch/arm64/kernel/process.c +9 -0

index 6391485..d7d8439 100644
@@ -283,6 +283,13 @@ static void tls_thread_switch(struct task_struct *next)
 : : "r" (tpidr), "r" (tpidrro));
 }
+static void tlb_flush_thread(struct task_struct *prev)
+{
+/* Flush the prev task's TLB entries */
+if (prev->mm)
+flush_tlb_mm(prev->mm);
+}
+
 /*
  * Thread switching.
  */

@@ -296,6 +303,8 @@ struct task_struct *__switch_to(struct task_struct *prev,
 hw_breakpoint_thread_switch(next);
 contextidr_thread_switch(next);
+tlb_flush_thread(prev);
+
 /*
 * Complete any pending TLB or cache maintenance on this CPU in case
 * the thread migrates to a different CPU.
 

Quoted 3 lines

The hackbench would work fine after this patch, so I guess that the old thread tlb may not be invalidate as soon as possible, but I don't know why, everything is fine on A57, Does I miss something?

It looks like the TLB invalidation messages may not get across the CCI between clusters. I don't have the TRMs at hand but make sure all the relevant bits in the CPUs and CCI are enabled.

BTW, which kernel version are you running? Is the firmware your own or built around ARM Trusted Firmware?

Quoted 48 lines · Ding Tianhong

Quoted 7 lines · Catalin Marinas

Quoted 6 lines · Ding Tianhong

Quoted 5 lines

I met this problem when running the hackbench test on A72 chip board:

sh[4779]: unhandled level 2 translation fault (11) at 0x7f96be0c80, esr 0x83000006 pgd = ffffffc01a1f0000
[7f96be0c80] *pgd=0000000084a20003, *pud=0000000084a20003, *pmd=0000000000000000

[...]

Quoted 39 lines

Quoted 7 lines

I can't tell for sure it's a TLB issue. The kernel page table dump shows *pmd being 0, so the fault is correctly called "level 2 translation fault". It also seems that there is no vma at this address, hence the kernel reports it as unhandled. It looks like data corruption which could be caused by cache or TLB incoherence. Just make sure the interconnect linking the two clusters is configured correctly by _firmware_ before Linux starts.

Thanks for the apply, I have try to apply this patch to test:

--- arch/arm64/kernel/process.c | 9 +++++++++

1 file changed, 9 insertions(+)

arch/arm64/kernel/process.c +9 -0

index 6391485..d7d8439 100644
@@ -283,6 +283,13 @@ static void tls_thread_switch(struct task_struct *next)
 : : "r" (tpidr), "r" (tpidrro));
 }
+static void tlb_flush_thread(struct task_struct *prev)
+{
+/* Flush the prev task's TLB entries */
+if (prev->mm)
+flush_tlb_mm(prev->mm);
+}
+
 /*
  * Thread switching.
  */

@@ -296,6 +303,8 @@ struct task_struct *__switch_to(struct task_struct *prev,
 hw_breakpoint_thread_switch(next);
 contextidr_thread_switch(next);
+tlb_flush_thread(prev);
+
 /*
 * Complete any pending TLB or cache maintenance on this CPU in case
 * the thread migrates to a different CPU.
 

Quoted 8 lines

Quoted 3 lines

The hackbench would work fine after this patch, so I guess that the old thread tlb may not be invalidate as soon as possible, but I don't know why, everything is fine on A57, Does I miss something?

It looks like the TLB invalidation messages may not get across the CCI between clusters. I don't have the TRMs at hand but make sure all the relevant bits in the CPUs and CCI are enabled.

Indeed check them several times, and need more information, check it again.

Quoted 4 lines

BTW, which kernel version are you running? Is the firmware your own or built around ARM Trusted Firmware?

I use 4.1 kernel version, and the firmware is our own.

Ding

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel at lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

2016-01-26 21:18 GMT+08:00 Ding Tianhong <[email protected]>:

Quoted 57 lines · Catalin Marinas

Quoted 56 lines · Ding Tianhong

Quoted 7 lines · Catalin Marinas

Quoted 6 lines · Ding Tianhong

Quoted 5 lines

I met this problem when running the hackbench test on A72 chip board:

sh[4779]: unhandled level 2 translation fault (11) at 0x7f96be0c80, esr 0x83000006 pgd = ffffffc01a1f0000
[7f96be0c80] *pgd=0000000084a20003, *pud=0000000084a20003, *pmd=0000000000000000

[...]

Quoted 39 lines

Quoted 7 lines

I can't tell for sure it's a TLB issue. The kernel page table dump shows *pmd being 0, so the fault is correctly called "level 2 translation fault". It also seems that there is no vma at this address, hence the kernel reports it as unhandled. It looks like data corruption which could be caused by cache or TLB incoherence. Just make sure the interconnect linking the two clusters is configured correctly by _firmware_ before Linux starts.

Thanks for the apply, I have try to apply this patch to test:

--- arch/arm64/kernel/process.c | 9 +++++++++

1 file changed, 9 insertions(+)

arch/arm64/kernel/process.c +9 -0

index 6391485..d7d8439 100644
@@ -283,6 +283,13 @@ static void tls_thread_switch(struct task_struct *next)
 : : "r" (tpidr), "r" (tpidrro));
 }
+static void tlb_flush_thread(struct task_struct *prev)
+{
+/* Flush the prev task's TLB entries */
+if (prev->mm)
+flush_tlb_mm(prev->mm);
+}
+
 /*
  * Thread switching.
  */

@@ -296,6 +303,8 @@ struct task_struct *__switch_to(struct task_struct *prev,
 hw_breakpoint_thread_switch(next);
 contextidr_thread_switch(next);
+tlb_flush_thread(prev);
+
 /*
 * Complete any pending TLB or cache maintenance on this CPU in case
 * the thread migrates to a different CPU.
 

Quoted 3 lines

The hackbench would work fine after this patch, so I guess that the old thread tlb may not be invalidate as soon as possible, but I don't know why, everything is fine on A57, Does I miss something?

It looks like the TLB invalidation messages may not get across the CCI between clusters. I don't have the TRMs at hand but make sure all the relevant bits in the CPUs and CCI are enabled.

Quoted 1 line

Indeed check them several times, and need more information, check it again.

How this issue is resolved finally? I search the mail-list and find this old email thread. Any response will be appreciated.

Jason Liu

Quoted 11 lines

Quoted 4 lines

BTW, which kernel version are you running? Is the firmware your own or built around ARM Trusted Firmware?

I use 4.1 kernel version, and the firmware is our own.

Ding

Quoted 4 lines

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel at lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel at lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel