Skip to contentSkip to conversation

powerpc/fsl-booke: don't load early TLB at once

PATCH In review David Lamparter 5 participants 10 messages Sep 20, 2018 – Oct 1, 2018 PowerPC

lore
Share & tools
lore
#0 lore raw

PATCH 1/1 powerpc/fsl-booke: don't load early TLB at once

This is a *partial* revert of "powerpc/85xx: Load all early TLB entries at once" (d9e1831a420267a7ced708bb259d65b0a3c0344d.)

My dusty old P4080DS just completely fails to boot (no output at all) without this revert. I have no clue what's going on here, I just bisected it down and since it looks like an optimization to me I just reverted it - and voilá, the P4080 boots again.

---
I'm not subscribed to linuxppc-dev, please keep me in Cc:. Also I contribute to the kernel only very rarely and have no clue how to find out when the appropriate time in the merge window is to submit things to linuxppc-dev... my apologies.

---
 arch/powerpc/mm/fsl_booke_mmu.c  |  3 +-
 arch/powerpc/mm/mmu_decl.h       |  1 -
 arch/powerpc/mm/tlb_nohash_low.S | 63 --------------------------------
 3 files changed, 2 insertions(+), 65 deletions(-)
arch/powerpc/mm/fsl_booke_mmu.c +2 -1
index 080d49b26c3a..f301a4c32c66 100644
@@ -143,6 +143,8 @@ static void settlbcam(int index, unsigned long virt, phys_addr_t phys,
 	tlbcam_addrs[index].start = virt;
 	tlbcam_addrs[index].limit = virt + size - 1;
 	tlbcam_addrs[index].phys = phys;
+
+	loadcam_entry(index);
 }
 
 unsigned long calc_cam_sz(unsigned long ram, unsigned long virt,
@@ -195,7 +197,6 @@ static unsigned long map_mem_in_cams_addr(phys_addr_t phys, unsigned long virt,
 	if (dryrun)
 		return amount_mapped;
 
-	loadcam_multi(0, i, max_cam_idx);
 	tlbcam_index = i;
 
 #ifdef CONFIG_PPC64
arch/powerpc/mm/mmu_decl.h +0 -1
index e5d779eed181..9a14de6a3ca9 100644
@@ -140,7 +140,6 @@ extern int switch_to_as1(void);
 extern void restore_to_as0(int esel, int offset, void *dt_ptr, int bootcpu);
 #endif
 extern void loadcam_entry(unsigned int index);
-extern void loadcam_multi(int first_idx, int num, int tmp_idx);
 
 struct tlbcam {
 	u32	MAS0;
arch/powerpc/mm/tlb_nohash_low.S +0 -63

73 lines not on this page · raw message

Signature · 1 line
2.18.0
#1 lore raw
Quoted 7 lines · David Lamparter

This is a *partial* revert of "powerpc/85xx: Load all early TLB entries at once" (d9e1831a420267a7ced708bb259d65b0a3c0344d.)

My dusty old P4080DS just completely fails to boot (no output at all) without this revert. I have no clue what's going on here, I just bisected it down and since it looks like an optimization to me I just reverted it - and voilá, the P4080 boots again.

It's not an optimization; it was required to get kdump working, at least for certain choices of crash kernel location. I just tried booting a 32- bit kernel and did not see this problem -- but I don't have access to a p4080ds anymore. I tried with qemu e500mc, and also running a 32-bit kernel on e6500 (needs a tiny change to get past SMP init, since 32-bit isn't really supported on e6500, but you do get output even without that).

Do you have a JTAG that can be used to find out where it's hanging? If not, I can try to get early debug working (needs an early IOMMU mapping).

York, can you try booting the latest kernel on p4080ds?

-Scott

#2 lore raw
Quoted 10 lines · Scott Wood
Quoted 7 lines · David Lamparter

This is a *partial* revert of "powerpc/85xx: Load all early TLB entries at once" (d9e1831a420267a7ced708bb259d65b0a3c0344d.)

My dusty old P4080DS just completely fails to boot (no output at all) without this revert. I have no clue what's going on here, I just bisected it down and since it looks like an optimization to me I just reverted it - and voilá, the P4080 boots again.

It's not an optimization; it was required to get kdump working, at least

Oh, my apologies for misunderstanding the code.

Quoted 5 lines

for certain choices of crash kernel location. I just tried booting a 32- bit kernel and did not see this problem -- but I don't have access to a p4080ds anymore. I tried with qemu e500mc, and also running a 32-bit kernel on e6500 (needs a tiny change to get past SMP init, since 32-bit isn't really supported on e6500, but you do get output even without that).

Hrm, maybe I should just make it use the old mechanism under an #ifdef CONFIG_PPC32? Better to boot and have kdump not work than not boot. (But obviously finding the actual problem would be best.)

Quoted 2 lines

Do you have a JTAG that can be used to find out where it's hanging? If not, I can try to get early debug working (needs an early IOMMU mapping).

I only have JTAG tools for ARM chips available; hardware wise I could probably solder up an adapter but software wise I have absoutely no clue how to fire up a session on anything PPC... I'm a novice openocd user, that's it.

Quoted 1 line

York, can you try booting the latest kernel on p4080ds?

d9e1831a42 has been around for quite some time, 4.4 already has it. I was a bit surprised noone has run into this, but then again P4080 is not exactly the most recent/interesting hardware.

Cheers,

-David

#3 lore raw
Quoted 3 lines · David Lamparter
Quoted 2 lines · Scott Wood

Do you have a JTAG that can be used to find out where it's hanging? If not, I can try to get early debug working (needs an early IOMMU mapping).

s/IOMMU mapping/MMU mapping for MMIO/ :-P

Quoted 10 lines

I only have JTAG tools for ARM chips available; hardware wise I could probably solder up an adapter but software wise I have absoutely no clue how to fire up a session on anything PPC... I'm a novice openocd user, that's it.

Quoted 1 line

York, can you try booting the latest kernel on p4080ds?

d9e1831a42 has been around for quite some time, 4.4 already has it. I was a bit surprised noone has run into this, but then again P4080 is not exactly the most recent/interesting hardware.

I don't suppose you're running a relocatable kernel at a non-zero address, and/or are running in an environment that sets HID0[EN_L2MMU_MHD] (neither standard U-boot nor Linux sets this bit, though they probably should)? On 32-bit, we're already running in an AS1 trampoline when loadcam_multi() is called, but loadcam_multi() sets up its own. This happens to not be catastrophic in standard scenarios, but it does add a duplicate TLB entry, and we return to AS0 sooner than expected. I think your patch, plus ifdefs to make the change 32-bit only, is the appropriate fix.

I also got an earlier udbg for e500 working (and happened to decide to test it with a relocatable kernel); I'll send that out once I've cleaned it up (or sooner with the extra TLB dumping included if the above doesn't explain why you're hitting this bug).

-Scott

#4 lore raw
Quoted 1 line · Scott Wood

I don't suppose you're running a relocatable kernel at a non-zero

# CONFIG_RELOCATABLE is not set

Quoted 2 lines

address, and/or are running in an environment that sets HID0[EN_L2MMU_MHD] (neither standard U-boot nor Linux sets this bit,

It's u-boot v2018.09-rc3; bit "33" / 30 isn't even defined as a constant, much less touched anywhere. HID0 in u-boot seems to be 0x80000080 aka EMCP | EN_MAS7_UPDATE.

Quoted 6 lines

though they probably should)? On 32-bit, we're already running in an AS1 trampoline when loadcam_multi() is called, but loadcam_multi() sets up its own. This happens to not be catastrophic in standard scenarios, but it does add a duplicate TLB entry, and we return to AS0 sooner than expected. I think your patch, plus ifdefs to make the change 32-bit only, is the appropriate fix.

I'll resend it with ifdefs inserted. Meanwhile, still trying the default config as mentioned in my previous mail.

Quoted 4 lines

I also got an earlier udbg for e500 working (and happened to decide to test it with a relocatable kernel); I'll send that out once I've cleaned it up (or sooner with the extra TLB dumping included if the above doesn't explain why you're hitting this bug).

Thanks & Cheers,

-David

#5 lore raw
In reply to Scott Wood

On 09/20/2018 05:31 PM, Scott Wood wrote:=0A=

Quoted 2 lines

On Fri, 2018-09-21 at 00:48 +0200, David Lamparter wrote:=0A=

Quoted 1 line

This is a *partial* revert of "powerpc/85xx: Load all early TLB entries=

=0A=

Quoted 8 lines
Quoted 6 lines

at once" (d9e1831a420267a7ced708bb259d65b0a3c0344d.)=0A=
=0A=
My dusty old P4080DS just completely fails to boot (no output at all)=0A= without this revert. I have no clue what's going on here, I just=0A= bisected it down and since it looks like an optimization to me I just=0A= reverted it - and voil=E1, the P4080 boots again.=0A=

=0A=
It's not an optimization; it was required to get kdump working, at least=

=0A=

Quoted 1 line

for certain choices of crash kernel location. I just tried booting a 32-=

=0A=

Quoted 1 line

bit kernel and did not see this problem -- but I don't have access to a=

=0A=

Quoted 2 lines

p4080ds anymore. I tried with qemu e500mc, and also running a 32-bit=0A= kernel on e6500 (needs a tiny change to get past SMP init, since 32-bit=

=0A=

Quoted 1 line

isn't really supported on e6500, but you do get output even without that)=

.=0A=

Quoted 2 lines

=0A=
Do you have a JTAG that can be used to find out where it's hanging? If=

=0A=

Quoted 1 line

not, I can try to get early debug working (needs an early IOMMU mapping).=

=0A=

Quoted 3 lines

=0A=
York, can you try booting the latest kernel on p4080ds?=0A= =0A=

=0A=
Scott,=0A=
=0A=
I haven't tried P4080DS for a long time. What defconfig do you use for=0A= this board? I tried latest master branch (commit a27fb6d983c7b5) with=0A= corenet_basic_defconfig, it didn't boot up. Kernel has an exception very=0A= early (pc ffffae80).=0A=
=0A=
However, before I claimed the board, someone booted Linux 4.14.71 on=0A= this board. I need to track down where the image came from.=0A= =0A=
York=0A=

#6 lore raw
Quoted 25 lines · York Sun
Quoted 16 lines · Scott Wood
Quoted 7 lines · David Lamparter

This is a *partial* revert of "powerpc/85xx: Load all early TLB entries at once" (d9e1831a420267a7ced708bb259d65b0a3c0344d.)

My dusty old P4080DS just completely fails to boot (no output at all) without this revert. I have no clue what's going on here, I just bisected it down and since it looks like an optimization to me I just reverted it - and voilá, the P4080 boots again.

It's not an optimization; it was required to get kdump working, at least for certain choices of crash kernel location. I just tried booting a 32- bit kernel and did not see this problem -- but I don't have access to a p4080ds anymore. I tried with qemu e500mc, and also running a 32-bit kernel on e6500 (needs a tiny change to get past SMP init, since 32-bit isn't really supported on e6500, but you do get output even without that).

Quoted 5 lines

Do you have a JTAG that can be used to find out where it's hanging? If not, I can try to get early debug working (needs an early IOMMU mapping).

York, can you try booting the latest kernel on p4080ds?

Scott,

Quoted 7 lines

I haven't tried P4080DS for a long time. What defconfig do you use for this board? I tried latest master branch (commit a27fb6d983c7b5) with corenet_basic_defconfig, it didn't boot up. Kernel has an exception very early (pc ffffae80).

However, before I claimed the board, someone booted Linux 4.14.71 on this board. I need to track down where the image came from.

Use corenet32_smp_defconfig

corenet_basic_defconfig is just a fragment used by the makefiles in assembling the final config.

-Scott

#7 lore raw

On 09/21/2018 10:47 AM, Scott Wood wrote:=0A=

Quoted 4 lines

On Fri, 2018-09-21 at 17:40 +0000, York Sun wrote:=0A=

Quoted 3 lines

On 09/20/2018 05:31 PM, Scott Wood wrote:=0A=

Quoted 2 lines

On Fri, 2018-09-21 at 00:48 +0200, David Lamparter wrote:=0A=

Quoted 1 line

This is a *partial* revert of "powerpc/85xx: Load all early TLB entrie=

s=0A=

Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines

at once" (d9e1831a420267a7ced708bb259d65b0a3c0344d.)=0A= =0A=
My dusty old P4080DS just completely fails to boot (no output at all)=

=0A=

Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines

without this revert. I have no clue what's going on here, I just=0A= bisected it down and since it looks like an optimization to me I just=

=0A=

Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 1 line

reverted it - and voil=E1, the P4080 boots again.=0A=

=0A=
It's not an optimization; it was required to get kdump working, at leas=

t=0A=

Quoted 1 line
Quoted 1 line
Quoted 1 line

for certain choices of crash kernel location. I just tried booting a 3=

2-=0A=

Quoted 1 line
Quoted 1 line
Quoted 1 line

bit kernel and did not see this problem -- but I don't have access to a=

=0A=

Quoted 1 line
Quoted 1 line
Quoted 1 line

p4080ds anymore. I tried with qemu e500mc, and also running a 32-bit=

=0A=

Quoted 1 line
Quoted 1 line
Quoted 1 line

kernel on e6500 (needs a tiny change to get past SMP init, since 32-bit=

=0A=

Quoted 1 line
Quoted 1 line
Quoted 1 line

isn't really supported on e6500, but you do get output even without tha=

t).=0A=

Quoted 2 lines
Quoted 2 lines
Quoted 2 lines

=0A=
Do you have a JTAG that can be used to find out where it's hanging? If=

=0A=

Quoted 1 line
Quoted 1 line
Quoted 1 line

not, I can try to get early debug working (needs an early IOMMU mapping=

).=0A=

Quoted 7 lines
Quoted 7 lines
Quoted 3 lines

=0A=
York, can you try booting the latest kernel on p4080ds?=0A= =0A=

=0A=
Scott,=0A=
=0A=
I haven't tried P4080DS for a long time. What defconfig do you use for=

=0A=

Quoted 2 lines
Quoted 2 lines

this board? I tried latest master branch (commit a27fb6d983c7b5) with=0A= corenet_basic_defconfig, it didn't boot up. Kernel has an exception very=

=0A=

Quoted 8 lines
Quoted 4 lines

early (pc ffffae80).=0A=
=0A=
However, before I claimed the board, someone booted Linux 4.14.71 on=0A= this board. I need to track down where the image came from.=0A=

=0A=
Use corenet32_smp_defconfig=0A=
=0A=
corenet_basic_defconfig is just a fragment used by the makefiles in assem=

bling=0A=

Quoted 2 lines

the final config.=0A=
=0A=

=0A=
Thanks for the instruction. Linux comes up OK with corenet32_smp_defconfig.= =0A=
=0A=

root@p4080ds:~# uname -a=0A=
Linux p4080ds 4.19.0-rc4-00206-ga27fb6d #1 SMP Fri Sep 21 10:56:36 PDT=0A=
2018 ppc GNU/Linux=0A=
=0A=
York=0A=
=0A=
#8 lore raw

(Sorry about the delay on my end, deadlines ...)

Quoted 11 lines · York Sun
Quoted 10 lines · Scott Wood
Quoted 9 lines · York Sun
Quoted 8 lines · Scott Wood
Quoted 4 lines · David Lamparter

My dusty old P4080DS just completely fails to boot (no output at all) without this revert. I have no clue what's going on here, I just bisected it down and since it looks like an optimization to me I just reverted it - and voilá, the P4080 boots again.

It's not an optimization; it was required to get kdump working, at least for certain choices of crash kernel location.

[...]

Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

York, can you try booting the latest kernel on p4080ds?

[...]

Quoted 6 lines

Thanks for the instruction. Linux comes up OK with corenet32_smp_defconfig.

root@p4080ds:~# uname -a
Linux p4080ds 4.19.0-rc4-00206-ga27fb6d #1 SMP Fri Sep 21 10:56:36 PDT
2018 ppc GNU/Linux

Well, shoot. I guess the next likely thing is that I have something weird enabled in my .config, so I'll try booting corenet32_smp_defconfig on my P4080DS. (I vaguely think I did that at some point but I can't swear to it so I'll just retry.) If that doesn't boot, could either of you provide me with your built uImage so I can exclude toolchain stupidity?

=> I'll send another mail when I got to try corenet32_smp_defconfig.

Aaaand, during the past week I noticed this is a rev1.0 P4080 chip, which apparently only exists on some early P4080DS boards - and Freescale even did a replacement program to get rev2.0 boards out (I guess this one was sitting in some broom closet.) I don't have access to erratas for this, I only know the entire QMan/FMan stuff is royally f*cked - no idea whether the PPC core has issues too.

FWIW, my board is running perfectly stable with that patch I posted and I'll just carry it locally if it's an issue for this one specific board I have here. I just don't have sufficient information to tell if that is indeed the case.

Thanks a lot for your input and help,

-David

#9 lore raw
In reply to David Lamparter
Quoted 8 lines · David Lamparter

This is a *partial* revert of "powerpc/85xx: Load all early TLB entries at once" (d9e1831a420267a7ced708bb259d65b0a3c0344d.)

My dusty old P4080DS just completely fails to boot (no output at all) without this revert. I have no clue what's going on here, I just bisected it down and since it looks like an optimization to me I just reverted it - and voil=C3=A1, the P4080 boots again.

Quoted 7 lines

---
I'm not subscribed to linuxppc-dev, please keep me in Cc:. Also I contribute to the kernel only very rarely and have no clue how to find out when the appropriate time in the merge window is to submit things to linuxppc-dev... my apologies.

Don't worry about the timing of sending patches.

Your patch will be tracked in patchwork, so if it's not a good time we'll just ignore it until it is a good time :)

cheers