ARM: move __fixup_smp out of init section
PATCH 1/1 ARM: move __fixup_smp out of init section
From: Rob Herring <[email protected]>
With large kernel builds such as allyesconfig exceeding maximum relative branch offsets, the init section will be too far away to branch to directly. This causes veneers to be added by the compiler, but veneers don't work before the MMU is enabled. Fix this by moving __fixup_smp to the .head.text section as it is not very big.
--- arch/arm/kernel/head.S | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
arch/arm/kernel/head.S +1 -1
@@ -475,7 +475,7 @@ ENDPROC(__turn_mmu_on) #ifdef CONFIG_SMP_ON_UP - __INIT + __HEAD __fixup_smp: and r3, r9, #0x000f0000 @ architecture version teq r3, #0x000f0000 @ CPU ID supported?
Signature · 1 line
1.8.3.2
PATCH 1/1 ARM: move __fixup_smp out of init section
Quoted 9 lines · On Tuesday 25 February 2014 12:16:33 Rob Herring wrote:
From: Rob Herring <[email protected]>
With large kernel builds such as allyesconfig exceeding maximum relative branch offsets, the init section will be too far away to branch to directly. This causes veneers to be added by the compiler, but veneers don't work before the MMU is enabled. Fix this by moving
__fixup_smpto the .head.text section as it is not very big.Signed-off-by Rob Herring <[email protected]>
This looks good to me, but I have some related questions:
* I needed to use -mlong-calls for large kernels. Did you manage without?
* Do you (or anyone) know how to force the use of a veneer? I see a problem with some driver calling __do_div_asm from IIRC an init section, and that creates a link error when the kernel gets too big
* If FUNCTION_TRACER is enabled, we get calls from every function using 'bl __gnu_mcount_nc'. Do you think it's possible to fix those?
* Same thing but simpler for svc_preempt calling preempt_schedule_irq and lookup_processor_type calling __lookup_processor_type. In those cases I guess we should be able to trivially rewrite the assembly to jump through an extra register.
The complete patch I have is at http://pastebin.com/5yVaUC4u, but it's likely that it's completely broken ;-)
Arnd
PATCH 1/1 ARM: move __fixup_smp out of init section
Quoted 9 lines · Arnd Bergmann
Quoted 8 lines · On Tuesday 25 February 2014 12:16:33 Rob Herring wrote:
From: Rob Herring <[email protected]>
With large kernel builds such as allyesconfig exceeding maximum relative branch offsets, the init section will be too far away to branch to directly. This causes veneers to be added by the compiler, but veneers don't work before the MMU is enabled. Fix this by moving
__fixup_smpto the .head.text section as it is not very big.
Quoted 5 lines
Quoted 1 line
Signed-off-by Rob Herring <[email protected]>This looks good to me, but I have some related questions:
* I needed to use -mlong-calls for large kernels. Did you manage without?
Yes, but I started with allyesconfig and then disabled things until it would link. This was mainly PCI, various debug, and filesystems. This particular failure you don't see until you try to boot.
I also saw intermittent inconsistent kallsym errors which KALLSYMS_EXTRA_PASS=1 did not fix.
Quoted 3 lines
* Do you (or anyone) know how to force the use of a veneer? I see a problem with some driver calling
__do_div_asmfrom IIRC an init section, and that creates a link error when the kernel gets too big
That is surprising given that I saw veneers be inserted for some asm to asm calls. Maybe it is something I turned off or something to do with the types of sections.
Quoted 2 lines
* If
FUNCTION_TRACERis enabled, we get calls from every function using 'bl__gnu_mcount_nc'. Do you think it's possible to fix those?
Put it in the middle of the image? ;)
Quoted 4 lines
* Same thing but simpler for
svc_preemptcallingpreempt_schedule_irqandlookup_processor_typecalling__lookup_processor_type. In those cases I guess we should be able to trivially rewrite the assembly to jump through an extra register.
I did see that lookup_processor_type had a veneer in my build.
Rob
PATCH 1/1 ARM: move __fixup_smp out of init section
Quoted 14 lines · On Tuesday 25 February 2014 15:05:01 Rob Herring wrote:
Quoted 13 lines · Arnd Bergmann
Quoted 9 lines · On Tuesday 25 February 2014 12:16:33 Rob Herring wrote:
From: Rob Herring <[email protected]>
With large kernel builds such as allyesconfig exceeding maximum relative branch offsets, the init section will be too far away to branch to directly. This causes veneers to be added by the compiler, but veneers don't work before the MMU is enabled. Fix this by moving
__fixup_smpto the .head.text section as it is not very big.Signed-off-by Rob Herring <[email protected]>This looks good to me, but I have some related questions:
Quoted 8 lines
Quoted 1 line
* I needed to use -mlong-calls for large kernels. Did you manage without?
Yes, but I started with allyesconfig and then disabled things until it would link. This was mainly PCI, various debug, and filesystems. This particular failure you don't see until you try to boot.
I also saw intermittent inconsistent kallsym errors which KALLSYMS_EXTRA_PASS=1 did not fix.
Yes, I saw those too. With XIP_KERNEL enabled, I've seen a randconfig case that needed 12 extra passes before converging. Without that option, it's much better but there are still cases that need more than 3 passes and I haven't gotten to the bottom of that. It seems to be the same configuration that refuses to link at all with ld.gold.
Quoted 11 lines
Quoted 3 lines
* Do you (or anyone) know how to force the use of a veneer? I see a problem with some driver calling
__do_div_asmfrom IIRC an init section, and that creates a link error when the kernel gets too bigThat is surprising given that I saw veneers be inserted for some asm to asm calls. Maybe it is something I turned off or something to do with the types of sections.
Quoted 2 lines
* If
FUNCTION_TRACERis enabled, we get calls from every function using 'bl__gnu_mcount_nc'. Do you think it's possible to fix those?
Quoted 8 lines
Put it in the middle of the image? ;)
Quoted 4 lines
* Same thing but simpler for
svc_preemptcallingpreempt_schedule_irqandlookup_processor_typecalling__lookup_processor_type. In those cases I guess we should be able to trivially rewrite the assembly to jump through an extra register.I did see that
lookup_processor_typehad a veneer in my build.
Strange. Could it be that -mlong-calls turns off veneers?
Which gcc and binutils version do you use?
Arnd
PATCH 1/1 ARM: move __fixup_smp out of init section
Quoted 48 lines · Arnd Bergmann
Quoted 14 lines · On Tuesday 25 February 2014 15:05:01 Rob Herring wrote:
Quoted 13 lines · Arnd Bergmann
Quoted 9 lines · On Tuesday 25 February 2014 12:16:33 Rob Herring wrote:
From: Rob Herring <[email protected]>
With large kernel builds such as allyesconfig exceeding maximum relative branch offsets, the init section will be too far away to branch to directly. This causes veneers to be added by the compiler, but veneers don't work before the MMU is enabled. Fix this by moving
__fixup_smpto the .head.text section as it is not very big.Signed-off-by Rob Herring <[email protected]>This looks good to me, but I have some related questions:
Quoted 8 lines
Quoted 1 line
* I needed to use -mlong-calls for large kernels. Did you manage without?
Yes, but I started with allyesconfig and then disabled things until it would link. This was mainly PCI, various debug, and filesystems. This particular failure you don't see until you try to boot.
I also saw intermittent inconsistent kallsym errors which KALLSYMS_EXTRA_PASS=1 did not fix.
Yes, I saw those too. With
XIP_KERNELenabled, I've seen a randconfig case that needed 12 extra passes before converging. Without that option, it's much better but there are still cases that need more than 3 passes and I haven't gotten to the bottom of that. It seems to be the same configuration that refuses to link at all with ld.gold.Quoted 18 lines
Quoted 3 lines
* Do you (or anyone) know how to force the use of a veneer? I see a problem with some driver calling
__do_div_asmfrom IIRC an init section, and that creates a link error when the kernel gets too bigThat is surprising given that I saw veneers be inserted for some asm to asm calls. Maybe it is something I turned off or something to do with the types of sections.
Quoted 2 lines
* If
FUNCTION_TRACERis enabled, we get calls from every function using 'bl__gnu_mcount_nc'. Do you think it's possible to fix those?Put it in the middle of the image? ;)
Quoted 4 lines
* Same thing but simpler for
svc_preemptcallingpreempt_schedule_irqandlookup_processor_typecalling__lookup_processor_type. In those cases I guess we should be able to trivially rewrite the assembly to jump through an extra register.
Quoted 5 lines
Quoted 1 line
I did see that
lookup_processor_typehad a veneer in my build.Strange. Could it be that -mlong-calls turns off veneers?
Which gcc and binutils version do you use?
It's the arm cross compiler in ubuntu 13.10:
$ arm-linux-gnueabihf-ld -v GNU ld (GNU Binutils for Ubuntu) 2.23.52.20130913 $ arm-linux-gnueabihf-gcc -v Using built-in specs. COLLECT_GCC=arm-linux-gnueabihf-gcc COLLECT_LTO_WRAPPER=/usr/lib/gcc-cross/arm-linux-gnueabihf/4.8/lto-wrapper Target: arm-linux-gnueabihf Configured with: ../src/configure -v --with-pkgversion='Ubuntu/Linaro 4.8.1-10ubuntu7' --with-bugurl=file:///usr/share/doc/gcc-4.8/README.Bugs --enable-languages=c,c++,java,go,d,fortran,objc,obj-c++ --prefix=/usr --program-suffix=-4.8 --enable-shared --enable-linker-build-id --libexecdir=/usr/lib --without-included-gettext --enable-threads=posix --with-gxx-include-dir=/usr/arm-linux-gnueabihf/include/c++/4.8.1 --libdir=/usr/lib --enable-nls --with-sysroot=/ --enable-clocale=gnu --enable-libstdcxx-debug --enable-libstdcxx-time=yes --enable-gnu-unique-object --disable-libitm --disable-libquadmath --enable-plugin --with-system-zlib --disable-browser-plugin --enable-java-awt=gtk --enable-gtk-cairo --with-java-home=/usr/lib/jvm/java-1.5.0-gcj-4.8-armhf-cross/jre --enable-java-home --with-jvm-root-dir=/usr/lib/jvm/java-1.5.0-gcj-4.8-armhf-cross --with-jvm-jar-dir=/usr/lib/jvm-exports/java-1.5.0-gcj-4.8-armhf-cross --with-arch-directory=arm --with-ecj-jar=/usr/share/java/eclipse-ecj.jar --disable-libgcj --enable-objc-gc --enable-multiarch --enable-multilib --disable-sjlj-exceptions --with-arch=armv7-a --with-fpu=vfpv3-d16 --with-float=hard --with-mode=thumb --disable-werror --enable-checking=release --build=x86_64-linux-gnu --host=x86_64-linux-gnu --target=arm-linux-gnueabihf --program-prefix=arm-linux-gnueabihf- --includedir=/usr/arm-linux-gnueabihf/include Thread model: posix gcc version 4.8.1 (Ubuntu/Linaro 4.8.1-10ubuntu7)
Rob
PATCH 1/1 ARM: move __fixup_smp out of init section
Quoted 8 lines · On Tuesday 25 February 2014 15:46:15 Rob Herring wrote:
It's the arm cross compiler in ubuntu 13.10:
$ arm-linux-gnueabihf-ld -v GNU ld (GNU Binutils for Ubuntu) 2.23.52.20130913 $ arm-linux-gnueabihf-gcc -v gcc version 4.8.1 (Ubuntu/Linaro 4.8.1-10ubuntu7)
Ok, I'm using the exact same one here.
Arnd
PATCH 1/1 ARM: move __fixup_smp out of init section
In reply to Rob HerringQuoted 5 lines · Rob Herring
From: Rob Herring <[email protected]>
With large kernel builds such as allyesconfig exceeding maximum relative branch offsets, the init section will be too far away to branch to directly. This causes veneers to be added by the compiler, but veneers
Don't you mean the linker? The compiler isn't involved here. Can you show the disassembly of the veneer please?
I'm confused why the linker would add a veneer for this, but not for the stuff built by the compiler.
Signature · 2 lines
FTTC broadband for 0.8mile line: now at 9.7Mbps down 460kbps up... slowly improving, and getting towards what was expected from it.
PATCH 1/1 ARM: move __fixup_smp out of init section
Quoted 9 lines · Russell King - ARM Linux
Quoted 5 lines · Rob Herring
From: Rob Herring <[email protected]>
With large kernel builds such as allyesconfig exceeding maximum relative branch offsets, the init section will be too far away to branch to directly. This causes veneers to be added by the compiler, but veneers
Don't you mean the linker? The compiler isn't involved here. Can you show the disassembly of the veneer please?
Here is the call to fixup_smp:
c0008054: eb000058 bl c00081bc <__vet_atags> c0008058: eb00006a bl c0008208 <____fixup_smp_veneer> c000805c: eb000042 bl c000816c <__fixup_pv_table> c0008060: eb000007 bl c0008084 <__create_page_tables>
Here's the veneer:
c0008208 <____fixup_smp_veneer>:c0008208: e51ff004 ldr pc, [pc, #-4] ; c000820c
<____fixup_smp_veneer+0x4>c000820c: c2659260 .word 0xc2659260
Quoted 3 lines
I'm confused why the linker would add a veneer for this, but not for the stuff built by the compiler.
There are 1413 other symbols with veneer in my image. So it is not that the linker can't create them, but why it cannot for some as the image gets bigger. Does "text" in a section name have any functional impact on the linker? It seems veneers between .head.text and and init.text are created, but between *.text and .fixup, .proc.info, etc. they don't get created.
Rob