objtool warnings "sibling call from callable instruction with modified stack frame" with CONFIG_LTO_CLANG_THIN
Hi LoongArch folks,
The kernel test robot recently generated several reports when building with LLVM that appear to be related (the "warning: objtool:" reports):
https://lore.kernel.org/all/?q=loongarch-randconfig-002-20250730
This can be easily reproduced by enabling CONFIG_LTO_CLANG_THIN on top of defconfig:
$ make -skj"$(nproc)" ARCH=loongarch LLVM=1 clean defconfig
$ scripts/config -d LTO_NONE -e LTO_CLANG_THIN
$ make -skj"$(nproc)" ARCH=loongarch LLVM=1 olddefconfig all drivers/char/ipmi/ipmi_devintf.o: warning: objtool: ipmi_ioctl+0x7c: sibling call from callable instruction with modified stack frame net/802/garp.o: warning: objtool: garp_pdu_rcv+0x170: sibling call from callable instruction with modified stack frame drivers/rapidio/devices/rio_mport_cdev.o: warning: objtool: mport_cdev_ioctl+0xe4: sibling call from callable instruction with modified stack frame net/802/mrp.o: warning: objtool: mrp_attr_event+0x5c: sibling call from callable instruction with modified stack frame net/netfilter/nfnetlink_osf.o: warning: objtool: nf_osf_match_one+0x1ac: unreachable instruction arch/loongarch/kvm/kvm.o: warning: objtool: kvm_set_memslot+0x22c: sibling call from callable instruction with modified stack frame arch/loongarch/kvm/kvm.o: warning: objtool: kvm_dev_ioctl+0x68: sibling call from callable instruction with modified stack frame arch/loongarch/kvm/kvm.o: warning: objtool: kvm_emu_iocsr+0x74: sibling call from callable instruction with modified stack frame arch/loongarch/kvm/kvm.o: warning: objtool: kvm_emu_mmio_read+0x58: sibling call from callable instruction with modified stack frame arch/loongarch/kvm/kvm.o: warning: objtool: kvm_emu_mmio_write+0x114: sibling call from callable instruction with modified stack frame arch/loongarch/kvm/kvm.o: warning: objtool: kvm_handle_gspr.llvm.8483686505109456213+0x230: sibling call from callable instruction with modified stack frame arch/loongarch/kvm/kvm.o: warning: objtool: kvm_arch_vcpu_ioctl+0x3ac: sibling call from callable instruction with modified stack frame arch/loongarch/kvm/kvm.o: warning: objtool: kvm_arch_vm_ioctl+0xec: sibling call from callable instruction with modified stack frame arch/loongarch/kvm/kvm.o: warning: objtool: kvm_ipi_read+0xe8: sibling call from callable instruction with modified stack frame arch/loongarch/kvm/kvm.o: warning: objtool: kvm_ipi_write+0x12c: sibling call from callable instruction with modified stack frame arch/loongarch/kvm/kvm.o: warning: objtool: kvm_ipi_regs_access+0x84: sibling call from callable instruction with modified stack frame arch/loongarch/kvm/kvm.o: warning: objtool: kvm_eiointc_regs_access+0x50: sibling call from callable instruction with modified stack frame arch/loongarch/kvm/kvm.o: warning: objtool: kvm_pch_pic_regs_access+0x54: sibling call from callable instruction with modified stack frame arch/loongarch/kvm/kvm.o: warning: objtool: kvm_pch_pic_read+0x64: sibling call from callable instruction with modified stack frame arch/loongarch/kvm/kvm.o: warning: objtool: kvm_pch_pic_write+0x7c: sibling call from callable instruction with modified stack frame drivers/char/ipmi/ipmi_si.o: warning: objtool: smi_event_handler+0x120: sibling call from callable instruction with modified stack frame drivers/char/ipmi/ipmi_si.o: warning: objtool: kcs_event.llvm.13474012466233770392+0x140: sibling call from callable instruction with modified stack frame ...
x86_64 does not see these objtool warnings when LTO is enabled, is this a backend issue or does objtool need to account for something that LLVM does yet GCC does not?
Cheers,
Nathan
Quoted 4 lines · Nathan Chancellor
Hi LoongArch folks,
The kernel test robot recently generated several reports when building with LLVM that appear to be related (the "warning: objtool:" reports):
...
Quoted 3 lines
x86_64does not see these objtool warnings when LTO is enabled, is this a backend issue or does objtool need to account for something that LLVM does yet GCC does not?
I can reproduce this problem.
After a quick look, objdump and readelf shows that there is no generated relocation section '.rela.discard.tablejump_annotate' for the table jump instruction when compiling with optimization config LTO_CLANG_THIN, thus objtool can not know that what is the actual destination address for the dynamic jump instruction jirl.
It needs to do something on the LLVM side to make sure that there is the relocation section '.rela.discard.tablejump_annotate' if LTO is enabled, but it seems not so easy.
I disscussed with Wang Rui offline and he suggested that we should ensure it is aware of linker with LTO, '--loongarch-annotate-tablejump' needs to be passed via '-mllvm' to ld.lld.
For the above reasons, in order to maintain downward compatibility for the current older LLVM compiler, it is safe to do the following changes in the kernel Makefile for now:
-----8<-----
arch/loongarch/Makefile +6 -0
@@ -103,6 +103,12 @@ KBUILD_CFLAGS += $(call cc-option,-mthin-add-sub) $(call cc-option,-Wa$(comma) ifdef CONFIG_OBJTOOL ifdef CONFIG_CC_HAS_ANNOTATE_TABLEJUMP KBUILD_CFLAGS += -mannotate-tablejump +# The annotate-tablejump option can not be passed to LLVM backend when LTO is enabled. +# Ensure it is aware of linker with LTO, '--loongarch-annotate-tablejump' needs to be +# passed via '-mllvm' to ld.lld. +ifdef CONFIG_LTO_CLANG +KBUILD_LDFLAGS += -mllvm --loongarch-annotate-tablejump +endif else KBUILD_CFLAGS += -fno-jump-tables # keep
compatibility with older compilers
endif
With the above changes, there are still some other objtool warnings and build errors if the config LTO_CLANG_THIN is set, maybe it needs to adjust the objtool code too, I will take some time to analysis.
Thanks,
Tiezhu