Skip to contentSkip to conversation

06685975c2: BUG:kernel_failed_in_early-boot_stage,last_printk:Booting_the_kernel(entry_offset:#)

tip:x86/sev kernel test robot 2 participants 6 messages Jul 9, 2024 – Jul 11, 2024 x86

lore
Share & tools
lore

kernel test robot

author build bot
#0 lore raw

Hello,

kernel test robot noticed "BUG:kernel_failed_in_early-boot_stage,last_printk:Booting_the_kernel(entry_offset:#)" on:

commit: 06685975c2090e180851a0ff175c140188b6b54a ("x86/sev: Move SEV compilation units")
https://git.kernel.org/cgit/linux/kernel/git/tip/tip.git x86/sev

[test failed on linux-next/master 0b58e108042b0ed28a71cd7edf5175999955b233]

in testcase: boot

compiler: gcc-13
test machine: qemu-system-x86_64 -enable-kvm -cpu SandyBridge -smp 2 -m 16G

(please refer to attached dmesg/kmsg for entire log/backtrace)

we don't have idea how this commit could cause early boot issue, so we rebuild kernel for both this commit and its parent with the config in [1], then rerun the tests, the issue is still persistent while keeping clean on parent.

3991b04d4870fd33 06685975c2090e180851a0ff175
---------------- ---------------------------
fail:runs %reproduction fail:runs
| | |
:200 25% 50:50 dmesg.BUG:kernel_failed_in_early-boot_stage,last_printk:Booting_the_kernel(entry_offset:#)

since the config in [1] is a random config, we also try another x86_64 config which we use for various functional tests, still can reproduce the issue.

we also try two i386 configs, but not reproduce the issue. FYI

If you fix the issue in a separate patch/commit (i.e. not just a new version of the same patch/commit), kindly add following tags
| Reported-by: kernel test robot <[email protected]>
| Closes: https://lore.kernel.org/oe-lkp/[email protected]

Decompressing Linux... No EFI environment detected. Parsing ELF... done.
Booting the kernel (entry_offset: 0x0000000000000ffa). convert early boot stage from reboot-without-warning to failed

BUG: kernel failed in early-boot stage, last printk: Booting the kernel (entry_offset: 0x0000000000000ffa).

Linux version 6.10.0-rc2-00015-g06685975c209 #1
Command line: ip=::::vm-meta-70::dhcp root=/dev/ram0 RESULT_ROOT=/result/boot/1/vm-snb/debian-12-x86_64-20240206.cgz/x86_64-randconfig-r016-20230701/gcc-13/06685975c2090e180851a0ff175c140188b6b54a/21 BOOT_IMAGE=/pkg/linux/x86_64-randconfig-r016-20230701/gcc-13/06685975c2090e180851a0ff175c140188b6b54a/vmlinuz-6.10.0-rc2-00015-g06685975c209 branch=tip/x86/sev job=/lkp/jobs/scheduled/vm-meta-70/boot-1-debian-12-x86_64-20240206.cgz-x86_64-randconfig-r016-20230701-06685975c209-20240708-45480-br2l55-20.yaml user=lkp ARCH=x86_64 kconfig=x86_64-randconfig-r016-20230701 commit=06685975c2090e180851a0ff175c140188b6b54a nmi_watchdog=0 intremap=posted_msi vmalloc=256M initramfs_async=0 page_owner=on carrier_timeout=60 max_uptime=600 LKP_SERVER=internal-lkp-server selinux=0 debug apic=debug sysrq_always_enabled rcupdate.rcu_cpu_stall_timeout=100 net.ifnames=0 printk.devkmsg=on panic=-1 softlockup_panic=1 nmi_watchdog=panic oops=panic load_ramdisk=2 prompt_ramdisk=0 drbd.minor_count=8 systemd.log_level=err ignore_loglevel console=tty0 earlyprintk=ttyS0,115200 console=ttyS0,115200 vga=normal rw rcuperf.shutdown=0 rcuscale.shutdown=0 refscale.shutdown=0 watchdog_thresh=240 audit=0 kunit.enable=0 ia32_emulation=on riscv_isa_fallback=1

Kboot worker: lkp-worker32
Elapsed time: 60

[1]
The kernel config and materials to reproduce are available at:
https://download.01.org/0day-ci/archive/20240709/[email protected]

Signature · 2 lines
0-DAY CI Kernel Test Service
https://github.com/intel/lkp-tests/wiki
maintainer
#1 lore raw
Quoted 4 lines · kernel test robot

kernel test robot noticed "BUG:kernel_failed_in_early-boot_stage,last_printk:Booting_the_kernel(entry_offset:#)" on:

commit: 06685975c2090e180851a0ff175c140188b6b54a ("x86/sev: Move SEV compilation units")
https://git.kernel.org/cgit/linux/kernel/git/tip/tip.git x86/sev

Does it go away when you try two patches after it:

a7e7185ccd2a ("x86/sev: Move the instrumentation exclusion bits too")

?

I might have to merge the two if so.

Thx.

Signature · 4 lines
Regards/Gruss,
    Boris.

https://people.kernel.org/tglx/notes-about-netiquette
#2 lore raw

hi, Boris,

Quoted 11 lines · Borislav Petkov
Quoted 4 lines · kernel test robot

kernel test robot noticed "BUG:kernel_failed_in_early-boot_stage,last_printk:Booting_the_kernel(entry_offset:#)" on:

commit: 06685975c2090e180851a0ff175c140188b6b54a ("x86/sev: Move SEV compilation units")
https://git.kernel.org/cgit/linux/kernel/git/tip/tip.git x86/sev

Does it go away when you try two patches after it:

a7e7185ccd2a ("x86/sev: Move the instrumentation exclusion bits too")

?

no. when this bisect done, we tested the branch tip 8434cf006ceae, the issue is still persistent.

* 8434cf006ceae (tip/x86/sev) Documentation/ABI/configfs-tsm: Fix an unexpected indentation silly
* a7e7185ccd2ac x86/sev: Move the instrumentation exclusion bits too
* 6a7d32f3a205f x86/sev: Do RMP memory coverage check after max_pfn has been set
* 06685975c2090 x86/sev: Move SEV compilation units
* 3991b04d4870f virt: sev-guest: Mark driver struct with __refdata to prevent section mismatch

Quoted 10 lines

I might have to merge the two if so.

Thx.

Signature · 4 lines
Regards/Gruss,
    Boris.

https://people.kernel.org/tglx/notes-about-netiquette
maintainer
#3 lore raw
Quoted 2 lines · Oliver Sang

no. when this bisect done, we tested the branch tip 8434cf006ceae, the issue is still persistent.

Ok, I think that should fix it:

From: "Borislav Petkov (AMD)" <[email protected]>
Date: Wed, 10 Jul 2024 10:16:18 +0200
Subject: [PATCH] x86/coco/sev: Disable KCSAN too

The SEV guest code was moved to x86/coco/ along with the instrumentation exclusion bits. However, there is a blanket KCSAN disable in arch/x86/kernel/Makefile due to boot hangs.

Copy that disable to the coco's sev Makefile too.

Fixes Fixes: 06685975c209 ("x86/sev: Move SEV compilation units")
---
 arch/x86/coco/sev/Makefile | 4 ++++
 1 file changed, 4 insertions(+)
arch/x86/coco/sev/Makefile +4 -0
index 5f72e92b37ff..4e375e7305ac 100644
@@ -9,3 +9,7 @@ endif
 KASAN_SANITIZE_core.o	:= n
 KMSAN_SANITIZE_core.o	:= n
 KCOV_INSTRUMENT_core.o	:= n
+
+# With some compiler versions the generated code results in boot hangs, caused
+# by several compilation units. To be safe, disable all instrumentation.
+KCSAN_SANITIZE		:= n
Signature · 7 lines
2.43.0

-- 
Regards/Gruss,
    Boris.

https://people.kernel.org/tglx/notes-about-netiquette
author Tested-by
#4 lore raw

hi, Boris,

Quoted 5 lines · Borislav Petkov
Quoted 2 lines · Oliver Sang

no. when this bisect done, we tested the branch tip 8434cf006ceae, the issue is still persistent.

Ok, I think that should fix it:

yes, your patch fixed the issue.

Quoted 39 lines

From: "Borislav Petkov (AMD)" <[email protected]>
Date: Wed, 10 Jul 2024 10:16:18 +0200
Subject: [PATCH] x86/coco/sev: Disable KCSAN too

The SEV guest code was moved to x86/coco/ along with the instrumentation exclusion bits. However, there is a blanket KCSAN disable in arch/x86/kernel/Makefile due to boot hangs.

Copy that disable to the coco's sev Makefile too.

Fixes Fixes: 06685975c209 ("x86/sev: Move SEV compilation units")
---
 arch/x86/coco/sev/Makefile | 4 ++++
 1 file changed, 4 insertions(+)
arch/x86/coco/sev/Makefile +4 -0
index 5f72e92b37ff..4e375e7305ac 100644
@@ -9,3 +9,7 @@ endif
 KASAN_SANITIZE_core.o	:= n
 KMSAN_SANITIZE_core.o	:= n
 KCOV_INSTRUMENT_core.o	:= n
+
+# With some compiler versions the generated code results in boot hangs, caused
+# by several compilation units. To be safe, disable all instrumentation.
+KCSAN_SANITIZE		:= n
Signature · 7 lines
2.43.0

-- 
Regards/Gruss,
    Boris.

https://people.kernel.org/tglx/notes-about-netiquette
maintainer
#5 lore raw
Quoted 3 lines · Oliver Sang

yes, your patch fixed the issue.

Thanks, I went and merged the other build issue reported by 0day into the original patch so that there's no build failures during bisection.

We should be good to go now, knock on wood.

Thx.

Signature · 4 lines
Regards/Gruss,
    Boris.

https://people.kernel.org/tglx/notes-about-netiquette