efi boot failures due to PTI with 32 bit builds and Intel CPUs
Hi all,
I see boot failures on mainline when trying to boot x86 images with an efi bios on Intel CPUs in qemu. Behavior is quite unusual: qemu dies silently after the kernel displays "Run /sbin/init as init process". With debugging enabled, qemu reports a CR3 update followed by a triple fault. Here is the end of the log file:
----------------
IN:
0xc75f1d1a: 66 90 nop 0xc75f1d1c: 0f 20 d8 movl %cr3, %eax 0xc75f1d1f: 0d 00 10 00 00 orl $0x1000, %eax 0xc75f1d24: 0f 22 d8 movl %eax, %cr3
CR3 update: CR3=0e39b000
----------------
IN:
0xc75f1d27: 5b popl %ebx 0xc75f1d28: 59 popl %ecx 0xc75f1d29: 5a popl %edx 0xc75f1d2a: 5e popl %esi 0xc75f1d2b: 5f popl %edi 0xc75f1d2c: 5d popl %ebp 0xc75f1d2d: 58 popl %eax 0xc75f1d2e: 1f popl %ds
Triple fault
This happens with both qemu 2.12 and 3.0. More detailed logs (not really showing anything) are at http://kerneltests.org/builders; look for x86 boot reports for master and next towards the end of the page.
Here is an example qemu command line:
qemu-system-i386 -kernel arch/x86/boot/bzImage -M q35 -cpu core2duo \ -no-reboot -m 256 \ -bios OVMF-pure-efi-32.fd \ -usb -device usb-storage,drive=d0 \ -drive file=rootfs.ext2,if=none,id=d0,format=raw \ --append 'root=/dev/sda rw rootwait mem=256M console=ttyS0 console=tty noreboot' \ -nographic
The problem is only seen in mainline (v4.19-rc1). It is not seen in earlier kernels. It does not really matter what to boot from as long as the boot is with an efi bios and an Intel CPU (AMD CPUs boot fine). Bisect was a bit tricky (see multiple runs below), but ultimately points to commit 7757d607c6b31 ("x86/pti: Allow CONFIG_PAGE_TABLE_ISOLATION for x86_32") as the culprit. Reverting this commit indeed fixes the problem.
Please let me know if I can help tracking down the underlying issue.
Thanks,
Guenter
---
# bad: [3f16503b7d2274ac8cbab11163047ac0b4c66cfe] Merge branch 'fixes' of git://git.kernel.org/pub/scm/linux/kernel/git/evalenti/linux-soc-thermal
# good: [94710cac0ef4ee177a63b5227664b38c95bbf703] Linux 4.18
git bisect start 'HEAD' 'v4.18'
# bad: [54dbe75bbf1e189982516de179147208e90b5e45] Merge tag 'drm-next-2018-08-15' of git://anongit.freedesktop.org/drm/drm
git bisect bad 54dbe75bbf1e189982516de179147208e90b5e45
# bad: [0a957467c5fd46142bc9c52758ffc552d4c5e2f7] x86: i8259: Add missing include file
git bisect bad 0a957467c5fd46142bc9c52758ffc552d4c5e2f7
# bad: [958f338e96f874a0d29442396d6adf9c1e17aa2d] Merge branch 'l1tf-final' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip
git bisect bad 958f338e96f874a0d29442396d6adf9c1e17aa2d
# bad: [85a0b791bc17f7a49280b33e2905d109c062a47b] Merge branch 'for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/s390/linux
git bisect bad 85a0b791bc17f7a49280b33e2905d109c062a47b
# good: [8603596a327c978534f5c45db135e6c36b4b1425] Merge branch 'perf-core-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip
git bisect good 8603596a327c978534f5c45db135e6c36b4b1425
# bad: [eac341194426ba7ead3444923b9eba491ae4feeb] Merge branch 'x86/pti' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip
git bisect bad eac341194426ba7ead3444923b9eba491ae4feeb
# good: [30de24c7dd21348b142ee977b687afc70b392af6] Merge branch 'x86-cache-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip
git bisect good 30de24c7dd21348b142ee977b687afc70b392af6
# bad: [8c934e01a7ce685d98e970880f5941d79272c654] x86/pti: Check the return value of pti_user_pagetable_walk_pmd()
git bisect bad 8c934e01a7ce685d98e970880f5941d79272c654
# bad: [fcbbd977572cfe5a3dcc97d663bf7480431a07ca] x86/pgtable: Move pti_set_user_pgtbl() to pgtable.h
git bisect bad fcbbd977572cfe5a3dcc97d663bf7480431a07ca
# bad: [e5862d0515ad970ccec6208ecf5bb0cffe291ea3] x86/entry/32: Leave the kernel via trampoline stack
git bisect bad e5862d0515ad970ccec6208ecf5bb0cffe291ea3
# bad: [a6b744f3ce9d017dd86b28355de2d8e0d36496d4] x86/entry/32: Load task stack from x86_tss.sp1 in SYSENTER handler
git bisect bad a6b744f3ce9d017dd86b28355de2d8e0d36496d4
# bad: [d9f4426c73002957be5dd39936f44a09498f7560] x86/speculation: Remove SPECTRE_V2_IBRS in enum spectre_v2_mitigation git bisect bad d9f4426c73002957be5dd39936f44a09498f7560
# bad: [21279157efffe5e7258483809942d576cb802768] x86/pti: Make pti_set_kernel_image_nonglobal() static
git bisect bad 21279157efffe5e7258483809942d576cb802768
# first bad commit: [21279157efffe5e7258483809942d576cb802768] x86/pti: Make pti_set_kernel_image_nonglobal() static
This doesn't really mean anything: The incoming merge is already broken due to commit e181ae0c5db9, but that should be fixed in mainline.
---
# bad: [eac341194426ba7ead3444923b9eba491ae4feeb] Merge branch 'x86/pti' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip
# good: [d191c82d4d9bd0bb3b945fc458cc65053ef868a0] Merge branch 'x86-vdso-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip
git bisect start 'eac341194426' 'd191c82d4d9b'
# bad: [b976690f5db26fbc7c2be413bfa0fbd270547a94] x86/mm/pti: Introduce pti_finalize()
git bisect bad b976690f5db26fbc7c2be413bfa0fbd270547a94
# bad: [b65bef400689ceee7108c2d47fb97ae91f4d1440] x86/entry/32: Add PTI CR3 switches to NMI handler code
git bisect bad b65bef400689ceee7108c2d47fb97ae91f4d1440
# bad: [8e676ced31e9d1448d3ffc4159586a259cc67f30] x86/entry/32: Unshare NMI return path
git bisect bad 8e676ced31e9d1448d3ffc4159586a259cc67f30
# bad: [9e97b73fdb235345a826519862a52a7398c89eb8] x86/asm-offsets: Move TSS_sp0 and TSS_sp1 to asm-offsets.c
git bisect bad 9e97b73fdb235345a826519862a52a7398c89eb8
# bad: [d9f4426c73002957be5dd39936f44a09498f7560] x86/speculation: Remove SPECTRE_V2_IBRS in enum spectre_v2_mitigation git bisect bad d9f4426c73002957be5dd39936f44a09498f7560
# bad: [21279157efffe5e7258483809942d576cb802768] x86/pti: Make pti_set_kernel_image_nonglobal() static
git bisect bad 21279157efffe5e7258483809942d576cb802768
# first bad commit: [21279157efffe5e7258483809942d576cb802768] x86/pti: Make pti_set_kernel_image_nonglobal() static
---
# bad: [21279157efffe5e7258483809942d576cb802768] x86/pti: Make pti_set_kernel_image_nonglobal() static
# good: [1e4b044d22517cae7047c99038abb444423243ca] Linux 4.18-rc4
git bisect start '21279157efffe5e7258483809942d576cb802768' 'v4.18-rc4'
# good: [35a84f34cf41915a0b2d0a3688b20761580f8ce4] Merge tag 'trace-v4.18-rc3-3' of git://git.kernel.org/pub/scm/linux/kernel/git/rostedt/linux-trace
git bisect good 35a84f34cf41915a0b2d0a3688b20761580f8ce4
# good: [75adbd1386796c1234035996c6aec3ede4060eb2] Merge branch 'efi-urgent-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip
git bisect good 75adbd1386796c1234035996c6aec3ede4060eb2
# good: [2db39a2f491a48ec740e0214a7dd584eefc2137d] Merge branch 'i2c/for-current' of git://git.kernel.org/pub/scm/linux/kernel/git/wsa/linux
git bisect good 2db39a2f491a48ec740e0214a7dd584eefc2137d
# good: [fe10e398e860955bac4d28ec031b701d358465e4] reiserfs: fix buffer overflow with long warning messages
git bisect good fe10e398e860955bac4d28ec031b701d358465e4
# bad: [c31496dbacc2b6352750937afc20a8dbe22b27a4] Merge tag 'for-linus-4.18-rc5-tag' of git://git.kernel.org/pub/scm/linux/kernel/git/xen/tip
git bisect bad c31496dbacc2b6352750937afc20a8dbe22b27a4
# bad: [2da8c426d90355eef1d42d974d2dccf0f5f7f21d] Merge tag 'for-linus-20180713' of git://git.kernel.dk/linux-block
git bisect bad 2da8c426d90355eef1d42d974d2dccf0f5f7f21d
# bad: [f353078f028fbfe9acd4b747b4a19c69ef6846cd] Merge branch 'akpm' (patches from Andrew)
git bisect bad f353078f028fbfe9acd4b747b4a19c69ef6846cd
# bad: [e181ae0c5db9544de9c53239eb22bc012ce75033] mm: zero unavailable pages before memmap init
git bisect bad e181ae0c5db9544de9c53239eb22bc012ce75033
# first bad commit: [e181ae0c5db9544de9c53239eb22bc012ce75033] mm: zero unavailable pages before memmap init
This 'bad' patch created a problem with 32-bit images which was later fixed with commit d1b47a7c9efc ("mm: don't do zero_resv_unavail if memmap is not allocated"). Reverting those two patches in mainline does _not_ fix the problem. With that in mind, I applied commit d1b47a7c9efc on top of 21279157efffe and ran another test. This test passed, so the above is a false positive.
---
Another bisect run, this time applying d1b47a7c9efc if 21279157efffe is in the image but d1b47a7c9efc isn't.
# bad: [781fca5b104693bc9242199cc47c690dcaf6a4cb] Merge tag 'xfs-4.19-merge-6' of git://git.kernel.org/pub/scm/fs/xfs/xfs-linux
# good: [94710cac0ef4ee177a63b5227664b38c95bbf703] Linux 4.18
git bisect start 'HEAD' 'v4.18'
# bad: [85a0b791bc17f7a49280b33e2905d109c062a47b] Merge branch 'for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/s390/linux
git bisect bad 85a0b791bc17f7a49280b33e2905d109c062a47b
# good: [8603596a327c978534f5c45db135e6c36b4b1425] Merge branch 'perf-core-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip
git bisect good 8603596a327c978534f5c45db135e6c36b4b1425
# bad: [eac341194426ba7ead3444923b9eba491ae4feeb] Merge branch 'x86/pti' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip
git bisect bad eac341194426ba7ead3444923b9eba491ae4feeb
# good: [30de24c7dd21348b142ee977b687afc70b392af6] Merge branch 'x86-cache-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip
git bisect good 30de24c7dd21348b142ee977b687afc70b392af6
# bad: [8c934e01a7ce685d98e970880f5941d79272c654] x86/pti: Check the return value of pti_user_pagetable_walk_pmd()
git bisect bad 8c934e01a7ce685d98e970880f5941d79272c654
# good: [fcbbd977572cfe5a3dcc97d663bf7480431a07ca] x86/pgtable: Move pti_set_user_pgtbl() to pgtable.h
git bisect good fcbbd977572cfe5a3dcc97d663bf7480431a07ca
# good: [ba0364e260ab37c02975557dbecc014a26072236] x86/mm/pti: Clone entry-text again in pti_finalize()
git bisect good ba0364e260ab37c02975557dbecc014a26072236
# good: [9bae3197e15dd5e03ce8e237db6fe4486b08a775] x86/ldt: Split out sanity check in map_ldt_struct()
git bisect good 9bae3197e15dd5e03ce8e237db6fe4486b08a775
# bad: [5e8105950a8b3e03e805299b4d05020ee4eda31a] x86/mm/pti: Add Warning when booting on a PCID capable CPU
git bisect bad 5e8105950a8b3e03e805299b4d05020ee4eda31a
# bad: [7757d607c6b31867777de42e1fb0210b9c5d8b70] x86/pti: Allow CONFIG_PAGE_TABLE_ISOLATION for x86_32
git bisect bad 7757d607c6b31867777de42e1fb0210b9c5d8b70
# good: [6df934b92a549cb3badb6d576f71aeb133e2f110] x86/ldt: Enable LDT user-mapping for PAE
git bisect good 6df934b92a549cb3badb6d576f71aeb133e2f110
# first bad commit: [7757d607c6b31867777de42e1fb0210b9c5d8b70] x86/pti: Allow CONFIG_PAGE_TABLE_ISOLATION for x86_32
Quoted 5 lines · Guenter Roeck
I see boot failures on mainline when trying to boot x86 images with an efi bios on Intel CPUs in qemu. Behavior is quite unusual: qemu dies silently after the kernel displays "Run
/sbin/initas init process". With debugging enabled, qemu reports a CR3 update followed by a triple fault.
My first thought would be that the EFI pgd is broken somehow.
Is 0e39b000 in your kernel binary, or was it dynamically allocated?
What was CR2 when things went bad? Could you just share a full register dump?
Quoted 11 lines · Dave Hansen
Quoted 5 lines · Guenter Roeck
I see boot failures on mainline when trying to boot x86 images with an efi bios on Intel CPUs in qemu. Behavior is quite unusual: qemu dies silently after the kernel displays "Run
/sbin/initas init process". With debugging enabled, qemu reports a CR3 update followed by a triple fault.My first thought would be that the EFI pgd is broken somehow.
Is
0e39b000in your kernel binary, or was it dynamically allocated?
No idea. In the log below (taken after I recompiled with the latest upstream kernel) the CR3 value is completely different. Where is the value expected to come from ?
Quoted 2 lines
What was CR2 when things went bad? Could you just share a full register dump?
Here you are. I have a complete log file, but its size is about 1.7GB (21MB compressed). Let me know if you need it, and I'll publish it somewhere.
Thanks,
Guenter
---
0xce1f1cd9: 66 90 nop 0xce1f1cdb: 8b 44 24 38 movl 0x38(%esp), %eax 0xce1f1cdf: 8a 64 24 40 movb 0x40(%esp), %ah 0xce1f1ce3: 8a 44 24 34 movb 0x34(%esp), %al 0xce1f1ce7: 25 03 04 02 00 andl $0x20403, %eax 0xce1f1cec: 3d 03 04 00 00 cmpl $0x403, %eax 0xce1f1cf1: 75 27 jne 0xce1f1d1a
EAX=f60b8000 EBX=ff8020bc ECX=00000000 EDX=00000000 ESI=f60c1ff8 EDI=ff802100 EBP=00000000 ESP=f60c1fb4 EIP=ce1f1cd7 EFL=00200006 [-----P-] CPL=0 II=0 A20=1 SMM=0 HLT=0
ES =007b 00000000 ffffffff 00cff300 DPL=3 DS [-WA]
CS =0060 00000000 ffffffff 00cf9a00 DPL=0 CS32 [-R-]
SS =0068 00000000 ffffffff 00cf9300 DPL=0 DS [-WA]
DS =007b 00000000 ffffffff 00cff300 DPL=3 DS [-WA]
FS =00d8 2819b000 ffffffff 008f9300 DPL=0 DS16 [-WA]
GS =00e0 f67f29c0 00000018 00409100 DPL=0 DS [--A]
LDT=0000 00000000 00000000 00008200 DPL=0 LDT
TR =0080 ff803000 0000206b 00008900 DPL=0 TSS32-avl
GDT= f67e2000 000000ff
IDT= ff800000 000007ff
CR0=80050033 CR2=b7f30854 CR3=35402000 CR4=000006d0 DR0=00000000 DR1=00000000 DR2=00000000 DR3=00000000 DR6=ffff0ff0 DR7=00000400
CCS=00000022 CCD=00000011 CCO=SARL EFER=0000000000000000
----------------
IN:
0xce1f1d1a: 66 90 nop 0xce1f1d1c: 0f 20 d8 movl %cr3, %eax 0xce1f1d1f: 0d 00 10 00 00 orl $0x1000, %eax 0xce1f1d24: 0f 22 d8 movl %eax, %cr3
EAX=00000003 EBX=ff8020bc ECX=00000000 EDX=00000000 ESI=f60c1ff8 EDI=ff802100 EBP=00000000 ESP=ff8020bc EIP=ce1f1d1a EFL=00200087 [--S--PC] CPL=0 II=0 A20=1 SMM=0 HLT=0
ES =007b 00000000 ffffffff 00cff300 DPL=3 DS [-WA]
CS =0060 00000000 ffffffff 00cf9a00 DPL=0 CS32 [-R-]
SS =0068 00000000 ffffffff 00cf9300 DPL=0 DS [-WA]
DS =007b 00000000 ffffffff 00cff300 DPL=3 DS [-WA]
FS =00d8 2819b000 ffffffff 008f9300 DPL=0 DS16 [-WA]
GS =00e0 f67f29c0 00000018 00409100 DPL=0 DS [--A]
LDT=0000 00000000 00000000 00008200 DPL=0 LDT
TR =0080 ff803000 0000206b 00008900 DPL=0 TSS32-avl
GDT= f67e2000 000000ff
IDT= ff800000 000007ff
CR0=80050033 CR2=b7f30854 CR3=35402000 CR4=000006d0 DR0=00000000 DR1=00000000 DR2=00000000 DR3=00000000 DR6=ffff0ff0 DR7=00000400
CCS=00000403 CCD=fffffc00 CCO=SUBL EFER=0000000000000000
CR3 update: CR3=35403000
----------------
IN:
0xce1f1d27: 5b popl %ebx 0xce1f1d28: 59 popl %ecx 0xce1f1d29: 5a popl %edx 0xce1f1d2a: 5e popl %esi 0xce1f1d2b: 5f popl %edi 0xce1f1d2c: 5d popl %ebp 0xce1f1d2d: 58 popl %eax 0xce1f1d2e: 1f popl %ds
EAX=35403000 EBX=ff8020bc ECX=00000000 EDX=00000000 ESI=f60c1ff8 EDI=ff802100 EBP=00000000 ESP=ff8020bc EIP=ce1f1d27 EFL=00200006 [-----P-] CPL=0 II=0 A20=1 SMM=0 HLT=0
ES =007b 00000000 ffffffff 00cff300 DPL=3 DS [-WA]
CS =0060 00000000 ffffffff 00cf9a00 DPL=0 CS32 [-R-]
SS =0068 00000000 ffffffff 00cf9300 DPL=0 DS [-WA]
DS =007b 00000000 ffffffff 00cff300 DPL=3 DS [-WA]
FS =00d8 2819b000 ffffffff 008f9300 DPL=0 DS16 [-WA]
GS =00e0 f67f29c0 00000018 00409100 DPL=0 DS [--A]
LDT=0000 00000000 00000000 00008200 DPL=0 LDT
TR =0080 ff803000 0000206b 00008900 DPL=0 TSS32-avl
GDT= f67e2000 000000ff
IDT= ff800000 000007ff
CR0=80050033 CR2=b7f30854 CR3=35403000 CR4=000006d0 DR0=00000000 DR1=00000000 DR2=00000000 DR3=00000000 DR6=ffff0ff0 DR7=00000400
CCS=00000403 CCD=35403000 CCO=LOGICL EFER=0000000000000000
EAX=35403000 EBX=ff8020bc ECX=00000000 EDX=00000000 ESI=f60c1ff8 EDI=ff802100 EBP=00000000 ESP=ff8020bc EIP=ce1f1d27 EFL=00200006 [-----P-] CPL=0 II=0 A20=1 SMM=0 HLT=0
ES =007b 00000000 ffffffff 00cff300 DPL=3 DS [-WA]
CS =0060 00000000 ffffffff 00cf9a00 DPL=0 CS32 [-R-]
SS =0068 00000000 ffffffff 00cf9300 DPL=0 DS [-WA]
DS =007b 00000000 ffffffff 00cff300 DPL=3 DS [-WA]
FS =00d8 2819b000 ffffffff 008f9300 DPL=0 DS16 [-WA]
GS =00e0 f67f29c0 00000018 00409100 DPL=0 DS [--A]
LDT=0000 00000000 00000000 00008200 DPL=0 LDT
TR =0080 ff803000 0000206b 00008900 DPL=0 TSS32-avl
GDT= f67e2000 000000ff
IDT= ff800000 000007ff
CR0=80050033 CR2=b7f30854 CR3=35403000 CR4=000006d0 DR0=00000000 DR1=00000000 DR2=00000000 DR3=00000000 DR6=ffff0ff0 DR7=00000400
CCS=00000004 CCD=35403000 CCO=EFLAGS EFER=0000000000000000
Triple fault
Hey Guenter,
Quoted 6 lines · Guenter Roeck
I see boot failures on mainline when trying to boot x86 images with an efi bios on Intel CPUs in qemu. Behavior is quite unusual: qemu dies silently after the kernel displays "Run
/sbin/initas init process". With debugging enabled, qemu reports a CR3 update followed by a triple fault. Here is the end of the log file:
Quoted 8 lines
----------------
IN:0xc75f1d1a: 66 90 nop 0xc75f1d1c: 0f 20 d8 movl %cr3, %eax 0xc75f1d1f: 0d 00 10 00 00 orl $0x1000, %eax 0xc75f1d24: 0f 22 d8 movl %eax, %cr3CR3 update: CR3=0e39b000
Thanks for the bug report, from a quick glance it looks like the kernel is switching to an empty page-table. But I need to debug that further.
Quoted 7 lines
qemu-system-i386 -kernel arch/x86/boot/bzImage -M q35 -cpu core2duo \ -no-reboot -m 256 \ -bios OVMF-pure-efi-32.fd \ -usb -device usb-storage,drive=d0 \ -drive file=rootfs.ext2,if=none,id=d0,format=raw \ --append 'root=/dev/sda rw rootwait mem=256M console=ttyS0 console=tty noreboot' \ -nographic
Can you please provide the exact OVMF-pure-efi-32.fd image and the .config of the kernel?
Thanks a lot,
Joerg
Quoted 29 lines · Joerg Roedel
Hey Guenter,
Quoted 6 lines · Guenter Roeck
I see boot failures on mainline when trying to boot x86 images with an efi bios on Intel CPUs in qemu. Behavior is quite unusual: qemu dies silently after the kernel displays "Run
/sbin/initas init process". With debugging enabled, qemu reports a CR3 update followed by a triple fault. Here is the end of the log file:Quoted 8 lines
----------------
IN:0xc75f1d1a: 66 90 nop 0xc75f1d1c: 0f 20 d8 movl %cr3, %eax 0xc75f1d1f: 0d 00 10 00 00 orl $0x1000, %eax 0xc75f1d24: 0f 22 d8 movl %eax, %cr3CR3 update: CR3=0e39b000
Thanks for the bug report, from a quick glance it looks like the kernel is switching to an empty page-table. But I need to debug that further.
Quoted 7 lines
qemu-system-i386 -kernel arch/x86/boot/bzImage -M q35 -cpu core2duo \ -no-reboot -m 256 \ -bios OVMF-pure-efi-32.fd \ -usb -device usb-storage,drive=d0 \ -drive file=rootfs.ext2,if=none,id=d0,format=raw \ --append 'root=/dev/sda rw rootwait mem=256M console=ttyS0 console=tty noreboot' \ -nographic
Quoted 3 lines
Can you please provide the exact OVMF-pure-efi-32.fd image and the .config of the kernel?
OVMF image:
https://github.com/groeck/linux-build-test/blob/master/rootfs/firmware/OVMF-pure-efi-32.fd
root file system:
https://github.com/groeck/linux-build-test/blob/master/rootfs/x86/rootfs.ext2
Configuration is attached.
Guenter
# CONFIG_LOCALVERSION_AUTO is not setCONFIG_SYSVIPC=yCONFIG_POSIX_MQUEUE=yCONFIG_AUDIT=yCONFIG_NO_HZ=yCONFIG_HIGH_RES_TIMERS=yCONFIG_PREEMPT_VOLUNTARY=yCONFIG_BSD_PROCESS_ACCT=yCONFIG_TASKSTATS=yCONFIG_TASK_DELAY_ACCT=yCONFIG_TASK_XACCT=yCONFIG_TASK_IO_ACCOUNTING=yCONFIG_LOG_BUF_SHIFT=18CONFIG_CGROUPS=yCONFIG_CGROUP_SCHED=yCONFIG_CGROUP_FREEZER=yCONFIG_CPUSETS=yCONFIG_CGROUP_CPUACCT=yCONFIG_BLK_DEV_INITRD=y
# CONFIG_COMPAT_BRK is not setCONFIG_PROFILING=yCONFIG_SMP=yCONFIG_X86_GENERIC=yCONFIG_HPET_TIMER=yCONFIG_SCHED_SMT=yCONFIG_X86_REROUTE_FOR_BROKEN_BOOT_IRQS=yCONFIG_X86_REBOOTFIXUPS=yCONFIG_MICROCODE_AMD=yCONFIG_X86_MSR=yCONFIG_X86_CPUID=yCONFIG_HIGHPTE=yCONFIG_X86_CHECK_BIOS_CORRUPTION=y
# CONFIG_MTRR_SANITIZER is not setCONFIG_EFI=yCONFIG_HZ_1000=yCONFIG_KEXEC=yCONFIG_CRASH_DUMP=yCONFIG_HIBERNATION=yCONFIG_PM_DEBUG=yCONFIG_PM_TRACE_RTC=yCONFIG_ACPI_DOCK=yCONFIG_CPU_FREQ_DEFAULT_GOV_USERSPACE=yCONFIG_CPU_FREQ_GOV_PERFORMANCE=yCONFIG_CPU_FREQ_GOV_ONDEMAND=yCONFIG_X86_ACPI_CPUFREQ=yCONFIG_PCIEPORTBUS=yCONFIG_PCI_MSI=yCONFIG_HOTPLUG_PCI=yCONFIG_PCCARD=yCONFIG_YENTA=yCONFIG_EFI_VARS=yCONFIG_KPROBES=yCONFIG_JUMP_LABEL=yCONFIG_MODULES=yCONFIG_MODULE_UNLOAD=yCONFIG_MODULE_FORCE_UNLOAD=yCONFIG_PARTITION_ADVANCED=yCONFIG_OSF_PARTITION=yCONFIG_AMIGA_PARTITION=yCONFIG_MAC_PARTITION=yCONFIG_BSD_DISKLABEL=yCONFIG_MINIX_SUBPARTITION=yCONFIG_SOLARIS_X86_PARTITION=yCONFIG_UNIXWARE_DISKLABEL=yCONFIG_SGI_PARTITION=yCONFIG_SUN_PARTITION=yCONFIG_KARMA_PARTITION=yCONFIG_BINFMT_MISC=yCONFIG_NET=yCONFIG_PACKET=yCONFIG_UNIX=yCONFIG_XFRM_USER=yCONFIG_INET=yCONFIG_IP_MULTICAST=yCONFIG_IP_ADVANCED_ROUTER=yCONFIG_IP_MULTIPLE_TABLES=yCONFIG_IP_ROUTE_MULTIPATH=yCONFIG_IP_ROUTE_VERBOSE=yCONFIG_IP_PNP=yCONFIG_IP_PNP_DHCP=yCONFIG_IP_PNP_BOOTP=yCONFIG_IP_PNP_RARP=yCONFIG_IP_MROUTE=yCONFIG_IP_PIMSM_V1=yCONFIG_IP_PIMSM_V2=yCONFIG_SYN_COOKIES=y
# CONFIG_INET_XFRM_MODE_TRANSPORT is not set
# CONFIG_INET_XFRM_MODE_TUNNEL is not set
# CONFIG_INET_XFRM_MODE_BEET is not set
# CONFIG_INET_DIAG is not setCONFIG_TCP_CONG_ADVANCED=y
# CONFIG_TCP_CONG_BIC is not set
# CONFIG_TCP_CONG_WESTWOOD is not set
# CONFIG_TCP_CONG_HTCP is not setCONFIG_TCP_MD5SIG=yCONFIG_INET6_AH=yCONFIG_INET6_ESP=yCONFIG_NETLABEL=yCONFIG_NETFILTER=y
# CONFIG_NETFILTER_ADVANCED is not setCONFIG_NF_CONNTRACK=yCONFIG_NF_CONNTRACK_FTP=yCONFIG_NF_CONNTRACK_IRC=yCONFIG_NF_CONNTRACK_SIP=yCONFIG_NF_CT_NETLINK=yCONFIG_NETFILTER_XT_TARGET_CONNSECMARK=yCONFIG_NETFILTER_XT_TARGET_NFLOG=yCONFIG_NETFILTER_XT_TARGET_SECMARK=yCONFIG_NETFILTER_XT_TARGET_TCPMSS=yCONFIG_NETFILTER_XT_MATCH_CONNTRACK=yCONFIG_NETFILTER_XT_MATCH_POLICY=yCONFIG_NETFILTER_XT_MATCH_STATE=yCONFIG_IP_NF_IPTABLES=yCONFIG_IP_NF_FILTER=yCONFIG_IP_NF_TARGET_REJECT=yCONFIG_IP_NF_MANGLE=yCONFIG_IP6_NF_IPTABLES=yCONFIG_IP6_NF_MATCH_IPV6HEADER=yCONFIG_IP6_NF_FILTER=yCONFIG_IP6_NF_TARGET_REJECT=yCONFIG_IP6_NF_MANGLE=yCONFIG_NET_SCHED=yCONFIG_NET_EMATCH=yCONFIG_NET_CLS_ACT=yCONFIG_HAMRADIO=yCONFIG_CFG80211=yCONFIG_MAC80211=yCONFIG_MAC80211_LEDS=yCONFIG_RFKILL=yCONFIG_UEVENT_HELPER_PATH="/sbin/hotplug"CONFIG_DEVTMPFS=yCONFIG_DEVTMPFS_MOUNT=yCONFIG_DEBUG_DEVRES=yCONFIG_CONNECTOR=yCONFIG_BLK_DEV_LOOP=yCONFIG_VIRTIO_BLK=yCONFIG_VIRTIO_BLK_SCSI=yCONFIG_BLK_DEV_NVME=yCONFIG_BLK_DEV_SD=yCONFIG_BLK_DEV_SR=yCONFIG_BLK_DEV_SR_VENDOR=yCONFIG_CHR_DEV_SG=yCONFIG_SCSI_CONSTANTS=yCONFIG_MEGARAID_SAS=yCONFIG_SCSI_SYM53C8XX_2=y
CONFIG_SCSI_DC395x=yCONFIG_SCSI_AM53C974=yCONFIG_SCSI_VIRTIO=yCONFIG_ATA=yCONFIG_SATA_AHCI=yCONFIG_ATA_PIIX=yCONFIG_PATA_AMD=yCONFIG_PATA_OLDPIIX=yCONFIG_PATA_SCH=yCONFIG_PATA_MPIIX=yCONFIG_ATA_GENERIC=yCONFIG_MD=yCONFIG_BLK_DEV_MD=yCONFIG_BLK_DEV_DM=yCONFIG_DM_MIRROR=yCONFIG_DM_ZERO=yCONFIG_FUSION=yCONFIG_FUSION_SAS=yCONFIG_MACINTOSH_DRIVERS=yCONFIG_MAC_EMUMOUSEBTN=yCONFIG_NETDEVICES=yCONFIG_NETCONSOLE=yCONFIG_BNX2=yCONFIG_TIGON3=yCONFIG_NET_TULIP=yCONFIG_E100=yCONFIG_E1000=yCONFIG_E1000E=yCONFIG_SKY2=yCONFIG_NE2K_PCI=yCONFIG_FORCEDETH=yCONFIG_8139TOO=y
# CONFIG_8139TOO_PIO is not setCONFIG_R8169=yCONFIG_FDDI=yCONFIG_INPUT_POLLDEV=yCONFIG_INPUT_EVDEV=yCONFIG_INPUT_JOYSTICK=yCONFIG_INPUT_TABLET=yCONFIG_INPUT_TOUCHSCREEN=yCONFIG_INPUT_MISC=y
# CONFIG_LEGACY_PTYS is not setCONFIG_SERIAL_NONSTANDARD=yCONFIG_SERIAL_8250=yCONFIG_SERIAL_8250_CONSOLE=yCONFIG_SERIAL_8250_NR_UARTS=32CONFIG_SERIAL_8250_EXTENDED=yCONFIG_SERIAL_8250_MANY_PORTS=yCONFIG_SERIAL_8250_SHARE_IRQ=yCONFIG_SERIAL_8250_DETECT_IRQ=yCONFIG_SERIAL_8250_RSA=yCONFIG_HW_RANDOM=yCONFIG_NVRAM=yCONFIG_HPET=y
# CONFIG_HPET_MMAP is not setCONFIG_I2C_I801=yCONFIG_WATCHDOG=yCONFIG_AGP=yCONFIG_AGP_AMD64=yCONFIG_AGP_INTEL=yCONFIG_DRM=yCONFIG_DRM_I915=yCONFIG_FB_MODE_HELPERS=yCONFIG_FB_TILEBLITTING=yCONFIG_FB_EFI=y
# CONFIG_LCD_CLASS_DEVICE is not setCONFIG_VGACON_SOFT_SCROLLBACK=yCONFIG_LOGO=y
# CONFIG_LOGO_LINUX_MONO is not set
# CONFIG_LOGO_LINUX_VGA16 is not setCONFIG_SOUND=yCONFIG_SND=yCONFIG_SND_HRTIMER=yCONFIG_SND_SEQUENCER=yCONFIG_SND_SEQ_DUMMY=yCONFIG_SND_HDA_INTEL=yCONFIG_SND_HDA_HWDEP=yCONFIG_HIDRAW=yCONFIG_HID_GYRATION=yCONFIG_LOGITECH_FF=yCONFIG_HID_NTRIG=yCONFIG_HID_PANTHERLORD=yCONFIG_PANTHERLORD_FF=yCONFIG_HID_PETALYNX=yCONFIG_HID_SAMSUNG=yCONFIG_HID_SONY=yCONFIG_HID_SUNPLUS=yCONFIG_HID_TOPSEED=yCONFIG_HID_PID=yCONFIG_USB_HIDDEV=yCONFIG_USB=yCONFIG_USB_ANNOUNCE_NEW_DEVICES=yCONFIG_USB_MON=yCONFIG_USB_XHCI_HCD=yCONFIG_USB_EHCI_HCD=yCONFIG_USB_OHCI_HCD=yCONFIG_USB_UHCI_HCD=yCONFIG_USB_PRINTER=yCONFIG_USB_STORAGE=yCONFIG_USB_UAS=yCONFIG_MMC=yCONFIG_MMC_SDHCI=yCONFIG_MMC_SDHCI_PCI=yCONFIG_EDAC=yCONFIG_RTC_CLASS=y
# CONFIG_RTC_HCTOSYS is not setCONFIG_DMADEVICES=yCONFIG_VIRTIO_PCI=yCONFIG_VIRTIO_BALLOON=yCONFIG_VIRTIO_MMIO=yCONFIG_EEEPC_LAPTOP=yCONFIG_EXT4_FS=yCONFIG_EXT4_FS_POSIX_ACL=yCONFIG_EXT4_FS_SECURITY=yCONFIG_QUOTA=yCONFIG_QUOTA_NETLINK_INTERFACE=y
# CONFIG_PRINT_QUOTA_WARNING is not setCONFIG_QFMT_V2=yCONFIG_AUTOFS4_FS=yCONFIG_ISO9660_FS=yCONFIG_JOLIET=yCONFIG_ZISOFS=yCONFIG_MSDOS_FS=yCONFIG_VFAT_FS=yCONFIG_PROC_KCORE=yCONFIG_TMPFS_POSIX_ACL=yCONFIG_HUGETLBFS=yCONFIG_NFS_FS=yCONFIG_NFS_V3_ACL=yCONFIG_NFS_V4=yCONFIG_ROOT_NFS=yCONFIG_NLS_DEFAULT="utf8"CONFIG_NLS_CODEPAGE_437=yCONFIG_NLS_ASCII=yCONFIG_NLS_ISO8859_1=yCONFIG_NLS_UTF8=yCONFIG_SECURITY=yCONFIG_SECURITY_NETWORK=yCONFIG_SECURITY_SELINUX=yCONFIG_SECURITY_SELINUX_BOOTPARAM=yCONFIG_SECURITY_SELINUX_DISABLE=yCONFIG_CRYPTO_AES_586=yCONFIG_PRINTK_TIME=y
# CONFIG_UNUSED_SYMBOLS is not setCONFIG_MAGIC_SYSRQ=yCONFIG_DEBUG_KERNEL=yCONFIG_DEBUG_STACK_USAGE=yCONFIG_DEBUG_STACKOVERFLOW=y
# CONFIG_SCHED_DEBUG is not setCONFIG_SCHEDSTATS=yCONFIG_BLK_DEV_IO_TRACE=yCONFIG_PROVIDE_OHCI1394_DMA_INIT=yCONFIG_EARLY_PRINTK_DBGP=yCONFIG_DEBUG_BOOT_PARAMS=yCONFIG_OPTIMIZE_INLINING=y
Quoted 3 lines · Guenter Roeck
OVMF image:
https://github.com/groeck/linux-build-test/blob/master/rootfs/firmware/OVMF-pure-efi-32.fd
Thanks, with this image I can reproduce the issue. Investigating now...
Regards,
Joerg
Quoted 7 lines · Joerg Roedel
Quoted 3 lines · Guenter Roeck
OVMF image:
https://github.com/groeck/linux-build-test/blob/master/rootfs/firmware/OVMF-pure-efi-32.fd
Thanks, with this image I can reproduce the issue. Investigating now...
Excellent.
In case it matters: I can use the same 32-bit efi image to boot a 64-bit kernel without problems.
Guenter
Quoted 4 lines · Guenter Roeck
Excellent.
In case it matters: I can use the same 32-bit efi image to boot a 64-bit kernel without problems.
Okay, I tracked it down. The 32-bit EFI code switches the CPU to the read-write version of the GDT which is not mapped in the user page-table. This obviously faults when the kernel entry/exit code restores the segment registers.
Without a mapped GDT the #PF and #DF handlers also can't be started, so the machine triple-faults. Below diff fixes it for me, I'll send a proper patch tomorrow.
arch/x86/platform/efi/efi_32.c +1 -1
@@ -87,7 +87,7 @@ void __init efi_call_phys_epilog(pgd_t *save_pgd) {struct desc_ptr gdt_descr; - gdt_descr.address = (unsigned long )get_cpu_gdt_rw (0 ); + gdt_descr.address = (unsigned long )get_cpu_gdt_ro (0 ); gdt_descr.size =GDT_SIZE -1 ;load_gdt (&gdt_descr);
Quoted 4 lines · Joerg Roedel
Without a mapped GDT the #PF and #DF handlers also can't be started, so the machine triple-faults. Below diff fixes it for me, I'll send a proper patch tomorrow.
Hmm. Is there any reason why this code doesn't just use
load_fixmap_gdt(0);
and shouldn't it do it after loading the new %cr3?
Linus
Quoted 9 lines · Linus Torvalds
Quoted 4 lines · Joerg Roedel
Without a mapped GDT the #PF and #DF handlers also can't be started, so the machine triple-faults. Below diff fixes it for me, I'll send a proper patch tomorrow.
Hmm. Is there any reason why this code doesn't just use
load_fixmap_gdt(0);
No idea, probably the function didn't exist when the code was written? I can change that when writing the patch.
Quoted 1 line
and shouldn't it do it after loading the new %cr3?
That seems more robust, yes. No sure if the old %cr3 (initial_page_table) has the fixmap gdt mapped at all.
Joerg
Guenter Roeck
Tested-byQuoted 11 lines · Joerg Roedel
Quoted 9 lines · Linus Torvalds
Quoted 4 lines · Joerg Roedel
Without a mapped GDT the #PF and #DF handlers also can't be started, so the machine triple-faults. Below diff fixes it for me, I'll send a proper patch tomorrow.
Hmm. Is there any reason why this code doesn't just use
load_fixmap_gdt(0);
Quoted 7 lines
No idea, probably the function didn't exist when the code was written? I can change that when writing the patch.
Quoted 1 line
and shouldn't it do it after loading the new %cr3?
That seems more robust, yes. No sure if the old %cr3 (
initial_page_table) has the fixmap gdt mapped at all.
All three variants (hardcoded, call load_fixmap_gdt(0) first, call load_fixmap_gdt(0) after load_cr3()) work for me. Feel free to add
when you submit the patch.
Thanks a lot for tracking this down!
Guenter