x86/purgatory: Switch to the position-independent small code model
Subsystems (2)
PATCH 1/1 x86/purgatory: Switch to the position-independent small code model
From: Ard Biesheuvel <[email protected]>
On x86, the ordinary, position dependent 'small' and 'kernel' code models only support placement of the executable in 32-bit addressable memory, due to the use of 32-bit signed immediates to generate references to global variables. For the kernel, this implies that all global variables must reside in the top 2 GiB of the kernel virtual address space, where the implicit address bits 63:32 are equal to sign bit 31.
This means the kernel code model is not suitable for other bare metal executables such as the kexec purgatory, which can be placed arbitrarily in the physical address space, where its address may no longer be representable as a sign extended 32-bit quantity. For this reason, commit
e16c2983fba0 ("x86/purgatory: Change compiler flags from -mcmodel=kernel to -mcmodel=large to fix kexec relocation errors")
switched to the 'large' code model, which uses 64-bit immediates for all symbol references, including function calls, in order to avoid relying on any assumptions regarding proximity of symbols in the final executable.
The large code model is rarely used, clunky and the least likely to operate in a similar fashion when comparing GCC and Clang, so it is best avoided. This is especially true now that Clang 18 has started to emit executable code in two separate sections (.text and .ltext), which triggers an issue in the kexec loading code at runtime.
Instead, use the position independent small code model, which makes no assumptions about placement but only about proximity, where all referenced symbols must be within -/+ 2 GiB, i.e., in range for a RIP-relative reference. Use hidden visibility to suppress the use of a GOT, which carries absolute addresses that are not covered by static ELF relocations, and is therefore incompatible with the kexec loader's relocation logic.
--- arch/x86/purgatory/Makefile | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-)
arch/x86/purgatory/Makefile +2 -1
@@ -42,7 +42,8 @@ KCOV_INSTRUMENT := n # make up the standalone purgatory.ro PURGATORY_CFLAGS_REMOVE := -mcmodel=kernel -PURGATORY_CFLAGS := -mcmodel=large -ffreestanding -fno-zero-initialized-in-bss -g0 +PURGATORY_CFLAGS := -mcmodel=small -ffreestanding -fno-zero-initialized-in-bss -g0 +PURGATORY_CFLAGS += -fpic -fvisibility=hidden PURGATORY_CFLAGS += $(DISABLE_STACKLEAK_PLUGIN) -DDISABLE_BRANCH_PROFILING PURGATORY_CFLAGS += -fno-stack-protector
Signature · 1 line
2.44.0.769.g3c40516874-goog
Quoted 37 lines · Ard Biesheuvel
From: Ard Biesheuvel <[email protected]>
On x86, the ordinary, position dependent 'small' and 'kernel' code models only support placement of the executable in 32-bit addressable memory, due to the use of 32-bit signed immediates to generate references to global variables. For the kernel, this implies that all global variables must reside in the top 2 GiB of the kernel virtual address space, where the implicit address bits 63:32 are equal to sign bit 31.
This means the kernel code model is not suitable for other bare metal executables such as the kexec purgatory, which can be placed arbitrarily in the physical address space, where its address may no longer be representable as a sign extended 32-bit quantity. For this reason, commit
e16c2983fba0("x86/purgatory: Change compiler flags from -mcmodel=kernel to -mcmodel=large to fix kexec relocation errors")switched to the 'large' code model, which uses 64-bit immediates for all symbol references, including function calls, in order to avoid relying on any assumptions regarding proximity of symbols in the final executable.
The large code model is rarely used, clunky and the least likely to operate in a similar fashion when comparing GCC and Clang, so it is best avoided. This is especially true now that Clang 18 has started to emit executable code in two separate sections (.text and .ltext), which triggers an issue in the kexec loading code at runtime.
Instead, use the position independent small code model, which makes no assumptions about placement but only about proximity, where all referenced symbols must be within -/+ 2 GiB, i.e., in range for a RIP-relative reference. Use hidden visibility to suppress the use of a GOT, which carries absolute addresses that are not covered by static ELF relocations, and is therefore incompatible with the kexec loader's relocation logic.
Quoted 2 lines
Cc Nathan Chancellor <[email protected]>Cc Nick Desaulniers <[email protected]>
Thanks Ard!
-mcmodel=kernel to -mcmodel=large to fix kexec relocation errors")
(I don't have a kexec setup ready to go; maybe someone that does can help test it.)
Quoted 30 lines
Cc Bill Wendling <[email protected]>Cc Justin Stitt <[email protected]>Cc Song Liu <[email protected]>Cc Ricardo Ribalda <[email protected]>Cc Fangrui Song <[email protected]>Cc Arthur Eubanks <[email protected]>Cc <[email protected]>Signed-off-by Ard Biesheuvel <[email protected]>--- arch/x86/purgatory/Makefile | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-)arch/x86/purgatory/Makefile +2
-2@@ -42,7 +42,8 @@ KCOV_INSTRUMENT := n # make up the standalone purgatory.ro PURGATORY_CFLAGS_REMOVE := -mcmodel=kernel -PURGATORY_CFLAGS := -mcmodel=large -ffreestanding -fno-zero-initialized-in-bss -g0 +PURGATORY_CFLAGS := -mcmodel=small -ffreestanding -fno-zero-initialized-in-bss -g0 +PURGATORY_CFLAGS += -fpic -fvisibility=hidden PURGATORY_CFLAGS += $(DISABLE_STACKLEAK_PLUGIN) -DDISABLE_BRANCH_PROFILING PURGATORY_CFLAGS += -fno-stack-protector --2.44.0.769.g3c40516874-goog
Signature · 2 lines
Thanks, ~Nick Desaulniers
Fangrui Song
Reviewed-byQuoted 43 lines · Nick Desaulniers
Quoted 37 lines · Ard Biesheuvel
From: Ard Biesheuvel <[email protected]>
On x86, the ordinary, position dependent 'small' and 'kernel' code models only support placement of the executable in 32-bit addressable memory, due to the use of 32-bit signed immediates to generate references to global variables. For the kernel, this implies that all global variables must reside in the top 2 GiB of the kernel virtual address space, where the implicit address bits 63:32 are equal to sign bit 31.
This means the kernel code model is not suitable for other bare metal executables such as the kexec purgatory, which can be placed arbitrarily in the physical address space, where its address may no longer be representable as a sign extended 32-bit quantity. For this reason, commit
e16c2983fba0("x86/purgatory: Change compiler flags from -mcmodel=kernel to -mcmodel=large to fix kexec relocation errors")switched to the 'large' code model, which uses 64-bit immediates for all symbol references, including function calls, in order to avoid relying on any assumptions regarding proximity of symbols in the final executable.
The large code model is rarely used, clunky and the least likely to operate in a similar fashion when comparing GCC and Clang, so it is best avoided. This is especially true now that Clang 18 has started to emit executable code in two separate sections (.text and .ltext), which triggers an issue in the kexec loading code at runtime.
Instead, use the position independent small code model, which makes no assumptions about placement but only about proximity, where all referenced symbols must be within -/+ 2 GiB, i.e., in range for a RIP-relative reference. Use hidden visibility to suppress the use of a GOT, which carries absolute addresses that are not covered by static ELF relocations, and is therefore incompatible with the kexec loader's relocation logic.
Quoted 2 lines
Cc Nathan Chancellor <[email protected]>Cc Nick Desaulniers <[email protected]>Thanks Ard!
Quoted 8 lines
Acked-by Nick Desaulniers <[email protected]>Reported-by ns <[email protected]>Fixes e16c2983fba0 ("x86/purgatory: Change compiler flags from-mcmodel=kernel to -mcmodel=large to fix kexec relocation errors")
(I don't have a kexec setup ready to go; maybe someone that does can help test it.)
LGTM.
Position-dependent small code model may generate R_X86_64_32S relocations with a range of [0,2GiB) (the negative half cannot be used). Position-independent small code model with hidden visibility will generate R_X86_64_PC32 and can typically be quite larger than 2G without hitting an overflow issue.
(I have some notes about R_X86_64_32S at
https://maskray.me/blog/2023-05-14-relocation-overflow-and-code-models#x86-64-linker-requirement)
Quoted 35 lines
Quoted 30 lines
Cc Bill Wendling <[email protected]>Cc Justin Stitt <[email protected]>Cc Song Liu <[email protected]>Cc Ricardo Ribalda <[email protected]>Cc Fangrui Song <[email protected]>Cc Arthur Eubanks <[email protected]>Cc <[email protected]>Signed-off-by Ard Biesheuvel <[email protected]>--- arch/x86/purgatory/Makefile | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-)arch/x86/purgatory/Makefile +2
-2@@ -42,7 +42,8 @@ KCOV_INSTRUMENT := n # make up the standalone purgatory.ro PURGATORY_CFLAGS_REMOVE := -mcmodel=kernel -PURGATORY_CFLAGS := -mcmodel=large -ffreestanding -fno-zero-initialized-in-bss -g0 +PURGATORY_CFLAGS := -mcmodel=small -ffreestanding -fno-zero-initialized-in-bss -g0 +PURGATORY_CFLAGS += -fpic -fvisibility=hidden PURGATORY_CFLAGS += $(DISABLE_STACKLEAK_PLUGIN) -DDISABLE_BRANCH_PROFILING PURGATORY_CFLAGS += -fno-stack-protector --2.44.0.769.g3c40516874-goog
Signature · 2 lines
Thanks, ~Nick Desaulniers
Borislav Petkov
maintainerQuoted 1 line · Nick Desaulniers
Reported-by ns <[email protected]>
I don't know what the goal with this tag is but this'll keep bouncing:
<[email protected]>: Host or domain name not found. Name service
error for name=users.noreply.github.com type=AAAA: Host not found
Maybe use a non-void email...?
Signature · 4 lines
Regards/Gruss,
Boris.
https://people.kernel.org/tglx/notes-about-netiquetteQuoted 11 lines · Borislav Petkov
Quoted 1 line · Nick Desaulniers
Reported-by ns <[email protected]>I don't know what the goal with this tag is but this'll keep bouncing:
<[email protected]>: Host or domain name not found. Name service
error for name=users.noreply.github.com type=AAAA: Host not foundMaybe use a non-void email...?
Let's just drop that.
If people cannot be bothered to use their real name, I don't think they care about being credited either.
Quoted 36 lines · Ard Biesheuvel
From: Ard Biesheuvel <[email protected]>
On x86, the ordinary, position dependent 'small' and 'kernel' code models only support placement of the executable in 32-bit addressable memory, due to the use of 32-bit signed immediates to generate references to global variables. For the kernel, this implies that all global variables must reside in the top 2 GiB of the kernel virtual address space, where the implicit address bits 63:32 are equal to sign bit 31.
This means the kernel code model is not suitable for other bare metal executables such as the kexec purgatory, which can be placed arbitrarily in the physical address space, where its address may no longer be representable as a sign extended 32-bit quantity. For this reason, commit
e16c2983fba0("x86/purgatory: Change compiler flags from -mcmodel=kernel to -mcmodel=large to fix kexec relocation errors")switched to the 'large' code model, which uses 64-bit immediates for all symbol references, including function calls, in order to avoid relying on any assumptions regarding proximity of symbols in the final executable.
The large code model is rarely used, clunky and the least likely to operate in a similar fashion when comparing GCC and Clang, so it is best avoided. This is especially true now that Clang 18 has started to emit executable code in two separate sections (.text and .ltext), which triggers an issue in the kexec loading code at runtime.
Instead, use the position independent small code model, which makes no assumptions about placement but only about proximity, where all referenced symbols must be within -/+ 2 GiB, i.e., in range for a RIP-relative reference. Use hidden visibility to suppress the use of a GOT, which carries absolute addresses that are not covered by static ELF relocations, and is therefore incompatible with the kexec loader's relocation logic.
Quoted 11 lines
Cc Nathan Chancellor <[email protected]>Cc Nick Desaulniers <[email protected]>Cc Bill Wendling <[email protected]>Cc Justin Stitt <[email protected]>Cc Song Liu <[email protected]>Cc Ricardo Ribalda <[email protected]>Cc Fangrui Song <[email protected]>Cc Arthur Eubanks <[email protected]>Cc <[email protected]>Signed-off-by Ard Biesheuvel <[email protected]>
This resolves the warning and relocation overflow error that I see with LLVM 18.1.4 (the relocation error was fixed in LLVM but dropping -mcmodel=large resolves it for toolchains without that fix).
Quoted 21 lines
--- arch/x86/purgatory/Makefile | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-)arch/x86/purgatory/Makefile +2
-1@@ -42,7 +42,8 @@ KCOV_INSTRUMENT := n # make up the standalone purgatory.ro PURGATORY_CFLAGS_REMOVE := -mcmodel=kernel -PURGATORY_CFLAGS := -mcmodel=large -ffreestanding -fno-zero-initialized-in-bss -g0 +PURGATORY_CFLAGS := -mcmodel=small -ffreestanding -fno-zero-initialized-in-bss -g0 +PURGATORY_CFLAGS += -fpic -fvisibility=hidden PURGATORY_CFLAGS += $(DISABLE_STACKLEAK_PLUGIN) -DDISABLE_BRANCH_PROFILING PURGATORY_CFLAGS += -fno-stack-protectorSignature · 1 line
2.44.0.769.g3c40516874-goog
Quoted 28 lines · Ard Biesheuvel
From: Ard Biesheuvel <[email protected]>
On x86, the ordinary, position dependent 'small' and 'kernel' code models only support placement of the executable in 32-bit addressable memory, due to the use of 32-bit signed immediates to generate references to global variables. For the kernel, this implies that all global variables must reside in the top 2 GiB of the kernel virtual address space, where the implicit address bits 63:32 are equal to sign bit 31.
This means the kernel code model is not suitable for other bare metal executables such as the kexec purgatory, which can be placed arbitrarily in the physical address space, where its address may no longer be representable as a sign extended 32-bit quantity. For this reason, commit
e16c2983fba0("x86/purgatory: Change compiler flags from -mcmodel=kernel to -mcmodel=large to fix kexec relocation errors")switched to the 'large' code model, which uses 64-bit immediates for all symbol references, including function calls, in order to avoid relying on any assumptions regarding proximity of symbols in the final executable.
The large code model is rarely used, clunky and the least likely to operate in a similar fashion when comparing GCC and Clang, so it is best avoided. This is especially true now that Clang 18 has started to emit executable code in two separate sections (.text and .ltext), which triggers an issue in the kexec loading code at runtime.
Quoted 7 lines
Instead, use the position independent small code model, which makes no assumptions about placement but only about proximity, where all referenced symbols must be within -/+ 2 GiB, i.e., in range for a RIP-relative reference. Use hidden visibility to suppress the use of a GOT, which carries absolute addresses that are not covered by static ELF relocations, and is therefore incompatible with the kexec loader's relocation logic.
FWIW:
thanks,
Signature · 2 lines
js suse labs
Borislav Petkov
maintainerQuoted 37 lines · Jiri Slaby
Quoted 28 lines · Ard Biesheuvel
From: Ard Biesheuvel <[email protected]>
On x86, the ordinary, position dependent 'small' and 'kernel' code models only support placement of the executable in 32-bit addressable memory, due to the use of 32-bit signed immediates to generate references to global variables. For the kernel, this implies that all global variables must reside in the top 2 GiB of the kernel virtual address space, where the implicit address bits 63:32 are equal to sign bit 31.
This means the kernel code model is not suitable for other bare metal executables such as the kexec purgatory, which can be placed arbitrarily in the physical address space, where its address may no longer be representable as a sign extended 32-bit quantity. For this reason, commit
e16c2983fba0("x86/purgatory: Change compiler flags from -mcmodel=kernel to -mcmodel=large to fix kexec relocation errors")switched to the 'large' code model, which uses 64-bit immediates for all symbol references, including function calls, in order to avoid relying on any assumptions regarding proximity of symbols in the final executable.
The large code model is rarely used, clunky and the least likely to operate in a similar fashion when comparing GCC and Clang, so it is best avoided. This is especially true now that Clang 18 has started to emit executable code in two separate sections (.text and .ltext), which triggers an issue in the kexec loading code at runtime.
Quoted 7 lines
Instead, use the position independent small code model, which makes no assumptions about placement but only about proximity, where all referenced symbols must be within -/+ 2 GiB, i.e., in range for a RIP-relative reference. Use hidden visibility to suppress the use of a GOT, which carries absolute addresses that are not covered by static ELF relocations, and is therefore incompatible with the kexec loader's relocation logic.
Quoted 3 lines
FWIW:
Interesting. I thought gcc doesn't have problems here yet and was talking to Matz on Thu about it and it seems he's forgotten about his statement too that "you should simply stop using -mcmodel=large. Noone should use it." :-)
Ok then, lemme queue Ard's fix.
I guess we want it in stable too but as it is pretty much the case with toolchain fixes I'd prefer if we queued it the normal path for 6.10 and then backport it after it sees some testing and nothing breaks. And not rush it to Linus now.
Thx.
Signature · 4 lines
Regards/Gruss,
Boris.
https://people.kernel.org/tglx/notes-about-netiquetteHello,
Quoted 4 lines · Borislav Petkov
Interesting. I thought gcc doesn't have problems here yet and was talking to Matz on Thu about it and it seems he's forgotten about his statement too that "you should simply stop using -mcmodel=large. Noone should use it." :-)
It may be so ingrained in my brain that I'm not _always_ saying it when talking about the large code model over a beer. And indeed I know of no particular problems with it vis GCC, but that doesn't mean it's a good idea to use :-)
So once again: "everyone should simply stop using -mcmodel=large. Noone should use it."
So the patch goes strictly into the direction of betterment of the universe. :)
Ciao,
Michael.
Borislav Petkov
maintainerQuoted 2 lines · Michael Matz
It may be so ingrained in my brain that I'm not _always_ saying it when talking about the large code model over a beer.
Doh, you should. This is what you start with! :-P
Quoted 8 lines
And indeed I know of no particular problems with it vis GCC, but that doesn't mean it's a good idea to use :-)
So once again: "everyone should simply stop using -mcmodel=large. Noone should use it."
So the patch goes strictly into the direction of betterment of the universe. :)
Yeah, it is already on its way to every kernel near you. And looka here:
$ git grep mcmodel=large arch/powerpc/Makefile:125: # 64bit relocation for this to work, hence -mcmodel=large. arch/powerpc/Makefile:126: KBUILD_CFLAGS_MODULE += -mcmodel=large arch/um/Makefile:34: KBUILD_CFLAGS += -mcmodel=large
x86 is all free of the large model now.
One less thing to worry about - gazillion more to go.
:-P
See ya on Thu.
Signature · 4 lines
Regards/Gruss,
Boris.
https://people.kernel.org/tglx/notes-about-netiquetteHi,
Quoted 10 lines · Michael Matz
Quoted 4 lines · Borislav Petkov
Interesting. I thought gcc doesn't have problems here yet and was talking to Matz on Thu about it and it seems he's forgotten about his statement too that "you should simply stop using -mcmodel=large. Noone should use it." :-)
It may be so ingrained in my brain that I'm not _always_ saying it when talking about the large code model over a beer. And indeed I know of no particular problems with it vis GCC,
Of course you do :). That bsc#1211853 I linked earlier. I.e. gcc-13 + -fstrict-flex-arrays=3 + -mcmodel=large + some asm() expecting __FILE__ to be constant (not true with the large model).
regards,
Signature · 2 lines
js suse labs
Hello,
Quoted 10 lines · Jiri Slaby
Quoted 8 lines
Quoted 4 lines
Interesting. I thought gcc doesn't have problems here yet and was talking to Matz on Thu about it and it seems he's forgotten about his statement too that "you should simply stop using -mcmodel=large. Noone should use it." :-)
It may be so ingrained in my brain that I'm not _always_ saying it when talking about the large code model over a beer. And indeed I know of no particular problems with it vis GCC,
Of course you do :).
:-P
Quoted 3 lines
That bsc#1211853 I linked earlier. I.e. gcc-13 + -fstrict-flex-arrays=3 + -mcmodel=large + some
asm()expecting__FILE__to be constant (not true with the large model).
"asm() expecting $whatever" - clearly a user problem, not a GCC problem ;-)
Ciao,
Michael.