x86/purgatory: do not use __builtin_memcpy and __builtin_memset
Subsystems (2)
PATCH v3 1/2 x86/purgatory: do not use __builtin_memcpy and __builtin_memset
Implementing memcpy and memset in terms of __builtin_memcpy and __builtin_memset is problematic.
GCC at -O2 will replace calls to the builtins with calls to memcpy and memset (but will generate an inline implementation at -Os). Clang will replace the builtins with these calls regardless of optimization level.
$ llvm-objdump -dr arch/x86/purgatory/string.o | tail
0000000000000339 memcpy:
339: 48 b8 00 00 00 00 00 00 00 00 movabsq $0, %rax
000000000000033b: R_X86_64_64 memcpy
343: ff e0 jmpq *%rax0000000000000345 memset:
345: 48 b8 00 00 00 00 00 00 00 00 movabsq $0, %rax
0000000000000347: R_X86_64_64 memset
34f: ff e0Such code results in infinite recursion at runtime. This is observed when doing kexec.
Instead, reuse an implementation from arch/x86/boot/compressed/string.c if we define warn as a symbol. Also, Clang may lower memcmp's that compare against 0 to bcmp's, so add a small definition, too. See also:
commit 5f074f3e192f ("lib/string.c: implement a basic bcmp")
---
Changes v2 -> v3:
* Add bcmp implementation.
* Drop tested-by tag (Vaibhav will help retest).
* Cc stable
Changes v1 -> v2:
* Add Fixes tag.
* Move this patch to first in the series.
arch/x86/boot/string.c | 7 +++++++ arch/x86/purgatory/Makefile | 3 +++ arch/x86/purgatory/purgatory.c | 6 ++++++ arch/x86/purgatory/string.c | 23 ----------------------- 4 files changed, 16 insertions(+), 23 deletions(-) delete mode 100644 arch/x86/purgatory/string.c
arch/x86/boot/string.c +7 -0
@@ -37,6 +37,13 @@ int memcmp(const void *s1, const void *s2, size_t len)return diff; } +/* +* Clang may lower `memcmp == 0` to `bcmp == 0`. +*/ +int bcmp (const void *s1,const void *s2,size_t len) { +return memcmp (s1, s2, len); +} +int strcmp (const char *str1,const char *str2) {const unsigned char *s1 = (const unsigned char *)str1;
arch/x86/purgatory/Makefile +3 -0
@@ -6,6 +6,9 @@ purgatory-y := purgatory.o stack.o setup-x86_$(BITS).o sha256.o entry64.o string targets += $(purgatory-y) PURGATORY_OBJS = $(addprefix $(obj)/,$(purgatory-y)) +$(obj)/string.o: $(srctree)/arch/x86/boot/compressed/string.c FORCE + $(call if_changed_rule,cc_o_c) + $(obj)/sha256.o: $(srctree)/lib/sha256.c FORCE $(call if_changed_rule,cc_o_c)
arch/x86/purgatory/purgatory.c +6 -0
@@ -68,3 +68,9 @@ void purgatory(void) }copy_backup_region (); } + +/* +* Defined in order to reuse memcpy() and memset() from +* arch/x86/boot/compressed/string.c +*/ +void warn (const char *msg) {}
arch/x86/purgatory/string.c deleted +0 -23
@@ -1,23 +0,0 @@ -// SPDX-License-Identifier: GPL-2.0-only -/* -* Simple string functions. -* -* Copyright (C) 2014 Red Hat Inc. -* -* Author: -* Vivek Goyal <[email protected]> -*/ - -#include <linux/types.h> - -#include "../boot/string.c" - -void *memcpy (void *dst,const void *src,size_t len) -{ -return __builtin_memcpy (dst, src, len); -} - -void *memset (void *dst,int c,size_t len) -{ -return __builtin_memset (dst, c, len); -}
Signature · 1 line
2.22.0.709.g102302147b-goog
PATCH v3 2/2 x86/purgatory: use CFLAGS_REMOVE rather than reset KBUILD_CFLAGS
KBUILD_CFLAGS is very carefully built up in the top level Makefile, particularly when cross compiling or using different build tools. Resetting KBUILD_CFLAGS via := assignment is an antipattern.
The comment above the reset mentions that -pg is problematic. Other Makefiles use CFLAGS_REMOVE_file.o = $(CC_FLAGS_FTRACE) when CONFIG_FUNCTION_TRACER is set. Prefer that pattern to wiping out all of the important KBUILD_CFLAGS then manually having to re-add them. Seems also that __stack_chk_fail references are generated when using CONFIG_STACKPROTECTOR or CONFIG_STACKPROTECTOR_STRONG.
---
Alternatively, we could put these in all in one variable and remove it without any conditional checks (I think that's ok to do so with CFLAGS_REMOVE).
Changes v2 -> v3:
* Prefer $(CC_FLAGS_FTRACE) which is exported to -pg.
* Also check CONFIG_STACKPROTECTOR and CONFIG_STACKPROTECTOR_STRONG.
* Cc stable.
Changes v1 -> v2:
Rather than manually add -mno-sse, -mno-mmx, -mno-sse2, prefer to filter -pg flags.
arch/x86/purgatory/Makefile | 26 +++++++++++++++++++++----- 1 file changed, 21 insertions(+), 5 deletions(-)
arch/x86/purgatory/Makefile +21 -5
@@ -20,11 +20,27 @@ KCOV_INSTRUMENT := n # Default KBUILD_CFLAGS can have -pg option set when FTRACE is enabled. That # in turn leaves some undefined symbols like __fentry__ in purgatory and not -# sure how to relocate those. Like kexec-tools, use custom flags. - -KBUILD_CFLAGS := -fno-strict-aliasing -Wall -Wstrict-prototypes -fno-zero-initialized-in-bss -fno-builtin -ffreestanding -c -Os -mcmodel=large -KBUILD_CFLAGS += -m$(BITS) -KBUILD_CFLAGS += $(call cc-option,-fno-PIE) +# sure how to relocate those. +ifdef CONFIG_FUNCTION_TRACER +CFLAGS_REMOVE_sha256.o += $(CC_FLAGS_FTRACE) +CFLAGS_REMOVE_purgatory.o += $(CC_FLAGS_FTRACE) +CFLAGS_REMOVE_string.o += $(CC_FLAGS_FTRACE) +CFLAGS_REMOVE_kexec-purgatory.o += $(CC_FLAGS_FTRACE) +endif + +ifdef CONFIG_STACKPROTECTOR +CFLAGS_REMOVE_sha256.o += -fstack-protector +CFLAGS_REMOVE_purgatory.o += -fstack-protector +CFLAGS_REMOVE_string.o += -fstack-protector +CFLAGS_REMOVE_kexec-purgatory.o += -fstack-protector +endif + +ifdef CONFIG_STACKPROTECTOR_STRONG +CFLAGS_REMOVE_sha256.o += -fstack-protector-strong +CFLAGS_REMOVE_purgatory.o += -fstack-protector-strong +CFLAGS_REMOVE_string.o += -fstack-protector-strong +CFLAGS_REMOVE_kexec-purgatory.o += -fstack-protector-strong +endif $(obj)/purgatory.ro: $(PURGATORY_OBJS) FORCE $(call if_changed,ld)
Signature · 1 line
2.22.0.709.g102302147b-goog
Re: [PATCH v3 2/2] x86/purgatory: use CFLAGS_REMOVE rather than reset KBUILD_CFLAGS
arch/x86/purgatory/Makefile KCOV_INSTRUMENT := n · line 28
Quoted 5 lines · Nick Desaulniers
arch/x86/purgatory/Makefile +5
-0+ifdef CONFIG_FUNCTION_TRACER +CFLAGS_REMOVE_sha256.o += $(CC_FLAGS_FTRACE) +CFLAGS_REMOVE_purgatory.o += $(CC_FLAGS_FTRACE) +CFLAGS_REMOVE_string.o += $(CC_FLAGS_FTRACE) +CFLAGS_REMOVE_kexec-purgatory.o += $(CC_FLAGS_FTRACE)
Nit. Can you please make that tabular?
CFLAGS_REMOVE_sha256.o += $(CC_FLAGS_FTRACE)
CFLAGS_REMOVE_purgatory.o += $(CC_FLAGS_FTRACE)
CFLAGS_REMOVE_string.o += $(CC_FLAGS_FTRACE)
CFLAGS_REMOVE_kexec-purgatory.o += $(CC_FLAGS_FTRACE)
The above confused my pattern recognition engine :)
Thanks,
tglx
Re: [PATCH v3 2/2] x86/purgatory: use CFLAGS_REMOVE rather than reset KBUILD_CFLAGS
In reply to Nick DesaulniersQuoted 71 lines · Nick Desaulniers
KBUILD_CFLAGSis very carefully built up in the top levelMakefile, particularly when cross compiling or using different build tools. ResettingKBUILD_CFLAGSvia := assignment is an antipattern.The comment above the reset mentions that -pg is problematic. Other Makefiles use
CFLAGS_REMOVE_file.o = $(CC_FLAGS_FTRACE)whenCONFIG_FUNCTION_TRACERis set. Prefer that pattern to wiping out all of the importantKBUILD_CFLAGSthen manually having to re-add them. Seems also that__stack_chk_failreferences are generated when usingCONFIG_STACKPROTECTORorCONFIG_STACKPROTECTOR_STRONG.Fixes 8fc5b4d4121c ("purgatory: core purgatory functionality")Reported-by Vaibhav Rustagi <[email protected]>Suggested-by Peter Zijlstra <[email protected]>Signed-off-by Nick Desaulniers <[email protected]>---
Alternatively, we could put these in all in one variable and remove it without any conditional checks (I think that's ok to do so withCFLAGS_REMOVE).Changes v2 -> v3:
* Prefer $(CC_FLAGS_FTRACE) which is exported to -pg.
* Also checkCONFIG_STACKPROTECTORandCONFIG_STACKPROTECTOR_STRONG.
* Cc stable.
Changes v1 -> v2:
Rather than manually add -mno-sse, -mno-mmx, -mno-sse2, prefer to filter(quoted diff) +0
-1-pg flags. arch/x86/purgatory/Makefile | 26 +++++++++++++++++++++----- 1 file changed, 21 insertions(+), 5 deletions(-)arch/x86/purgatory/Makefile +21
-6@@ -20,11 +20,27 @@ KCOV_INSTRUMENT := n # Default KBUILD_CFLAGS can have -pg option set when FTRACE is enabled. That # in turn leaves some undefined symbols like __fentry__ in purgatory and not -# sure how to relocate those. Like kexec-tools, use custom flags. - -KBUILD_CFLAGS := -fno-strict-aliasing -Wall -Wstrict-prototypes -fno-zero-initialized-in-bss -fno-builtin -ffreestanding -c -Os -mcmodel=large -KBUILD_CFLAGS += -m$(BITS) -KBUILD_CFLAGS += $(call cc-option,-fno-PIE) +# sure how to relocate those. +ifdef CONFIG_FUNCTION_TRACER +CFLAGS_REMOVE_sha256.o += $(CC_FLAGS_FTRACE) +CFLAGS_REMOVE_purgatory.o += $(CC_FLAGS_FTRACE) +CFLAGS_REMOVE_string.o += $(CC_FLAGS_FTRACE) +CFLAGS_REMOVE_kexec-purgatory.o += $(CC_FLAGS_FTRACE) +endif + +ifdef CONFIG_STACKPROTECTOR +CFLAGS_REMOVE_sha256.o += -fstack-protector +CFLAGS_REMOVE_purgatory.o += -fstack-protector +CFLAGS_REMOVE_string.o += -fstack-protector +CFLAGS_REMOVE_kexec-purgatory.o += -fstack-protector +endif + +ifdef CONFIG_STACKPROTECTOR_STRONG +CFLAGS_REMOVE_sha256.o += -fstack-protector-strong +CFLAGS_REMOVE_purgatory.o += -fstack-protector-strong +CFLAGS_REMOVE_string.o += -fstack-protector-strong +CFLAGS_REMOVE_kexec-purgatory.o += -fstack-protector-strong +endif $(obj)/purgatory.ro: $(PURGATORY_OBJS) FORCE $(call if_changed,ld) --
Quoted 2 lines
2.22.0.709.g102302147b-goog
I tested the v3 patch series with clang compiled kernel for below scenarios:
1. kexec'ing into a new kernel.
2. Purposely crashing the running kernel to generate kdump logs.
Thanks,
Vaibhav
v3 cover Support kexec/kdump for clang built kernel
In reply to Nick Desaulniers1. Reuse the implementation of memcpy and memset instead of relying on __builtin_memcpy and __builtin_memset as it causes infinite recursion in Clang (at any opt level) or GCC at -O2.
2. Don't reset KBUILD_CFLAGS, rather filter CONFIG_FUNCTION_TRACER, CONFIG_STACKPROTECTOR, and CONFIG_STACKPROTECTOR_STRONG flags via `CFLAGS_REMOVE_<file>.o'
A good test of this series (besides boot testing a kexec kernel):
* There should be no undefined symbols in arch/x86/purgatory/purgatory.ro:
$ nm arch/x86/purgatory/purgatory.ro particularly `warn`, `bcmp`, `__stack_chk_fail`, `memcpy` or `memset`. * `-pg`, `-fstack-protector`, `-fstack-protector-strong` should not be added to the command line for the c source files under arch/x86/purgatory/ when compiling with CONFIG_FUNCTION_TRACER=y, CONFIG_STACKPROTECTOR=y, and CONFIG_STACKPROTECTOR_STRONG=y.
V3 of: https://groups.google.com/forum/#!msg/clang-built-linux/GzThkAkZqqI/J727I6vpBwAJ
Nick Desaulniers (2):
x86/purgatory: do not use __builtin_memcpy and __builtin_memset x86/purgatory: use CFLAGS_REMOVE rather than reset KBUILD_CFLAGS
arch/x86/boot/string.c | 7 +++++++ arch/x86/purgatory/Makefile | 29 ++++++++++++++++++++++++----- arch/x86/purgatory/purgatory.c | 6 ++++++ arch/x86/purgatory/string.c | 23 ----------------------- 4 files changed, 37 insertions(+), 28 deletions(-) delete mode 100644 arch/x86/purgatory/string.c
Signature · 1 line
2.22.0.709.g102302147b-goog
Quoted 5 lines · Nick Desaulniers
Instead, reuse an implementation from
arch/x86/boot/compressed/string.cif we define warn as a symbol. Also, Clang may lower memcmp's that compare against 0 to bcmp's, so add a small definition, too. See also:
commit5f074f3e192f("lib/string.c: implement a basic bcmp")
Quoted 9 lines
Fixes 8fc5b4d4121c ("purgatory: core purgatory functionality")Reported-by Vaibhav Rustagi <[email protected]>Debugged-by Vaibhav Rustagi <[email protected]>Debugged-by Manoj Gupta <[email protected]>Suggested-by Alistair Delva <[email protected]>Signed-off-by Vaibhav Rustagi <[email protected]>Signed-off-by Nick Desaulniers <[email protected]>
That SOB chain is weird. Is Vaibhav the author?
arch/x86/boot/string.c int memcmp(const void *s1, const void *s2, size_t len) · line 45
Quoted 6 lines
arch/x86/boot/string.c +6
-0+/* +* Clang may lower `memcmp == 0` to `bcmp == 0`. +*/ +int bcmp (const void *s1,const void *s2,size_t len) { +return memcmp (s1, s2, len); +}
foo()
{
}
please.
Thanks,
tglx
Quoted 17 lines · Thomas Gleixner
Quoted 5 lines · Nick Desaulniers
Instead, reuse an implementation from
arch/x86/boot/compressed/string.cif we define warn as a symbol. Also, Clang may lower memcmp's that compare against 0 to bcmp's, so add a small definition, too. See also:
commit5f074f3e192f("lib/string.c: implement a basic bcmp")Quoted 9 lines
Fixes 8fc5b4d4121c ("purgatory: core purgatory functionality")Reported-by Vaibhav Rustagi <[email protected]>Debugged-by Vaibhav Rustagi <[email protected]>Debugged-by Manoj Gupta <[email protected]>Suggested-by Alistair Delva <[email protected]>Signed-off-by Vaibhav Rustagi <[email protected]>Signed-off-by Nick Desaulniers <[email protected]>
Quoted 2 lines
That SOB chain is weird. Is Vaibhav the author?
No, Nick Desaulniers is the author of v3.
Quoted 11 lines
arch/x86/boot/string.c int memcmp(const void *s1, const void *s2, size_t len) · line 45
Quoted 6 lines
arch/x86/boot/string.c +6
-0+/* +* Clang may lower `memcmp == 0` to `bcmp == 0`. +*/ +int bcmp (const void *s1,const void *s2,size_t len) { +return memcmp (s1, s2, len); +}
foo()
{
}
Quoted 5 lines
please.
Thanks,
tglx
Thanks,
Vaibhav