Skip to contentSkip to conversation

x86/purgatory: do not use __builtin_memcpy and __builtin_memset

PATCH Superseded Nick Desaulniers 2 participants 8 messages Jul 25, 2019 – Aug 7, 2019 x86

v4 of 5 Superseded Latest is v5 →

lore
Share & tools
lore
#0 lore raw

PATCH v4 1/2 x86/purgatory: do not use __builtin_memcpy and __builtin_memset

Implementing memcpy and memset in terms of __builtin_memcpy and __builtin_memset is problematic.

GCC at -O2 will replace calls to the builtins with calls to memcpy and memset (but will generate an inline implementation at -Os). Clang will replace the builtins with these calls regardless of optimization level.

$ llvm-objdump -dr arch/x86/purgatory/string.o | tail

0000000000000339 memcpy:

     339: 48 b8 00 00 00 00 00 00 00 00 movabsq $0, %rax
                000000000000033b:  R_X86_64_64  memcpy
     343: ff e0                         jmpq    *%rax

0000000000000345 memset:

     345: 48 b8 00 00 00 00 00 00 00 00 movabsq $0, %rax
                0000000000000347:  R_X86_64_64  memset
     34f: ff e0

Such code results in infinite recursion at runtime. This is observed when doing kexec.

Instead, reuse an implementation from arch/x86/boot/compressed/string.c if we define warn as a symbol. Also, Clang may lower memcmp's that compare against 0 to bcmp's, so add a small definition, too. See also:
commit 5f074f3e192f ("lib/string.c: implement a basic bcmp")

---
Changes v3 -> v4:
* (style) open brace on newline
* drop Vaibhav's SOB tag that was accidentally copy+pasta'd from v1.
* Carry Vaibhav's tested by tag from v3 since v4 is strictly stylistic change from v3.
* Drop cc'ing stable. Sasha's bot reports v1 doesn't cherry pick cleanly 5.1, so this series will require manual backports.
Changes v2 -> v3:
* Add bcmp implementation.
* Drop tested-by tag (Vaibhav will help retest).
* Cc stable
Changes v1 -> v2:
* Add Fixes tag.
* Move this patch to first in the series.

 arch/x86/boot/string.c         |  8 ++++++++
 arch/x86/purgatory/Makefile    |  3 +++
 arch/x86/purgatory/purgatory.c |  6 ++++++
 arch/x86/purgatory/string.c    | 23 -----------------------
 4 files changed, 17 insertions(+), 23 deletions(-)
 delete mode 100644 arch/x86/purgatory/string.c
arch/x86/boot/string.c +8 -0
index 401e30ca0a75..8272a4492844 100644
@@ -37,6 +37,14 @@ int memcmp(const void *s1, const void *s2, size_t len)
 	return diff;
 }
 
+/*
+ * Clang may lower `memcmp == 0` to `bcmp == 0`.
+ */
+int bcmp(const void *s1, const void *s2, size_t len)
+{
+	return memcmp(s1, s2, len);
+}
+
 int strcmp(const char *str1, const char *str2)
 {
 	const unsigned char *s1 = (const unsigned char *)str1;
arch/x86/purgatory/Makefile +3 -0
index 3cf302b26332..91ef244026d2 100644
@@ -6,6 +6,9 @@ purgatory-y := purgatory.o stack.o setup-x86_$(BITS).o sha256.o entry64.o string
 targets += $(purgatory-y)
 PURGATORY_OBJS = $(addprefix $(obj)/,$(purgatory-y))
 
+$(obj)/string.o: $(srctree)/arch/x86/boot/compressed/string.c FORCE
+	$(call if_changed_rule,cc_o_c)
+
 $(obj)/sha256.o: $(srctree)/lib/sha256.c FORCE
 	$(call if_changed_rule,cc_o_c)
 
arch/x86/purgatory/purgatory.c +6 -0
index 6d8d5a34c377..b607bda786f6 100644
@@ -68,3 +68,9 @@ void purgatory(void)
 	}
 	copy_backup_region();
 }
+
+/*
+ * Defined in order to reuse memcpy() and memset() from
+ * arch/x86/boot/compressed/string.c
+ */
+void warn(const char *msg) {}
arch/x86/purgatory/string.c deleted +0 -23
deleted file mode 100644
index 01ad43873ad9..000000000000
@@ -1,23 +0,0 @@
-// SPDX-License-Identifier: GPL-2.0-only
-/*
- * Simple string functions.
- *
- * Copyright (C) 2014 Red Hat Inc.
- *
- * Author:
- *       Vivek Goyal <[email protected]>
- */
-
-#include <linux/types.h>
-
-#include "../boot/string.c"
-
-void *memcpy(void *dst, const void *src, size_t len)
-{
-	return __builtin_memcpy(dst, src, len);
-}
-
-void *memset(void *dst, int c, size_t len)
-{
-	return __builtin_memset(dst, c, len);
-}
Signature · 1 line
2.22.0.709.g102302147b-goog
#1 lore raw

PATCH v4 2/2 x86/purgatory: use CFLAGS_REMOVE rather than reset KBUILD_CFLAGS

KBUILD_CFLAGS is very carefully built up in the top level Makefile, particularly when cross compiling or using different build tools. Resetting KBUILD_CFLAGS via := assignment is an antipattern.

The comment above the reset mentions that -pg is problematic. Other Makefiles use CFLAGS_REMOVE_file.o = $(CC_FLAGS_FTRACE) when CONFIG_FUNCTION_TRACER is set. Prefer that pattern to wiping out all of the important KBUILD_CFLAGS then manually having to re-add them. Seems also that __stack_chk_fail references are generated when using CONFIG_STACKPROTECTOR or CONFIG_STACKPROTECTOR_STRONG.

Fixes 8fc5b4d4121c ("purgatory: core purgatory functionality")

---
Changes v3 -> v4:
* Use tabs to align flags (stylistic change only).
* Drop stable tag, patch 01/02 doesn't apply earlier than 5.1.
* Add tglx's suggested by tag for the tabs.
* Carry Vaibhav's tested by tag from v3 since v4 is simply stylistic. Changes v2 -> v3:
* Prefer $(CC_FLAGS_FTRACE) which is exported to -pg.
* Also check CONFIG_STACKPROTECTOR and CONFIG_STACKPROTECTOR_STRONG.
* Cc stable.
Changes v1 -> v2:
Rather than manually add -mno-sse, -mno-mmx, -mno-sse2, prefer to filter -pg flags.

 arch/x86/purgatory/Makefile | 26 +++++++++++++++++++++-----
 1 file changed, 21 insertions(+), 5 deletions(-)
arch/x86/purgatory/Makefile +21 -5
index 91ef244026d2..940f043a4d97 100644
@@ -20,11 +20,27 @@ KCOV_INSTRUMENT := n
 
 # Default KBUILD_CFLAGS can have -pg option set when FTRACE is enabled. That
 # in turn leaves some undefined symbols like __fentry__ in purgatory and not
-# sure how to relocate those. Like kexec-tools, use custom flags.
-
-KBUILD_CFLAGS := -fno-strict-aliasing -Wall -Wstrict-prototypes -fno-zero-initialized-in-bss -fno-builtin -ffreestanding -c -Os -mcmodel=large
-KBUILD_CFLAGS += -m$(BITS)
-KBUILD_CFLAGS += $(call cc-option,-fno-PIE)
+# sure how to relocate those.
+ifdef CONFIG_FUNCTION_TRACER
+CFLAGS_REMOVE_sha256.o		+= $(CC_FLAGS_FTRACE)
+CFLAGS_REMOVE_purgatory.o	+= $(CC_FLAGS_FTRACE)
+CFLAGS_REMOVE_string.o		+= $(CC_FLAGS_FTRACE)
+CFLAGS_REMOVE_kexec-purgatory.o	+= $(CC_FLAGS_FTRACE)
+endif
+
+ifdef CONFIG_STACKPROTECTOR
+CFLAGS_REMOVE_sha256.o		+= -fstack-protector
+CFLAGS_REMOVE_purgatory.o	+= -fstack-protector
+CFLAGS_REMOVE_string.o		+= -fstack-protector
+CFLAGS_REMOVE_kexec-purgatory.o	+= -fstack-protector
+endif
+
+ifdef CONFIG_STACKPROTECTOR_STRONG
+CFLAGS_REMOVE_sha256.o		+= -fstack-protector-strong
+CFLAGS_REMOVE_purgatory.o	+= -fstack-protector-strong
+CFLAGS_REMOVE_string.o		+= -fstack-protector-strong
+CFLAGS_REMOVE_kexec-purgatory.o	+= -fstack-protector-strong
+endif
 
 $(obj)/purgatory.ro: $(PURGATORY_OBJS) FORCE
 		$(call if_changed,ld)
Signature · 1 line
2.22.0.709.g102302147b-goog
#2 lore raw

Re: [PATCH v4 2/2] x86/purgatory: use CFLAGS_REMOVE rather than reset KBUILD_CFLAGS

Quoted 10 lines · Nick Desaulniers

KBUILD_CFLAGS is very carefully built up in the top level Makefile, particularly when cross compiling or using different build tools. Resetting KBUILD_CFLAGS via := assignment is an antipattern.

The comment above the reset mentions that -pg is problematic. Other Makefiles use CFLAGS_REMOVE_file.o = $(CC_FLAGS_FTRACE) when CONFIG_FUNCTION_TRACER is set. Prefer that pattern to wiping out all of the important KBUILD_CFLAGS then manually having to re-add them. Seems also that __stack_chk_fail references are generated when using CONFIG_STACKPROTECTOR or CONFIG_STACKPROTECTOR_STRONG.

Looking at the resulting build flags. Most stuff looks correct but there are a few which need to be looked at twice.

removes:

-ffreestanding
-fno-builtin
-fno-zero-initialized-in-bss

changes:

-mcmodel=large to -mcmodel=kernel

adds:

-mindirect-branch-register
-mindirect-branch=thunk-extern

The latter makes me nervous. That probably wants to have retpoline disabled as well. It's not having an instance right now, but ...

Thanks,

tglx

#3 lore raw

Re: [PATCH v4 2/2] x86/purgatory: use CFLAGS_REMOVE rather than reset KBUILD_CFLAGS

On Fri, 26 Jul 2019, Thomas Gleixner wrote:

Ping...

Quoted 38 lines · Nick Desaulniers
Quoted 11 lines

KBUILD_CFLAGS is very carefully built up in the top level Makefile, particularly when cross compiling or using different build tools. Resetting KBUILD_CFLAGS via := assignment is an antipattern.

The comment above the reset mentions that -pg is problematic. Other Makefiles use CFLAGS_REMOVE_file.o = $(CC_FLAGS_FTRACE) when CONFIG_FUNCTION_TRACER is set. Prefer that pattern to wiping out all of the important KBUILD_CFLAGS then manually having to re-add them. Seems also that __stack_chk_fail references are generated when using CONFIG_STACKPROTECTOR or CONFIG_STACKPROTECTOR_STRONG.

Looking at the resulting build flags. Most stuff looks correct but there are a few which need to be looked at twice.

removes:

-ffreestanding
-fno-builtin
-fno-zero-initialized-in-bss

changes:

-mcmodel=large to -mcmodel=kernel

adds:

-mindirect-branch-register
-mindirect-branch=thunk-extern

The latter makes me nervous. That probably wants to have retpoline disabled as well. It's not having an instance right now, but ...

Thanks,

tglx

#4 lore raw

v4 cover Support kexec/kdump for clang built kernel

In reply to Nick Desaulniers

1. Reuse the implementation of memcpy and memset instead of relying on __builtin_memcpy and __builtin_memset as it causes infinite recursion in Clang (at any opt level) or GCC at -O2.
2. Don't reset KBUILD_CFLAGS, rather filter CONFIG_FUNCTION_TRACER, CONFIG_STACKPROTECTOR, and CONFIG_STACKPROTECTOR_STRONG flags via `CFLAGS_REMOVE_<file>.o'

A good test of this series (besides boot testing a kexec kernel):
* There should be no undefined symbols in arch/x86/purgatory/purgatory.ro:

$ nm arch/x86/purgatory/purgatory.ro
  particularly `warn`, `bcmp`, `__stack_chk_fail`, `memcpy` or `memset`.
* `-pg`, `-fstack-protector`, `-fstack-protector-strong` should not be
  added to the command line for the c source files under arch/x86/purgatory/
  when compiling with CONFIG_FUNCTION_TRACER=y, CONFIG_STACKPROTECTOR=y,
  and CONFIG_STACKPROTECTOR_STRONG=y.

V4 of: https://lkml.org/lkml/2019/7/23/864

Nick Desaulniers (2):
x86/purgatory: do not use __builtin_memcpy and __builtin_memset x86/purgatory: use CFLAGS_REMOVE rather than reset KBUILD_CFLAGS

 arch/x86/boot/string.c         |  7 +++++++
 arch/x86/purgatory/Makefile    | 29 ++++++++++++++++++++++++-----
 arch/x86/purgatory/purgatory.c |  6 ++++++
 arch/x86/purgatory/string.c    | 23 -----------------------
 4 files changed, 37 insertions(+), 28 deletions(-)
 delete mode 100644 arch/x86/purgatory/string.c
Signature · 1 line
2.22.0.709.g102302147b-goog
#5 lore raw

Re: [PATCH v4 0/2] Support kexec/kdump for clang built kernel

On Thu, 25 Jul 2019, Nick Desaulniers wrote:

I'm really impressed how you manage to make the cover letter (0/N) a reply to 1/N instead of 1..N/N being a reply to 0/N.

In-Reply-To: <[email protected]>
Message-Id: <[email protected]>

Is that a new git feature to be $corp top-posting compliant?

Quoted 9 lines

1. Reuse the implementation of memcpy and memset instead of relying on __builtin_memcpy and __builtin_memset as it causes infinite recursion in Clang (at any opt level) or GCC at -O2.
2. Don't reset KBUILD_CFLAGS, rather filter CONFIG_FUNCTION_TRACER, CONFIG_STACKPROTECTOR, and CONFIG_STACKPROTECTOR_STRONG flags via `CFLAGS_REMOVE_<file>.o'

A good test of this series (besides boot testing a kexec kernel):
* There should be no undefined symbols in arch/x86/purgatory/purgatory.ro:

Quoted 8 lines
$ nm arch/x86/purgatory/purgatory.ro
  particularly `warn`, `bcmp`, `__stack_chk_fail`, `memcpy` or `memset`.
* `-pg`, `-fstack-protector`, `-fstack-protector-strong` should not be
  added to the command line for the c source files under arch/x86/purgatory/
  when compiling with CONFIG_FUNCTION_TRACER=y, CONFIG_STACKPROTECTOR=y,
  and CONFIG_STACKPROTECTOR_STRONG=y.

V4 of: https://lkml.org/lkml/2019/7/23/864

Please don't use lkml.org references. I know it's popular but equally unreliable at times.

The long term reliable reference is message id based, i.e.:

lkml.kernel.org/r/$MSGID

or

lore.kernel.org/lkml/$MSGID

even if the base URLs would cease to exist, the message id will give you a trivial way to find the relevant thread, but if '2019/7/23/864' stops to work, good luck in finding the original post. I wasted hours on that just because a subject line changed enough to confuse the big internet stalking machines.

Thanks,

tglx

#6 lore raw

Re: [PATCH v4 0/2] Support kexec/kdump for clang built kernel

Quoted 10 lines · Thomas Gleixner

On Thu, 25 Jul 2019, Nick Desaulniers wrote:

I'm really impressed how you manage to make the cover letter (0/N) a reply to 1/N instead of 1..N/N being a reply to 0/N.

In-Reply-To: <[email protected]>
Message-Id: <[email protected]>

Is that a new git feature to be $corp top-posting compliant?

It appears to be a hidden bonus feature of:

$ git-send-email purgatory/v4-000*
Quoted 4 lines
Quoted 1 line

V4 of: https://lkml.org/lkml/2019/7/23/864

Please don't use lkml.org references. I know it's popular but equally unreliable at times.

Oh?

Quoted 7 lines

The long term reliable reference is message id based, i.e.:

lkml.kernel.org/r/$MSGID

or

Quoted 7 lines

lore.kernel.org/lkml/$MSGID

even if the base URLs would cease to exist, the message id will give you a trivial way to find the relevant thread, but if '2019/7/23/864' stops to work, good luck in finding the original post. I wasted hours on that just because a subject line changed enough to confuse the big internet stalking machines.

Thanks for the tips; I'll try to use lore.kernel.org going forward.

Signature · 2 lines
Thanks,
~Nick Desaulniers
#7 lore raw

Re: [PATCH v4 0/2] Support kexec/kdump for clang built kernel

Quoted 6 lines · Nick Desaulniers
Quoted 5 lines · Thomas Gleixner

On Thu, 25 Jul 2019, Nick Desaulniers wrote:

I'm really impressed how you manage to make the cover letter (0/N) a reply to 1/N instead of 1..N/N being a reply to 0/N.

Quoted 7 lines
Quoted 4 lines

In-Reply-To: <[email protected]>
Message-Id: <[email protected]>

Is that a new git feature to be $corp top-posting compliant?

It appears to be a hidden bonus feature of:

$ git-send-email purgatory/v4-000*

Care to poke the git folks?

Quoted 6 lines
Quoted 4 lines
Quoted 1 line

V4 of: https://lkml.org/lkml/2019/7/23/864

Please don't use lkml.org references. I know it's popular but equally unreliable at times.

Oh?

404 or silent fail are happening on a regular base and of course always when you really need that information. Murphy's law ...

Thanks,

tglx