Skip to contentSkip to conversation

kcov, objtool: Make runtime functions noinstr-compatible

PATCH Superseded Marco Elver 3 participants 5 messages Jun 4, 2020 – Aug 12, 2021

Subsystems (2)

v2 of 3 Superseded Latest is v3 →

lore
Share & tools
lore
#0 lore raw

PATCH v2 1/2 kcov, objtool: Make runtime functions noinstr-compatible

While we lack a compiler attribute to add to noinstr that would disable KCOV, make the KCOV runtime functions return if the caller is in a noinstr section. We then whitelist __sanitizer_cov_*() functions in objtool. __sanitizer_cov_*() cannot safely become safe noinstr functions as-is, as they may fault due to accesses to vmalloc's memory.

Declare write_comp_data() as __always_inline to ensure it is inlined, and reduce stack usage and remove one extra call from the fast-path.

In future, our compilers may provide an attribute to implement __no_sanitize_coverage, which can then be added to noinstr, and the checks added in this patch can be guarded by an #ifdef checking if the compiler has such an attribute or not.

---
Apply after:
https://lkml.kernel.org/r/[email protected]

v2:
* Rewrite based on Peter's and Andrey's feedback -- v1 worked because we got lucky. Let's not rely on luck, as it will be difficult to ensure the same conditions remain true in future.

v1: https://lkml.kernel.org/r/[email protected]

Note: There are a set of KCOV patches from Andrey in -next:
https://lkml.kernel.org/r/[email protected] -- Git cleanly merges this patch with those patches, and no merge conflict is expected.

---
 kernel/kcov.c         | 19 +++++++++++++++++--
 tools/objtool/check.c |  7 +++++++
 2 files changed, 24 insertions(+), 2 deletions(-)
kernel/kcov.c +17 -2
index 8accc9722a81..3329a0fdb868 100644
@@ -24,6 +24,7 @@
 #include <linux/refcount.h>
 #include <linux/log2.h>
 #include <asm/setup.h>
+#include <asm/sections.h>
 
 #define kcov_debug(fmt, ...) pr_debug("%s: " fmt, __func__, ##__VA_ARGS__)
 
@@ -172,6 +173,12 @@ static notrace unsigned long canonicalize_ip(unsigned long ip)
 	return ip;
 }
 
+static __always_inline bool in_noinstr_section(unsigned long ip)
+{
+	return (unsigned long)__noinstr_text_start <= ip &&
+	       ip < (unsigned long)__noinstr_text_end;
+}
+
 /*
  * Entry point from instrumented code.
  * This is called once per basic-block/edge.
@@ -180,13 +187,18 @@ void notrace __sanitizer_cov_trace_pc(void)
 {
 	struct task_struct *t;
 	unsigned long *area;
-	unsigned long ip = canonicalize_ip(_RET_IP_);
+	unsigned long ip;
 	unsigned long pos;
 
+	if (unlikely(in_noinstr_section(_RET_IP_)))
+		return;
+
 	t = current;
 	if (!check_kcov_mode(KCOV_MODE_TRACE_PC, t))
 		return;
 
+	ip = canonicalize_ip(_RET_IP_);
+
 	area = t->kcov_area;
 	/* The first 64-bit word is the number of subsequent PCs. */
 	pos = READ_ONCE(area[0]) + 1;
@@ -198,12 +210,15 @@ void notrace __sanitizer_cov_trace_pc(void)
 EXPORT_SYMBOL(__sanitizer_cov_trace_pc);
 
 #ifdef CONFIG_KCOV_ENABLE_COMPARISONS
-static void notrace write_comp_data(u64 type, u64 arg1, u64 arg2, u64 ip)
+static __always_inline void write_comp_data(u64 type, u64 arg1, u64 arg2, u64 ip)
 {
 	struct task_struct *t;
 	u64 *area;
 	u64 count, start_index, end_pos, max_pos;
 
+	if (unlikely(in_noinstr_section(ip)))
+		return;
+
 	t = current;
 	if (!check_kcov_mode(KCOV_MODE_TRACE_CMP, t))
 		return;
tools/objtool/check.c +7 -0
index 3e214f879ada..cb208959f560 100644
@@ -2213,6 +2213,13 @@ static inline bool noinstr_call_dest(struct symbol *func)
 	if (!strncmp(func->name, "__ubsan_handle_", 15))
 		return true;
 
+	/*
+	 * The __sanitizer_cov_*() calls include a check if the caller is in the
+	 * noinstr section, and simply return if that is the case.
+	 */
+	if (!strncmp(func->name, "__sanitizer_cov_", 16))
+		return true;
+
 	return false;
 }
 
Signature · 1 line
2.27.0.rc2.251.g90737beb825-goog
#1 lore raw

PATCH v2 2/2 kcov: Pass -fno-stack-protector with Clang

For Clang, correctly pass -fno-stack-protector via a separate cc-option, as -fno-conserve-stack does not exist with Clang.

---
 kernel/Makefile | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
kernel/Makefile +1 -1
index ce8716a04d0e..82153c47d2a6 100644
@@ -35,7 +35,7 @@ KCOV_INSTRUMENT_stacktrace.o := n
 KCOV_INSTRUMENT_kcov.o := n
 KASAN_SANITIZE_kcov.o := n
 KCSAN_SANITIZE_kcov.o := n
-CFLAGS_kcov.o := $(call cc-option, -fno-conserve-stack -fno-stack-protector)
+CFLAGS_kcov.o := $(call cc-option, -fno-conserve-stack) $(call cc-option, -fno-stack-protector)
 
 # cond_syscall is currently not LTO compatible
 CFLAGS_sys_ni.o = $(DISABLE_LTO)
Signature · 1 line
2.27.0.rc2.251.g90737beb825-goog
#2 lore raw

Re: [PATCH v2 2/2] kcov: Pass -fno-stack-protector with Clang

Quoted 14 lines · Marco Elver' via Clang Built Linux

For Clang, correctly pass -fno-stack-protector via a separate cc-option, as -fno-conserve-stack does not exist with Clang.

---
 kernel/Makefile | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
kernel/Makefile +0 -0
index ce8716a04d0e..82153c47d2a6 100644

kernel/Makefile KCOV_INSTRUMENT_stacktrace.o := n · line 38

Quoted 5 lines
kernel/Makefile +1 -1
@@ -35,7 +35,7 @@ KCOV_INSTRUMENT_stacktrace.o := n
 KCOV_INSTRUMENT_kcov.o := n
 KASAN_SANITIZE_kcov.o := n
 KCSAN_SANITIZE_kcov.o := n
-CFLAGS_kcov.o := $(call cc-option, -fno-conserve-stack -fno-stack-protector)
+CFLAGS_kcov.o := $(call cc-option, -fno-conserve-stack) $(call cc-option, -fno-stack-protector)

All supported compiler versions understand -fno-stack-protector. Please unwrap it from cc-option. That's one less compiler invocation at build time.

Quoted 4 lines
kernel/Makefile +0 -1
 # cond_syscall is currently not LTO compatible
 CFLAGS_sys_ni.o = $(DISABLE_LTO)
--
Signature · 2 lines
Thanks,
~Nick Desaulniers
maintainer
#3 lore raw
In reply to Marco Elver
Quoted 7 lines · Marco Elver

While we lack a compiler attribute to add to noinstr that would disable KCOV, make the KCOV runtime functions return if the caller is in a noinstr section. We then whitelist __sanitizer_cov_*() functions in objtool.

__sanitizer_cov_*() cannot safely become safe noinstr functions as-is, as they may fault due to accesses to vmalloc's memory.

I would feel very much better with those actually in noinstr, because without it, there is nothing stopping us from adding a kprobe/hw- breakpoint or other funny to the function.

Even if they almost instra-return, having a kprobe on the function entry or condition check is enough to utterly wreck things.

So something like:

void noinstr __sanitizer_cov_trace_*(...)

{
	if (within_noinstr_section(ip))
		return;

	instrumentation_begin();
	write_comp_data(...);
	instrumentation_end();
}

Would make me feel a whole lot better.

kernel/kcov.c static notrace unsigned long canonicalize_ip(unsigned long ip) · line 180

Quoted 5 lines
kernel/kcov.c +5 -0
+static __always_inline bool in_noinstr_section(unsigned long ip)
+{
+	return (unsigned long)__noinstr_text_start <= ip &&
+	       ip < (unsigned long)__noinstr_text_end;
+}

.entry.text is also considered noinstr, although I suppose that all being in .S files avoids it having annotations inserted, but perhaps a comment?

#4 lore raw

[-Cc most]
[+Cc Mark]

On Thu, 4 Jun 2020 at 17:25, Peter Zijlstra <[email protected]> wrote:
[...]

Quoted 10 lines

I would feel very much better with those actually in noinstr, because without it, there is nothing stopping us from adding a kprobe/hw- breakpoint or other funny to the function.

Even if they almost instra-return, having a kprobe on the function entry or condition check is enough to utterly wreck things.

So something like:

void noinstr __sanitizer_cov_trace_*(...)

Quoted 8 lines
{
        if (within_noinstr_section(ip))
                return;

        instrumentation_begin();
        write_comp_data(...);
        instrumentation_end();
}

Apologies for resurrecting this. :-)

It seems I'll need to use this approach soon for upcoming KCSAN instrumentation for memory barriers. I'm able to use the same objtool feature that erases __sanitizer_cov* calls on x86 to erase memory barrier instrumentation, but arm64 will still be a problem because of lack of objtool support.

Mark, on arm64, is the approach above that Peter proposed ~1y ago acceptable in general to make instrumentation noinstr-safe?

Thanks,
-- Marco