Skip to contentSkip to conversation

kmsan: Enable on s390

PATCH Superseded Ilya Leoshkevich 4 participants 70 messages Nov 21, 2023 – Dec 13, 2023

Subsystems (14)

v2 of 7 Superseded Latest is v7 →

lore
Share & tools
lore
#0 lore raw

v2 cover kmsan: Enable on s390

v1: https://lore.kernel.org/lkml/[email protected]/
v1 -> v2: Add comments, sort #includes, introduce
memset_no_sanitize_memory() and use it to avoid unpoisoning of redzones, change vmalloc alignment to _REGION3_SIZE, add R-bs (Alexander P.).

Fix building
[PATCH 28/33] s390/string: Add KMSAN support with FORTIFY_SOURCE.

Hi,

This series provides the minimal support for Kernel Memory Sanitizer on s390. Kernel Memory Sanitizer is clang-only instrumentation for finding accesses to uninitialized memory. The clang support for s390 has already been merged [1].

With this series, I can successfully boot s390 defconfig and debug_defconfig with kmsan.panic=1. The tool found one real s390-specific bug (fixed in master).

Best regards,
Ilya

[1] https://reviews.llvm.org/D148596

Ilya Leoshkevich (33):
ftrace: Unpoison ftrace_regs in ftrace_ops_list_func()
kmsan: Make the tests compatible with kmsan.panic=1
kmsan: Disable KMSAN when DEFERRED_STRUCT_PAGE_INIT is enabled
kmsan: Increase the maximum store size to 4096
kmsan: Fix is_bad_asm_addr() on arches with overlapping address spaces
kmsan: Fix kmsan_copy_to_user() on arches with overlapping address
spaces
kmsan: Remove a useless assignment from
kmsan_vmap_pages_range_noflush()
kmsan: Remove an x86-specific #include from kmsan.h
kmsan: Introduce kmsan_memmove_metadata()
kmsan: Expose kmsan_get_metadata()
kmsan: Export panic_on_kmsan
kmsan: Allow disabling KMSAN checks for the current task
kmsan: Introduce memset_no_sanitize_memory()
kmsan: Support SLAB_POISON
kmsan: Use ALIGN_DOWN() in kmsan_get_metadata()
mm: slub: Let KMSAN access metadata
mm: kfence: Disable KMSAN when checking the canary lib/string: Add KMSAN support to strlcpy() and strlcat() lib/zlib: Unpoison DFLTCC output buffers
kmsan: Accept ranges starting with 0 on s390
s390: Turn off KMSAN for boot, vdso and purgatory
s390: Use a larger stack for KMSAN
s390/boot: Add the KMSAN runtime stub
s390/checksum: Add a KMSAN check
s390/cpacf: Unpoison the results of cpacf_trng() s390/ftrace: Unpoison ftrace_regs in kprobe_ftrace_handler() s390/mm: Define KMSAN metadata for vmalloc and modules s390/string: Add KMSAN support
s390/traps: Unpoison the kernel_stack_overflow()'s pt_regs s390/uaccess: Add KMSAN support to put_user() and get_user() s390/unwind: Disable KMSAN checks
s390: Implement the architecture-specific kmsan functions
kmsan: Enable on s390

 Documentation/dev-tools/kmsan.rst   |   4 +-
 arch/s390/Kconfig                   |   1 +
 arch/s390/Makefile                  |   2 +-
 arch/s390/boot/Makefile             |   3 +
 arch/s390/boot/kmsan.c              |   6 ++
 arch/s390/boot/startup.c            |   8 ++
 arch/s390/boot/string.c             |  16 ++++
 arch/s390/include/asm/checksum.h    |   2 +
 arch/s390/include/asm/cpacf.h       |   2 +
 arch/s390/include/asm/kmsan.h       |  36 +++++++++
 arch/s390/include/asm/pgtable.h     |  10 +++
 arch/s390/include/asm/string.h      |  20 +++--
 arch/s390/include/asm/thread_info.h |   2 +-
 arch/s390/include/asm/uaccess.h     | 110 ++++++++++++++++++++--------
 arch/s390/kernel/ftrace.c           |   1 +
 arch/s390/kernel/traps.c            |   6 ++
 arch/s390/kernel/unwind_bc.c        |   4 +
 arch/s390/kernel/vdso32/Makefile    |   3 +-
 arch/s390/kernel/vdso64/Makefile    |   3 +-
 arch/s390/purgatory/Makefile        |   2 +
 include/linux/kmsan-checks.h        |  26 +++++++
 include/linux/kmsan.h               |  23 ++++++
 include/linux/kmsan_types.h         |   2 +-
 kernel/trace/ftrace.c               |   1 +
 lib/string.c                        |   6 ++
 lib/zlib_dfltcc/dfltcc.h            |   1 +
 lib/zlib_dfltcc/dfltcc_util.h       |  23 ++++++
 mm/Kconfig                          |   1 +
 mm/kfence/core.c                    |   5 +-
 mm/kmsan/core.c                     |   2 +-
 mm/kmsan/hooks.c                    |  30 +++++++-
 mm/kmsan/init.c                     |   5 +-
 mm/kmsan/instrumentation.c          |  11 +--
 mm/kmsan/kmsan.h                    |   9 +--
 mm/kmsan/kmsan_test.c               |   5 ++
 mm/kmsan/report.c                   |   7 +-
 mm/kmsan/shadow.c                   |   9 +--
 mm/slub.c                           |  12 ++-
 38 files changed, 345 insertions(+), 74 deletions(-)
 create mode 100644 arch/s390/boot/kmsan.c
 create mode 100644 arch/s390/include/asm/kmsan.h
Signature · 1 line
2.41.0
#1 lore raw

PATCH v2 1/33 ftrace: Unpoison ftrace_regs in ftrace_ops_list_func()

Architectures use assembly code to initialize ftrace_regs and call ftrace_ops_list_func(). Therefore, from the KMSAN's point of view, ftrace_regs is poisoned on ftrace_ops_list_func entry(). This causes KMSAN warnings when running the ftrace testsuite.

Fix by trusting the architecture-specific assembly code and always unpoisoning ftrace_regs in ftrace_ops_list_func.

---
 kernel/trace/ftrace.c | 1 +
 1 file changed, 1 insertion(+)
kernel/trace/ftrace.c +1 -0
index 8de8bec5f366..dfb8b26966aa 100644
@@ -7399,6 +7399,7 @@ __ftrace_ops_list_func(unsigned long ip, unsigned long parent_ip,
 void arch_ftrace_ops_list_func(unsigned long ip, unsigned long parent_ip,
 			       struct ftrace_ops *op, struct ftrace_regs *fregs)
 {
+	kmsan_unpoison_memory(fregs, sizeof(*fregs));
 	__ftrace_ops_list_func(ip, parent_ip, NULL, fregs);
 }
 #else
Signature · 1 line
2.41.0
maintainer Acked-by on v2 1/33
#2 lore raw

Re: [PATCH v2 01/33] ftrace: Unpoison ftrace_regs in ftrace_ops_list_func()

Quoted 7 lines · On Tue, 21 Nov 2023 23:00:55 +0100 Ilya Leoshkevich <[email protected]> wrote:

Architectures use assembly code to initialize ftrace_regs and call ftrace_ops_list_func(). Therefore, from the KMSAN's point of view, ftrace_regs is poisoned on ftrace_ops_list_func entry(). This causes KMSAN warnings when running the ftrace testsuite.

Fix by trusting the architecture-specific assembly code and always unpoisoning ftrace_regs in ftrace_ops_list_func.

You must be very trusting to trust architecture-specific assembly code ;-)

-- Steve

Quoted 18 lines
---
 kernel/trace/ftrace.c | 1 +
 1 file changed, 1 insertion(+)
kernel/trace/ftrace.c +1 -0
index 8de8bec5f366..dfb8b26966aa 100644
@@ -7399,6 +7399,7 @@ __ftrace_ops_list_func(unsigned long ip, unsigned long parent_ip,
 void arch_ftrace_ops_list_func(unsigned long ip, unsigned long parent_ip,
 			       struct ftrace_ops *op, struct ftrace_regs *fregs)
 {
+	kmsan_unpoison_memory(fregs, sizeof(*fregs));
 	__ftrace_ops_list_func(ip, parent_ip, NULL, fregs);
 }
 #else
maintainer Reviewed-by on v2 1/33
#3 lore raw

Re: [PATCH v2 01/33] ftrace: Unpoison ftrace_regs in ftrace_ops_list_func()

In reply to Ilya Leoshkevich
Quoted 5 lines · Ilya Leoshkevich

Architectures use assembly code to initialize ftrace_regs and call ftrace_ops_list_func(). Therefore, from the KMSAN's point of view, ftrace_regs is poisoned on ftrace_ops_list_func entry(). This causes KMSAN warnings when running the ftrace testsuite.

I couldn't reproduce these warnings on x86, hope you really need this change on s390 :)

Quoted 4 lines

Fix by trusting the architecture-specific assembly code and always unpoisoning ftrace_regs in ftrace_ops_list_func.

maintainer
#4 lore raw

Re: [PATCH v2 01/33] ftrace: Unpoison ftrace_regs in ftrace_ops_list_func()

Quoted 9 lines · On Fri, 8 Dec 2023 15:16:10 +0100 Alexander Potapenko <[email protected]> wrote:
Quoted 5 lines · Ilya Leoshkevich

Architectures use assembly code to initialize ftrace_regs and call ftrace_ops_list_func(). Therefore, from the KMSAN's point of view, ftrace_regs is poisoned on ftrace_ops_list_func entry(). This causes KMSAN warnings when running the ftrace testsuite.

I couldn't reproduce these warnings on x86, hope you really need this change on s390 :)

On x86, ftrace_regs sits on the stack. And IIUC, s390 doesn't have the same concept of a "stack" as other architectures. Perhaps that's the reason s390 needs this?

-- Steve

#5 lore raw

Re: [PATCH v2 01/33] ftrace: Unpoison ftrace_regs in ftrace_ops_list_func()

Quoted 14 lines · Steven Rostedt
Quoted 11 lines · On Fri, 8 Dec 2023 15:16:10 +0100 Alexander Potapenko <[email protected]> wrote:
Quoted 8 lines · Ilya Leoshkevich

Architectures use assembly code to initialize ftrace_regs and call
ftrace_ops_list_func(). Therefore, from the KMSAN's point of view,
ftrace_regs is poisoned on ftrace_ops_list_func entry(). This causes
KMSAN warnings when running the ftrace testsuite.

Quoted 3 lines
Quoted 3 lines

I couldn't reproduce these warnings on x86, hope you really need this
change on s390 :)

I just double-checked, and it's still needed. Without it, I get:

[    4.140184] =====================================================  
[    4.140416] BUG: KMSAN: uninit-value in
arch_ftrace_ops_list_func+0x8e6/0x14b0            
[    4.140484]  arch_ftrace_ops_list_func+0x8e6/0x14b0                
[    4.140546]  ftrace_graph_caller+0x0/0x34                          
[    4.140614]  read_tod_clock+0x6/0x1e0                              
[    4.140671]  ktime_get+0x3a4/0x670                                 
[    4.140727]  clockevents_program_event+0x1c8/0xb10                 
[    4.140785]  tick_program_event+0x11e/0x230                        
[    4.140842]  hrtimer_interrupt+0x118a/0x1d10                       
[    4.140898]  do_IRQ+0x108/0x150                                    
[    4.140959]  do_irq_async+0xfc/0x270                               
[    4.141021]  do_ext_irq+0x98/0x120                                 
[    4.141080]  ext_int_handler+0xc4/0xf0                             
[    4.141141]  _raw_spin_unlock_irqrestore+0xfa/0x190                
[    4.141207]  _raw_spin_unlock_irqrestore+0xf6/0x190                
[    4.141271]  s390_kernel_write+0x218/0x250                         
[    4.141328]  ftrace_make_call+0x362/0x4a0                          
[    4.141386]  __ftrace_replace_code+0xb44/0xbd0                     
[    4.141442]  ftrace_replace_code+0x1d8/0x440                       
[    4.141497]  ftrace_modify_all_code+0xfe/0x510                     
[    4.141555]  ftrace_startup+0x4f0/0xcf0                            
[    4.141609]  register_ftrace_function+0x1316/0x1440                
[    4.141670]  function_trace_init+0x2c0/0x3d0                       
[    4.141732]  tracer_init+0x282/0x370                               
[    4.141789]  trace_selftest_startup_function+0x104/0x19d0          
[    4.141857]  run_tracer_selftest+0x7c8/0xab0                       
[    4.141918]  init_trace_selftests+0x200/0x820
[    4.141977]  do_one_initcall+0x35e/0x1090
[    4.142032]  do_initcall_level+0x276/0x660
[    4.142095]  do_initcalls+0x16a/0x2d0
[    4.142153]  kernel_init_freeable+0x632/0x960
[    4.142216]  kernel_init+0x36/0x1810
[    4.142277]  __ret_from_fork+0xc0/0x180
[    4.142333]  ret_from_fork+0xa/0x30
[    4.142431] Local variable agg.tmp.i.i created at:                 
02:06:55 [30/1836]
[    4.142476]  timekeeping_advance+0x79a/0x2870
[    4.142394] 
[    4.142431] Local variable agg.tmp.i.i created at:
[    4.142476]  timekeeping_advance+0x79a/0x2870
[    4.142534] 
[    4.142573] CPU: 0 PID: 1 Comm: swapper/0 Tainted: G        W      
6.7.0-rc4-g7657d31dc545 #4
[    4.142638] Hardware name: IBM 3931 A01 704 (KVM/Linux)
[    4.142686] =====================================================
[    4.142734] Kernel panic - not syncing: kmsan.panic set ...
[    4.142734] =====================================================
Quoted 5 lines

On x86, ftrace_regs sits on the stack. And IIUC, s390 doesn't have the same
concept of a "stack" as other architectures. Perhaps that's the reason s390
needs this?

It's not that different on s390x. There is indeed no architecture- mandated stack pointer and no push/pop, but other than that it's fairly normal. Linux uses %r15 as a stack pointer.

On s390x ftrace_regs is allocated on stack by mcount.S. From what I can see, Intel's ftrace_64.S does the same thing, so I don't immediately see why uninit-value is not detected on Intel, even though I think it should.

Quoted 1 line

-- Steve

#6 lore raw

PATCH v2 2/33 kmsan: Make the tests compatible with kmsan.panic=1

In reply to Ilya Leoshkevich

It's useful to have both tests and kmsan.panic=1 during development, but right now the warnings, that the tests cause, lead to kernel panics.

Temporarily set kmsan.panic=0 for the duration of the KMSAN testing.

---
 mm/kmsan/kmsan_test.c | 5 +++++
 1 file changed, 5 insertions(+)
mm/kmsan/kmsan_test.c +5 -0
index 07d3a3a5a9c5..9bfd11674fe3 100644
@@ -659,9 +659,13 @@ static void test_exit(struct kunit *test)
 {
 }
 
+static int orig_panic_on_kmsan;
+
 static int kmsan_suite_init(struct kunit_suite *suite)
 {
 	register_trace_console(probe_console, NULL);
+	orig_panic_on_kmsan = panic_on_kmsan;
+	panic_on_kmsan = 0;
 	return 0;
 }
 
@@ -669,6 +673,7 @@ static void kmsan_suite_exit(struct kunit_suite *suite)
 {
 	unregister_trace_console(probe_console, NULL);
 	tracepoint_synchronize_unregister();
+	panic_on_kmsan = orig_panic_on_kmsan;
 }
 
 static struct kunit_suite kmsan_test_suite = {
Signature · 1 line
2.41.0
#7 lore raw

PATCH v2 3/33 kmsan: Disable KMSAN when DEFERRED_STRUCT_PAGE_INIT is enabled

In reply to Ilya Leoshkevich

KMSAN relies on memblock returning all available pages to it (see kmsan_memblock_free_pages()). It partitions these pages into 3
categories: pages available to the buddy allocator, shadow pages and origin pages. This partitioning is static.

If new pages appear after kmsan_init_runtime(), it is considered an error. DEFERRED_STRUCT_PAGE_INIT causes this, so mark it as incompatible with KMSAN.

---
 mm/Kconfig | 1 +
 1 file changed, 1 insertion(+)
mm/Kconfig +1 -0
index 89971a894b60..4f2f99339fc7 100644
@@ -985,6 +985,7 @@ config DEFERRED_STRUCT_PAGE_INIT
 	depends on SPARSEMEM
 	depends on !NEED_PER_CPU_KM
 	depends on 64BIT
+	depends on !KMSAN
 	select PADATA
 	help
 	  Ordinarily all struct pages are initialised during early boot in a
Signature · 1 line
2.41.0
#8 lore raw

PATCH v2 4/33 kmsan: Increase the maximum store size to 4096

In reply to Ilya Leoshkevich

The inline assembly block in s390's chsc() stores that much.

---
 mm/kmsan/instrumentation.c | 7 +++----
 1 file changed, 3 insertions(+), 4 deletions(-)
mm/kmsan/instrumentation.c +3 -4
index cc3907a9c33a..470b0b4afcc4 100644
@@ -110,11 +110,10 @@ void __msan_instrument_asm_store(void *addr, uintptr_t size)
 
 	ua_flags = user_access_save();
 	/*
-	 * Most of the accesses are below 32 bytes. The two exceptions so far
-	 * are clwb() (64 bytes) and FPU state (512 bytes).
-	 * It's unlikely that the assembly will touch more than 512 bytes.
+	 * Most of the accesses are below 32 bytes. The exceptions so far are
+	 * clwb() (64 bytes), FPU state (512 bytes) and chsc() (4096 bytes).
 	 */
-	if (size > 512) {
+	if (size > 4096) {
 		WARN_ONCE(1, "assembly store size too big: %ld\n", size);
 		size = 8;
 	}
Signature · 1 line
2.41.0
#10 lore raw

PATCH v2 5/33 kmsan: Fix is_bad_asm_addr() on arches with overlapping address spaces

In reply to Ilya Leoshkevich

Comparing pointers with TASK_SIZE does not make sense when kernel and userspace overlap. Skip the comparison when this is the case.

---
 mm/kmsan/instrumentation.c | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)
mm/kmsan/instrumentation.c +2 -1
index 470b0b4afcc4..8a1bbbc723ab 100644
@@ -20,7 +20,8 @@
 
 static inline bool is_bad_asm_addr(void *addr, uintptr_t size, bool is_store)
 {
-	if ((u64)addr < TASK_SIZE)
+	if (IS_ENABLED(CONFIG_ARCH_HAS_NON_OVERLAPPING_ADDRESS_SPACE) &&
+	    (u64)addr < TASK_SIZE)
 		return true;
 	if (!kmsan_get_metadata(addr, KMSAN_META_SHADOW))
 		return true;
Signature · 1 line
2.41.0
#12 lore raw

PATCH v2 6/33 kmsan: Fix kmsan_copy_to_user() on arches with overlapping address spaces

In reply to Ilya Leoshkevich

Comparing pointers with TASK_SIZE does not make sense when kernel and userspace overlap. Assume that we are handling user memory access in this case.

---
 mm/kmsan/hooks.c | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)
mm/kmsan/hooks.c +2 -1
index 5d6e2dee5692..eafc45f937eb 100644
@@ -267,7 +267,8 @@ void kmsan_copy_to_user(void __user *to, const void *from, size_t to_copy,
 		return;
 
 	ua_flags = user_access_save();
-	if ((u64)to < TASK_SIZE) {
+	if (!IS_ENABLED(CONFIG_ARCH_HAS_NON_OVERLAPPING_ADDRESS_SPACE) ||
+	    (u64)to < TASK_SIZE) {
 		/* This is a user memory access, check it. */
 		kmsan_internal_check_memory((void *)from, to_copy - left, to,
 					    REASON_COPY_TO_USER);
Signature · 1 line
2.41.0
#13 lore raw

PATCH v2 7/33 kmsan: Remove a useless assignment from kmsan_vmap_pages_range_noflush()

In reply to Ilya Leoshkevich

The value assigned to prot is immediately overwritten on the next line with PAGE_KERNEL. The right hand side of the assignment has no side-effects.

Fixes b073d7f8aee4 ("mm: kmsan: maintain KMSAN metadata for page operations")
---
 mm/kmsan/shadow.c | 1 -
 1 file changed, 1 deletion(-)
mm/kmsan/shadow.c +0 -1
index b9d05aff313e..2d57408c78ae 100644
@@ -243,7 +243,6 @@ int kmsan_vmap_pages_range_noflush(unsigned long start, unsigned long end,
 		s_pages[i] = shadow_page_for(pages[i]);
 		o_pages[i] = origin_page_for(pages[i]);
 	}
-	prot = __pgprot(pgprot_val(prot) | _PAGE_NX);
 	prot = PAGE_KERNEL;
 
 	origin_start = vmalloc_meta((void *)start, KMSAN_META_ORIGIN);
Signature · 1 line
2.41.0
#14 lore raw

PATCH v2 8/33 kmsan: Remove an x86-specific #include from kmsan.h

In reply to Ilya Leoshkevich

Replace the x86-specific asm/pgtable_64_types.h #include with the linux/pgtable.h one, which all architectures have.

While at it, sort the headers alphabetically for the sake of consistency with other KMSAN code.

Fixes f80be4571b19 ("kmsan: add KMSAN runtime core")
---
 mm/kmsan/kmsan.h | 8 ++++----
 1 file changed, 4 insertions(+), 4 deletions(-)
mm/kmsan/kmsan.h +4 -4
index a14744205435..adf443bcffe8 100644
@@ -10,14 +10,14 @@
 #ifndef __MM_KMSAN_KMSAN_H
 #define __MM_KMSAN_KMSAN_H
 
-#include <asm/pgtable_64_types.h>
 #include <linux/irqflags.h>
+#include <linux/mm.h>
+#include <linux/nmi.h>
+#include <linux/pgtable.h>
+#include <linux/printk.h>
 #include <linux/sched.h>
 #include <linux/stackdepot.h>
 #include <linux/stacktrace.h>
-#include <linux/nmi.h>
-#include <linux/mm.h>
-#include <linux/printk.h>
 
 #define KMSAN_ALLOCA_MAGIC_ORIGIN 0xabcd0100
 #define KMSAN_CHAIN_MAGIC_ORIGIN 0xabcd0200
Signature · 1 line
2.41.0
#15 lore raw

PATCH v2 9/33 kmsan: Introduce kmsan_memmove_metadata()

In reply to Ilya Leoshkevich

It is useful to manually copy metadata in order to describe the effects of memmove()-like logic in uninstrumented code or inline asm. Introduce kmsan_memmove_metadata() for this purpose.

---
 include/linux/kmsan-checks.h | 14 ++++++++++++++
 mm/kmsan/hooks.c             | 11 +++++++++++
 2 files changed, 25 insertions(+)
include/linux/kmsan-checks.h +14 -0
index c4cae333deec..5218973f0ad0 100644
@@ -61,6 +61,17 @@ void kmsan_check_memory(const void *address, size_t size);
 void kmsan_copy_to_user(void __user *to, const void *from, size_t to_copy,
 			size_t left);
 
+/**
+ * kmsan_memmove_metadata() - Copy kernel memory range metadata.
+ * @dst: start of the destination kernel memory range.
+ * @src: start of the source kernel memory range.
+ * @n:   size of the memory ranges.
+ *
+ * KMSAN will treat the destination range as if its contents were memmove()d
+ * from the source range.
+ */
+void kmsan_memmove_metadata(void *dst, const void *src, size_t n);
+
 #else
 
 static inline void kmsan_poison_memory(const void *address, size_t size,
@@ -77,6 +88,9 @@ static inline void kmsan_copy_to_user(void __user *to, const void *from,
 				      size_t to_copy, size_t left)
 {
 }
+static inline void kmsan_memmove_metadata(void *dst, const void *src, size_t n)
+{
+}
 
 #endif
 
mm/kmsan/hooks.c +11 -0
index eafc45f937eb..4d477a0a356c 100644
@@ -286,6 +286,17 @@ void kmsan_copy_to_user(void __user *to, const void *from, size_t to_copy,
 }
 EXPORT_SYMBOL(kmsan_copy_to_user);
 
+void kmsan_memmove_metadata(void *dst, const void *src, size_t n)
+{
+	if (!kmsan_enabled || kmsan_in_runtime())
+		return;
+
+	kmsan_enter_runtime();
+	kmsan_internal_memmove_metadata(dst, (void *)src, n);
+	kmsan_leave_runtime();
+}
+EXPORT_SYMBOL(kmsan_memmove_metadata);
+
 /* Helper function to check an URB. */
 void kmsan_handle_urb(const struct urb *urb, bool is_out)
 {
Signature · 1 line
2.41.0
#16 lore raw

Re: [PATCH v2 09/33] kmsan: Introduce kmsan_memmove_metadata()

Quoted 19 lines · Ilya Leoshkevich

It is useful to manually copy metadata in order to describe the effects of memmove()-like logic in uninstrumented code or inline asm. Introduce kmsan_memmove_metadata() for this purpose.

---
 include/linux/kmsan-checks.h | 14 ++++++++++++++
 mm/kmsan/hooks.c             | 11 +++++++++++
 2 files changed, 25 insertions(+)
include/linux/kmsan-checks.h +0 -0
index c4cae333deec..5218973f0ad0 100644
@@ -61,6 +61,17 @@ void kmsan_check_memory(const void *address, size_t size);
 void kmsan_copy_to_user(void __user *to, const void *from, size_t to_copy,
                        size_t left);
 

include/linux/kmsan-checks.h void kmsan_check_memory(const void *address, size_t size); · line 73

Quoted 10 lines
include/linux/kmsan-checks.h +10 -0
+/**
+ * kmsan_memmove_metadata() - Copy kernel memory range metadata.
+ * @dst: start of the destination kernel memory range.
+ * @src: start of the source kernel memory range.
+ * @n:   size of the memory ranges.
+ *
+ * KMSAN will treat the destination range as if its contents were memmove()d
+ * from the source range.
+ */
+void kmsan_memmove_metadata(void *dst, const void *src, size_t n);

As noted in patch 18/33, I am pretty sure we shouldn't need this function.

#17 lore raw

PATCH v2 10/33 kmsan: Expose kmsan_get_metadata()

In reply to Ilya Leoshkevich

Each s390 CPU has lowcore pages associated with it. Each CPU sees its own lowcore at virtual address 0 through a hardware mechanism called prefixing. Additionally, all lowcores are mapped to non-0 virtual addresses stored in the lowcore_ptr[] array.

When lowcore is accessed through virtual address 0, one needs to resolve metadata for lowcore_ptr[raw_smp_processor_id()].

Expose kmsan_get_metadata() to make it possible to do this from the arch code.

---
 include/linux/kmsan.h      | 14 ++++++++++++++
 mm/kmsan/instrumentation.c |  1 +
 mm/kmsan/kmsan.h           |  1 -
 3 files changed, 15 insertions(+), 1 deletion(-)
include/linux/kmsan.h +14 -0
index e0c23a32cdf0..ff8fd95733fa 100644
@@ -230,6 +230,15 @@ void kmsan_handle_urb(const struct urb *urb, bool is_out);
  */
 void kmsan_unpoison_entry_regs(const struct pt_regs *regs);
 
+/**
+ * kmsan_get_metadata() - Return a pointer to KMSAN shadow or origins.
+ * @addr:      kernel address.
+ * @is_origin: whether to return origins or shadow.
+ *
+ * Return NULL if metadata cannot be found.
+ */
+void *kmsan_get_metadata(void *addr, bool is_origin);
+
 #else
 
 static inline void kmsan_init_shadow(void)
@@ -329,6 +338,11 @@ static inline void kmsan_unpoison_entry_regs(const struct pt_regs *regs)
 {
 }
 
+static inline void *kmsan_get_metadata(void *addr, bool is_origin)
+{
+	return NULL;
+}
+
 #endif
 
 #endif /* _LINUX_KMSAN_H */
mm/kmsan/instrumentation.c +1 -0
index 8a1bbbc723ab..94b49fac9d8b 100644
@@ -14,6 +14,7 @@
 
 #include "kmsan.h"
 #include <linux/gfp.h>
+#include <linux/kmsan.h>
 #include <linux/kmsan_string.h>
 #include <linux/mm.h>
 #include <linux/uaccess.h>
mm/kmsan/kmsan.h +0 -1
index adf443bcffe8..34b83c301d57 100644
@@ -66,7 +66,6 @@ struct shadow_origin_ptr {
 
 struct shadow_origin_ptr kmsan_get_shadow_origin_ptr(void *addr, u64 size,
 						     bool store);
-void *kmsan_get_metadata(void *addr, bool is_origin);
 void __init kmsan_init_alloc_meta_for_range(void *start, void *end);
 
 enum kmsan_bug_reason {
Signature · 1 line
2.41.0
#18 lore raw

Re: [PATCH v2 10/33] kmsan: Expose kmsan_get_metadata()

include/linux/kmsan.h static inline void kmsan_unpoison_entry_regs(const struct pt_regs *regs) · line 346

Quoted 6 lines
include/linux/kmsan.h +5 -0
+static inline void *kmsan_get_metadata(void *addr, bool is_origin)
+{
+       return NULL;
+}
+
 #endif

We shouldn't need this part, as kmsan_get_metadata() should never be called in non-KMSAN builds.

#19 lore raw

PATCH v2 11/33 kmsan: Export panic_on_kmsan

In reply to Ilya Leoshkevich

When building the kmsan test as a module, modpost fails with the following error message:

ERROR: modpost: "panic_on_kmsan" [mm/kmsan/kmsan_test.ko] undefined!

Export panic_on_kmsan in order to improve the KMSAN usability for modules.

---
 mm/kmsan/report.c | 1 +
 1 file changed, 1 insertion(+)
mm/kmsan/report.c +1 -0
index 02736ec757f2..c79d3b0d2d0d 100644
@@ -20,6 +20,7 @@ static DEFINE_RAW_SPINLOCK(kmsan_report_lock);
 /* Protected by kmsan_report_lock */
 static char report_local_descr[DESCR_SIZE];
 int panic_on_kmsan __read_mostly;
+EXPORT_SYMBOL_GPL(panic_on_kmsan);
 
 #ifdef MODULE_PARAM_PREFIX
 #undef MODULE_PARAM_PREFIX
Signature · 1 line
2.41.0
#20 lore raw

PATCH v2 12/33 kmsan: Allow disabling KMSAN checks for the current task

In reply to Ilya Leoshkevich

Like for KASAN, it's useful to temporarily disable KMSAN checks around, e.g., redzone accesses. Introduce kmsan_disable_current() and kmsan_enable_current(), which are similar to their KASAN counterparts.

Even though it's not strictly necessary, make them reentrant, in order to match the KASAN behavior. Repurpose the allow_reporting field for this.

---
 Documentation/dev-tools/kmsan.rst |  4 ++--
 include/linux/kmsan-checks.h      | 12 ++++++++++++
 include/linux/kmsan_types.h       |  2 +-
 mm/kmsan/core.c                   |  2 +-
 mm/kmsan/hooks.c                  | 14 +++++++++++++-
 mm/kmsan/report.c                 |  6 +++---
 6 files changed, 32 insertions(+), 8 deletions(-)
Documentation/dev-tools/kmsan.rst +2 -2
index 323eedad53cd..022a823f5f1b 100644
@@ -338,11 +338,11 @@ Per-task KMSAN state
 ~~~~~~~~~~~~~~~~~~~~
 
 Every task_struct has an associated KMSAN task state that holds the KMSAN
-context (see above) and a per-task flag disallowing KMSAN reports::
+context (see above) and a per-task counter disallowing KMSAN reports::
 
   struct kmsan_context {
     ...
-    bool allow_reporting;
+    unsigned int depth;
     struct kmsan_context_state cstate;
     ...
   }
include/linux/kmsan-checks.h +12 -0
index 5218973f0ad0..bab2603685f7 100644
@@ -72,6 +72,10 @@ void kmsan_copy_to_user(void __user *to, const void *from, size_t to_copy,
  */
 void kmsan_memmove_metadata(void *dst, const void *src, size_t n);
 
+void kmsan_enable_current(void);
+
+void kmsan_disable_current(void);
+
 #else
 
 static inline void kmsan_poison_memory(const void *address, size_t size,
@@ -92,6 +96,14 @@ static inline void kmsan_memmove_metadata(void *dst, const void *src, size_t n)
 {
 }
 
+static inline void kmsan_enable_current(void)
+{
+}
+
+static inline void kmsan_disable_current(void)
+{
+}
+
 #endif
 
 #endif /* _LINUX_KMSAN_CHECKS_H */
include/linux/kmsan_types.h +1 -1
index 8bfa6c98176d..27bb146ece95 100644
@@ -29,7 +29,7 @@ struct kmsan_context_state {
 struct kmsan_ctx {
 	struct kmsan_context_state cstate;
 	int kmsan_in_runtime;
-	bool allow_reporting;
+	unsigned int depth;
 };
 
 #endif /* _LINUX_KMSAN_TYPES_H */
mm/kmsan/core.c +1 -1
index c19f47af0424..b8767378cf8a 100644
@@ -43,7 +43,7 @@ void kmsan_internal_task_create(struct task_struct *task)
 	struct thread_info *info = current_thread_info();
 
 	__memset(ctx, 0, sizeof(*ctx));
-	ctx->allow_reporting = true;
+	ctx->depth = 0;
 	kmsan_internal_unpoison_memory(info, sizeof(*info), false);
 }
 
mm/kmsan/hooks.c +13 -1
index 4d477a0a356c..7b5814412e9f 100644
@@ -44,7 +44,7 @@ void kmsan_task_exit(struct task_struct *task)
 	if (!kmsan_enabled || kmsan_in_runtime())
 		return;
 
-	ctx->allow_reporting = false;
+	ctx->depth++;
 }
 
 void kmsan_slab_alloc(struct kmem_cache *s, void *object, gfp_t flags)
@@ -434,3 +434,15 @@ void kmsan_check_memory(const void *addr, size_t size)
 					   REASON_ANY);
 }
 EXPORT_SYMBOL(kmsan_check_memory);
+
+void kmsan_enable_current(void)
+{
+	current->kmsan_ctx.depth--;
+}
+EXPORT_SYMBOL(kmsan_enable_current);
+
+void kmsan_disable_current(void)
+{
+	current->kmsan_ctx.depth++;
+}
+EXPORT_SYMBOL(kmsan_disable_current);
mm/kmsan/report.c +3 -3
index c79d3b0d2d0d..edcf53ca428e 100644
@@ -158,12 +158,12 @@ void kmsan_report(depot_stack_handle_t origin, void *address, int size,
 
 	if (!kmsan_enabled)
 		return;
-	if (!current->kmsan_ctx.allow_reporting)
+	if (current->kmsan_ctx.depth)
 		return;
 	if (!origin)
 		return;
 
-	current->kmsan_ctx.allow_reporting = false;
+	current->kmsan_ctx.depth++;
 	ua_flags = user_access_save();
 	raw_spin_lock(&kmsan_report_lock);
 	pr_err("=====================================================\n");
@@ -216,5 +216,5 @@ void kmsan_report(depot_stack_handle_t origin, void *address, int size,
 	if (panic_on_kmsan)
 		panic("kmsan.panic set ...\n");
 	user_access_restore(ua_flags);
-	current->kmsan_ctx.allow_reporting = true;
+	current->kmsan_ctx.depth--;
 }
Signature · 1 line
2.41.0
#21 lore raw

Re: [PATCH v2 12/33] kmsan: Allow disabling KMSAN checks for the current task

Quoted 4 lines · Ilya Leoshkevich

Like for KASAN, it's useful to temporarily disable KMSAN checks around, e.g., redzone accesses. Introduce kmsan_disable_current() and kmsan_enable_current(), which are similar to their KASAN counterparts.

Initially we used to have this disablement counter in KMSAN, but adding it uncontrollably can result in KMSAN not functioning properly. E.g. forgetting to call kmsan_disable_current() or underflowing the counter will break reporting.
We'd better put this API in include/linux/kmsan.h to indicate it should be discouraged.

Quoted 2 lines

Even though it's not strictly necessary, make them reentrant, in order to match the KASAN behavior.

Until this becomes strictly necessary, I think we'd better KMSAN_WARN_ON if the counter is re-entered.

#22 lore raw

Re: [PATCH v2 12/33] kmsan: Allow disabling KMSAN checks for the current task

Quoted 17 lines · Alexander Potapenko
Quoted 6 lines · Ilya Leoshkevich

Like for KASAN, it's useful to temporarily disable KMSAN checks around,
e.g., redzone accesses. Introduce kmsan_disable_current() and kmsan_enable_current(), which are similar to their KASAN counterparts.

Initially we used to have this disablement counter in KMSAN, but adding it uncontrollably can result in KMSAN not functioning properly.
E.g. forgetting to call kmsan_disable_current() or underflowing the counter will break reporting.
We'd better put this API in include/linux/kmsan.h to indicate it should be discouraged.

Quoted 6 lines
Quoted 3 lines

Even though it's not strictly necessary, make them reentrant, in order
to match the KASAN behavior.

Until this becomes strictly necessary, I think we'd better KMSAN_WARN_ON if the counter is re-entered.

I encountered a case when we are freeing memory from an interrupt handler:

[  149.840553] ------------[ cut here ]------------                   
[  149.840649] WARNING: CPU: 1 PID: 181 at mm/kmsan/hooks.c:447
kmsan_disable_current+0x2e/0x40                                       
[  149.840790] Modules linked in:                                     
[  149.840894] CPU: 1 PID: 181 Comm: (direxec) Tainted: G    B   W    
N 6.7.0-rc5-gd34a4b46f382 #13
[  149.841003] Hardware name: IBM 3931 A01 704 (KVM/Linux)     
[  149.841094] Krnl PSW : 0404c00180000000 000000000197dbc2
(kmsan_disable_current+0x32/0x40)
[  149.841276]            R:0 T:1 IO:0 EX:0 Key:0 M:1 W:0 P:0 AS:3 CC:0
PM:0 RI:0 EA:3
[  149.841420] Krnl GPRS: 0000000000000040 0000000096914100
0000000000001000 0000000000000001
[  149.841518]            0000036d827daee0 0000000007c97008
0000000080096500 0000000092f4f000
[  149.841617]            0000036d00000000 0000000000000000
0000000000000040 0000000000000000
[  149.841712]            0000000092f4efc0 00000001ff710f60
000000000193acba 0000037f0008f710
[  149.841893] Krnl Code: 000000000197dbb6: eb0018640352        mviy  
14436(%r1),0
[  149.841893]            000000000197dbbc: 07fe                bcr   
15,%r14
[  149.841893]           #000000000197dbbe: af000000            mc    
0,0
[  149.841893]           >000000000197dbc2: a7f4fffa            brc   
15,000000000197dbb6
[  149.841893]            000000000197dbc6: 0700                bcr   
0,%r0
[  149.841893]            000000000197dbc8: 0700                bcr   
0,%r0
[  149.841893]            000000000197dbca: 0700                bcr   
0,%r0
[  149.841893]            000000000197dbcc: 0700                bcr   
0,%r0
[  149.842438] Call Trace:                                            
15:37:25 [90/1838]
[  149.842510]  [<000000000197dbc2>] kmsan_disable_current+0x32/0x40 
[  149.842631] ([<000000000193ac14>] slab_pad_check+0x1d4/0xac0)
[  149.842738]  [<0000000001949222>] free_to_partial_list+0x1d72/0x3b80
[  149.842850]  [<0000000001947066>] __slab_free+0xd86/0x11d0 
[  149.842956]  [<00000000019111e8>] kmem_cache_free+0x15d8/0x25d0 
[  149.843062]  [<0000000000229e3a>] __tlb_remove_table+0x20a/0xa50 
[  149.843174]  [<00000000016c7f98>] tlb_remove_table_rcu+0x98/0x120 
[  149.843291]  [<000000000083e1c6>] rcu_core+0x15b6/0x54b0 
[  149.843406]  [<00000000069c3c0e>] __do_softirq+0xa1e/0x2178 
[  149.843514]  [<00000000003467b4>] irq_exit_rcu+0x2c4/0x630 
[  149.843623]  [<0000000006949f6e>] do_ext_irq+0x9e/0x120 
[  149.843736]  [<00000000069c18d4>] ext_int_handler+0xc4/0xf0 
[  149.843841]  [<000000000197e428>] kmsan_get_metadata+0x68/0x280 
[  149.843950]  [<000000000197e344>]
kmsan_get_shadow_origin_ptr+0x74/0xf0 
[  149.844071]  [<000000000197ba3a>]
__msan_metadata_ptr_for_load_8+0x2a/0x40 
[  149.844192]  [<0000000000184e4a>]
unwind_get_return_address+0xda/0x150 
[  149.844313]  [<000000000018fd12>] arch_stack_walk+0x172/0x2f0 
[  149.844417]  [<00000000008f1af0>] stack_trace_save+0x100/0x160 
[  149.844529]  [<000000000197af22>]
kmsan_internal_chain_origin+0x62/0xe0 
[  149.844647]  [<000000000197c1f0>] __msan_chain_origin+0xd0/0x160 
[  149.844763]  [<00000000068b3ba4>] memchr_inv+0x5b4/0xb20 
[  149.844877]  [<000000000193e730>] check_bytes_and_report+0xa0/0xd30 
[  149.844986]  [<000000000193b920>] check_object+0x420/0x17d0 
[  149.845092]  [<000000000194aa8a>] free_to_partial_list+0x35da/0x3b80
[  149.845202]  [<0000000001947066>] __slab_free+0xd86/0x11d0 
[  149.845308]  [<00000000019111e8>] kmem_cache_free+0x15d8/0x25d0 
[  149.845414]  [<00000000016bc2fe>] exit_mmap+0x87e/0x1200 
[  149.845524]  [<00000000002f315c>] mmput+0x13c/0x5b0 
[  149.845632]  [<0000000001b9d634>] exec_mmap+0xc34/0x1230 
[  149.845744]  [<0000000001b996c2>] begin_new_exec+0xcf2/0x2520 
[  149.845857]  [<0000000001f6a084>] load_elf_binary+0x2364/0x67d0 
[  149.845971]  [<0000000001ba5ba4>] bprm_execve+0x25b4/0x4010 
[  149.846083]  [<0000000001baa7e6>] do_execveat_common+0x2436/0x2600 
[  149.846200]  [<0000000001ba78f8>] __s390x_sys_execve+0x108/0x140 
[  149.846314]  [<000000000011b192>] do_syscall+0x4c2/0x690 
[  149.846424]  [<0000000006949d78>] __do_syscall+0x98/0xe0 
[  149.846536]  [<00000000069c1640>] system_call+0x70/0xa0 
[  149.846638] INFO: lockdep is turned off.
[  149.846846] Last Breaking-Event-Address:
[  149.846916]  [<000000000197dbb2>] kmsan_disable_current+0x22/0x40
[  149.847057] irq event stamp: 0
[  149.847128] hardirqs last  enabled at (0): [<0000000000000000>] 0x0
[  149.847227] hardirqs last disabled at (0): [<00000000002f8f46>]
copy_process+0x21f6/0x8b20
[  149.847344] softirqs last  enabled at (0): [<00000000002f8f80>]
copy_process+0x2230/0x8b20
[  149.847461] softirqs last disabled at (0): [<0000000000000000>] 0x0
[  149.847559] ---[ end trace 0000000000000000 ]---
[  149.865485] =====================================================

Using a counter resolves this issue, but, of course, at the expense of not reporting valid issues in the interrupt handler.

Unfortunately I don't see another easy way to solve this problem. The possibilities that come to mind are providing uninstrumented memchr_inv() or disablement flags for each context, but I'm not sure if we want to go there, especially since KASAN already has this limitation.

#23 lore raw

PATCH v2 13/33 kmsan: Introduce memset_no_sanitize_memory()

In reply to Ilya Leoshkevich

Add a wrapper for memset() that prevents unpoisoning. This is useful for filling memory allocator redzones.

---
 include/linux/kmsan.h | 9 +++++++++
 1 file changed, 9 insertions(+)
include/linux/kmsan.h +9 -0
index ff8fd95733fa..439df72c8dc6 100644
@@ -345,4 +345,13 @@ static inline void *kmsan_get_metadata(void *addr, bool is_origin)
 
 #endif
 
+/**
+ * memset_no_sanitize_memory() - memset() without the KMSAN instrumentation.
+ */
+__no_sanitize_memory
+static inline void *memset_no_sanitize_memory(void *s, int c, size_t n)
+{
+	return memset(s, c, n);
+}
+
 #endif /* _LINUX_KMSAN_H */
Signature · 1 line
2.41.0
#24 lore raw

Re: [PATCH v2 13/33] kmsan: Introduce memset_no_sanitize_memory()

Quoted 2 lines · Ilya Leoshkevich

Add a wrapper for memset() that prevents unpoisoning.

We have __memset() already, won't it work for this case? On the other hand, I am not sure you want to preserve the redzone in its previous state (unless it's known to be poisoned). You might consider explicitly unpoisoning the redzone instead.

...

include/linux/kmsan.h static inline void *kmsan_get_metadata(void *addr, bool is_origin) · line 355

Quoted 5 lines
include/linux/kmsan.h +5 -0
+__no_sanitize_memory
+static inline void *memset_no_sanitize_memory(void *s, int c, size_t n)
+{
+       return memset(s, c, n);
+}

I think depending on the compiler optimizations this might end up being a call to normal memset, that would still change the shadow bytes.

#25 lore raw

Re: [PATCH v2 13/33] kmsan: Introduce memset_no_sanitize_memory()

Quoted 6 lines · Alexander Potapenko
Quoted 2 lines · Ilya Leoshkevich

Add a wrapper for memset() that prevents unpoisoning.

We have __memset() already, won't it work for this case?

A problem with __memset() is that, at least for me, it always ends up being a call. There is a use case where we need to write only 1 byte, so I thought that introducing a call there (when compiling without KMSAN) would be unacceptable.

Quoted 2 lines

On the other hand, I am not sure you want to preserve the redzone in its previous state (unless it's known to be poisoned).

That's exactly the problem with unpoisoning: it removes the distinction between a new allocation and a UAF.

Quoted 1 line

You might consider explicitly unpoisoning the redzone instead.

That was my first attempt, but it resulted in test failures due to the above.

Quoted 12 lines

...

include/linux/kmsan.h static inline void *kmsan_get_metadata(void *addr, bool is_origin) · line 355

Quoted 6 lines
(quoted diff) +2 -0
+__no_sanitize_memory
+static inline void *memset_no_sanitize_memory(void *s, int c,

size_t n)

include/linux/kmsan.h +3 -0
+{
+       return memset(s, c, n);
+}

I think depending on the compiler optimizations this might end up being a call to normal memset, that would still change the shadow bytes.

Interesting, do you have some specific scenario in mind? I vaguely remember that in the past there were cases when sanitizer annotations were lost after inlining, but I thought they were sorted out?

And, in any case, if this were to happen, would not it be considered a compiler bug that needs fixing there, and not in the kernel?

#26 lore raw

Re: [PATCH v2 13/33] kmsan: Introduce memset_no_sanitize_memory()

Quoted 4 lines

A problem with __memset() is that, at least for me, it always ends up being a call. There is a use case where we need to write only 1 byte, so I thought that introducing a call there (when compiling without KMSAN) would be unacceptable.

Wonder what happens with that use case if we e.g. build with fortify-source. Calling memset() for a single byte might be indicating the code is not hot.

Quoted 13 lines
Quoted 12 lines

...

include/linux/kmsan.h static inline void *kmsan_get_metadata(void *addr, bool is_origin) · line 355

Quoted 6 lines
(quoted diff) +2 -0
+__no_sanitize_memory
+static inline void *memset_no_sanitize_memory(void *s, int c,

size_t n)

include/linux/kmsan.h +3 -0
+{
+       return memset(s, c, n);
+}

I think depending on the compiler optimizations this might end up being a call to normal memset, that would still change the shadow bytes.

Quoted 3 lines

Interesting, do you have some specific scenario in mind? I vaguely remember that in the past there were cases when sanitizer annotations were lost after inlining, but I thought they were sorted out?

Sanitizer annotations are indeed lost after inlining, and we cannot do much about that.
They are implemented using function attributes, and if a function dissolves after inlining, we cannot possibly know which instructions belonged to it.

Consider the following example (also available at
https://godbolt.org/z/5r7817G8e):

==================================
void *kmalloc(int size);

__attribute__((no_sanitize("kernel-memory"))) __attribute__((always_inline))
static void *memset_nosanitize(void *s, int c, int n) {
return __builtin_memset(s, c, n);
}

void *do_something_nosanitize(int size) {
void *ptr = kmalloc(size);
memset_nosanitize(ptr, 0, size);
return ptr;
}

void *do_something_sanitize(int size) {
void *ptr = kmalloc(size);
__builtin_memset(ptr, 0, size);
return ptr;
}
==================================

If memset_nosanitize() has __attribute__((always_inline)), the compiler generates the same LLVM IR calling __msan_memset() for both do_something_nosanitize() and do_something_sanitize().
If we comment out this attribute, do_something_nosanitize() calls memset_nosanitize(), which doesn't have the sanitize_memory attribute.

But even now __builtin_memset() is still calling __msan_memset(), because __attribute__((no_sanitize("kernel-memory"))) somewhat counterintuitively still preserves some instrumentation (see include/linux/compiler-clang.h for details).
Replacing __attribute__((no_sanitize("kernel-memory"))) with __attribute__((disable_sanitizer_instrumentation)) fixes this situation:

define internal fastcc noundef ptr @memset_nosanitize(void*, int, int)(ptr noundef returned writeonly %s, i32 noundef %n) unnamed_addr #2 {
entry:
%conv = sext i32 %n to i64
tail call void @llvm.memset.p0.i64(ptr align 1 %s, i8 0, i64 %conv, i1 false) ret ptr %s
}

Quoted 3 lines

And, in any case, if this were to happen, would not it be considered a compiler bug that needs fixing there, and not in the kernel?

As stated above, I don't think this is more or less working as intended. If we really want the ability to inline __memset(), we could transform it into memset() in non-sanitizer builds, but perhaps having a call is also acceptable?

#27 lore raw

Re: [PATCH v2 13/33] kmsan: Introduce memset_no_sanitize_memory()

Quoted 9 lines · Alexander Potapenko
Quoted 4 lines

A problem with __memset() is that, at least for me, it always ends up being a call. There is a use case where we need to write only 1 byte, so I thought that introducing a call there (when compiling without KMSAN) would be unacceptable.

Wonder what happens with that use case if we e.g. build with fortify- source.
Calling memset() for a single byte might be indicating the code is not hot.

The original code has a simple assignment. Here is the relevant diff:

        if (s->flags & __OBJECT_POISON) {
-               memset(p, POISON_FREE, poison_size - 1);
-               p[poison_size - 1] = POISON_END;
+               memset_no_sanitize_memory(p, POISON_FREE, poison_size -
1);
+               memset_no_sanitize_memory(p + poison_size - 1,
POISON_END, 1);
        }

[...]

Quoted 7 lines

As stated above, I don't think this is more or less working as intended.
If we really want the ability to inline __memset(), we could transform
it into memset() in non-sanitizer builds, but perhaps having a call is
also acceptable?

Thanks for the detailed explanation and analysis. I will post a version with a __memset() and let the slab maintainers decide if the additional overhead is acceptable.

#28 lore raw

Re: [PATCH v2 13/33] kmsan: Introduce memset_no_sanitize_memory()

Quoted 7 lines · Ilya Leoshkevich
Quoted 6 lines · Alexander Potapenko
Quoted 6 lines

A problem with __memset() is that, at least for me, it always ends
up being a call. There is a use case where we need to write only 1
byte, so I thought that introducing a call there (when compiling without KMSAN) would be unacceptable.

[...]

Quoted 11 lines
Quoted 7 lines

As stated above, I don't think this is more or less working as intended.
If we really want the ability to inline __memset(), we could transform
it into memset() in non-sanitizer builds, but perhaps having a call is
also acceptable?

Thanks for the detailed explanation and analysis. I will post a version with a __memset() and let the slab maintainers decide if the additional overhead is acceptable.

I noticed I had the same problem in the get_user()/put_user() and check_canary() patches.

The annotation being silently ignored is never what a programmer intends, so what do you think about adding noinline to __no_kmsan_checks and __no_sanitize_memory?

#29 lore raw

PATCH v2 14/33 kmsan: Support SLAB_POISON

In reply to Ilya Leoshkevich

Avoid false KMSAN negatives with SLUB_DEBUG by allowing kmsan_slab_free() to poison the freed memory, and by preventing init_object() from unpoisoning new allocations. The usage of memset_no_sanitize_memory() does not degrade the generated code quality.

There are two alternatives to this approach. First, init_object() can be marked with __no_sanitize_memory. This annotation should be used with great care, because it drops all instrumentation from the function, and any shadow writes will be lost. Even though this is not a concern with the current init_object() implementation, this may change in the future.

Second, kmsan_poison_memory() calls may be added after memset() calls. The downside is that init_object() is called from free_debug_processing(), in which case poisoning will erase the distinction between simply uninitialized memory and UAF.

---
 mm/kmsan/hooks.c |  2 +-
 mm/slub.c        | 10 ++++++----
 2 files changed, 7 insertions(+), 5 deletions(-)
mm/kmsan/hooks.c +1 -1
index 7b5814412e9f..7a30274b893c 100644
@@ -76,7 +76,7 @@ void kmsan_slab_free(struct kmem_cache *s, void *object)
 		return;
 
 	/* RCU slabs could be legally used after free within the RCU period */
-	if (unlikely(s->flags & (SLAB_TYPESAFE_BY_RCU | SLAB_POISON)))
+	if (unlikely(s->flags & SLAB_TYPESAFE_BY_RCU))
 		return;
 	/*
 	 * If there's a constructor, freed memory must remain in the same state
mm/slub.c +6 -4
index 63d281dfacdb..169e5f645ea8 100644
@@ -1030,7 +1030,8 @@ static void init_object(struct kmem_cache *s, void *object, u8 val)
 	unsigned int poison_size = s->object_size;
 
 	if (s->flags & SLAB_RED_ZONE) {
-		memset(p - s->red_left_pad, val, s->red_left_pad);
+		memset_no_sanitize_memory(p - s->red_left_pad, val,
+					  s->red_left_pad);
 
 		if (slub_debug_orig_size(s) && val == SLUB_RED_ACTIVE) {
 			/*
@@ -1043,12 +1044,13 @@ static void init_object(struct kmem_cache *s, void *object, u8 val)
 	}
 
 	if (s->flags & __OBJECT_POISON) {
-		memset(p, POISON_FREE, poison_size - 1);
-		p[poison_size - 1] = POISON_END;
+		memset_no_sanitize_memory(p, POISON_FREE, poison_size - 1);
+		memset_no_sanitize_memory(p + poison_size - 1, POISON_END, 1);
 	}
 
 	if (s->flags & SLAB_RED_ZONE)
-		memset(p + poison_size, val, s->inuse - poison_size);
+		memset_no_sanitize_memory(p + poison_size, val,
+					  s->inuse - poison_size);
 }
 
 static void restore_bytes(struct kmem_cache *s, char *message, u8 data,
Signature · 1 line
2.41.0
#30 lore raw

Re: [PATCH v2 14/33] kmsan: Support SLAB_POISON

Quoted 43 lines · Ilya Leoshkevich

Avoid false KMSAN negatives with SLUB_DEBUG by allowing kmsan_slab_free() to poison the freed memory, and by preventing init_object() from unpoisoning new allocations. The usage of memset_no_sanitize_memory() does not degrade the generated code quality.

There are two alternatives to this approach. First, init_object() can be marked with __no_sanitize_memory. This annotation should be used with great care, because it drops all instrumentation from the function, and any shadow writes will be lost. Even though this is not a concern with the current init_object() implementation, this may change in the future.

Second, kmsan_poison_memory() calls may be added after memset() calls. The downside is that init_object() is called from free_debug_processing(), in which case poisoning will erase the distinction between simply uninitialized memory and UAF.

---
 mm/kmsan/hooks.c |  2 +-
 mm/slub.c        | 10 ++++++----
 2 files changed, 7 insertions(+), 5 deletions(-)
mm/kmsan/hooks.c +1 -1
index 7b5814412e9f..7a30274b893c 100644
@@ -76,7 +76,7 @@ void kmsan_slab_free(struct kmem_cache *s, void *object)
                return;
 
        /* RCU slabs could be legally used after free within the RCU period */
-       if (unlikely(s->flags & (SLAB_TYPESAFE_BY_RCU | SLAB_POISON)))
+       if (unlikely(s->flags & SLAB_TYPESAFE_BY_RCU))
                return;
        /*
         * If there's a constructor, freed memory must remain in the same state
mm/slub.c +0 -0
index 63d281dfacdb..169e5f645ea8 100644

mm/slub.c static void init_object(struct kmem_cache *s, void *object, u8 val) · line 1033

Quoted 5 lines
mm/slub.c +1 -1
@@ -1030,7 +1030,8 @@ static void init_object(struct kmem_cache *s, void *object, u8 val)
        unsigned int poison_size = s->object_size;
 
        if (s->flags & SLAB_RED_ZONE) {
-               memset(p - s->red_left_pad, val, s->red_left_pad);
+               memset_no_sanitize_memory(p - s->red_left_pad, val,

As I wrote in patch 13/33, let's try to use __memset() here (with a comment that we want to preserve the previously poisoned memory)

#31 lore raw

PATCH v2 15/33 kmsan: Use ALIGN_DOWN() in kmsan_get_metadata()

In reply to Ilya Leoshkevich

Improve the readability by replacing the custom aligning logic with ALIGN_DOWN(). Unlike other places where a similar sequence is used, there is no size parameter that needs to be adjusted, so the standard macro fits.

---
 mm/kmsan/shadow.c | 8 +++-----
 1 file changed, 3 insertions(+), 5 deletions(-)
mm/kmsan/shadow.c +3 -5
index 2d57408c78ae..9c58f081d84f 100644
@@ -123,14 +123,12 @@ struct shadow_origin_ptr kmsan_get_shadow_origin_ptr(void *address, u64 size,
  */
 void *kmsan_get_metadata(void *address, bool is_origin)
 {
-	u64 addr = (u64)address, pad, off;
+	u64 addr = (u64)address, off;
 	struct page *page;
 	void *ret;
 
-	if (is_origin && !IS_ALIGNED(addr, KMSAN_ORIGIN_SIZE)) {
-		pad = addr % KMSAN_ORIGIN_SIZE;
-		addr -= pad;
-	}
+	if (is_origin)
+		addr = ALIGN_DOWN(addr, KMSAN_ORIGIN_SIZE);
 	address = (void *)addr;
 	if (kmsan_internal_is_vmalloc_addr(address) ||
 	    kmsan_internal_is_module_addr(address))
Signature · 1 line
2.41.0
#32 lore raw

PATCH v2 16/33 mm: slub: Let KMSAN access metadata

In reply to Ilya Leoshkevich

Building the kernel with CONFIG_SLUB_DEBUG and CONFIG_KMSAN causes KMSAN to complain about touching redzones in kfree().

Fix by extending the existing KASAN-related metadata_access_enable() and metadata_access_disable() functions to KMSAN.

---
 mm/slub.c | 2 ++
 1 file changed, 2 insertions(+)
mm/slub.c +2 -0
index 169e5f645ea8..6e61c27951a4 100644
@@ -700,10 +700,12 @@ static int disable_higher_order_debug;
 static inline void metadata_access_enable(void)
 {
 	kasan_disable_current();
+	kmsan_disable_current();
 }
 
 static inline void metadata_access_disable(void)
 {
+	kmsan_enable_current();
 	kasan_enable_current();
 }
 
Signature · 1 line
2.41.0
Acked-by on v2 16/33
#33 lore raw

Re: [PATCH v2 16/33] mm: slub: Let KMSAN access metadata

Quoted 7 lines · Ilya Leoshkevich

Building the kernel with CONFIG_SLUB_DEBUG and CONFIG_KMSAN causes KMSAN to complain about touching redzones in kfree().

Fix by extending the existing KASAN-related metadata_access_enable() and metadata_access_disable() functions to KMSAN.

Quoted 21 lines
---
 mm/slub.c | 2 ++
 1 file changed, 2 insertions(+)
mm/slub.c +2 -0
index 169e5f645ea8..6e61c27951a4 100644
@@ -700,10 +700,12 @@ static int disable_higher_order_debug;
 static inline void metadata_access_enable(void)
 {
 	kasan_disable_current();
+	kmsan_disable_current();
 }
 
 static inline void metadata_access_disable(void)
 {
+	kmsan_enable_current();
 	kasan_enable_current();
 }
 
#34 lore raw

PATCH v2 17/33 mm: kfence: Disable KMSAN when checking the canary

In reply to Ilya Leoshkevich

KMSAN warns about check_canary() accessing the canary.

The reason is that, even though set_canary() is properly instrumented and sets shadow, slub explicitly poisons the canary's address range afterwards.

Unpoisoning the canary is not the right thing to do: only check_canary() is supposed to ever touch it. Instead, disable KMSAN checks around canary read accesses.

---
 mm/kfence/core.c | 5 +++--
 1 file changed, 3 insertions(+), 2 deletions(-)
mm/kfence/core.c +3 -2
index 3872528d0963..a2ea8e5a1ad9 100644
@@ -306,7 +306,7 @@ metadata_update_state(struct kfence_metadata *meta, enum kfence_object_state nex
 }
 
 /* Check canary byte at @addr. */
-static inline bool check_canary_byte(u8 *addr)
+__no_kmsan_checks static inline bool check_canary_byte(u8 *addr)
 {
 	struct kfence_metadata *meta;
 	unsigned long flags;
@@ -341,7 +341,8 @@ static inline void set_canary(const struct kfence_metadata *meta)
 		*((u64 *)addr) = KFENCE_CANARY_PATTERN_U64;
 }
 
-static inline void check_canary(const struct kfence_metadata *meta)
+__no_kmsan_checks static inline void
+check_canary(const struct kfence_metadata *meta)
 {
 	const unsigned long pageaddr = ALIGN_DOWN(meta->addr, PAGE_SIZE);
 	unsigned long addr = pageaddr;
Signature · 1 line
2.41.0
maintainer Reviewed-by on v2 17/33
#35 lore raw

Re: [PATCH v2 17/33] mm: kfence: Disable KMSAN when checking the canary

Quoted 7 lines · Ilya Leoshkevich

KMSAN warns about check_canary() accessing the canary.

The reason is that, even though set_canary() is properly instrumented and sets shadow, slub explicitly poisons the canary's address range afterwards.

Quoted 5 lines

Unpoisoning the canary is not the right thing to do: only check_canary() is supposed to ever touch it. Instead, disable KMSAN checks around canary read accesses.

maintainer Tested-by
#36 lore raw

Re: [PATCH v2 17/33] mm: kfence: Disable KMSAN when checking the canary

Quoted 9 lines · Alexander Potapenko
Quoted 7 lines · Ilya Leoshkevich

KMSAN warns about check_canary() accessing the canary.

The reason is that, even though set_canary() is properly instrumented and sets shadow, slub explicitly poisons the canary's address range afterwards.

Quoted 6 lines
Quoted 5 lines

Unpoisoning the canary is not the right thing to do: only check_canary() is supposed to ever touch it. Instead, disable KMSAN checks around canary read accesses.

and even

#37 lore raw

PATCH v2 18/33 lib/string: Add KMSAN support to strlcpy() and strlcat()

In reply to Ilya Leoshkevich

Currently KMSAN does not fully propagate metadata in strlcpy() and strlcat(), because they are built with -ffreestanding and call memcpy(). In this combination memcpy() calls are not instrumented.

Fix by copying the metadata manually. Add the __STDC_HOSTED__ #ifdef in case the code is compiled with different flags in the future.

---
 lib/string.c | 6 ++++++
 1 file changed, 6 insertions(+)
lib/string.c +6 -0
index be26623953d2..e83c6dd77ec6 100644
@@ -111,6 +111,9 @@ size_t strlcpy(char *dest, const char *src, size_t size)
 	if (size) {
 		size_t len = (ret >= size) ? size - 1 : ret;
 		__builtin_memcpy(dest, src, len);
+#if __STDC_HOSTED__ == 0
+		kmsan_memmove_metadata(dest, src, len);
+#endif
 		dest[len] = '\0';
 	}
 	return ret;
@@ -261,6 +264,9 @@ size_t strlcat(char *dest, const char *src, size_t count)
 	if (len >= count)
 		len = count-1;
 	__builtin_memcpy(dest, src, len);
+#if __STDC_HOSTED__ == 0
+	kmsan_memmove_metadata(dest, src, len);
+#endif
 	dest[len] = 0;
 	return res;
 }
Signature · 1 line
2.41.0
#38 lore raw

Re: [PATCH v2 18/33] lib/string: Add KMSAN support to strlcpy() and strlcat()

Quoted 4 lines · Ilya Leoshkevich

Currently KMSAN does not fully propagate metadata in strlcpy() and strlcat(), because they are built with -ffreestanding and call memcpy(). In this combination memcpy() calls are not instrumented.

Is this something specific to s390?

Quoted 13 lines

Fix by copying the metadata manually. Add the __STDC_HOSTED__ #ifdef in case the code is compiled with different flags in the future.

---
 lib/string.c | 6 ++++++
 1 file changed, 6 insertions(+)
lib/string.c +0 -0
index be26623953d2..e83c6dd77ec6 100644

lib/string.c size_t strlcpy(char *dest, const char *src, size_t size) · line 113

Quoted 3 lines
lib/string.c +0 -0
@@ -111,6 +111,9 @@ size_t strlcpy(char *dest, const char *src, size_t size)
        if (size) {
                size_t len = (ret >= size) ? size - 1 : ret;
                __builtin_memcpy(dest, src, len);

On x86, I clearly see this __builtin_memcpy() being replaced with __msan_memcpy().

#39 lore raw

Re: [PATCH v2 18/33] lib/string: Add KMSAN support to strlcpy() and strlcat()

Quoted 8 lines · Alexander Potapenko
Quoted 4 lines · Ilya Leoshkevich

Currently KMSAN does not fully propagate metadata in strlcpy() and strlcat(), because they are built with -ffreestanding and call memcpy(). In this combination memcpy() calls are not instrumented.

Is this something specific to s390?

Nice catch - I can't reproduce this behavior anymore. Even if I go back to the clang version that first introduced KMSAN on s390x, the memset() instrumentation with -ffreestanding is still there. I should have written down more detailed notes after investigating this, but here we are. I will drop this patch as well as 10/33.

[...]

#40 lore raw

PATCH v2 19/33 lib/zlib: Unpoison DFLTCC output buffers

In reply to Ilya Leoshkevich

The constraints of the DFLTCC inline assembly are not precise: they do not communicate the size of the output buffers to the compiler, so it cannot automatically instrument it.

Add the manual kmsan_unpoison_memory() calls for the output buffers. The logic is the same as in [1].

[1] https://github.com/zlib-ng/zlib-ng/commit/1f5ddcc009ac3511e99fc88736a9e1a6381168c5

---
 lib/zlib_dfltcc/dfltcc.h      |  1 +
 lib/zlib_dfltcc/dfltcc_util.h | 23 +++++++++++++++++++++++
 2 files changed, 24 insertions(+)
lib/zlib_dfltcc/dfltcc.h +1 -0
index b96232bdd44d..0f2a16d7a48a 100644
@@ -80,6 +80,7 @@ struct dfltcc_param_v0 {
     uint8_t csb[1152];
 };
 
+static_assert(offsetof(struct dfltcc_param_v0, csb) == 384);
 static_assert(sizeof(struct dfltcc_param_v0) == 1536);
 
 #define CVT_CRC32 0
lib/zlib_dfltcc/dfltcc_util.h +23 -0
index 4a46b5009f0d..ce2e039a55b5 100644
@@ -2,6 +2,7 @@
 #ifndef DFLTCC_UTIL_H
 #define DFLTCC_UTIL_H
 
+#include "dfltcc.h"
 #include <linux/zutil.h>
 
 /*
@@ -20,6 +21,7 @@ typedef enum {
 #define DFLTCC_CMPR 2
 #define DFLTCC_XPND 4
 #define HBT_CIRCULAR (1 << 7)
+#define DFLTCC_FN_MASK ((1 << 7) - 1)
 #define HB_BITS 15
 #define HB_SIZE (1 << HB_BITS)
 
@@ -34,6 +36,7 @@ static inline dfltcc_cc dfltcc(
 )
 {
     Byte *t2 = op1 ? *op1 : NULL;
+    unsigned char *orig_t2 = t2;
     size_t t3 = len1 ? *len1 : 0;
     const Byte *t4 = op2 ? *op2 : NULL;
     size_t t5 = len2 ? *len2 : 0;
@@ -59,6 +62,26 @@ static inline dfltcc_cc dfltcc(
                      : "cc", "memory");
     t2 = r2; t3 = r3; t4 = r4; t5 = r5;
 
+    switch (fn & DFLTCC_FN_MASK) {
+    case DFLTCC_QAF:
+        kmsan_unpoison_memory(param, sizeof(struct dfltcc_qaf_param));
+        break;
+    case DFLTCC_GDHT:
+        kmsan_unpoison_memory(param, offsetof(struct dfltcc_param_v0, csb));
+        break;
+    case DFLTCC_CMPR:
+        kmsan_unpoison_memory(param, sizeof(struct dfltcc_param_v0));
+        kmsan_unpoison_memory(
+                orig_t2,
+                t2 - orig_t2 +
+                    (((struct dfltcc_param_v0 *)param)->sbb == 0 ? 0 : 1));
+        break;
+    case DFLTCC_XPND:
+        kmsan_unpoison_memory(param, sizeof(struct dfltcc_param_v0));
+        kmsan_unpoison_memory(orig_t2, t2 - orig_t2);
+        break;
+    }
+
     if (op1)
         *op1 = t2;
     if (len1)
Signature · 1 line
2.41.0
#41 lore raw

Re: [PATCH v2 19/33] lib/zlib: Unpoison DFLTCC output buffers

Quoted 4 lines · Ilya Leoshkevich

The constraints of the DFLTCC inline assembly are not precise: they do not communicate the size of the output buffers to the compiler, so it cannot automatically instrument it.

KMSAN usually does a poor job instrumenting inline assembly. Wouldn't be it better to switch to pure C ZLIB implementation, making ZLIB_DFLTCC depend on !KMSAN?

#42 lore raw

Re: [PATCH v2 19/33] lib/zlib: Unpoison DFLTCC output buffers

Quoted 11 lines · Alexander Potapenko
Quoted 5 lines · Ilya Leoshkevich

The constraints of the DFLTCC inline assembly are not precise: they do not communicate the size of the output buffers to the compiler, so
it cannot automatically instrument it.

KMSAN usually does a poor job instrumenting inline assembly. Wouldn't be it better to switch to pure C ZLIB implementation, making ZLIB_DFLTCC depend on !KMSAN?

Normally I would agree, but the kernel DFLTCC code base is synced with the zlib-ng code base to the extent that it uses the zlib-ng code style instead of the kernel code style, and MSAN annotations are already a part of the zlib-ng code base. So I would prefer to keep them for consistency.

The code is also somewhat tricky in the are of buffer management, so I find it beneficial to have it checked for uninitialized memory accesses.

#43 lore raw

Re: [PATCH v2 19/33] lib/zlib: Unpoison DFLTCC output buffers

Quoted 14 lines · Ilya Leoshkevich
Quoted 11 lines · Alexander Potapenko
Quoted 5 lines · Ilya Leoshkevich

The constraints of the DFLTCC inline assembly are not precise: they do not communicate the size of the output buffers to the compiler, so
it cannot automatically instrument it.

KMSAN usually does a poor job instrumenting inline assembly. Wouldn't be it better to switch to pure C ZLIB implementation, making ZLIB_DFLTCC depend on !KMSAN?

Quoted 5 lines

Normally I would agree, but the kernel DFLTCC code base is synced with the zlib-ng code base to the extent that it uses the zlib-ng code style instead of the kernel code style, and MSAN annotations are already a part of the zlib-ng code base. So I would prefer to keep them for consistency.

Hm, I didn't realize this code is being taken from elsewhere. If so, maybe we should come up with an annotation that can be contributed to zlib-ng, so that it doesn't cause merge conflicts every time Mikhail is doing an update?
(leaving this up to you to decide).

If you decide to go with the current solution, please consider adding an #include for kmsan-checks.h, which introduces kmsan_unpoison_memory().

#44 lore raw

PATCH v2 20/33 kmsan: Accept ranges starting with 0 on s390

In reply to Ilya Leoshkevich

On s390 the virtual address 0 is valid (current CPU's lowcore is mapped there), therefore KMSAN should not complain about it.

Disable the respective check on s390. There doesn't seem to be a Kconfig option to describe this situation, so explicitly check for s390.

---
 mm/kmsan/init.c | 5 ++++-
 1 file changed, 4 insertions(+), 1 deletion(-)
mm/kmsan/init.c +4 -1
index ffedf4dbc49d..7a3df4d359f8 100644
@@ -33,7 +33,10 @@ static void __init kmsan_record_future_shadow_range(void *start, void *end)
 	bool merged = false;
 
 	KMSAN_WARN_ON(future_index == NUM_FUTURE_RANGES);
-	KMSAN_WARN_ON((nstart >= nend) || !nstart || !nend);
+	KMSAN_WARN_ON((nstart >= nend) ||
+		      /* Virtual address 0 is valid on s390. */
+		      (!IS_ENABLED(CONFIG_S390) && !nstart) ||
+		      !nend);
 	nstart = ALIGN_DOWN(nstart, PAGE_SIZE);
 	nend = ALIGN(nend, PAGE_SIZE);
 
Signature · 1 line
2.41.0
#45 lore raw

PATCH v2 21/33 s390: Turn off KMSAN for boot, vdso and purgatory

In reply to Ilya Leoshkevich

All other sanitizers are disabled for these components as well. While at it, add a comment to boot and purgatory.

---
 arch/s390/boot/Makefile          | 2 ++
 arch/s390/kernel/vdso32/Makefile | 3 ++-
 arch/s390/kernel/vdso64/Makefile | 3 ++-
 arch/s390/purgatory/Makefile     | 2 ++
 4 files changed, 8 insertions(+), 2 deletions(-)
arch/s390/boot/Makefile +2 -0
index c7c81e5f9218..fb10fcd21221 100644
@@ -3,11 +3,13 @@
 # Makefile for the linux s390-specific parts of the memory manager.
 #
 
+# Tooling runtimes are unavailable and cannot be linked for early boot code
 KCOV_INSTRUMENT := n
 GCOV_PROFILE := n
 UBSAN_SANITIZE := n
 KASAN_SANITIZE := n
 KCSAN_SANITIZE := n
+KMSAN_SANITIZE := n
 
 KBUILD_AFLAGS := $(KBUILD_AFLAGS_DECOMPRESSOR)
 KBUILD_CFLAGS := $(KBUILD_CFLAGS_DECOMPRESSOR)
arch/s390/kernel/vdso32/Makefile +2 -1
index caec7db6f966..7cbec6b0b11f 100644
@@ -32,11 +32,12 @@ obj-y += vdso32_wrapper.o
 targets += vdso32.lds
 CPPFLAGS_vdso32.lds += -P -C -U$(ARCH)
 
-# Disable gcov profiling, ubsan and kasan for VDSO code
+# Disable gcov profiling, ubsan, kasan and kmsan for VDSO code
 GCOV_PROFILE := n
 UBSAN_SANITIZE := n
 KASAN_SANITIZE := n
 KCSAN_SANITIZE := n
+KMSAN_SANITIZE := n
 
 # Force dependency (incbin is bad)
 $(obj)/vdso32_wrapper.o : $(obj)/vdso32.so
arch/s390/kernel/vdso64/Makefile +2 -1
index e3c9085f8fa7..6f3252712f64 100644
@@ -36,11 +36,12 @@ obj-y += vdso64_wrapper.o
 targets += vdso64.lds
 CPPFLAGS_vdso64.lds += -P -C -U$(ARCH)
 
-# Disable gcov profiling, ubsan and kasan for VDSO code
+# Disable gcov profiling, ubsan, kasan and kmsan for VDSO code
 GCOV_PROFILE := n
 UBSAN_SANITIZE := n
 KASAN_SANITIZE := n
 KCSAN_SANITIZE := n
+KMSAN_SANITIZE := n
 
 # Force dependency (incbin is bad)
 $(obj)/vdso64_wrapper.o : $(obj)/vdso64.so
arch/s390/purgatory/Makefile +2 -0
index 4e930f566878..4e421914e50f 100644
@@ -15,11 +15,13 @@ CFLAGS_sha256.o := -D__DISABLE_EXPORTS -D__NO_FORTIFY
 $(obj)/mem.o: $(srctree)/arch/s390/lib/mem.S FORCE
 	$(call if_changed_rule,as_o_S)
 
+# Tooling runtimes are unavailable and cannot be linked for purgatory code
 KCOV_INSTRUMENT := n
 GCOV_PROFILE := n
 UBSAN_SANITIZE := n
 KASAN_SANITIZE := n
 KCSAN_SANITIZE := n
+KMSAN_SANITIZE := n
 
 KBUILD_CFLAGS := -fno-strict-aliasing -Wall -Wstrict-prototypes
 KBUILD_CFLAGS += -Wno-pointer-sign -Wno-sign-compare
Signature · 1 line
2.41.0
#46 lore raw

PATCH v2 22/33 s390: Use a larger stack for KMSAN

In reply to Ilya Leoshkevich

Adjust the stack size for the KMSAN-enabled kernel like it was done for the KASAN-enabled one in commit 7fef92ccadd7 ("s390/kasan: double the stack size"). Both tools have similar requirements.

---
 arch/s390/Makefile                  | 2 +-
 arch/s390/include/asm/thread_info.h | 2 +-
 2 files changed, 2 insertions(+), 2 deletions(-)
arch/s390/Makefile +1 -1
index 73873e451686..a7f5386d25ad 100644
@@ -34,7 +34,7 @@ KBUILD_CFLAGS_DECOMPRESSOR += $(if $(CONFIG_DEBUG_INFO_DWARF4), $(call cc-option
 KBUILD_CFLAGS_DECOMPRESSOR += $(if $(CONFIG_CC_NO_ARRAY_BOUNDS),-Wno-array-bounds)
 
 UTS_MACHINE	:= s390x
-STACK_SIZE	:= $(if $(CONFIG_KASAN),65536,16384)
+STACK_SIZE	:= $(if $(CONFIG_KASAN),65536,$(if $(CONFIG_KMSAN),65536,16384))
 CHECKFLAGS	+= -D__s390__ -D__s390x__
 
 export LD_BFD
arch/s390/include/asm/thread_info.h +1 -1
index a674c7d25da5..d02a709717b8 100644
@@ -16,7 +16,7 @@
 /*
  * General size of kernel stacks
  */
-#ifdef CONFIG_KASAN
+#if defined(CONFIG_KASAN) || defined(CONFIG_KMSAN)
 #define THREAD_SIZE_ORDER 4
 #else
 #define THREAD_SIZE_ORDER 2
Signature · 1 line
2.41.0
#47 lore raw

PATCH v2 23/33 s390/boot: Add the KMSAN runtime stub

In reply to Ilya Leoshkevich

It should be possible to have inline functions in the s390 header files, which call kmsan_unpoison_memory(). The problem is that these header files might be included by the decompressor, which does not contain KMSAN runtime, causing linker errors.

Not compiling these calls if __SANITIZE_MEMORY__ is not defined - either by changing kmsan-checks.h or at the call sites - may cause unintended side effects, since calling these functions from an uninstrumented code that is linked into the kernel is valid use case.

One might want to explicitly distinguish between the kernel and the decompressor. Checking for a decompressor-specific #define is quite heavy-handed, and will have to be done at all call sites.

A more generic approach is to provide a dummy kmsan_unpoison_memory() definition. This produces some runtime overhead, but only when building with CONFIG_KMSAN. The benefit is that it does not disturb the existing KMSAN build logic and call sites don't need to be changed.

---
 arch/s390/boot/Makefile | 1 +
 arch/s390/boot/kmsan.c  | 6 ++++++
 2 files changed, 7 insertions(+)
 create mode 100644 arch/s390/boot/kmsan.c
arch/s390/boot/Makefile +1 -0
index fb10fcd21221..096216a72e98 100644
@@ -44,6 +44,7 @@ obj-$(findstring y, $(CONFIG_PROTECTED_VIRTUALIZATION_GUEST) $(CONFIG_PGSTE))	+=
 obj-$(CONFIG_RANDOMIZE_BASE)	+= kaslr.o
 obj-y	+= $(if $(CONFIG_KERNEL_UNCOMPRESSED),,decompressor.o) info.o
 obj-$(CONFIG_KERNEL_ZSTD) += clz_ctz.o
+obj-$(CONFIG_KMSAN) += kmsan.o
 obj-all := $(obj-y) piggy.o syms.o
 
 targets	:= bzImage section_cmp.boot.data section_cmp.boot.preserved.data $(obj-y)
arch/s390/boot/kmsan.c new file +6 -0
new file mode 100644
index 000000000000..e7b3ac48143e
@@ -0,0 +1,6 @@
+// SPDX-License-Identifier: GPL-2.0
+#include <linux/kmsan-checks.h>
+
+void kmsan_unpoison_memory(const void *address, size_t size)
+{
+}
Signature · 1 line
2.41.0
maintainer Reviewed-by on v2 23/33
#48 lore raw

Re: [PATCH v2 23/33] s390/boot: Add the KMSAN runtime stub

Quoted 15 lines · Ilya Leoshkevich

It should be possible to have inline functions in the s390 header files, which call kmsan_unpoison_memory(). The problem is that these header files might be included by the decompressor, which does not contain KMSAN runtime, causing linker errors.

Not compiling these calls if __SANITIZE_MEMORY__ is not defined - either by changing kmsan-checks.h or at the call sites - may cause unintended side effects, since calling these functions from an uninstrumented code that is linked into the kernel is valid use case.

One might want to explicitly distinguish between the kernel and the decompressor. Checking for a decompressor-specific #define is quite heavy-handed, and will have to be done at all call sites.

Quoted 6 lines

A more generic approach is to provide a dummy kmsan_unpoison_memory() definition. This produces some runtime overhead, but only when building with CONFIG_KMSAN. The benefit is that it does not disturb the existing KMSAN build logic and call sites don't need to be changed.

#49 lore raw

PATCH v2 24/33 s390/checksum: Add a KMSAN check

In reply to Ilya Leoshkevich

Add a KMSAN check to the CKSM inline assembly, similar to how it was done for ASAN in commit e42ac7789df6 ("s390/checksum: always use cksm instruction").

---
 arch/s390/include/asm/checksum.h | 2 ++
 1 file changed, 2 insertions(+)
arch/s390/include/asm/checksum.h +2 -0
index 69837eec2ff5..55ba0ddd8eab 100644
@@ -13,6 +13,7 @@
 #define _S390_CHECKSUM_H
 
 #include <linux/kasan-checks.h>
+#include <linux/kmsan-checks.h>
 #include <linux/in6.h>
 
 /*
@@ -35,6 +36,7 @@ static inline __wsum csum_partial(const void *buff, int len, __wsum sum)
 	};
 
 	kasan_check_read(buff, len);
+	kmsan_check_memory(buff, len);
 	asm volatile(
 		"0:	cksm	%[sum],%[rp]\n"
 		"	jo	0b\n"
Signature · 1 line
2.41.0
#51 lore raw

PATCH v2 25/33 s390/cpacf: Unpoison the results of cpacf_trng()

In reply to Ilya Leoshkevich

Prevent KMSAN from complaining about buffers filled by cpacf_trng() being uninitialized.

---
 arch/s390/include/asm/cpacf.h | 2 ++
 1 file changed, 2 insertions(+)
arch/s390/include/asm/cpacf.h +2 -0
index b378e2b57ad8..a72b92770c4b 100644
@@ -473,6 +473,8 @@ static inline void cpacf_trng(u8 *ucbuf, unsigned long ucbuf_len,
 		: [ucbuf] "+&d" (u.pair), [cbuf] "+&d" (c.pair)
 		: [fc] "K" (CPACF_PRNO_TRNG), [opc] "i" (CPACF_PRNO)
 		: "cc", "memory", "0");
+	kmsan_unpoison_memory(ucbuf, ucbuf_len);
+	kmsan_unpoison_memory(cbuf, cbuf_len);
 }
 
 /**
Signature · 1 line
2.41.0
#53 lore raw

PATCH v2 26/33 s390/ftrace: Unpoison ftrace_regs in kprobe_ftrace_handler()

In reply to Ilya Leoshkevich

s390 uses assembly code to initialize ftrace_regs and call kprobe_ftrace_handler(). Therefore, from the KMSAN's point of view, ftrace_regs is poisoned on kprobe_ftrace_handler() entry. This causes KMSAN warnings when running the ftrace testsuite.

Fix by trusting the assembly code and always unpoisoning ftrace_regs in kprobe_ftrace_handler().

---
 arch/s390/kernel/ftrace.c | 1 +
 1 file changed, 1 insertion(+)
arch/s390/kernel/ftrace.c +1 -0
index c46381ea04ec..3bad34eaa51e 100644
@@ -300,6 +300,7 @@ void kprobe_ftrace_handler(unsigned long ip, unsigned long parent_ip,
 	if (bit < 0)
 		return;
 
+	kmsan_unpoison_memory(fregs, sizeof(*fregs));
 	regs = ftrace_get_regs(fregs);
 	p = get_kprobe((kprobe_opcode_t *)ip);
 	if (!regs || unlikely(!p) || kprobe_disabled(p))
Signature · 1 line
2.41.0
maintainer Reviewed-by on v2 26/33
#54 lore raw

Re: [PATCH v2 26/33] s390/ftrace: Unpoison ftrace_regs in kprobe_ftrace_handler()

Quoted 10 lines · Ilya Leoshkevich

s390 uses assembly code to initialize ftrace_regs and call kprobe_ftrace_handler(). Therefore, from the KMSAN's point of view, ftrace_regs is poisoned on kprobe_ftrace_handler() entry. This causes KMSAN warnings when running the ftrace testsuite.

Fix by trusting the assembly code and always unpoisoning ftrace_regs in kprobe_ftrace_handler().

#55 lore raw

PATCH v2 27/33 s390/mm: Define KMSAN metadata for vmalloc and modules

In reply to Ilya Leoshkevich

The pages for the KMSAN metadata associated with most kernel mappings are taken from memblock by the common code. However, vmalloc and module metadata needs to be defined by the architectures.

Be a little bit more careful than x86: allocate exactly MODULES_LEN for the module shadow and origins, and then take 2/3 of vmalloc for the vmalloc shadow and origins. This ensures that users passing small vmalloc= values on the command line do not cause module metadata collisions.

---
 arch/s390/boot/startup.c        |  8 ++++++++
 arch/s390/include/asm/pgtable.h | 10 ++++++++++
 2 files changed, 18 insertions(+)
arch/s390/boot/startup.c +8 -0
index 8104e0e3d188..e37e7ffda430 100644
@@ -253,9 +253,17 @@ static unsigned long setup_kernel_memory_layout(void)
 	MODULES_END = round_down(__abs_lowcore, _SEGMENT_SIZE);
 	MODULES_VADDR = MODULES_END - MODULES_LEN;
 	VMALLOC_END = MODULES_VADDR;
+#ifdef CONFIG_KMSAN
+	VMALLOC_END -= MODULES_LEN * 2;
+#endif
 
 	/* allow vmalloc area to occupy up to about 1/2 of the rest virtual space left */
 	vmalloc_size = min(vmalloc_size, round_down(VMALLOC_END / 2, _REGION3_SIZE));
+#ifdef CONFIG_KMSAN
+	/* take 2/3 of vmalloc area for KMSAN shadow and origins */
+	vmalloc_size = round_down(vmalloc_size / 3, _REGION3_SIZE);
+	VMALLOC_END -= vmalloc_size * 2;
+#endif
 	VMALLOC_START = VMALLOC_END - vmalloc_size;
 
 	/* split remaining virtual space between 1:1 mapping & vmemmap array */
arch/s390/include/asm/pgtable.h +10 -0
index 601e87fa8a9a..d764abeb9e6d 100644
@@ -107,6 +107,16 @@ static inline int is_module_addr(void *addr)
 	return 1;
 }
 
+#ifdef CONFIG_KMSAN
+#define KMSAN_VMALLOC_SIZE (VMALLOC_END - VMALLOC_START)
+#define KMSAN_VMALLOC_SHADOW_START VMALLOC_END
+#define KMSAN_VMALLOC_ORIGIN_START (KMSAN_VMALLOC_SHADOW_START + \
+				    KMSAN_VMALLOC_SIZE)
+#define KMSAN_MODULES_SHADOW_START (KMSAN_VMALLOC_ORIGIN_START + \
+				    KMSAN_VMALLOC_SIZE)
+#define KMSAN_MODULES_ORIGIN_START (KMSAN_MODULES_SHADOW_START + MODULES_LEN)
+#endif
+
 /*
  * A 64 bit pagetable entry of S390 has following format:
  * |			 PFRA			      |0IPC|  OS  |
Signature · 1 line
2.41.0
maintainer Reviewed-by on v2 27/33
#56 lore raw

Re: [PATCH v2 27/33] s390/mm: Define KMSAN metadata for vmalloc and modules

Quoted 5 lines · Ilya Leoshkevich

The pages for the KMSAN metadata associated with most kernel mappings are taken from memblock by the common code. However, vmalloc and module metadata needs to be defined by the architectures.

Quoted 7 lines

Be a little bit more careful than x86: allocate exactly MODULES_LEN for the module shadow and origins, and then take 2/3 of vmalloc for the vmalloc shadow and origins. This ensures that users passing small vmalloc= values on the command line do not cause module metadata collisions.

(hope some s390 maintainer acks this as well)

#57 lore raw

PATCH v2 28/33 s390/string: Add KMSAN support

In reply to Ilya Leoshkevich

Add KMSAN support for the s390 implementations of the string functions. Do this similar to how it's already done for KASAN, except that the optimized memset{16,32,64}() functions need to be disabled: it's important for KMSAN to know that they initialized something.

The way boot code is built with regard to string functions is problematic, since most files think it's configured with sanitizers, but boot/string.c doesn't. This creates various problems with the memset64() definitions, depending on whether the code is built with sanitizers or fortify. This should probably be streamlined, but in the meantime resolve the issues by introducing the IN_BOOT_STRING_C macro, similar to the existing IN_ARCH_STRING_C macro.

---
 arch/s390/boot/string.c        | 16 ++++++++++++++++
 arch/s390/include/asm/string.h | 20 +++++++++++++++-----
 2 files changed, 31 insertions(+), 5 deletions(-)
arch/s390/boot/string.c +16 -0
index faccb33b462c..f6b9b1df48a8 100644
@@ -1,11 +1,18 @@
 // SPDX-License-Identifier: GPL-2.0
+#define IN_BOOT_STRING_C 1
 #include <linux/ctype.h>
 #include <linux/kernel.h>
 #include <linux/errno.h>
 #undef CONFIG_KASAN
 #undef CONFIG_KASAN_GENERIC
+#undef CONFIG_KMSAN
 #include "../lib/string.c"
 
+/*
+ * Duplicate some functions from the common lib/string.c
+ * instead of fully including it.
+ */
+
 int strncmp(const char *cs, const char *ct, size_t count)
 {
 	unsigned char c1, c2;
@@ -22,6 +29,15 @@ int strncmp(const char *cs, const char *ct, size_t count)
 	return 0;
 }
 
+void *memset64(uint64_t *s, uint64_t v, size_t count)
+{
+	uint64_t *xs = s;
+
+	while (count--)
+		*xs++ = v;
+	return s;
+}
+
 char *skip_spaces(const char *str)
 {
 	while (isspace(*str))
arch/s390/include/asm/string.h +15 -5
index 351685de53d2..2ab868cbae6c 100644
@@ -15,15 +15,12 @@
 #define __HAVE_ARCH_MEMCPY	/* gcc builtin & arch function */
 #define __HAVE_ARCH_MEMMOVE	/* gcc builtin & arch function */
 #define __HAVE_ARCH_MEMSET	/* gcc builtin & arch function */
-#define __HAVE_ARCH_MEMSET16	/* arch function */
-#define __HAVE_ARCH_MEMSET32	/* arch function */
-#define __HAVE_ARCH_MEMSET64	/* arch function */
 
 void *memcpy(void *dest, const void *src, size_t n);
 void *memset(void *s, int c, size_t n);
 void *memmove(void *dest, const void *src, size_t n);
 
-#ifndef CONFIG_KASAN
+#if !defined(CONFIG_KASAN) && !defined(CONFIG_KMSAN)
 #define __HAVE_ARCH_MEMCHR	/* inline & arch function */
 #define __HAVE_ARCH_MEMCMP	/* arch function */
 #define __HAVE_ARCH_MEMSCAN	/* inline & arch function */
@@ -36,6 +33,9 @@ void *memmove(void *dest, const void *src, size_t n);
 #define __HAVE_ARCH_STRNCPY	/* arch function */
 #define __HAVE_ARCH_STRNLEN	/* inline & arch function */
 #define __HAVE_ARCH_STRSTR	/* arch function */
+#define __HAVE_ARCH_MEMSET16	/* arch function */
+#define __HAVE_ARCH_MEMSET32	/* arch function */
+#define __HAVE_ARCH_MEMSET64	/* arch function */
 
 /* Prototypes for non-inlined arch strings functions. */
 int memcmp(const void *s1, const void *s2, size_t n);
@@ -44,7 +44,7 @@ size_t strlcat(char *dest, const char *src, size_t n);
 char *strncat(char *dest, const char *src, size_t n);
 char *strncpy(char *dest, const char *src, size_t n);
 char *strstr(const char *s1, const char *s2);
-#endif /* !CONFIG_KASAN */
+#endif /* !defined(CONFIG_KASAN) && !defined(CONFIG_KMSAN) */
 
 #undef __HAVE_ARCH_STRCHR
 #undef __HAVE_ARCH_STRNCHR
@@ -74,20 +74,30 @@ void *__memset16(uint16_t *s, uint16_t v, size_t count);
 void *__memset32(uint32_t *s, uint32_t v, size_t count);
 void *__memset64(uint64_t *s, uint64_t v, size_t count);
 
+#ifdef __HAVE_ARCH_MEMSET16
 static inline void *memset16(uint16_t *s, uint16_t v, size_t count)
 {
 	return __memset16(s, v, count * sizeof(v));
 }
+#endif
 
+#ifdef __HAVE_ARCH_MEMSET32
 static inline void *memset32(uint32_t *s, uint32_t v, size_t count)
 {
 	return __memset32(s, v, count * sizeof(v));
 }
+#endif
 
+#ifdef __HAVE_ARCH_MEMSET64
+#ifdef IN_BOOT_STRING_C
+void *memset64(uint64_t *s, uint64_t v, size_t count);
+#else
 static inline void *memset64(uint64_t *s, uint64_t v, size_t count)
 {
 	return __memset64(s, v, count * sizeof(v));
 }
+#endif
+#endif
 
 #if !defined(IN_ARCH_STRING_C) && (!defined(CONFIG_FORTIFY_SOURCE) || defined(__NO_FORTIFY))
 
Signature · 1 line
2.41.0
maintainer Reviewed-by on v2 28/33
#58 lore raw

Re: [PATCH v2 28/33] s390/string: Add KMSAN support

Quoted 14 lines · Ilya Leoshkevich

Add KMSAN support for the s390 implementations of the string functions. Do this similar to how it's already done for KASAN, except that the optimized memset{16,32,64}() functions need to be disabled: it's important for KMSAN to know that they initialized something.

The way boot code is built with regard to string functions is problematic, since most files think it's configured with sanitizers, but boot/string.c doesn't. This creates various problems with the memset64() definitions, depending on whether the code is built with sanitizers or fortify. This should probably be streamlined, but in the meantime resolve the issues by introducing the IN_BOOT_STRING_C macro, similar to the existing IN_ARCH_STRING_C macro.

Quoted 1 line
#59 lore raw

PATCH v2 29/33 s390/traps: Unpoison the kernel_stack_overflow()'s pt_regs

In reply to Ilya Leoshkevich

This is normally done by the generic entry code, but the kernel_stack_overflow() flow bypasses it.

---
 arch/s390/kernel/traps.c | 6 ++++++
 1 file changed, 6 insertions(+)
arch/s390/kernel/traps.c +6 -0
index 1d2aa448d103..f299b1203a20 100644
@@ -27,6 +27,7 @@
 #include <linux/uaccess.h>
 #include <linux/cpu.h>
 #include <linux/entry-common.h>
+#include <linux/kmsan.h>
 #include <asm/asm-extable.h>
 #include <asm/fpu/api.h>
 #include <asm/vtime.h>
@@ -260,6 +261,11 @@ static void monitor_event_exception(struct pt_regs *regs)
 
 void kernel_stack_overflow(struct pt_regs *regs)
 {
+	/*
+	 * Normally regs are unpoisoned by the generic entry code, but
+	 * kernel_stack_overflow() is a rare case that is called bypassing it.
+	 */
+	kmsan_unpoison_entry_regs(regs);
 	bust_spinlocks(1);
 	printk("Kernel stack overflow.\n");
 	show_regs(regs);
Signature · 1 line
2.41.0
#60 lore raw

PATCH v2 30/33 s390/uaccess: Add KMSAN support to put_user() and get_user()

In reply to Ilya Leoshkevich

put_user() uses inline assembly with precise constraints, so Clang is in principle capable of instrumenting it automatically. Unfortunately, one of the constraints contains a dereferenced user pointer, and Clang does not currently distinguish user and kernel pointers. Therefore KMSAN attempts to access shadow for user pointers, which is not a right thing to do.

An obvious fix to add __no_sanitize_memory to __put_user_fn() does not work, since it's __always_inline. And __always_inline cannot be removed due to the __put_user_bad() trick.

A different obvious fix of using the "a" instead of the "+Q" constraint degrades the code quality, which is very important here, since it's a hot path.

Instead, repurpose the __put_user_asm() macro to define __put_user_{char,short,int,long}_noinstr() functions and mark them with __no_sanitize_memory. For the non-KMSAN builds make them __always_inline in order to keep the generated code quality. Also define __put_user_{char,short,int,long}() functions, which call the aforementioned ones and which *are* instrumented, because they call KMSAN hooks, which may be implemented as macros.

The same applies to get_user() as well.

---
 arch/s390/include/asm/uaccess.h | 110 ++++++++++++++++++++++----------
 1 file changed, 78 insertions(+), 32 deletions(-)
arch/s390/include/asm/uaccess.h +78 -32
index 81ae8a98e7ec..b0715b88b55a 100644
@@ -78,13 +78,23 @@ union oac {
 
 int __noreturn __put_user_bad(void);
 
-#define __put_user_asm(to, from, size)					\
-({									\
+#ifdef CONFIG_KMSAN
+#define GET_PUT_USER_NOINSTR_ATTRIBUTES inline __no_sanitize_memory
+#else
+#define GET_PUT_USER_NOINSTR_ATTRIBUTES __always_inline
+#endif
+
+#define DEFINE_PUT_USER(type)						\
+static GET_PUT_USER_NOINSTR_ATTRIBUTES int				\
+__put_user_##type##_noinstr(unsigned type __user *to,			\
+			    unsigned type *from,			\
+			    unsigned long size)				\
+{									\
 	union oac __oac_spec = {					\
 		.oac1.as = PSW_BITS_AS_SECONDARY,			\
 		.oac1.a = 1,						\
 	};								\
-	int __rc;							\
+	int rc;								\
 									\
 	asm volatile(							\
 		"	lr	0,%[spec]\n"				\
@@ -93,12 +103,28 @@ int __noreturn __put_user_bad(void);
 		"2:\n"							\
 		EX_TABLE_UA_STORE(0b, 2b, %[rc])			\
 		EX_TABLE_UA_STORE(1b, 2b, %[rc])			\
-		: [rc] "=&d" (__rc), [_to] "+Q" (*(to))			\
+		: [rc] "=&d" (rc), [_to] "+Q" (*(to))			\
 		: [_size] "d" (size), [_from] "Q" (*(from)),		\
 		  [spec] "d" (__oac_spec.val)				\
 		: "cc", "0");						\
-	__rc;								\
-})
+	return rc;							\
+}									\
+									\
+static __always_inline int						\
+__put_user_##type(unsigned type __user *to, unsigned type *from,	\
+		  unsigned long size)					\
+{									\
+	int rc;								\
+									\
+	rc = __put_user_##type##_noinstr(to, from, size);		\
+	instrument_put_user(*from, to, size);				\
+	return rc;							\
+}
+
+DEFINE_PUT_USER(char);
+DEFINE_PUT_USER(short);
+DEFINE_PUT_USER(int);
+DEFINE_PUT_USER(long);
 
 static __always_inline int __put_user_fn(void *x, void __user *ptr, unsigned long size)
 {
@@ -106,24 +132,24 @@ static __always_inline int __put_user_fn(void *x, void __user *ptr, unsigned lon
 
 	switch (size) {
 	case 1:
-		rc = __put_user_asm((unsigned char __user *)ptr,
-				    (unsigned char *)x,
-				    size);
+		rc = __put_user_char((unsigned char __user *)ptr,
+				     (unsigned char *)x,
+				     size);
 		break;
 	case 2:
-		rc = __put_user_asm((unsigned short __user *)ptr,
-				    (unsigned short *)x,
-				    size);
+		rc = __put_user_short((unsigned short __user *)ptr,
+				      (unsigned short *)x,
+				      size);
 		break;
 	case 4:
-		rc = __put_user_asm((unsigned int __user *)ptr,
+		rc = __put_user_int((unsigned int __user *)ptr,
 				    (unsigned int *)x,
 				    size);
 		break;
 	case 8:
-		rc = __put_user_asm((unsigned long __user *)ptr,
-				    (unsigned long *)x,
-				    size);
+		rc = __put_user_long((unsigned long __user *)ptr,
+				     (unsigned long *)x,
+				     size);
 		break;
 	default:
 		__put_user_bad();
@@ -134,13 +160,17 @@ static __always_inline int __put_user_fn(void *x, void __user *ptr, unsigned lon
 
 int __noreturn __get_user_bad(void);
 
-#define __get_user_asm(to, from, size)					\
-({									\
+#define DEFINE_GET_USER(type)						\
+static GET_PUT_USER_NOINSTR_ATTRIBUTES int				\
+__get_user_##type##_noinstr(unsigned type *to,				\
+			    unsigned type __user *from,			\
+			    unsigned long size)				\
+{									\
 	union oac __oac_spec = {					\
 		.oac2.as = PSW_BITS_AS_SECONDARY,			\
 		.oac2.a = 1,						\
 	};								\
-	int __rc;							\
+	int rc;								\
 									\
 	asm volatile(							\
 		"	lr	0,%[spec]\n"				\
@@ -149,13 +179,29 @@ int __noreturn __get_user_bad(void);
 		"2:\n"							\
 		EX_TABLE_UA_LOAD_MEM(0b, 2b, %[rc], %[_to], %[_ksize])	\
 		EX_TABLE_UA_LOAD_MEM(1b, 2b, %[rc], %[_to], %[_ksize])	\
-		: [rc] "=&d" (__rc), "=Q" (*(to))			\
+		: [rc] "=&d" (rc), "=Q" (*(to))				\
 		: [_size] "d" (size), [_from] "Q" (*(from)),		\
 		  [spec] "d" (__oac_spec.val), [_to] "a" (to),		\
 		  [_ksize] "K" (size)					\
 		: "cc", "0");						\
-	__rc;								\
-})
+	return rc;							\
+}									\
+									\
+static __always_inline int						\
+__get_user_##type(unsigned type *to, unsigned type __user *from,	\
+		  unsigned long size)					\
+{									\
+	int rc;								\
+									\
+	rc = __get_user_##type##_noinstr(to, from, size);		\
+	instrument_get_user(*to);					\
+	return rc;							\
+}
+
+DEFINE_GET_USER(char);
+DEFINE_GET_USER(short);
+DEFINE_GET_USER(int);
+DEFINE_GET_USER(long);
 
 static __always_inline int __get_user_fn(void *x, const void __user *ptr, unsigned long size)
 {
@@ -163,24 +209,24 @@ static __always_inline int __get_user_fn(void *x, const void __user *ptr, unsign
 
 	switch (size) {
 	case 1:
-		rc = __get_user_asm((unsigned char *)x,
-				    (unsigned char __user *)ptr,
-				    size);
+		rc = __get_user_char((unsigned char *)x,
+				     (unsigned char __user *)ptr,
+				     size);
 		break;
 	case 2:
-		rc = __get_user_asm((unsigned short *)x,
-				    (unsigned short __user *)ptr,
-				    size);
+		rc = __get_user_short((unsigned short *)x,
+				      (unsigned short __user *)ptr,
+				      size);
 		break;
 	case 4:
-		rc = __get_user_asm((unsigned int *)x,
+		rc = __get_user_int((unsigned int *)x,
 				    (unsigned int __user *)ptr,
 				    size);
 		break;
 	case 8:
-		rc = __get_user_asm((unsigned long *)x,
-				    (unsigned long __user *)ptr,
-				    size);
+		rc = __get_user_long((unsigned long *)x,
+				     (unsigned long __user *)ptr,
+				     size);
 		break;
 	default:
 		__get_user_bad();
Signature · 1 line
2.41.0
#61 lore raw

Re: [PATCH v2 30/33] s390/uaccess: Add KMSAN support to put_user() and get_user()

Quoted 24 lines · Ilya Leoshkevich

put_user() uses inline assembly with precise constraints, so Clang is in principle capable of instrumenting it automatically. Unfortunately, one of the constraints contains a dereferenced user pointer, and Clang does not currently distinguish user and kernel pointers. Therefore KMSAN attempts to access shadow for user pointers, which is not a right thing to do.

An obvious fix to add __no_sanitize_memory to __put_user_fn() does not work, since it's __always_inline. And __always_inline cannot be removed due to the __put_user_bad() trick.

A different obvious fix of using the "a" instead of the "+Q" constraint degrades the code quality, which is very important here, since it's a hot path.

Instead, repurpose the __put_user_asm() macro to define __put_user_{char,short,int,long}_noinstr() functions and mark them with __no_sanitize_memory. For the non-KMSAN builds make them __always_inline in order to keep the generated code quality. Also define __put_user_{char,short,int,long}() functions, which call the aforementioned ones and which *are* instrumented, because they call KMSAN hooks, which may be implemented as macros.

Quoted 4 lines

The same applies to get_user() as well.

I think this patch makes sense, but I don't feel myself qualified enough to stamp it. Hope Heiko's ack is enough.

#62 lore raw

PATCH v2 31/33 s390/unwind: Disable KMSAN checks

In reply to Ilya Leoshkevich

The unwind code can read uninitialized frames. Furthermore, even in the good case, KMSAN does not emit shadow for backchains. Therefore disable it for the unwinding functions.

---
 arch/s390/kernel/unwind_bc.c | 4 ++++
 1 file changed, 4 insertions(+)
arch/s390/kernel/unwind_bc.c +4 -0
index 0ece156fdd7c..cd44be2b6ce8 100644
@@ -49,6 +49,8 @@ static inline bool is_final_pt_regs(struct unwind_state *state,
 	       READ_ONCE_NOCHECK(regs->psw.mask) & PSW_MASK_PSTATE;
 }
 
+/* Avoid KMSAN false positives from touching uninitialized frames. */
+__no_kmsan_checks
 bool unwind_next_frame(struct unwind_state *state)
 {
 	struct stack_info *info = &state->stack_info;
@@ -118,6 +120,8 @@ bool unwind_next_frame(struct unwind_state *state)
 }
 EXPORT_SYMBOL_GPL(unwind_next_frame);
 
+/* Avoid KMSAN false positives from touching uninitialized frames. */
+__no_kmsan_checks
 void __unwind_start(struct unwind_state *state, struct task_struct *task,
 		    struct pt_regs *regs, unsigned long first_frame)
 {
Signature · 1 line
2.41.0
#63 lore raw

PATCH v2 32/33 s390: Implement the architecture-specific kmsan functions

In reply to Ilya Leoshkevich

arch_kmsan_get_meta_or_null() finds the lowcore shadow by querying the prefix and calling kmsan_get_metadata() again.

kmsan_virt_addr_valid() delegates to virt_addr_valid().

---
 arch/s390/include/asm/kmsan.h | 36 +++++++++++++++++++++++++++++++++++
 1 file changed, 36 insertions(+)
 create mode 100644 arch/s390/include/asm/kmsan.h
arch/s390/include/asm/kmsan.h new file +36 -0
new file mode 100644
index 000000000000..afec71e9e9ac
@@ -0,0 +1,36 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef _ASM_S390_KMSAN_H
+#define _ASM_S390_KMSAN_H
+
+#include <asm/lowcore.h>
+#include <asm/page.h>
+#include <linux/kmsan.h>
+#include <linux/mmzone.h>
+#include <linux/stddef.h>
+
+#ifndef MODULE
+
+static inline void *arch_kmsan_get_meta_or_null(void *addr, bool is_origin)
+{
+	if (addr >= (void *)&S390_lowcore &&
+	    addr < (void *)(&S390_lowcore + 1)) {
+		/*
+		 * Different lowcores accessed via S390_lowcore are described
+		 * by the same struct page. Resolve the prefix manually in
+		 * order to get a distinct struct page.
+		 */
+		addr += (void *)lowcore_ptr[raw_smp_processor_id()] -
+			(void *)&S390_lowcore;
+		return kmsan_get_metadata(addr, is_origin);
+	}
+	return NULL;
+}
+
+static inline bool kmsan_virt_addr_valid(void *addr)
+{
+	return virt_addr_valid(addr);
+}
+
+#endif /* !MODULE */
+
+#endif /* _ASM_S390_KMSAN_H */
Signature · 1 line
2.41.0
#64 lore raw

Re: [PATCH v2 32/33] s390: Implement the architecture-specific kmsan functions

Quoted 5 lines
arch/s390/include/asm/kmsan.h new file +5 -0
+static inline void *arch_kmsan_get_meta_or_null(void *addr, bool is_origin)
+{
+       if (addr >= (void *)&S390_lowcore &&
+           addr < (void *)(&S390_lowcore + 1)) {
+               /*

arch/s390/include/asm/kmsan.h line 27

Quoted 10 lines
arch/s390/include/asm/kmsan.h new file +10 -0
+                * Different lowcores accessed via S390_lowcore are described
+                * by the same struct page. Resolve the prefix manually in
+                * order to get a distinct struct page.
+                */
+               addr += (void *)lowcore_ptr[raw_smp_processor_id()] -
+                       (void *)&S390_lowcore;
+               return kmsan_get_metadata(addr, is_origin);
+       }
+       return NULL;
+}

Is there a possibility for infinite recursion here? E.g. can lowcore_ptr[raw_smp_processor_id()] point somewhere in between (void *)&S390_lowcore and (void *)(&S390_lowcore + 1))?

#65 lore raw

Re: [PATCH v2 32/33] s390: Implement the architecture-specific kmsan functions

Quoted 15 lines · Alexander Potapenko
Quoted 15 lines

+static inline void *arch_kmsan_get_meta_or_null(void *addr, bool is_origin)
+{
+       if (addr >= (void *)&S390_lowcore &&
+           addr < (void *)(&S390_lowcore + 1)) {
+               /*
+                * Different lowcores accessed via S390_lowcore are described
+                * by the same struct page. Resolve the prefix manually in
+                * order to get a distinct struct page. +                */
+               addr += (void *)lowcore_ptr[raw_smp_processor_id()] -
+                       (void *)&S390_lowcore;

Quoted 8 lines
Quoted 4 lines

+               return kmsan_get_metadata(addr, is_origin); +       }
+       return NULL;
+}

Is there a possibility for infinite recursion here? E.g. can lowcore_ptr[raw_smp_processor_id()] point somewhere in between (void *)&S390_lowcore and (void *)(&S390_lowcore + 1))?

No, it's allocated with __get_free_pages() or memblock_alloc_low(). But since this question came up, I should probably add a check and a WARN_ON_ONCE() here.

#66 lore raw

Re: [PATCH v2 32/33] s390: Implement the architecture-specific kmsan functions

Quoted 7 lines
Quoted 3 lines

Is there a possibility for infinite recursion here? E.g. can lowcore_ptr[raw_smp_processor_id()] point somewhere in between (void *)&S390_lowcore and (void *)(&S390_lowcore + 1))?

No, it's allocated with __get_free_pages() or memblock_alloc_low(). But since this question came up, I should probably add a check and a WARN_ON_ONCE() here.

Yes, please.

Signature · 10 lines
Alexander Potapenko
Software Engineer

Google Germany GmbH
Erika-Mann-Straße, 33
80636 München

Geschäftsführer: Paul Manicle, Liana Sebastian
Registergericht und -nummer: Hamburg, HRB 86891
Sitz der Gesellschaft: Hamburg
#67 lore raw

PATCH v2 33/33 kmsan: Enable on s390

In reply to Ilya Leoshkevich

Now that everything else is in place, enable KMSAN in Kconfig.

---
 arch/s390/Kconfig | 1 +
 1 file changed, 1 insertion(+)
arch/s390/Kconfig +1 -0
index 3bec98d20283..160ad2220c53 100644
@@ -153,6 +153,7 @@ config S390
 	select HAVE_ARCH_KASAN
 	select HAVE_ARCH_KASAN_VMALLOC
 	select HAVE_ARCH_KCSAN
+	select HAVE_ARCH_KMSAN
 	select HAVE_ARCH_KFENCE
 	select HAVE_ARCH_RANDOMIZE_KSTACK_OFFSET
 	select HAVE_ARCH_SECCOMP_FILTER
Signature · 1 line
2.41.0
#68 lore raw

Hi Ilya,

Sorry for this taking so long, I'll probably take a closer look next week. Overall, the s390 part looks good to me, but I wanted to check the x86 behavior once again (and perhaps figure out how to avoid introducing another way to disable KMSAN).
Do you happen to have a Git repo with your patches somewhere?

#69 lore raw
Quoted 9 lines · Alexander Potapenko

Hi Ilya,

Sorry for this taking so long, I'll probably take a closer look next week.
Overall, the s390 part looks good to me, but I wanted to check the x86
behavior once again (and perhaps figure out how to avoid introducing another way to disable KMSAN).
Do you happen to have a Git repo with your patches somewhere?

Hi, yes, the latest version of the patches is available at [1].

[1] https://github.com/iii-i/linux/tree/kmsan