mm: Add PAGE_ALIGN_DOWN macro

This is v2 of the KASAN/UML port. It should be ready to go.

It does benefit significantly from the following patches:
- Bugfix for memory corruption, needed for KASAN_STACK support:
https://lore.kernel.org/lkml/[email protected]/

+++ b/arch/um/os-Linux/user_syms.c

arch/um/os-Linux/user_syms.c +2 -2

@@ -27,10 +27,10 @@ EXPORT_SYMBOL(strstr);
 #ifndef __x86_64__
 extern void *memcpy(void *, const void *, size_t);
 EXPORT_SYMBOL(memcpy);
-#endif
-
 EXPORT_SYMBOL(memmove);
 EXPORT_SYMBOL(memset);
+#endif
+
 EXPORT_SYMBOL(printf);
 
 /* Here, instead, I can provide a fake prototype. Yes, someone cares: genksyms.

arch/x86/um/Makefile +0 -0

index ba5789c35809..f778e37494ba 100644

Quoted 2 lines · David Gow

This is v2 of the KASAN/UML port. It should be ready to go.

Nice, thanks a lot! :)

Quoted 3 lines

It does benefit significantly from the following patches:
- Bugfix for memory corruption, needed for KASAN_STACK support:
https://lore.kernel.org/lkml/[email protected]/

Btw, oddly enough, I don't seem to actually see this (tried gcc 10.3 and 11.3 so far) - is there anything you know about compiler versions related to this perhaps? Or clang only?

Quoted 2 lines · David Gow

This is v2 of the KASAN/UML port. It should be ready to go.

Nice, thanks a lot! :)

Quoted 3 lines

It does benefit significantly from the following patches:
- Bugfix for memory corruption, needed for KASAN_STACK support:
https://lore.kernel.org/lkml/[email protected]/

Btw, oddly enough, I don't seem to actually see this (tried gcc 10.3 and 11.3 so far) - is there anything you know about compiler versions related to this perhaps? Or clang only?

The kasan_stack_oob test passes though, and generally 45 tests pass and 10 are skipped.

arch/um/Makefile USER_CFLAGS = $(patsubst $(KERNEL_DEFINES),,$(patsubst -I%,,$(KBUILD_CFLAGS))) \ · line 82

Quoted 7 lines

+# Kernel config options are not included in USER_CFLAGS, but the option for KASAN

arch/um/Makefile +4 -0

+# should be included if the KASAN config option was set.
+ifdef CONFIG_KASAN
+     USER_CFLAGS+=-DCONFIG_KASAN=y
+endif

I'm not sure that's (still?) necessary - you don't #ifdef on it anywhere in the user code; perhaps the original intent had been to #ifdef kasan_map_memory()?

Quoted 19 lines

+++ b/arch/um/os-Linux/user_syms.c

arch/um/os-Linux/user_syms.c +2 -2

@@ -27,10 +27,10 @@ EXPORT_SYMBOL(strstr);
 #ifndef __x86_64__
 extern void *memcpy(void *, const void *, size_t);
 EXPORT_SYMBOL(memcpy);
-#endif
-
 EXPORT_SYMBOL(memmove);
 EXPORT_SYMBOL(memset);
+#endif
+
 EXPORT_SYMBOL(printf);
 
 /* Here, instead, I can provide a fake prototype. Yes, someone cares: genksyms.

arch/x86/um/Makefile +0 -0

index ba5789c35809..f778e37494ba 100644

arch/x86/um/Makefile else · line 32

Quoted 6 lines

arch/x86/um/Makefile +2 -1

@@ -28,7 +28,8 @@ else
 
 obj-y += syscalls_64.o vdso/
 
-subarch-y = ../lib/csum-partial_64.o ../lib/memcpy_64.o ../entry/thunk_64.o
+subarch-y = ../lib/csum-partial_64.o ../lib/memcpy_64.o ../entry/thunk_64.o \
+     ../lib/memmove_64.o ../lib/memset_64.o

I wonder if we should make these two changes contingent on KASAN too, I seem to remember that we had some patches from Anton flying around at some point to use glibc string routines, since they can be even more optimised (we're in user space, after all).

But I suppose for now this doesn't really matter, and even if we did use them, they'd come from libasan anyway?

Anyway, looks good to me, not sure the little not above about the user cflags matters.

Reviewed-by Johannes Berg <[email protected]>

The UML-specific KASAN initializer uses mmap to map the roughly 2.25TB

Quoted 2 lines · David Gow

The UML-specific KASAN initializer uses mmap to map the roughly 2.25TB

You say 2.25TB here, and

arch/um/Kconfig config UML_TIME_TRAVEL_SUPPORT · line 229

Quoted 6 lines

arch/um/Kconfig +6 -0

+config KASAN_SHADOW_OFFSET
+     hex
+     depends on KASAN
+     default 0x100000000000
+     help
+       This is the offset at which the ~2.25TB of shadow memory is

here too, of course.

But I notice that I get ~16TB address space use when running,

arch/um/include/asm/kasan.h line 17

Quoted 8 lines

arch/um/include/asm/kasan.h new file +8 -0

+/* used in kasan_mem_to_shadow to divide by 8 */
+#define KASAN_SHADOW_SCALE_SHIFT 3
+
+#ifdef CONFIG_X86_64
+#define KASAN_HOST_USER_SPACE_END_ADDR 0x00007fffffffffffUL
+/* KASAN_SHADOW_SIZE is the size of total address space divided by 8 */
+#define KASAN_SHADOW_SIZE ((KASAN_HOST_USER_SPACE_END_ADDR + 1) >> \
+                     KASAN_SHADOW_SCALE_SHIFT)

because this ends up being 0x100000000000, i.e. 16 TiB.

Is that intentional? Was something missed? Maybe
KASAN_HOST_USER_SPACE_END_ADDR was too big?

It doesn't really matter, but I guess then the documentation should be updated.

From: Patricia Alfonso <[email protected]>

Make KASAN run on User Mode Linux on x86_64.

The UML-specific KASAN initializer uses mmap to map the roughly 2.25TB of shadow memory to the location defined by KASAN_SHADOW_OFFSET. kasan_init() utilizes constructors to initialize KASAN before main().

The location of the KASAN shadow memory, starting at KASAN_SHADOW_OFFSET, can be configured using the KASAN_SHADOW_OFFSET option. UML uses roughly 18TB of address space, and KASAN requires 1/8th of this. The default location of this offset is 0x100000000000, which keeps it out-of-the-way even on UML setups with more "physical" memory.

For low-memory setups, 0x7fff8000 can be used instead, which fits in an immediate and is therefore faster, as suggested by Dmitry Vyukov. There is usually enough free space at this location; however, it is a config option so that it can be easily changed if needed.

Note that, unlike KASAN on other architectures, vmalloc allocations still use the shadow memory allocated upfront, rather than allocating and free-ing it per-vmalloc allocation.

Also note that, while UML supports both KASAN in inline mode (CONFIG_KASAN_INLINE) and static linking (CONFIG_STATIC_LINK), it does not support both at the same time.

Signed-off-by Patricia Alfonso <[email protected]>

Co-developed-by Vincent Whitchurch <[email protected]>

Signed-off-by Vincent Whitchurch <[email protected]>

Signed-off-by David Gow <[email protected]>

mm/kasan/shadow.c +0 -0

index a4f07de21771..c993d99116f2 100644

mm/kasan/shadow.c +23 -1

+        *
+        * For the time being, though, this check works. The remaining CONFIG_UML
+        * checks in this file exist for the same reason.
+        */
+       if (IS_ENABLED(CONFIG_UML)) {
+               __memset((void *)shadow_start, KASAN_VMALLOC_INVALID, shadow_end - shadow_start);
+               return 0;
+       }
+
+       shadow_start = PAGE_ALIGN_DOWN(shadow_start);
+       shadow_end = PAGE_ALIGN(shadow_end);
 
        ret = apply_to_page_range(&init_mm, shadow_start,
                                  shadow_end - shadow_start,

@@ -466,6 +486,10 @@ void kasan_release_vmalloc(unsigned long start, unsigned long end,
 
        if (shadow_end > shadow_start) {
                size = shadow_end - shadow_start;
+               if (IS_ENABLED(CONFIG_UML)) {
+                       __memset(shadow_start, KASAN_SHADOW_INIT, shadow_end - shadow_start);
+                       return;
+               }
                apply_to_existing_page_range(&init_mm,
                                             (unsigned long)shadow_start,
                                             size, kasan_depopulate_vmalloc_pte,

@@ -531,6 +555,11 @@ int kasan_alloc_module_shadow(void *addr, size_t size, gfp_t gfp_mask)
        if (WARN_ON(!PAGE_ALIGNED(shadow_start)))
                return -EINVAL;
 
+       if (IS_ENABLED(CONFIG_UML)) {
+               __memset((void *)shadow_start, KASAN_SHADOW_INIT, shadow_size);
+               return 0;
+       }
+
        ret = __vmalloc_node_range(shadow_size, 1, shadow_start,
                        shadow_start + shadow_size,
                        GFP_KERNEL,

@@ -554,6 +583,9 @@ int kasan_alloc_module_shadow(void *addr, size_t size, gfp_t gfp_mask)
 
 void kasan_free_module_shadow(const struct vm_struct *vm)
 {
+       if (IS_ENABLED(CONFIG_UML))
+               return;
+
        if (vm->flags & VM_KASAN)
                vfree(kasan_mem_to_shadow(vm->addr));
 }
--

2.36.1.124.g0e6072fb45-goog

Quoted 24 lines · David Gow

From: Patricia Alfonso <[email protected]>

Make KASAN run on User Mode Linux on x86_64.

The UML-specific KASAN initializer uses mmap to map the roughly 2.25TB of shadow memory to the location defined by KASAN_SHADOW_OFFSET. kasan_init() utilizes constructors to initialize KASAN before main().

The location of the KASAN shadow memory, starting at KASAN_SHADOW_OFFSET, can be configured using the KASAN_SHADOW_OFFSET option. UML uses roughly 18TB of address space, and KASAN requires 1/8th of this. The default location of this offset is 0x100000000000, which keeps it out-of-the-way even on UML setups with more "physical" memory.

For low-memory setups, 0x7fff8000 can be used instead, which fits in an immediate and is therefore faster, as suggested by Dmitry Vyukov. There is usually enough free space at this location; however, it is a config option so that it can be easily changed if needed.

Note that, unlike KASAN on other architectures, vmalloc allocations still use the shadow memory allocated upfront, rather than allocating and free-ing it per-vmalloc allocation.

Quoted 8 lines

Also note that, while UML supports both KASAN in inline mode (CONFIG_KASAN_INLINE) and static linking (CONFIG_STATIC_LINK), it does not support both at the same time.

Signed-off-by Patricia Alfonso <[email protected]>

Co-developed-by Vincent Whitchurch <[email protected]>

Signed-off-by Vincent Whitchurch <[email protected]>

Signed-off-by David Gow <[email protected]>

Hi David,

Thanks for working on this!

Quoted 5 lines

mm/kasan/shadow.c +0 -0

index a4f07de21771..c993d99116f2 100644

mm/kasan/shadow.c int kasan_populate_vmalloc(unsigned long addr, unsigned long size) · line 302

Quoted 10 lines

mm/kasan/shadow.c +4 -2

@@ -295,9 +295,29 @@ int kasan_populate_vmalloc(unsigned long addr, unsigned long size)
                return 0;
 
        shadow_start = (unsigned long)kasan_mem_to_shadow((void *)addr);
-       shadow_start = ALIGN_DOWN(shadow_start, PAGE_SIZE);
        shadow_end = (unsigned long)kasan_mem_to_shadow((void *)addr + size);
-       shadow_end = ALIGN(shadow_end, PAGE_SIZE);
+
+       /*
+        * User Mode Linux maps enough shadow memory for all of physical memory
+        * at boot, so doesn't need to allocate more on vmalloc, just clear it.

Should this say "for all of _virtual_ memory"?

Otherwise, this is confusing. All KASAN-enabled architectures map shadow for physical memory. And they still need map shadow for vmalloc() separately. This is what kasan_populate_vmalloc() is for.

mm/kasan/shadow.c int kasan_populate_vmalloc(unsigned long addr, unsigned long size) · line 309

Quoted 7 lines

mm/kasan/shadow.c +7 -0

+        *
+        * If another architecture chooses to go down the same path, we should
+        * replace this check for CONFIG_UML with something more generic, such
+        * as:
+        * - A CONFIG_KASAN_NO_SHADOW_ALLOC option, which architectures could set
+        * - or, a way of having architecture-specific versions of these vmalloc
+        *   and module shadow memory allocation options.

I think this part above and the first sentence below belong to the commit changelog, not to a comment.

Quoted 48 lines

mm/kasan/shadow.c +23 -1

+        *
+        * For the time being, though, this check works. The remaining CONFIG_UML
+        * checks in this file exist for the same reason.
+        */
+       if (IS_ENABLED(CONFIG_UML)) {
+               __memset((void *)shadow_start, KASAN_VMALLOC_INVALID, shadow_end - shadow_start);
+               return 0;
+       }
+
+       shadow_start = PAGE_ALIGN_DOWN(shadow_start);
+       shadow_end = PAGE_ALIGN(shadow_end);
 
        ret = apply_to_page_range(&init_mm, shadow_start,
                                  shadow_end - shadow_start,

@@ -466,6 +486,10 @@ void kasan_release_vmalloc(unsigned long start, unsigned long end,
 
        if (shadow_end > shadow_start) {
                size = shadow_end - shadow_start;
+               if (IS_ENABLED(CONFIG_UML)) {
+                       __memset(shadow_start, KASAN_SHADOW_INIT, shadow_end - shadow_start);
+                       return;
+               }
                apply_to_existing_page_range(&init_mm,
                                             (unsigned long)shadow_start,
                                             size, kasan_depopulate_vmalloc_pte,

@@ -531,6 +555,11 @@ int kasan_alloc_module_shadow(void *addr, size_t size, gfp_t gfp_mask)
        if (WARN_ON(!PAGE_ALIGNED(shadow_start)))
                return -EINVAL;
 
+       if (IS_ENABLED(CONFIG_UML)) {
+               __memset((void *)shadow_start, KASAN_SHADOW_INIT, shadow_size);
+               return 0;
+       }
+
        ret = __vmalloc_node_range(shadow_size, 1, shadow_start,
                        shadow_start + shadow_size,
                        GFP_KERNEL,

@@ -554,6 +583,9 @@ int kasan_alloc_module_shadow(void *addr, size_t size, gfp_t gfp_mask)
 
 void kasan_free_module_shadow(const struct vm_struct *vm)
 {
+       if (IS_ENABLED(CONFIG_UML))
+               return;
+
        if (vm->flags & VM_KASAN)
                vfree(kasan_mem_to_shadow(vm->addr));
 }
--

Quoted 2 lines

2.36.1.124.g0e6072fb45-goog

Thanks!

This is just the same as PAGE_ALIGN(), but rounds the address down, not up.