Skip to contentSkip to conversation

MIPS64: add KASAN support

PATCH In review Tommy Jin 3 participants 3 messages Aug 27, 2019 – Sep 5, 2019

Subsystems (3)
lore
Share & tools
lore

Tommy Jin

author
#0 lore raw

PATCH 1/1 MIPS64: add KASAN support

From: tjin <[email protected]>

This patch adds arch specific code for kernel address sanitizer

1/8 of kernel addresses reserved for shadow memory. But for misp64, There are a lot of holes between different segments and valid address space(256T available) is insufficient to map all these segments (Scattered in 8192P space) to kasan shadow memory with the common formula provided by kasan core, saying
addr >> KASAN_SHADOW_SCALE_SHIFT) + KASAN_SHADOW_OFFSET

So MIPS64 has a ARCH specific mapping formula,different segments are mapped individually, and only limited length of space of that specific segment is mapped to shadow. for example,XKPHYS CACHE starts from 0xa800000000000000, around 1T of this segment is going be mapped to shadow in case that 40-bit address is using.
XKPHYS UNCACHE starts from 0x9000000000000000, the gap between 0xa800000000000000 + 1T and 0x9000000000000000 is not going to be mapped to shadow.

At early boot stage the whole shadow region populated with just one physical page (kasan_early_shadow_page). Later, this page is reused as readonly zero shadow for some memory that Kasan currently don't track.
After mapping the physical memory, pages for shadow memory are allocated and mapped.

Functions like memset/memmove/memcpy do a lot of memory accesses. If bad pointer passed to one of these function it is important to catch this. Compiler's instrumentation cannot do this since these functions are written in assembly.
KASan replaces memory functions with manually instrumented variants. Original functions declared as weak symbols so strong definitions in mm/kasan/kasan.c could replace them. Original functions have aliases with '__' prefix in name, so we could call non-instrumented variant if needed.

Some files built without kasan instrumentation(e.g. mm/slub.c). Original mem* function replaced (via #define) with prefixed variants to disable memory access checks for such files.

With GCC, it requires 9.0 or later version for the basic support of MIPS address SANitizer.

---
 arch/mips/Kconfig                  |   1 +
 arch/mips/include/asm/kasan.h      | 201 ++++++++++++++++++++++++++++++++++
 arch/mips/include/asm/pgtable-64.h |  12 ++-
 arch/mips/include/asm/string.h     |  20 ++++
 arch/mips/kernel/Makefile          |   8 ++
 arch/mips/kernel/head.S            |   3 +
 arch/mips/kernel/setup.c           |   3 +
 arch/mips/kernel/traps.c           |  11 ++
 arch/mips/lib/memcpy.S             |  20 ++--
 arch/mips/lib/memset.S             |  18 ++--
 arch/mips/mm/Makefile              |   5 +
 arch/mips/mm/kasan_init.c          | 216 +++++++++++++++++++++++++++++++++++++
 arch/mips/vdso/Makefile            |   5 +
 include/linux/kasan.h              |   2 +
 mm/kasan/generic.c                 |  17 ++-
 mm/kasan/kasan.h                   |   2 +
 16 files changed, 522 insertions(+), 22 deletions(-)
 create mode 100644 arch/mips/include/asm/kasan.h
 create mode 100644 arch/mips/mm/kasan_init.c
arch/mips/Kconfig +1 -0
index 70d3200..0ed8eb0 100644
@@ -80,6 +80,7 @@ config MIPS
 	select RTC_LIB
 	select SYSCTL_EXCEPTION_TRACE
 	select VIRT_TO_BUS
+	select HAVE_ARCH_KASAN if 64BIT
 
 menu "Machine selection"
 
arch/mips/include/asm/kasan.h new file +201 -0

201 lines not on this page · raw message

arch/mips/include/asm/pgtable-64.h +10 -2
index 93a9dce..e3f5c4e 100644
@@ -144,10 +144,17 @@
  * reliably trap.
  */
 #define VMALLOC_START		(MAP_BASE + (2 * PAGE_SIZE))
+#ifdef CONFIG_KASAN
+#define VMALLOC_END	\
+	(MAP_BASE + \
+	 min(PTRS_PER_PGD * PTRS_PER_PUD * PTRS_PER_PMD * PTRS_PER_PTE * PAGE_SIZE / 2, \
+	     (1UL << cpu_vmbits)) - (1UL << 32))
+#else
 #define VMALLOC_END	\
 	(MAP_BASE + \
 	 min(PTRS_PER_PGD * PTRS_PER_PUD * PTRS_PER_PMD * PTRS_PER_PTE * PAGE_SIZE, \
 	     (1UL << cpu_vmbits)) - (1UL << 32))
+#endif
 
 #if defined(CONFIG_MODULES) && defined(KBUILD_64BIT_SYM32) && \
 	VMALLOC_START != CKSSEG
@@ -352,7 +359,8 @@ static inline pmd_t *pmd_offset(pud_t * pud, unsigned long address)
 #define pte_offset_map(dir, address)					\
 	((pte_t *)page_address(pmd_page(*(dir))) + __pte_offset(address))
 #define pte_unmap(pte) ((void)(pte))
-
+#define pte_index(addr)		(((addr) >> PAGE_SHIFT) & (PTRS_PER_PTE - 1))
+#define pte_none(pte)		(!(pte_val(pte) & ~_PAGE_GLOBAL))
 /*
  * Initialize a new pgd / pmd table with invalid pointers.
  */
@@ -372,5 +380,5 @@ static inline pte_t mk_swap_pte(unsigned long type, unsigned long offset)
 #define __swp_entry(type, offset) ((swp_entry_t) { pte_val(mk_swap_pte((type), (offset))) })
 #define __pte_to_swp_entry(pte) ((swp_entry_t) { pte_val(pte) })
 #define __swp_entry_to_pte(x)	((pte_t) { (x).val })
-
+#define sym_to_pfn(x)	    __phys_to_pfn(__pa_symbol(x))
 #endif /* _ASM_PGTABLE_64_H */
arch/mips/include/asm/string.h +20 -0
index 29030cb..19d3740 100644
@@ -133,11 +133,31 @@ strncmp(__const__ char *__cs, __const__ char *__ct, size_t __count)
 
 #define __HAVE_ARCH_MEMSET
 extern void *memset(void *__s, int __c, size_t __count);
+extern void *__memset(void *__s, int __c, size_t __count);
 
 #define __HAVE_ARCH_MEMCPY
 extern void *memcpy(void *__to, __const__ void *__from, size_t __n);
+extern void *__memcpy(void *__to, __const__ void *__from, size_t __n);
 
 #define __HAVE_ARCH_MEMMOVE
 extern void *memmove(void *__dest, __const__ void *__src, size_t __n);
+extern void *__memmove(void *__dest, __const__ void *__src, size_t __n);
+
+#if defined(CONFIG_KASAN) && !defined(__SANITIZE_ADDRESS__)
+
+/*
+ * For files that are not instrumented (e.g. mm/slub.c) we
+ * should use not instrumented version of mem* functions.
+ */
+
+#define memcpy(dst, src, len) __memcpy(dst, src, len)
+#define memmove(dst, src, len) __memmove(dst, src, len)
+#define memset(s, c, n) __memset(s, c, n)
+
+#ifndef __NO_FORTIFY
+#define __NO_FORTIFY /* FORTIFY_SOURCE uses __builtin_memcpy, etc. */
+#endif
+
+#endif
 
 #endif /* _ASM_STRING_H */
arch/mips/kernel/Makefile +8 -0
index 89b07ea..196564b 100644
@@ -17,6 +17,14 @@ CFLAGS_REMOVE_perf_event.o = -pg
 CFLAGS_REMOVE_perf_event_mipsxx.o = -pg
 endif
 
+KASAN_SANITIZE_head.o := n
+KASAN_SANITIZE_spram.o := n
+KASAN_SANITIZE_traps.o := n
+KASAN_SANITIZE_vdso.o := n
+KASAN_SANITIZE_watch.o := n
+KASAN_SANITIZE_stacktrace.o := n
+KASAN_SANITIZE_cpu-probe.o := n
+
 obj-$(CONFIG_CEVT_BCM1480)	+= cevt-bcm1480.o
 obj-$(CONFIG_CEVT_R4K)		+= cevt-r4k.o
 obj-$(CONFIG_CEVT_DS1287)	+= cevt-ds1287.o
arch/mips/kernel/head.S +3 -0
index 351d40f..f2e4910 100644
@@ -159,6 +159,9 @@ dtb_found:
 	 */
 	jr.hb		v0
 #else  /* !CONFIG_RELOCATABLE */
+#ifdef CONFIG_KASAN
+	jal kasan_early_init
+#endif /* CONFIG_KASAN */
 	j		start_kernel
 #endif /* !CONFIG_RELOCATABLE */
 	END(kernel_entry)
arch/mips/kernel/setup.c +3 -0
index ab349d2..7c28fe4 100644
@@ -942,6 +942,9 @@ void __init setup_arch(char **cmdline_p)
 
 	cpu_cache_init();
 	paging_init();
+#if defined(CONFIG_KASAN)
+	kasan_init();
+#endif
 }
 
 unsigned long kernelsp[NR_CPUS];
arch/mips/kernel/traps.c +11 -0
index c52766a..a63ff86 100644
@@ -2273,6 +2273,17 @@ void __init trap_init(void)
 	unsigned long i, vec_size;
 	phys_addr_t ebase_pa;
 
+	/*
+	 * If kasan is enabled, instrumented code may cause tlb excpetion.
+	 * trap_init will be called in kasan_init, once tarp is initialized,
+	 * ebase should be a non-zero value , so use it as a flag that trap
+	 * won't be initialized more than once.
+	 */
+#if defined(CONFIG_KASAN)
+	if (ebase)
+		return;
+#endif
+
 	check_wait();
 
 	if (!cpu_has_mips_r2_r6) {
arch/mips/lib/memcpy.S +13 -7
index cdd19d85..a150f0b 100644
@@ -271,10 +271,10 @@
 	 */
 	.macro __BUILD_COPY_USER mode, from, to
 
-	/* initialize __memcpy if this the first time we execute this macro */
-	.ifnotdef __memcpy
-	.set __memcpy, 1
-	.hidden __memcpy /* make sure it does not leak */
+	/* initialize _memcpy if this the first time we execute this macro */
+	.ifnotdef _memcpy
+	.set _memcpy, 1
+	.hidden _memcpy /* make sure it does not leak */
 	.endif
 
 	/*
@@ -535,10 +535,10 @@
 	b	1b
 	 ADD	dst, dst, 8
 #endif /* !CONFIG_CPU_HAS_LOAD_STORE_LR */
-	.if __memcpy == 1
+	.if _memcpy == 1
 	END(memcpy)
-	.set __memcpy, 0
-	.hidden __memcpy
+	.set _memcpy, 0
+	.hidden _memcpy
 	.endif
 
 .Ll_exc_copy\@:
@@ -599,6 +599,9 @@ SEXC(1)
 	.endm
 
 	.align	5
+    .weak memmove
+FEXPORT(__memmove)
+EXPORT_SYMBOL(__memmove)
 LEAF(memmove)
 EXPORT_SYMBOL(memmove)
 	ADD	t0, a0, a2
@@ -656,6 +659,9 @@ LEAF(__rmemcpy)					/* a0=dst a1=src a2=len */
  * memcpy sets v0 to dst.
  */
 	.align	5
+    .weak memcpy
+FEXPORT(__memcpy)
+EXPORT_SYMBOL(__memcpy)
 LEAF(memcpy)					/* a0=dst a1=src a2=len */
 EXPORT_SYMBOL(memcpy)
 	move	v0, dst				/* return value */
arch/mips/lib/memset.S +10 -8
index 418611e..0234328 100644
@@ -86,10 +86,10 @@
 	 * mode: LEGACY_MODE or EVA_MODE
 	 */
 	.macro __BUILD_BZERO mode
-	/* Initialize __memset if this is the first time we call this macro */
-	.ifnotdef __memset
-	.set __memset, 1
-	.hidden __memset /* Make sure it does not leak */
+	/* Initialize _memset if this is the first time we call this macro */
+	.ifnotdef _memset
+	.set _memset, 1
+	.hidden _memset /* Make sure it does not leak */
 	.endif
 
 	sltiu		t0, a2, STORSIZE	/* very small region? */
@@ -228,10 +228,10 @@
 
 2:	move		a2, zero
 	jr		ra			/* done */
-	.if __memset == 1
+	.if _memset == 1
 	END(memset)
-	.set __memset, 0
-	.hidden __memset
+	.set _memset, 0
+	.hidden _memset
 	.endif
 
 #ifndef CONFIG_CPU_HAS_LOAD_STORE_LR
@@ -295,7 +295,9 @@
  * a1: char to fill with
  * a2: size of area to clear
  */
-
+    .weak memset
+FEXPORT(__memset)
+EXPORT_SYMBOL(__memset)
 LEAF(memset)
 EXPORT_SYMBOL(memset)
 	move		v0, a0			/* result */
arch/mips/mm/Makefile +5 -0
index f34d7ff..cf2a4a9 100644
@@ -41,3 +41,8 @@ obj-$(CONFIG_R5000_CPU_SCACHE)	+= sc-r5k.o
 obj-$(CONFIG_RM7000_CPU_SCACHE) += sc-rm7k.o
 obj-$(CONFIG_MIPS_CPU_SCACHE)	+= sc-mips.o
 obj-$(CONFIG_SCACHE_DEBUGFS)	+= sc-debugfs.o
+obj-$(CONFIG_KASAN)     += kasan_init.o
+KASAN_SANITIZE_kasan_init.o     := n
+KASAN_SANITIZE_pgtable-64.o     := n
+KASAN_SANITIZE_tlb-r4k.o        := n
+KASAN_SANITIZE_tlbex.o          := n
arch/mips/mm/kasan_init.c new file +216 -0

216 lines not on this page · raw message

arch/mips/vdso/Makefile +5 -0
index 7221df2..b84ceb4 100644
@@ -1,5 +1,10 @@
 # SPDX-License-Identifier: GPL-2.0
 # Objects to go into the VDSO.
+
+ifdef CONFIG_KASAN
+KASAN_SANITIZE	:= n
+endif
+
 obj-vdso-y := elf.o gettimeofday.o sigreturn.o
 
 # Common compiler flags between ABIs.
include/linux/kasan.h +2 -0
index b40ea10..055eb85 100644
@@ -23,11 +23,13 @@ extern p4d_t kasan_early_shadow_p4d[MAX_PTRS_PER_P4D];
 int kasan_populate_early_shadow(const void *shadow_start,
 				const void *shadow_end);
 
+#ifndef __HAVE_ARCH_SHADOW_MAP
 static inline void *kasan_mem_to_shadow(const void *addr)
 {
 	return (void *)((unsigned long)addr >> KASAN_SHADOW_SCALE_SHIFT)
 		+ KASAN_SHADOW_OFFSET;
 }
+#endif /* __HAVE_ARCH_SHADOW_MAP*/
 
 /* Enable reporting bugs after kasan_disable_current() */
 extern void kasan_enable_current(void);
mm/kasan/generic.c +12 -5
index 504c7936..ce070fb 100644
@@ -173,11 +173,18 @@ static __always_inline void check_memory_region_inline(unsigned long addr,
 	if (unlikely(size == 0))
 		return;
 
-	if (unlikely((void *)addr <
-		kasan_shadow_to_mem((void *)KASAN_SHADOW_START))) {
-		kasan_report(addr, size, write, ret_ip);
-		return;
-	}
+#ifndef __HAVE_ARCH_SHADOW_MAP
+		if (unlikely((void *)addr <
+			kasan_shadow_to_mem((void *)KASAN_SHADOW_START))) {
+			kasan_report(addr, size, write, ret_ip);
+			return;
+		}
+#else
+		if (unlikely(kasan_mem_to_shadow((void *)addr) == NULL)) {
+			kasan_report(addr, size, write, ret_ip);
+			return;
+		}
+#endif
 
 	if (likely(!memory_is_poisoned(addr, size)))
 		return;
mm/kasan/kasan.h +2 -0
index 3ce956e..5f86724 100644
@@ -110,11 +110,13 @@ struct kasan_alloc_meta *get_alloc_info(struct kmem_cache *cache,
 struct kasan_free_meta *get_free_info(struct kmem_cache *cache,
 					const void *object);
 
+#ifndef __HAVE_ARCH_SHADOW_MAP
 static inline const void *kasan_shadow_to_mem(const void *shadow_addr)
 {
 	return (void *)(((unsigned long)shadow_addr - KASAN_SHADOW_OFFSET)
 		<< KASAN_SHADOW_SCALE_SHIFT);
 }
+#endif
 
 static inline bool addr_has_shadow(const void *addr)
 {
Signature · 1 line
2.7.4
#1 lore raw

arch/mips/include/asm/kasan.h line 39

Quoted 2 lines · Tommy Jin
arch/mips/include/asm/kasan.h new file +2 -0
+/* 64-bit segment value. */
+#define XKPHYS_CACHE_SEG	(0xa8)

that's just cachable coherent exclusive on write, what about cachable non coherent (0x98) and cachable exclusive (0xa0) ?

Thomas.

Signature · 2 lines
Crap can work. Given enough thrust pigs will fly, but it's not necessarily a
good idea.                                                [ RFC1925, 2.3 ]
#2 lore raw

Hi Thomas,
Tommy and me are responsible for mips KASAN issue. Thank you for your comments.
We didn't consider cachable non coherent (0x98) and cachable exclusive (0xa0) mainly because we had not seen them be used. Do you know in which case this two segment address will be applied/accessed? but if these two segments are really needed, we can add it to our code. And correspondingly the KASAN address space need to be extend.(from 512G to 1024G, that is change the PGD_ORDER from 1 to 2).
BR
Zhongwu

-----邮件原件-----
发件人: Thomas Bogendoerfer <[email protected]> 发送时间: 2019年8月29日 19:33
收件人: Tommy Jin <[email protected]>
抄送: [email protected]; Zhongwu Zhu <[email protected]> 主题: [EXTERNAL]Re: [PATCH] MIPS64: add KASAN support

arch/mips/include/asm/kasan.h line 39

Quoted 2 lines · Tommy Jin
arch/mips/include/asm/kasan.h new file +2 -0
+/* 64-bit segment value. */
+#define XKPHYS_CACHE_SEG	(0xa8)

that's just cachable coherent exclusive on write, what about cachable non coherent (0x98) and cachable exclusive (0xa0) ?

Thomas.

--
Crap can work. Given enough thrust pigs will fly, but it's not necessarily a
good idea. [ RFC1925, 2.3 ]