Add KFENCE support for LoongArch
Subsystems (3)
v3 cover Add KFENCE support for LoongArch
Hi all,
This patchset adds KFENCE support on LoongArch.
To run the testcases, you will need to enable the following options,
-> Kernel hacking
[*] Tracers
[*] Support for tracing block IO actions (NEW)
-> Kernel Testing and Coverage
<*> KUnit - Enable support for unit tests
and then,
-> Kernel hacking
-> Memory Debugging
[*] KFENCE: low-overhead sampling-based memory safety error detector (NEW)
<*> KFENCE integration test suite (NEW)
With these options enabled, KFENCE will be tested during kernel startup. And normally, you might get the following feedback,
========================================================
[ 35.326363 ] # kfence: pass:23 fail:0 skip:2 total:25 [ 35.326486 ] # Totals: pass:23 fail:0 skip:2 total:25 [ 35.326621 ] ok 1 kfence
========================================================
you might notice that 2 testcases have been skipped. If you tend to run all testcases, please enable CONFIG_INIT_ON_FREE_DEFAULT_ON, you can find it here,
-> Security options
-> Kernel hardening options
-> Memory initialization
[*] Enable heap memory zeroing on free by default
and you might get all testcases passed.
========================================================
[ 35.531860 ] # kfence: pass:25 fail:0 skip:0 total:25 [ 35.531999 ] # Totals: pass:25 fail:0 skip:0 total:25 [ 35.532135 ] ok 1 kfence
========================================================
v3:
* Address Huacai's comments.
* Fix a bug that Jackie Liu pointed out.
* Rewrite arch_stack_walk() with the suggestion of Jinyang He.
v2:
* Address Huacai's comments.
* Fix typos in commit message.
Thanks,
Enze
Enze Li (4):
KFENCE: Defer the assignment of the local variable addr
LoongArch: mm: Add page table mapped mode support
LoongArch: Get stack without NMI when providing regs parameter
LoongArch: Add KFENCE support
arch/loongarch/Kconfig | 1 + arch/loongarch/include/asm/kfence.h | 66 ++++++++++++++++++++++++++++ arch/loongarch/include/asm/page.h | 8 +++- arch/loongarch/include/asm/pgtable.h | 16 ++++++- arch/loongarch/kernel/stacktrace.c | 18 ++++---- arch/loongarch/mm/fault.c | 22 ++++++---- arch/loongarch/mm/pgtable.c | 7 +++ mm/kfence/core.c | 5 ++- 8 files changed, 123 insertions(+), 20 deletions(-) create mode 100644 arch/loongarch/include/asm/kfence.h
Signature · 1 line
2.34.1
PATCH v3 1/4 KFENCE: Defer the assignment of the local variable addr
The LoongArch architecture is different from other architectures. It needs to update __kfence_pool during arch_kfence_init_pool().
This patch modifies the assignment location of the local variable addr in the kfence_init_pool function to support the case of updating __kfence_pool in arch_kfence_init_pool().
--- mm/kfence/core.c | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-)
mm/kfence/core.c +3 -2
@@ -566,13 +566,14 @@ static void rcu_guarded_free(struct rcu_head *h)*/ static unsigned long kfence_init_pool (void ) { -unsigned long addr = (unsigned long )__kfence_pool; +unsigned long addr;struct page *pages;int i;if (!arch_kfence_init_pool ()) -return addr; +return (unsigned long )__kfence_pool; + addr = (unsigned long )__kfence_pool; pages =virt_to_page (__kfence_pool);/*
Signature · 1 line
2.34.1
PATCH v3 2/4 LoongArch: mm: Add page table mapped mode support
In reply to Enze LiAccording to LoongArch documentation online, there are two types of address translation modes: direct mapped address translation mode (direct mapped mode) and page table mapped address translation mode (page table mapped mode).
Currently, the upstream kernel only supports direct mapped mode. This patch adds a function that determines whether page table mapped mode should be used, and also adds the corresponding handler functions for both modes.
For more details on the two modes, see [1].
--- arch/loongarch/include/asm/page.h | 8 +++++++- arch/loongarch/include/asm/pgtable.h | 3 +++ arch/loongarch/mm/pgtable.c | 7 +++++++ 3 files changed, 17 insertions(+), 1 deletion(-)
arch/loongarch/include/asm/page.h +7 -1
@@ -84,7 +84,13 @@ typedef struct { unsigned long pgprot; } pgprot_t;#define sym_to_pfn (x) __phys_to_pfn(__pa_symbol(x))#define virt_to_pfn (kaddr) PFN_DOWN(PHYSADDR(kaddr)) -#define virt_to_page (kaddr) pfn_to_page(virt_to_pfn(kaddr)) + +#define virt_to_page (kaddr) \ +({ \ + ((unsigned long)kaddr >= vm_map_base) ? \ + tlb_virt_to_page((unsigned long)kaddr) : \ + dmw_virt_to_page((unsigned long)kaddr); \ +})extern int __virt_addr_valid (volatile void *kaddr);#define virt_addr_valid (kaddr) __virt_addr_valid((volatile void *)(kaddr))
arch/loongarch/include/asm/pgtable.h +3 -0
@@ -353,6 +353,9 @@ static inline void pte_clear(struct mm_struct *mm, unsigned long addr, pte_t *pt#define PMD_T_LOG2 (__builtin_ffs(sizeof(pmd_t)) - 1)#define PTE_T_LOG2 (__builtin_ffs(sizeof(pte_t)) - 1) +inline struct page *tlb_virt_to_page (unsigned long kaddr); +#define dmw_virt_to_page (kaddr) pfn_to_page(virt_to_pfn(kaddr)) +extern pgd_t swapper_pg_dir[];extern pgd_t invalid_pg_dir[];
arch/loongarch/mm/pgtable.c +7 -0
@@ -9,6 +9,13 @@#include <asm/pgtable.h> #include <asm/tlbflush.h> + +inline struct page *tlb_virt_to_page (unsigned long kaddr) +{ +return pte_page (*virt_to_kpte (kaddr)); +} +EXPORT_SYMBOL_GPL (tlb_virt_to_page); +pgd_t *pgd_alloc (struct mm_struct *mm) {pgd_t *ret, *init;
Signature · 1 line
2.34.1
PATCH v3 3/4 LoongArch: Get stack without NMI when providing regs parameter
In reply to Enze LiCurrently, arch_stack_walk() can only get the full stack information including NMI. This is because the implementation of arch_stack_walk() is forced to ignore the information passed by the regs parameter and use the current stack information instead.
For some detection systems like KFENCE, only partial stack information is needed. In particular, the stack frame where the interrupt occurred.
To support KFENCE, this patch modifies the implementation of the arch_stack_walk() function so that if this function is called with the regs argument passed, it retains all the stack information in regs and uses it to provide accurate information.
Before the patch applied, I get,
[ 1.531195 ] ================================================================== [ 1.531442 ] BUG: KFENCE: out-of-bounds read in stack_trace_save_regs+0x48/0x6c [ 1.531442 ] [ 1.531900 ] Out-of-bounds read at 0xffff800012267fff (1B left of kfence-#12): [ 1.532046 ] stack_trace_save_regs+0x48/0x6c [ 1.532169 ] kfence_report_error+0xa4/0x528 [ 1.532276 ] kfence_handle_page_fault+0x124/0x270 [ 1.532388 ] no_context+0x50/0x94 [ 1.532453 ] do_page_fault+0x1a8/0x36c [ 1.532524 ] tlb_do_page_fault_0+0x118/0x1b4 [ 1.532623 ] test_out_of_bounds_read+0xa0/0x1d8 [ 1.532745 ] kunit_generic_run_threadfn_adapter+0x1c/0x28 [ 1.532854 ] kthread+0x124/0x130 [ 1.532922 ] ret_from_kernel_thread+0xc/0xa4
<snip>
With this patch applied, I get the correct stack information.
[ 1.320220 ] ================================================================== [ 1.320401 ] BUG: KFENCE: out-of-bounds read in test_out_of_bounds_read+0xa8/0x1d8 [ 1.320401 ] [ 1.320898 ] Out-of-bounds read at 0xffff800012257fff (1B left of kfence-#10): [ 1.321134 ] test_out_of_bounds_read+0xa8/0x1d8 [ 1.321264 ] kunit_generic_run_threadfn_adapter+0x1c/0x28 [ 1.321392 ] kthread+0x124/0x130 [ 1.321459 ] ret_from_kernel_thread+0xc/0xa4
<snip>
--- arch/loongarch/kernel/stacktrace.c | 18 ++++++++++-------- 1 file changed, 10 insertions(+), 8 deletions(-)
arch/loongarch/kernel/stacktrace.c +10 -8
@@ -18,17 +18,19 @@ void arch_stack_walk(stack_trace_consume_fn consume_entry, void *cookie,struct pt_regs dummyregs;struct unwind_state state; - regs = &dummyregs; +if (!regs) { + regs = &dummyregs; -if (task == current) { - regs->regs[3 ] = (unsigned long )__builtin_frame_address (0 ); - regs->csr_era = (unsigned long )__builtin_return_address (0 ); - }else { - regs->regs[3 ] =thread_saved_fp (task); - regs->csr_era =thread_saved_ra (task); +if (task == current) { + regs->regs[3 ] = (unsigned long )__builtin_frame_address (0 ); + regs->csr_era = (unsigned long )__builtin_return_address (0 ); + }else { + regs->regs[3 ] =thread_saved_fp (task); + regs->csr_era =thread_saved_ra (task); + } + regs->regs[1 ] =0 ; } - regs->regs[1 ] =0 ;for (unwind_start (&state, task, regs); !unwind_done (&state) && !unwind_error (&state);unwind_next_frame (&state)) { addr =unwind_get_return_address (&state);
Signature · 1 line
2.34.1
PATCH v3 4/4 LoongArch: Add KFENCE support
In reply to Enze LiThe LoongArch architecture is quite different from other architectures. When the allocating of KFENCE itself is done, it is mapped to the direct mapping configuration window [1] by default on LoongArch. It means that it is not possible to use the page table mapped mode which required by the KFENCE system and therefore it should be remapped to the appropriate region.
This patch adds architecture specific implementation details for KFENCE. In particular, this implements the required interface in <asm/kfence.h>.
Tested this patch by running the testcases and all passed.
--- arch/loongarch/Kconfig | 1 + arch/loongarch/include/asm/kfence.h | 66 ++++++++++++++++++++++++++++ arch/loongarch/include/asm/pgtable.h | 13 +++++- arch/loongarch/mm/fault.c | 22 ++++++---- 4 files changed, 93 insertions(+), 9 deletions(-) create mode 100644 arch/loongarch/include/asm/kfence.h
arch/loongarch/Kconfig +1 -0
@@ -92,6 +92,7 @@ config LOONGARCH select HAVE_ARCH_AUDITSYSCALL select HAVE_ARCH_JUMP_LABEL select HAVE_ARCH_JUMP_LABEL_RELATIVE + select HAVE_ARCH_KFENCE select HAVE_ARCH_MMAP_RND_BITS if MMU select HAVE_ARCH_SECCOMP_FILTER select HAVE_ARCH_TRACEHOOK
arch/loongarch/include/asm/kfence.h new file +66 -0
66 lines not on this page · raw message
arch/loongarch/include/asm/pgtable.h +12 -1
@@ -77,6 +77,12 @@ extern unsigned long zero_page_mask; (virt_to_page((void *)(empty_zero_page + (((unsigned long)(vaddr)) & zero_page_mask))))#define __HAVE_COLOR_ZERO_PAGE +#ifdef CONFIG_KFENCE +#define KFENCE_AREA_SIZE (((CONFIG_KFENCE_NUM_OBJECTS + 1) * 2 + 2) * PAGE_SIZE) +#else +#define KFENCE_AREA_SIZE 0 +#endif +/* * TLB refill handlers may also map the vmalloc area into xkvrange. * Avoid the first couple of pages so NULL pointer dereferences will
@@ -88,11 +94,16 @@ extern unsigned long zero_page_mask;#define VMALLOC_START MODULES_END#define VMALLOC_END \ (vm_map_base + \ - min(PTRS_PER_PGD * PTRS_PER_PUD * PTRS_PER_PMD * PTRS_PER_PTE * PAGE_SIZE, (1UL << cpu_vabits)) - PMD_SIZE - VMEMMAP_SIZE) + min(PTRS_PER_PGD * PTRS_PER_PUD * PTRS_PER_PMD * PTRS_PER_PTE * PAGE_SIZE, (1UL << cpu_vabits)) - PMD_SIZE - VMEMMAP_SIZE - KFENCE_AREA_SIZE)#define vmemmap ((struct page *)((VMALLOC_END + PMD_SIZE) & PMD_MASK))#define VMEMMAP_END ((unsigned long)vmemmap + VMEMMAP_SIZE - 1) +#ifdef CONFIG_KFENCE +#define KFENCE_AREA_START VMEMMAP_END +#define KFENCE_AREA_END (KFENCE_AREA_START + KFENCE_AREA_SIZE) +#endif +#define pte_ERROR (e) \ pr_err("%s:%d: bad pte %016lx.\n", __FILE__, __LINE__, pte_val(e))#ifndef __PAGETABLE_PMD_FOLDED
arch/loongarch/mm/fault.c +14 -8
@@ -23,6 +23,7 @@#include <linux/kprobes.h> #include <linux/perf_event.h> #include <linux/uaccess.h> +#include <linux/kfence.h> #include <asm/branch.h> #include <asm/mmu_context.h>
@@ -30,7 +31,8 @@int show_unhandled_signals =1 ; -static void __kprobes no_context (struct pt_regs *regs,unsigned long address) +static void __kprobes no_context (struct pt_regs *regs,unsigned long address, +unsigned long write) {const int field =sizeof (unsigned long ) *2 ;
@@ -38,6 +40,9 @@ static void __kprobes no_context(struct pt_regs *regs, unsigned long address)if (fixup_exception (regs))return ; +if (kfence_handle_page_fault (address, write, regs)) +return ; +/* * Oops. The kernel tried to access some bad page. We'll have to * terminate things with extreme prejudice.
@@ -51,14 +56,15 @@ static void __kprobes no_context(struct pt_regs *regs, unsigned long address)die ("Oops" , regs); } -static void __kprobes do_out_of_memory (struct pt_regs *regs,unsigned long address) +static void __kprobes do_out_of_memory (struct pt_regs *regs,unsigned long address, +unsigned long write) {/* * We ran out of memory, call the OOM killer, and return the userspace * (which will retry the fault, or kill us if we got oom-killed). */ if (!user_mode (regs)) { -no_context (regs, address); +no_context (regs, address, write);return ; }pagefault_out_of_memory ();
@@ -69,7 +75,7 @@ static void __kprobes do_sigbus(struct pt_regs *regs, {/* Kernel mode? Handle exceptions or die */ if (!user_mode (regs)) { -no_context (regs, address); +no_context (regs, address, write);return ; }
@@ -90,7 +96,7 @@ static void __kprobes do_sigsegv(struct pt_regs *regs,/* Kernel mode? Handle exceptions or die */ if (!user_mode (regs)) { -no_context (regs, address); +no_context (regs, address, write);return ; }
@@ -149,7 +155,7 @@ static void __kprobes __do_page_fault(struct pt_regs *regs,*/ if (address & __UA_LIMIT) {if (!user_mode (regs)) -no_context (regs, address); +no_context (regs, address, write);else do_sigsegv (regs, write, address, si_code);return ;
@@ -211,7 +217,7 @@ static void __kprobes __do_page_fault(struct pt_regs *regs,if (fault_signal_pending (fault, regs)) {if (!user_mode (regs)) -no_context (regs, address); +no_context (regs, address, write);return ; }
@@ -232,7 +238,7 @@ static void __kprobes __do_page_fault(struct pt_regs *regs,if (unlikely (fault &VM_FAULT_ERROR )) {mmap_read_unlock (mm);if (fault &VM_FAULT_OOM ) { -do_out_of_memory (regs, address); +do_out_of_memory (regs, address, write);return ; }else if (fault &VM_FAULT_SIGSEGV ) {do_sigsegv (regs, write, address, si_code);
Signature · 1 line
2.34.1
Huacai Chen
maintainerHi, Enze,
I applied this series (with some small modifications) together with KASAN at:
https://github.com/chenhuacai/linux/commits/loongarch-next
Please confirm everything works well for you.
Huacai
Quoted 77 lines · Enze Li
Hi all,
This patchset adds KFENCE support on LoongArch.
To run the testcases, you will need to enable the following options,
-> Kernel hacking
[*] Tracers
[*] Support for tracing block IO actions (NEW)
-> Kernel Testing and Coverage
<*> KUnit - Enable support for unit testsand then,
-> Kernel hacking
-> Memory Debugging
[*] KFENCE: low-overhead sampling-based memory safety error detector (NEW)
<*> KFENCE integration test suite (NEW)With these options enabled, KFENCE will be tested during kernel startup. And normally, you might get the following feedback,
========================================================
[ 35.326363 ] # kfence: pass:23 fail:0 skip:2 total:25 [ 35.326486 ] # Totals: pass:23 fail:0 skip:2 total:25 [ 35.326621 ] ok 1 kfence========================================================
you might notice that 2 testcases have been skipped. If you tend to run all testcases, please enable
CONFIG_INIT_ON_FREE_DEFAULT_ON, you can find it here,-> Security options
-> Kernel hardening options
-> Memory initialization
[*] Enable heap memory zeroing on free by defaultand you might get all testcases passed.
========================================================[ 35.531860 ] # kfence: pass:25 fail:0 skip:0 total:25 [ 35.531999 ] # Totals: pass:25 fail:0 skip:0 total:25 [ 35.532135 ] ok 1 kfence========================================================
v3:
* Address Huacai's comments.
* Fix a bug that Jackie Liu pointed out.
* Rewritearch_stack_walk()with the suggestion of Jinyang He.v2:
* Address Huacai's comments.
* Fix typos in commit message.Thanks,
EnzeEnze Li (4):
KFENCE: Defer the assignment of the local variable addr
LoongArch: mm: Add page table mapped mode support
LoongArch: Get stack without NMI when providing regs parameter
LoongArch: Add KFENCE supportarch/loongarch/Kconfig | 1 + arch/loongarch/include/asm/kfence.h | 66 ++++++++++++++++++++++++++++ arch/loongarch/include/asm/page.h | 8 +++- arch/loongarch/include/asm/pgtable.h | 16 ++++++- arch/loongarch/kernel/stacktrace.c | 18 ++++---- arch/loongarch/mm/fault.c | 22 ++++++---- arch/loongarch/mm/pgtable.c | 7 +++ mm/kfence/core.c | 5 ++- 8 files changed, 123 insertions(+), 20 deletions(-) create mode 100644 arch/loongarch/include/asm/kfence.h--
2.34.1
Quoted 6 lines · Huacai Chen
Hi, Enze,
I applied this series (with some small modifications) together with KASAN at:
https://github.com/chenhuacai/linux/commits/loongarch-nextPlease confirm everything works well for you.
Hi Huacai,
Thanks for your patience these days.
I've tested this on both a physical machine and a qemu VM. It works well.
BTW, if there're any modifications, bugs or improvments to KFENCE on LoongArch in the future, feel free to Cc me. I'll appreciate it. :)
Best Regards,
Enze
Quoted 81 lines
Huacai
Quoted 77 lines · Enze Li
Hi all,
This patchset adds KFENCE support on LoongArch.
To run the testcases, you will need to enable the following options,
-> Kernel hacking
[*] Tracers
[*] Support for tracing block IO actions (NEW)
-> Kernel Testing and Coverage
<*> KUnit - Enable support for unit testsand then,
-> Kernel hacking
-> Memory Debugging
[*] KFENCE: low-overhead sampling-based memory safety error detector (NEW)
<*> KFENCE integration test suite (NEW)With these options enabled, KFENCE will be tested during kernel startup. And normally, you might get the following feedback,
========================================================
[ 35.326363 ] # kfence: pass:23 fail:0 skip:2 total:25 [ 35.326486 ] # Totals: pass:23 fail:0 skip:2 total:25 [ 35.326621 ] ok 1 kfence========================================================
you might notice that 2 testcases have been skipped. If you tend to run all testcases, please enable
CONFIG_INIT_ON_FREE_DEFAULT_ON, you can find it here,-> Security options
-> Kernel hardening options
-> Memory initialization
[*] Enable heap memory zeroing on free by defaultand you might get all testcases passed.
========================================================[ 35.531860 ] # kfence: pass:25 fail:0 skip:0 total:25 [ 35.531999 ] # Totals: pass:25 fail:0 skip:0 total:25 [ 35.532135 ] ok 1 kfence========================================================
v3:
* Address Huacai's comments.
* Fix a bug that Jackie Liu pointed out.
* Rewritearch_stack_walk()with the suggestion of Jinyang He.v2:
* Address Huacai's comments.
* Fix typos in commit message.Thanks,
EnzeEnze Li (4):
KFENCE: Defer the assignment of the local variable addr
LoongArch: mm: Add page table mapped mode support
LoongArch: Get stack without NMI when providing regs parameter
LoongArch: Add KFENCE supportarch/loongarch/Kconfig | 1 + arch/loongarch/include/asm/kfence.h | 66 ++++++++++++++++++++++++++++ arch/loongarch/include/asm/page.h | 8 +++- arch/loongarch/include/asm/pgtable.h | 16 ++++++- arch/loongarch/kernel/stacktrace.c | 18 ++++---- arch/loongarch/mm/fault.c | 22 ++++++---- arch/loongarch/mm/pgtable.c | 7 +++ mm/kfence/core.c | 5 ++- 8 files changed, 123 insertions(+), 20 deletions(-) create mode 100644 arch/loongarch/include/asm/kfence.h--
2.34.1