kasan: x86: arm64: KASAN tag-based mode for x86
Subsystems (13)
v5 cover kasan: x86: arm64: KASAN tag-based mode for x86
======= Introduction
The patchset aims to add a KASAN tag-based mode for the x86 architecture with the help of the new CPU feature called Linear Address Masking (LAM). Main improvement introduced by the series is 2x lower memory usage compared to KASAN's generic mode, the only currently available mode on x86. The tag based mode may also find errors that the generic mode couldn't because of differences in how these modes operate.
======= How does KASAN' tag-based mode work?
When enabled, memory accesses and allocations are augmented by the compiler during kernel compilation. Instrumentation functions are added to each memory allocation and each pointer dereference.
The allocation related functions generate a random tag and save it in two places: in shadow memory that maps to the allocated memory, and in the top bits of the pointer that points to the allocated memory. Storing the tag in the top of the pointer is possible because of Top-Byte Ignore (TBI) on arm64 architecture and LAM on x86.
The access related functions are performing a comparison between the tag stored in the pointer and the one stored in shadow memory. If the tags don't match an out of bounds error must have occurred and so an error report is generated.
The general idea for the tag-based mode is very well explained in the series with the original implementation [1].
[1] https://lore.kernel.org/all/[email protected]/
======= Differences summary compared to the arm64 tag-based mode
- Tag width:
- Tag width influences the chance of a tag mismatch due to two tags from different allocations having the same value. The bigger the possible range of tag values the lower the chance of that happening.
- Shortening the tag width from 8 bits to 4, while it can help with memory usage, it also increases the chance of not reporting an error. 4 bit tags have a ~7% chance of a tag mismatch.
- Address masking mechanism
- TBI in arm64 allows for storing metadata in the top 8 bits of the virtual address.
- LAM in x86 allows storing tags in bits [62:57] of the pointer. To maximize memory savings the tag width is reduced to bits [60:57].
- Inline mode mismatch reporting
- Arm64 inserts a BRK instruction to pass metadata about a tag mismatch to the KASAN report.
- On x86 the INT3 instruction is used for the same purpose.
======= Testing
Checked all the kunits for both software tags and generic KASAN after making changes.
In generic mode the results were:
kasan: pass:59 fail:0 skip:13 total:72
Totals: pass:59 fail:0 skip:13 total:72
ok 1 kasan
and for software tags:
kasan: pass:63 fail:0 skip:9 total:72
Totals: pass:63 fail:0 skip:9 total:72
ok 1 kasan
======= Benchmarks [1]
All tests were ran on a Sierra Forest server platform. The only differences between the tests were kernel options:
- CONFIG_KASAN
- CONFIG_KASAN_GENERIC
- CONFIG_KASAN_SW_TAGS
- CONFIG_KASAN_INLINE [1]
- CONFIG_KASAN_OUTLINE
Boot time (until login prompt):
* 02:55 for clean kernel
* 05:42 / 06:32 for generic KASAN (inline/outline)
* 05:58 for tag-based KASAN (outline) [2]
Total memory usage (512GB present on the system - MemAvailable just after boot):
* 12.56 GB for clean kernel
* 81.74 GB for generic KASAN
* 44.39 GB for tag-based KASAN
Kernel size:
* 14 MB for clean kernel
* 24.7 MB / 19.5 MB for generic KASAN (inline/outline)
* 27.1 MB / 18.1 MB for tag-based KASAN (inline/outline)
Work under load time comparison (compiling the mainline kernel) (200 cores):
* 62s for clean kernel
* 171s / 125s for generic KASAN (outline/inline)
* 145s for tag-based KASAN (outline) [2]
[1] Currently inline mode doesn't work on x86 due to things missing in the compiler. I have written a patch for clang that seems to fix the inline mode and I was able to boot and check that all patches regarding the inline mode work as expected. My hope is to post the patch to LLVM once this series is completed, and then make inline mode available in the kernel config.
[2] While I was able to boot the inline tag-based kernel with my compiler changes in a simulated environment, due to toolchain difficulties I couldn't get it to boot on the machine I had access to. Also boot time results from the simulation seem too good to be true, and they're much too worse for the generic case to be believable. Therefore I'm posting only results from the physical server platform.
======= Compilation
Clang was used to compile the series (make LLVM=1) since gcc doesn't seem to have support for KASAN tag-based compiler instrumentation on x86.
======= Dependencies
The base branch for the series is the mainline kernel, tag 6.17-rc3.
======= Enabling LAM for testing
Since LASS is needed for LAM and it can't be compiled without it I applied the LASS series [1] first, then applied my patches.
[1] https://lore.kernel.org/all/[email protected]/
Changes v5:
- Fix a bunch of arm64 compilation errors I didn't catch earlier. Thank You Ada for testing the series!
- Simplify the usage of the tag handling x86 functions (virt_to_page, phys_addr etc.).
- Remove within() and within_range() from the EXECMEM_ROX patch.
- Count time it takes to compile a kernel when running kernels with generic KASAN, tag based KASAN and a clean kernel. Put data in the cover letter benchmark section.
Changes v4:
- Revert x86 kasan_mem_to_shadow() scheme to the same on used in generic KASAN. Keep the arithmetic shift idea for the KASAN in general since it makes more sense for arm64 and in risc-v.
- Fix inline mode but leave it unavailable until a complementary compiler patch can be merged.
- Apply Dave Hansen's comments on series formatting, patch style and code simplifications.
Changes v3:
- Remove the runtime_const patch and setup a unified offset for both 5 and 4 paging levels.
- Add a fix for inline mode on x86 tag-based KASAN. Add a handler for int3 that is generated on inline tag mismatches.
- Fix scripts/gdb/linux/kasan.py so the new signed mem_to_shadow() is reflected there.
- Fix Documentation/arch/arm64/kasan-offsets.sh to take new offsets into account.
- Made changes to the kasan_non_canonical_hook() according to upstream discussion.
- Remove patches 2 and 3 since they related to risc-v and this series adds only x86 related things.
- Reorder __tag_*() functions so they're before arch_kasan_*(). Remove CONFIG_KASAN condition from __tag_set().
Changes v2:
- Split the series into one adding KASAN tag-based mode (this one) and another one that adds the dense mode to KASAN (will post later).
- Removed exporting kasan_poison() and used a wrapper instead in kasan_init_64.c
- Prepended series with 4 patches from the risc-v series and applied review comments to the first patch as the rest already are reviewed.
Maciej Wieczor-Retman (17):
kasan: Fix inline mode for x86 tag-based mode
x86: Add arch specific kasan functions
kasan: arm64: x86: Make special tags arch specific
x86: Reset tag for virtual to physical address conversions
mm: x86: Untag addresses in EXECMEM_ROX related pointer arithmetic
x86: Physical address comparisons in fill_p*d/pte
x86: KASAN raw shadow memory PTE init
x86: LAM compatible non-canonical definition
x86: LAM initialization
x86: Minimal SLAB alignment
kasan: x86: Handle int3 for inline KASAN reports
arm64: Unify software tag-based KASAN inline recovery path
kasan: x86: Apply multishot to the inline report handler
kasan: x86: Logical bit shift for kasan_mem_to_shadow
mm: Unpoison pcpu chunks with base address tag
mm: Unpoison vms[area] addresses with a common tag
x86: Make software tag-based kasan available
Samuel Holland (2):
kasan: sw_tags: Use arithmetic shift for shadow computation
kasan: sw_tags: Support tag widths less than 8 bits
Documentation/arch/arm64/kasan-offsets.sh | 8 ++- Documentation/arch/x86/x86_64/mm.rst | 6 +- MAINTAINERS | 4 +- arch/arm64/Kconfig | 10 ++-- arch/arm64/include/asm/kasan-tags.h | 13 +++++ arch/arm64/include/asm/kasan.h | 2 - arch/arm64/include/asm/memory.h | 14 ++++- arch/arm64/include/asm/uaccess.h | 1 + arch/arm64/kernel/traps.c | 17 +----- arch/arm64/mm/kasan_init.c | 7 ++- arch/x86/Kconfig | 4 +- arch/x86/boot/compressed/misc.h | 1 + arch/x86/include/asm/cache.h | 4 ++ arch/x86/include/asm/kasan-tags.h | 9 +++ arch/x86/include/asm/kasan.h | 71 ++++++++++++++++++++++- arch/x86/include/asm/page.h | 18 ++++++ arch/x86/include/asm/page_64.h | 1 + arch/x86/kernel/alternative.c | 4 +- arch/x86/kernel/head_64.S | 3 + arch/x86/kernel/setup.c | 2 + arch/x86/kernel/traps.c | 4 ++ arch/x86/mm/Makefile | 2 + arch/x86/mm/init.c | 3 + arch/x86/mm/init_64.c | 11 ++-- arch/x86/mm/kasan_init_64.c | 19 +++++- arch/x86/mm/kasan_inline.c | 26 +++++++++ arch/x86/mm/physaddr.c | 2 + include/linux/kasan-tags.h | 21 +++++-- include/linux/kasan.h | 51 +++++++++++++++- include/linux/mm.h | 6 +- include/linux/mmzone.h | 1 - include/linux/page-flags-layout.h | 9 +-- lib/Kconfig.kasan | 3 +- mm/execmem.c | 2 +- mm/kasan/hw_tags.c | 11 ++++ mm/kasan/report.c | 45 ++++++++++++-- mm/kasan/shadow.c | 18 ++++++ mm/vmalloc.c | 6 +- scripts/Makefile.kasan | 3 + scripts/gdb/linux/kasan.py | 5 +- scripts/gdb/linux/mm.py | 5 +- 41 files changed, 375 insertions(+), 77 deletions(-) mode change 100644 => 100755 Documentation/arch/arm64/kasan-offsets.sh create mode 100644 arch/arm64/include/asm/kasan-tags.h create mode 100644 arch/x86/include/asm/kasan-tags.h create mode 100644 arch/x86/mm/kasan_inline.c
Signature · 1 line
2.50.1
PATCH v5 1/19 kasan: sw_tags: Use arithmetic shift for shadow computation
From: Samuel Holland <[email protected]>
Currently, kasan_mem_to_shadow() uses a logical right shift, which turns canonical kernel addresses into non-canonical addresses by clearing the high KASAN_SHADOW_SCALE_SHIFT bits. The value of KASAN_SHADOW_OFFSET is then chosen so that the addition results in a canonical address for the shadow memory.
For KASAN_GENERIC, this shift/add combination is ABI with the compiler, because KASAN_SHADOW_OFFSET is used in compiler-generated inline tag checks[1], which must only attempt to dereference canonical addresses.
However, for KASAN_SW_TAGS we have some freedom to change the algorithm without breaking the ABI. Because TBI is enabled for kernel addresses, the top bits of shadow memory addresses computed during tag checks are irrelevant, and so likewise are the top bits of KASAN_SHADOW_OFFSET. This is demonstrated by the fact that LLVM uses a logical right shift in the tag check fast path[2] but a sbfx (signed bitfield extract) instruction in the slow path[3] without causing any issues.
Using an arithmetic shift in kasan_mem_to_shadow() provides a number of benefits:
1) The memory layout doesn't change but is easier to understand. KASAN_SHADOW_OFFSET becomes a canonical memory address, and the shifted pointer becomes a negative offset, so KASAN_SHADOW_OFFSET == KASAN_SHADOW_END regardless of the shift amount or the size of the virtual address space.
2) KASAN_SHADOW_OFFSET becomes a simpler constant, requiring only one instruction to load instead of two. Since it must be loaded in each function with a tag check, this decreases kernel text size by 0.5%.
3) This shift and the sign extension from kasan_reset_tag() can be combined into a single sbfx instruction. When this same algorithm change is applied to the compiler, it removes an instruction from each inline tag check, further reducing kernel text size by an additional 4.6%.
These benefits extend to other architectures as well. On RISC-V, where the baseline ISA does not shifted addition or have an equivalent to the sbfx instruction, loading KASAN_SHADOW_OFFSET is reduced from 3 to 2 instructions, and kasan_mem_to_shadow(kasan_reset_tag(addr)) similarly combines two consecutive right shifts.
---
Changelog v5: (Maciej)
- (u64) -> (unsigned long) in report.c
Changelog v4: (Maciej)
- Revert x86 to signed mem_to_shadow mapping.
- Remove last two paragraphs since they were just poorer duplication of the comments in kasan_non_canonical_hook().
Changelog v3: (Maciej)
- Fix scripts/gdb/linux/kasan.py so the new signed mem_to_shadow() is reflected there.
- Fix Documentation/arch/arm64/kasan-offsets.sh to take new offsets into account.
- Made changes to the kasan_non_canonical_hook() according to upstream discussion. Settled on overflow on both ranges and separate checks for x86 and arm.
Changelog v2: (Maciej)
- Correct address range that's checked in kasan_non_canonical_hook(). Adjust the comment inside.
- Remove part of comment from arch/arm64/include/asm/memory.h.
- Append patch message paragraph about the overflow in kasan_non_canonical_hook().
Documentation/arch/arm64/kasan-offsets.sh | 8 +++-- arch/arm64/Kconfig | 10 +++---- arch/arm64/include/asm/memory.h | 14 ++++++++- arch/arm64/mm/kasan_init.c | 7 +++-- include/linux/kasan.h | 10 +++++-- mm/kasan/report.c | 36 ++++++++++++++++++++--- scripts/gdb/linux/kasan.py | 3 ++ scripts/gdb/linux/mm.py | 5 ++-- 8 files changed, 75 insertions(+), 18 deletions(-) mode change 100644 => 100755 Documentation/arch/arm64/kasan-offsets.sh
Documentation/arch/arm64/kasan-offsets.sh +6 -2
@@ -5,8 +5,12 @@ print_kasan_offset () { printf "%02d\t" $1 - printf "0x%08x00000000\n" $(( (0xffffffff & (-1 << ($1 - 1 - 32))) \ - - (1 << (64 - 32 - $2)) )) + if [[ $2 -ne 4 ]] then + printf "0x%08x00000000\n" $(( (0xffffffff & (-1 << ($1 - 1 - 32))) \ + - (1 << (64 - 32 - $2)) )) + else + printf "0x%08x00000000\n" $(( (0xffffffff & (-1 << ($1 - 1 - 32))) )) + fi } echo KASAN_SHADOW_SCALE_SHIFT = 3
arch/arm64/Kconfig +5 -5
@@ -431,11 +431,11 @@ config KASAN_SHADOW_OFFSET default 0xdffffe0000000000 if ARM64_VA_BITS_42 && !KASAN_SW_TAGS default 0xdfffffc000000000 if ARM64_VA_BITS_39 && !KASAN_SW_TAGS default 0xdffffff800000000 if ARM64_VA_BITS_36 && !KASAN_SW_TAGS - default 0xefff800000000000 if (ARM64_VA_BITS_48 || (ARM64_VA_BITS_52 && !ARM64_16K_PAGES)) && KASAN_SW_TAGS - default 0xefffc00000000000 if (ARM64_VA_BITS_47 || ARM64_VA_BITS_52) && ARM64_16K_PAGES && KASAN_SW_TAGS - default 0xeffffe0000000000 if ARM64_VA_BITS_42 && KASAN_SW_TAGS - default 0xefffffc000000000 if ARM64_VA_BITS_39 && KASAN_SW_TAGS - default 0xeffffff800000000 if ARM64_VA_BITS_36 && KASAN_SW_TAGS + default 0xffff800000000000 if (ARM64_VA_BITS_48 || (ARM64_VA_BITS_52 && !ARM64_16K_PAGES)) && KASAN_SW_TAGS + default 0xffffc00000000000 if (ARM64_VA_BITS_47 || ARM64_VA_BITS_52) && ARM64_16K_PAGES && KASAN_SW_TAGS + default 0xfffffe0000000000 if ARM64_VA_BITS_42 && KASAN_SW_TAGS + default 0xffffffc000000000 if ARM64_VA_BITS_39 && KASAN_SW_TAGS + default 0xfffffff800000000 if ARM64_VA_BITS_36 && KASAN_SW_TAGS default 0xffffffffffffffff config UNWIND_TABLES
arch/arm64/include/asm/memory.h +13 -1
@@ -89,7 +89,15 @@* * KASAN_SHADOW_END is defined first as the shadow address that corresponds to * the upper bound of possible virtual kernel memory addresses UL(1) << 64 -* according to the mapping formula. +* according to the mapping formula. For Generic KASAN, the address in the +* mapping formula is treated as unsigned (part of the compiler's ABI), so the +* end of the shadow memory region is at a large positive offset from +* KASAN_SHADOW_OFFSET. For Software Tag-Based KASAN, the address in the +* formula is treated as signed. Since all kernel addresses are negative, they +* map to shadow memory below KASAN_SHADOW_OFFSET, making KASAN_SHADOW_OFFSET +* itself the end of the shadow memory region. (User pointers are positive and +* would map to shadow memory above KASAN_SHADOW_OFFSET, but shadow memory is +* not allocated for them.) * * KASAN_SHADOW_START is defined second based on KASAN_SHADOW_END. The shadow * memory start must map to the lowest possible kernel virtual memory address
@@ -100,7 +108,11 @@*/ #if defined(CONFIG_KASAN_GENERIC ) || defined(CONFIG_KASAN_SW_TAGS )#define KASAN_SHADOW_OFFSET _AC(CONFIG_KASAN_SHADOW_OFFSET, UL) +#ifdef CONFIG_KASAN_GENERIC #define KASAN_SHADOW_END ((UL(1) << (64 - KASAN_SHADOW_SCALE_SHIFT)) + KASAN_SHADOW_OFFSET) +#else +#define KASAN_SHADOW_END KASAN_SHADOW_OFFSET +#endif #define _KASAN_SHADOW_START (va) (KASAN_SHADOW_END - (UL(1) << ((va) - KASAN_SHADOW_SCALE_SHIFT)))#define KASAN_SHADOW_START _KASAN_SHADOW_START(vabits_actual)#define PAGE_END KASAN_SHADOW_START
arch/arm64/mm/kasan_init.c +5 -2
@@ -198,8 +198,11 @@ static bool __init root_level_aligned(u64 addr)/* The early shadow maps everything to a single page of zeroes */ asmlinkage void __init kasan_early_init(void) { -BUILD_BUG_ON (KASAN_SHADOW_OFFSET != -KASAN_SHADOW_END - (1UL << (64 -KASAN_SHADOW_SCALE_SHIFT ))); +if (IS_ENABLED (CONFIG_KASAN_GENERIC )) +BUILD_BUG_ON (KASAN_SHADOW_OFFSET != +KASAN_SHADOW_END - (1UL << (64 -KASAN_SHADOW_SCALE_SHIFT ))); +else +BUILD_BUG_ON (KASAN_SHADOW_OFFSET !=KASAN_SHADOW_END );BUILD_BUG_ON (!IS_ALIGNED (_KASAN_SHADOW_START (VA_BITS ),SHADOW_ALIGN ));BUILD_BUG_ON (!IS_ALIGNED (_KASAN_SHADOW_START (VA_BITS_MIN ),SHADOW_ALIGN ));BUILD_BUG_ON (!IS_ALIGNED (KASAN_SHADOW_END ,SHADOW_ALIGN ));
include/linux/kasan.h +8 -2
@@ -61,8 +61,14 @@ int kasan_populate_early_shadow(const void *shadow_start,#ifndef kasan_mem_to_shadowstatic inline void *kasan_mem_to_shadow (const void *addr) { -return (void *)((unsigned long )addr >>KASAN_SHADOW_SCALE_SHIFT ) - +KASAN_SHADOW_OFFSET ; +void *scaled; + +if (IS_ENABLED (CONFIG_KASAN_GENERIC )) + scaled = (void *)((unsigned long )addr >>KASAN_SHADOW_SCALE_SHIFT ); +else + scaled = (void *)((long )addr >>KASAN_SHADOW_SCALE_SHIFT ); + +return KASAN_SHADOW_OFFSET + scaled; }#endif
mm/kasan/report.c +32 -4
@@ -642,11 +642,39 @@ void kasan_non_canonical_hook(unsigned long addr)const char *bug_type;/* -* All addresses that came as a result of the memory-to-shadow mapping -* (even for bogus pointers) must be >= KASAN_SHADOW_OFFSET. +* For Generic KASAN, kasan_mem_to_shadow() uses the logical right shift +* and never overflows with the chosen KASAN_SHADOW_OFFSET values (on +* both x86 and arm64). Thus, the possible shadow addresses (even for +* bogus pointers) belong to a single contiguous region that is the +* result of kasan_mem_to_shadow() applied to the whole address space. */ -if (addr <KASAN_SHADOW_OFFSET ) -return ; +if (IS_ENABLED (CONFIG_KASAN_GENERIC )) { +if (addr < (unsigned long )kasan_mem_to_shadow ((void *)(0UL )) || + addr > (unsigned long )kasan_mem_to_shadow ((void *)(~0UL ))) +return ; + } + +/* +* For Software Tag-Based KASAN, kasan_mem_to_shadow() uses the +* arithmetic shift. Normally, this would make checking for a possible +* shadow address complicated, as the shadow address computation +* operation would overflow only for some memory addresses. However, due +* to the chosen KASAN_SHADOW_OFFSET values and the fact the +* kasan_mem_to_shadow() only operates on pointers with the tag reset, +* the overflow always happens. +* +* For arm64, the top byte of the pointer gets reset to 0xFF. Thus, the +* possible shadow addresses belong to a region that is the result of +* kasan_mem_to_shadow() applied to the memory range +* [0xFF000000000000, 0xFFFFFFFFFFFFFFFF]. Despite the overflow, the +* resulting possible shadow region is contiguous, as the overflow +* happens for both 0xFF000000000000 and 0xFFFFFFFFFFFFFFFF. +*/ +if (IS_ENABLED (CONFIG_KASAN_SW_TAGS ) &&IS_ENABLED (CONFIG_ARM64 )) { +if (addr < (unsigned long )kasan_mem_to_shadow ((void *)(0xFFUL <<56 )) || + addr > (unsigned long )kasan_mem_to_shadow ((void *)(~0UL ))) +return ; + } orig_addr = (unsigned long )kasan_shadow_to_mem ((void *)addr);
scripts/gdb/linux/kasan.py +3 -0
@@ -8,6 +8,7 @@ import gdb from linux import constants, mm +from ctypes import c_int64 as s64 def help(): t = """Usage: lx-kasan_mem_to_shadow [Hex memory addr]
@@ -39,6 +40,8 @@ class KasanMemToShadow(gdb.Command): else: help() def kasan_mem_to_shadow(self, addr): + if constants.CONFIG_KASAN_SW_TAGS: + addr = s64(addr) return (addr >> self.p_ops.KASAN_SHADOW_SCALE_SHIFT) + self.p_ops.KASAN_SHADOW_OFFSET KasanMemToShadow()
scripts/gdb/linux/mm.py +3 -2
@@ -110,12 +110,13 @@ class aarch64_page_ops(): self.KERNEL_END = gdb.parse_and_eval("_end") if constants.LX_CONFIG_KASAN_GENERIC or constants.LX_CONFIG_KASAN_SW_TAGS: + self.KASAN_SHADOW_OFFSET = constants.LX_CONFIG_KASAN_SHADOW_OFFSET if constants.LX_CONFIG_KASAN_GENERIC: self.KASAN_SHADOW_SCALE_SHIFT = 3 + self.KASAN_SHADOW_END = (1 << (64 - self.KASAN_SHADOW_SCALE_SHIFT)) + self.KASAN_SHADOW_OFFSET else: self.KASAN_SHADOW_SCALE_SHIFT = 4 - self.KASAN_SHADOW_OFFSET = constants.LX_CONFIG_KASAN_SHADOW_OFFSET - self.KASAN_SHADOW_END = (1 << (64 - self.KASAN_SHADOW_SCALE_SHIFT)) + self.KASAN_SHADOW_OFFSET + self.KASAN_SHADOW_END = self.KASAN_SHADOW_OFFSET self.PAGE_END = self.KASAN_SHADOW_END - (1 << (self.vabits_actual - self.KASAN_SHADOW_SCALE_SHIFT)) else: self.PAGE_END = self._PAGE_END(self.VA_BITS_MIN)
Signature · 1 line
2.50.1
Re: [PATCH v5 01/19] kasan: sw_tags: Use arithmetic shift for shadow computation
Quoted 62 lines · Maciej Wieczor-Retman
arch/arm64/Kconfig +5
-5@@ -431,11 +431,11 @@ config KASAN_SHADOW_OFFSET default 0xdffffe0000000000 if ARM64_VA_BITS_42 && !KASAN_SW_TAGS default 0xdfffffc000000000 if ARM64_VA_BITS_39 && !KASAN_SW_TAGS default 0xdffffff800000000 if ARM64_VA_BITS_36 && !KASAN_SW_TAGS - default 0xefff800000000000 if (ARM64_VA_BITS_48 || (ARM64_VA_BITS_52 && !ARM64_16K_PAGES)) && KASAN_SW_TAGS - default 0xefffc00000000000 if (ARM64_VA_BITS_47 || ARM64_VA_BITS_52) && ARM64_16K_PAGES && KASAN_SW_TAGS - default 0xeffffe0000000000 if ARM64_VA_BITS_42 && KASAN_SW_TAGS - default 0xefffffc000000000 if ARM64_VA_BITS_39 && KASAN_SW_TAGS - default 0xeffffff800000000 if ARM64_VA_BITS_36 && KASAN_SW_TAGS + default 0xffff800000000000 if (ARM64_VA_BITS_48 || (ARM64_VA_BITS_52 && !ARM64_16K_PAGES)) && KASAN_SW_TAGS + default 0xffffc00000000000 if (ARM64_VA_BITS_47 || ARM64_VA_BITS_52) && ARM64_16K_PAGES && KASAN_SW_TAGS + default 0xfffffe0000000000 if ARM64_VA_BITS_42 && KASAN_SW_TAGS + default 0xffffffc000000000 if ARM64_VA_BITS_39 && KASAN_SW_TAGS + default 0xfffffff800000000 if ARM64_VA_BITS_36 && KASAN_SW_TAGS default 0xffffffffffffffff config UNWIND_TABLESarch/arm64/include/asm/memory.h +13
-1@@ -89,7 +89,15 @@* * KASAN_SHADOW_END is defined first as the shadow address that corresponds to * the upper bound of possible virtual kernel memory addresses UL(1) << 64 -* according to the mapping formula. +* according to the mapping formula. For Generic KASAN, the address in the +* mapping formula is treated as unsigned (part of the compiler's ABI), so the +* end of the shadow memory region is at a large positive offset from +* KASAN_SHADOW_OFFSET. For Software Tag-Based KASAN, the address in the +* formula is treated as signed. Since all kernel addresses are negative, they +* map to shadow memory below KASAN_SHADOW_OFFSET, making KASAN_SHADOW_OFFSET +* itself the end of the shadow memory region. (User pointers are positive and +* would map to shadow memory above KASAN_SHADOW_OFFSET, but shadow memory is +* not allocated for them.) * * KASAN_SHADOW_START is defined second based on KASAN_SHADOW_END. The shadow * memory start must map to the lowest possible kernel virtual memory address @@ -100,7 +108,11 @@*/ #if defined(CONFIG_KASAN_GENERIC ) || defined(CONFIG_KASAN_SW_TAGS )#define KASAN_SHADOW_OFFSET _AC(CONFIG_KASAN_SHADOW_OFFSET, UL) +#ifdef CONFIG_KASAN_GENERIC #define KASAN_SHADOW_END ((UL(1) << (64 - KASAN_SHADOW_SCALE_SHIFT)) + KASAN_SHADOW_OFFSET) +#else +#define KASAN_SHADOW_END KASAN_SHADOW_OFFSET +#endif #define _KASAN_SHADOW_START (va) (KASAN_SHADOW_END - (UL(1) << ((va) - KASAN_SHADOW_SCALE_SHIFT)))#define KASAN_SHADOW_START _KASAN_SHADOW_START(vabits_actual)#define PAGE_END KASAN_SHADOW_STARTarch/arm64/mm/kasan_init.c +0
-0@@ -198,8 +198,11 @@ static bool __init root_level_aligned(u64 addr)/* The early shadow maps everything to a single page of zeroes */ asmlinkage void __init kasan_early_init(void) {
arch/arm64/mm/kasan_init.c static bool __init root_level_aligned(u64 addr) · line 208
Quoted 10 lines
arch/arm64/mm/kasan_init.c +5
-2-BUILD_BUG_ON (KASAN_SHADOW_OFFSET != -KASAN_SHADOW_END - (1UL << (64 -KASAN_SHADOW_SCALE_SHIFT ))); +if (IS_ENABLED (CONFIG_KASAN_GENERIC )) +BUILD_BUG_ON (KASAN_SHADOW_OFFSET != +KASAN_SHADOW_END - (1UL << (64 -KASAN_SHADOW_SCALE_SHIFT ))); +else +BUILD_BUG_ON (KASAN_SHADOW_OFFSET !=KASAN_SHADOW_END );BUILD_BUG_ON (!IS_ALIGNED (_KASAN_SHADOW_START (VA_BITS ),SHADOW_ALIGN ));BUILD_BUG_ON (!IS_ALIGNED (_KASAN_SHADOW_START (VA_BITS_MIN ),SHADOW_ALIGN ));BUILD_BUG_ON (!IS_ALIGNED (KASAN_SHADOW_END ,SHADOW_ALIGN ));
For the arm64 parts:
I wonder whether it's worth keeping the generic KASAN mode for arm64. We've had the hardware TBI from the start, so the architecture version is not an issue. The compiler support may differ though.
Anyway, that would be more suitable for a separate cleanup patch.
Signature · 1 line
Catalin
Re: [PATCH v5 01/19] kasan: sw_tags: Use arithmetic shift for shadow computation
Quoted 76 lines · Catalin Marinas
Quoted 62 lines · Maciej Wieczor-Retman
arch/arm64/Kconfig +5
-5@@ -431,11 +431,11 @@ config KASAN_SHADOW_OFFSET default 0xdffffe0000000000 if ARM64_VA_BITS_42 && !KASAN_SW_TAGS default 0xdfffffc000000000 if ARM64_VA_BITS_39 && !KASAN_SW_TAGS default 0xdffffff800000000 if ARM64_VA_BITS_36 && !KASAN_SW_TAGS - default 0xefff800000000000 if (ARM64_VA_BITS_48 || (ARM64_VA_BITS_52 && !ARM64_16K_PAGES)) && KASAN_SW_TAGS - default 0xefffc00000000000 if (ARM64_VA_BITS_47 || ARM64_VA_BITS_52) && ARM64_16K_PAGES && KASAN_SW_TAGS - default 0xeffffe0000000000 if ARM64_VA_BITS_42 && KASAN_SW_TAGS - default 0xefffffc000000000 if ARM64_VA_BITS_39 && KASAN_SW_TAGS - default 0xeffffff800000000 if ARM64_VA_BITS_36 && KASAN_SW_TAGS + default 0xffff800000000000 if (ARM64_VA_BITS_48 || (ARM64_VA_BITS_52 && !ARM64_16K_PAGES)) && KASAN_SW_TAGS + default 0xffffc00000000000 if (ARM64_VA_BITS_47 || ARM64_VA_BITS_52) && ARM64_16K_PAGES && KASAN_SW_TAGS + default 0xfffffe0000000000 if ARM64_VA_BITS_42 && KASAN_SW_TAGS + default 0xffffffc000000000 if ARM64_VA_BITS_39 && KASAN_SW_TAGS + default 0xfffffff800000000 if ARM64_VA_BITS_36 && KASAN_SW_TAGS default 0xffffffffffffffff config UNWIND_TABLESarch/arm64/include/asm/memory.h +13
-1@@ -89,7 +89,15 @@* * KASAN_SHADOW_END is defined first as the shadow address that corresponds to * the upper bound of possible virtual kernel memory addresses UL(1) << 64 -* according to the mapping formula. +* according to the mapping formula. For Generic KASAN, the address in the +* mapping formula is treated as unsigned (part of the compiler's ABI), so the +* end of the shadow memory region is at a large positive offset from +* KASAN_SHADOW_OFFSET. For Software Tag-Based KASAN, the address in the +* formula is treated as signed. Since all kernel addresses are negative, they +* map to shadow memory below KASAN_SHADOW_OFFSET, making KASAN_SHADOW_OFFSET +* itself the end of the shadow memory region. (User pointers are positive and +* would map to shadow memory above KASAN_SHADOW_OFFSET, but shadow memory is +* not allocated for them.) * * KASAN_SHADOW_START is defined second based on KASAN_SHADOW_END. The shadow * memory start must map to the lowest possible kernel virtual memory address @@ -100,7 +108,11 @@*/ #if defined(CONFIG_KASAN_GENERIC ) || defined(CONFIG_KASAN_SW_TAGS )#define KASAN_SHADOW_OFFSET _AC(CONFIG_KASAN_SHADOW_OFFSET, UL) +#ifdef CONFIG_KASAN_GENERIC #define KASAN_SHADOW_END ((UL(1) << (64 - KASAN_SHADOW_SCALE_SHIFT)) + KASAN_SHADOW_OFFSET) +#else +#define KASAN_SHADOW_END KASAN_SHADOW_OFFSET +#endif #define _KASAN_SHADOW_START (va) (KASAN_SHADOW_END - (UL(1) << ((va) - KASAN_SHADOW_SCALE_SHIFT)))#define KASAN_SHADOW_START _KASAN_SHADOW_START(vabits_actual)#define PAGE_END KASAN_SHADOW_STARTarch/arm64/mm/kasan_init.c +0
-0@@ -198,8 +198,11 @@ static bool __init root_level_aligned(u64 addr)/* The early shadow maps everything to a single page of zeroes */ asmlinkage void __init kasan_early_init(void) {arch/arm64/mm/kasan_init.c static bool __init root_level_aligned(u64 addr) · line 208
Quoted 10 lines
arch/arm64/mm/kasan_init.c +5
-2-BUILD_BUG_ON (KASAN_SHADOW_OFFSET != -KASAN_SHADOW_END - (1UL << (64 -KASAN_SHADOW_SCALE_SHIFT ))); +if (IS_ENABLED (CONFIG_KASAN_GENERIC )) +BUILD_BUG_ON (KASAN_SHADOW_OFFSET != +KASAN_SHADOW_END - (1UL << (64 -KASAN_SHADOW_SCALE_SHIFT ))); +else +BUILD_BUG_ON (KASAN_SHADOW_OFFSET !=KASAN_SHADOW_END );BUILD_BUG_ON (!IS_ALIGNED (_KASAN_SHADOW_START (VA_BITS ),SHADOW_ALIGN ));BUILD_BUG_ON (!IS_ALIGNED (_KASAN_SHADOW_START (VA_BITS_MIN ),SHADOW_ALIGN ));BUILD_BUG_ON (!IS_ALIGNED (KASAN_SHADOW_END ,SHADOW_ALIGN ));For the arm64 parts:
Quoted 1 line
Acked-by Catalin Marinas <[email protected]>
Thanks :)
Quoted 9 lines
I wonder whether it's worth keeping the generic KASAN mode for arm64. We've had the hardware TBI from the start, so the architecture version is not an issue. The compiler support may differ though.
Anyway, that would be more suitable for a separate cleanup patch.
Signature · 1 line
Catalin
I want to test it at some point, but I was always under the impression (that at least in theory) different modes should be able to catch slightly different errors. Not a big set but an example being accessing wrong address, but allocated memory - on Generic it should be okay since shadow memory only says if and how much is allocated. On sw-tags it will fault because randomized tags would mismatch. Now I can't think of any examples the other way around but I assume there is a few.
Signature · 2 lines
Kind regards Maciej Wieczór-Retman
PATCH v5 2/19 kasan: sw_tags: Support tag widths less than 8 bits
In reply to Maciej Wieczor-RetmanFrom: Samuel Holland <[email protected]>
Allow architectures to override KASAN_TAG_KERNEL in asm/kasan.h. This is needed on RISC-V, which supports 57-bit virtual addresses and 7-bit pointer tags. For consistency, move the arm64 MTE definition of KASAN_TAG_MIN to asm/kasan.h, since it is also architecture-dependent; RISC-V's equivalent extension is expected to support 7-bit hardware memory tags.
--- arch/arm64/include/asm/kasan.h | 6 ++++-- arch/arm64/include/asm/uaccess.h | 1 + include/linux/kasan-tags.h | 13 ++++++++----- 3 files changed, 13 insertions(+), 7 deletions(-)
arch/arm64/include/asm/kasan.h +4 -2
@@ -6,8 +6,10 @@#include <linux/linkage.h> #include <asm/memory.h> -#include <asm/mte-kasan.h> -#include <asm/pgtable-types.h> + +#ifdef CONFIG_KASAN_HW_TAGS +#define KASAN_TAG_MIN 0xF0/* minimum value for random tags */ +#endif #define arch_kasan_set_tag (addr, tag) __tag_set(addr, tag)#define arch_kasan_reset_tag (addr) __tag_reset(addr)
arch/arm64/include/asm/uaccess.h +1 -0
@@ -22,6 +22,7 @@#include <asm/cpufeature.h> #include <asm/mmu.h> #include <asm/mte.h> +#include <asm/mte-kasan.h> #include <asm/ptrace.h> #include <asm/memory.h> #include <asm/extable.h>
include/linux/kasan-tags.h +8 -5
@@ -2,13 +2,16 @@#ifndef _LINUX_KASAN_TAGS_H#define _LINUX_KASAN_TAGS_H +#include <asm/kasan.h> + +#ifndef KASAN_TAG_KERNEL #define KASAN_TAG_KERNEL 0xFF/* native kernel pointers tag */ -#define KASAN_TAG_INVALID 0xFE/* inaccessible memory tag */ -#define KASAN_TAG_MAX 0xFD/* maximum value for random tags */ +#endif + +#define KASAN_TAG_INVALID (KASAN_TAG_KERNEL - 1)/* inaccessible memory tag */ +#define KASAN_TAG_MAX (KASAN_TAG_KERNEL - 2)/* maximum value for random tags */ -#ifdef CONFIG_KASAN_HW_TAGS -#define KASAN_TAG_MIN 0xF0/* minimum value for random tags */ -#else +#ifndef KASAN_TAG_MIN #define KASAN_TAG_MIN 0x00/* minimum value for random tags */ #endif
Signature · 1 line
2.50.1
PATCH v5 3/19 kasan: Fix inline mode for x86 tag-based mode
In reply to Maciej Wieczor-RetmanThe LLVM compiler uses hwasan-instrument-with-calls parameter to setup inline or outline mode in tag-based KASAN. If zeroed, it means the instrumentation implementation will be pasted into each relevant location along with KASAN related constants during compilation. If set to one all function instrumentation will be done with function calls instead.
The default hwasan-instrument-with-calls value for the x86 architecture in the compiler is "1", which is not true for other architectures. Because of this, enabling inline mode in software tag-based KASAN doesn't work on x86 as the kernel script doesn't zero out the parameter and always sets up the outline mode.
Explicitly zero out hwasan-instrument-with-calls when enabling inline mode in tag-based KASAN.
---
Changelog v3:
- Add this patch to the series.
scripts/Makefile.kasan | 3 +++ 1 file changed, 3 insertions(+)
scripts/Makefile.kasan +3 -0
@@ -76,8 +76,11 @@ CFLAGS_KASAN := -fsanitize=kernel-hwaddress RUSTFLAGS_KASAN := -Zsanitizer=kernel-hwaddress \ -Zsanitizer-recover=kernel-hwaddress +# LLVM sets hwasan-instrument-with-calls to 1 on x86 by default. Set it to 0 +# when inline mode is enabled. ifdef CONFIG_KASAN_INLINE kasan_params += hwasan-mapping-offset=$(KASAN_SHADOW_OFFSET) + kasan_params += hwasan-instrument-with-calls=0 else kasan_params += hwasan-instrument-with-calls=1 endif
Signature · 1 line
2.50.1
Re: [PATCH v5 03/19] kasan: Fix inline mode for x86 tag-based mode
Quoted 42 lines · Maciej Wieczor-Retman
The LLVM compiler uses hwasan-instrument-with-calls parameter to setup inline or outline mode in tag-based KASAN. If zeroed, it means the instrumentation implementation will be pasted into each relevant location along with KASAN related constants during compilation. If set to one all function instrumentation will be done with function calls instead.
The default hwasan-instrument-with-calls value for the x86 architecture in the compiler is "1", which is not true for other architectures. Because of this, enabling inline mode in software tag-based KASAN doesn't work on x86 as the kernel script doesn't zero out the parameter and always sets up the outline mode.
Explicitly zero out hwasan-instrument-with-calls when enabling inline mode in tag-based KASAN.
Signed-off-by Maciej Wieczor-Retman <[email protected]>---
Changelog v3:
- Add this patch to the series.scripts/Makefile.kasan | 3 +++ 1 file changed, 3 insertions(+)scripts/Makefile.kasan +3
-1@@ -76,8 +76,11 @@ CFLAGS_KASAN := -fsanitize=kernel-hwaddress RUSTFLAGS_KASAN := -Zsanitizer=kernel-hwaddress \ -Zsanitizer-recover=kernel-hwaddress +# LLVM sets hwasan-instrument-with-calls to 1 on x86 by default. Set it to 0 +# when inline mode is enabled. ifdef CONFIG_KASAN_INLINE kasan_params += hwasan-mapping-offset=$(KASAN_SHADOW_OFFSET) + kasan_params += hwasan-instrument-with-calls=0 else kasan_params += hwasan-instrument-with-calls=1 endif --
Quoted 2 lines
2.50.1
PATCH v5 4/19 x86: Add arch specific kasan functions
In reply to Maciej Wieczor-RetmanKASAN's software tag-based mode needs multiple macros/functions to handle tag and pointer interactions - to set, retrieve and reset tags from the top bits of a pointer.
Mimic functions currently used by arm64 but change the tag's position to bits [60:57] in the pointer.
---
Changelog v4:
- Rewrite __tag_set() without pointless casts and make it more readable.
Changelog v3:
- Reorder functions so that __tag_*() etc are above the arch_kasan_*() ones.
- Remove CONFIG_KASAN condition from __tag_set()
arch/x86/include/asm/kasan.h | 36 ++++++++++++++++++++++++++++++++++-- 1 file changed, 34 insertions(+), 2 deletions(-)
arch/x86/include/asm/kasan.h +34 -2
@@ -3,6 +3,8 @@#define _ASM_X86_KASAN_H#include <linux/const.h> +#include <linux/kasan-tags.h> +#include <linux/types.h> #define KASAN_SHADOW_OFFSET _AC(CONFIG_KASAN_SHADOW_OFFSET, UL)#define KASAN_SHADOW_SCALE_SHIFT 3
@@ -24,8 +26,37 @@ KASAN_SHADOW_SCALE_SHIFT)))#ifndef __ASSEMBLER__ +#include <linux/bitops.h> +#include <linux/bitfield.h> +#include <linux/bits.h> + +#ifdef CONFIG_KASAN_SW_TAGS + +#define __tag_shifted (tag) FIELD_PREP(GENMASK_ULL(60, 57), tag) +#define __tag_reset (addr) (sign_extend64((u64)(addr), 56)) +#define __tag_get (addr) ((u8)FIELD_GET(GENMASK_ULL(60, 57), (u64)addr)) +#else +#define __tag_shifted (tag) 0UL +#define __tag_reset (addr) (addr) +#define __tag_get (addr) 0 +#endif /* CONFIG_KASAN_SW_TAGS */ + +static inline void *__tag_set (const void *__addr,u8 tag) +{ +u64 addr = (u64 )__addr; + + addr &= ~__tag_shifted (KASAN_TAG_MASK ); + addr |=__tag_shifted (tag); + +return (void *)addr; +} + +#define arch_kasan_set_tag (addr, tag) __tag_set(addr, tag) +#define arch_kasan_reset_tag (addr) __tag_reset(addr) +#define arch_kasan_get_tag (addr) __tag_get(addr)#ifdef CONFIG_KASAN +void __init kasan_early_init (void );void __init kasan_init (void );void __init kasan_populate_shadow_for_vaddr (void *va,size_t size,int nid);
@@ -34,8 +65,9 @@ static inline void kasan_early_init(void) { }static inline void kasan_init (void ) { }static inline void kasan_populate_shadow_for_vaddr (void *va,size_t size,int nid) { } -#endif -#endif +#endif /* CONFIG_KASAN */ + +#endif /* __ASSEMBLER__ */ #endif
Signature · 1 line
2.50.1
Re: [PATCH v5 04/19] x86: Add arch specific kasan functions
Quoted 35 lines · Maciej Wieczor-Retman
KASAN's software tag-based mode needs multiple macros/functions to handle tag and pointer interactions - to set, retrieve and reset tags from the top bits of a pointer.
Mimic functions currently used by arm64 but change the tag's position to bits [60:57] in the pointer.
Signed-off-by Maciej Wieczor-Retman <[email protected]>---
Changelog v4:
- Rewrite__tag_set()without pointless casts and make it more readable.Changelog v3:
- Reorder functions so that __tag_*() etc are above the arch_kasan_*() ones.
- RemoveCONFIG_KASANcondition from__tag_set()arch/x86/include/asm/kasan.h | 36 ++++++++++++++++++++++++++++++++++-- 1 file changed, 34 insertions(+), 2 deletions(-)arch/x86/include/asm/kasan.h +2
-0@@ -3,6 +3,8 @@#define _ASM_X86_KASAN_H#include <linux/const.h> +#include <linux/kasan-tags.h> +#include <linux/types.h> #define KASAN_SHADOW_OFFSET _AC(CONFIG_KASAN_SHADOW_OFFSET, UL)#define KASAN_SHADOW_SCALE_SHIFT 3
arch/x86/include/asm/kasan.h line 34
Quoted 9 lines
arch/x86/include/asm/kasan.h +6
-0@@ -24,8 +26,37 @@ KASAN_SHADOW_SCALE_SHIFT)))#ifndef __ASSEMBLER__ +#include <linux/bitops.h> +#include <linux/bitfield.h> +#include <linux/bits.h> + +#ifdef CONFIG_KASAN_SW_TAGS +
Nit: can remove this empty line.
Quoted 41 lines
arch/x86/include/asm/kasan.h +26
-3+#define __tag_shifted (tag) FIELD_PREP(GENMASK_ULL(60, 57), tag) +#define __tag_reset (addr) (sign_extend64((u64)(addr), 56)) +#define __tag_get (addr) ((u8)FIELD_GET(GENMASK_ULL(60, 57), (u64)addr)) +#else +#define __tag_shifted (tag) 0UL +#define __tag_reset (addr) (addr) +#define __tag_get (addr) 0 +#endif /* CONFIG_KASAN_SW_TAGS */ + +static inline void *__tag_set (const void *__addr,u8 tag) +{ +u64 addr = (u64 )__addr; + + addr &= ~__tag_shifted (KASAN_TAG_MASK ); + addr |=__tag_shifted (tag); + +return (void *)addr; +} + +#define arch_kasan_set_tag (addr, tag) __tag_set(addr, tag) +#define arch_kasan_reset_tag (addr) __tag_reset(addr) +#define arch_kasan_get_tag (addr) __tag_get(addr)#ifdef CONFIG_KASAN +void __init kasan_early_init (void );void __init kasan_init (void );void __init kasan_populate_shadow_for_vaddr (void *va,size_t size,int nid);@@ -34,8 +65,9 @@ static inline void kasan_early_init(void) { }static inline void kasan_init (void ) { }static inline void kasan_populate_shadow_for_vaddr (void *va,size_t size,int nid) { } -#endif -#endif +#endif /* CONFIG_KASAN */ + +#endif /* __ASSEMBLER__ */ #endif --
Quoted 2 lines
2.50.1
Re: [PATCH v5 04/19] x86: Add arch specific kasan functions
Quoted 39 lines · Andrey Konovalov
Quoted 37 lines · Maciej Wieczor-Retman
KASAN's software tag-based mode needs multiple macros/functions to handle tag and pointer interactions - to set, retrieve and reset tags from the top bits of a pointer.
Mimic functions currently used by arm64 but change the tag's position to bits [60:57] in the pointer.
Signed-off-by Maciej Wieczor-Retman <[email protected]>---
Changelog v4:
- Rewrite__tag_set()without pointless casts and make it more readable.Changelog v3:
- Reorder functions so that __tag_*() etc are above the arch_kasan_*() ones.
- RemoveCONFIG_KASANcondition from__tag_set()arch/x86/include/asm/kasan.h | 36 ++++++++++++++++++++++++++++++++++-- 1 file changed, 34 insertions(+), 2 deletions(-)arch/x86/include/asm/kasan.h +2
-0@@ -3,6 +3,8 @@#define _ASM_X86_KASAN_H#include <linux/const.h> +#include <linux/kasan-tags.h> +#include <linux/types.h> #define KASAN_SHADOW_OFFSET _AC(CONFIG_KASAN_SHADOW_OFFSET, UL)#define KASAN_SHADOW_SCALE_SHIFT 3@@ -24,8 +26,37 @@ KASAN_SHADOW_SCALE_SHIFT)))
Quoted 9 lines
arch/x86/include/asm/kasan.h line 34
Quoted 7 lines
arch/x86/include/asm/kasan.h +6
-0#ifndef __ASSEMBLER__ +#include <linux/bitops.h> +#include <linux/bitfield.h> +#include <linux/bits.h> + +#ifdef CONFIG_KASAN_SW_TAGS +Nit: can remove this empty line.
Sure, will do, thanks.
Quoted 46 lines
Quoted 42 lines
arch/x86/include/asm/kasan.h +26
-3+#define __tag_shifted (tag) FIELD_PREP(GENMASK_ULL(60, 57), tag) +#define __tag_reset (addr) (sign_extend64((u64)(addr), 56)) +#define __tag_get (addr) ((u8)FIELD_GET(GENMASK_ULL(60, 57), (u64)addr)) +#else +#define __tag_shifted (tag) 0UL +#define __tag_reset (addr) (addr) +#define __tag_get (addr) 0 +#endif /* CONFIG_KASAN_SW_TAGS */ + +static inline void *__tag_set (const void *__addr,u8 tag) +{ +u64 addr = (u64 )__addr; + + addr &= ~__tag_shifted (KASAN_TAG_MASK ); + addr |=__tag_shifted (tag); + +return (void *)addr; +} + +#define arch_kasan_set_tag (addr, tag) __tag_set(addr, tag) +#define arch_kasan_reset_tag (addr) __tag_reset(addr) +#define arch_kasan_get_tag (addr) __tag_get(addr)#ifdef CONFIG_KASAN +void __init kasan_early_init (void );void __init kasan_init (void );void __init kasan_populate_shadow_for_vaddr (void *va,size_t size,int nid);@@ -34,8 +65,9 @@ static inline void kasan_early_init(void) { }static inline void kasan_init (void ) { }static inline void kasan_populate_shadow_for_vaddr (void *va,size_t size,int nid) { } -#endif -#endif +#endif /* CONFIG_KASAN */ + +#endif /* __ASSEMBLER__ */ #endif --Quoted 2 lines
2.50.1
Acked-by Andrey Konovalov <[email protected]>
Signature · 2 lines
Kind regards Maciej Wieczór-Retman
Andrey Ryabinin
maintainerRe: [PATCH v5 04/19] x86: Add arch specific kasan functions
In reply to Maciej Wieczor-Retmanarch/x86/include/asm/kasan.h line 53
Quoted 10 lines · Maciej Wieczor-Retman
arch/x86/include/asm/kasan.h +10
-0+static inline void *__tag_set (const void *__addr,u8 tag) +{ +u64 addr = (u64 )__addr; + + addr &= ~__tag_shifted (KASAN_TAG_MASK ); + addr |=__tag_shifted (tag); + +return (void *)addr; +} +
This requires some ifdef magic to avoid getting this into vdso32 image build process, otherwise we'll get this warning:
CC arch/x86/entry/vdso/vdso32/vclock_gettime.o
In file included from ../arch/x86/include/asm/page.h:10,
from ../arch/x86/include/asm/processor.h:20,
from ../arch/x86/include/asm/timex.h:5,
from ../include/linux/timex.h:67,
from ../include/linux/time32.h:13,
from ../include/linux/time.h:60,
from ../arch/x86/entry/vdso/vdso32/../vclock_gettime.c:11,
from ../arch/x86/entry/vdso/vdso32/vclock_gettime.c:4:../arch/x86/include/asm/kasan.h: In function ‘__tag_set’:../arch/x86/include/asm/kasan.h:81:20: warning: cast from pointer to integer of different size [-Wpointer-to-int-cast]
81 | u64 addr = (u64)__addr;
| ^../arch/x86/include/asm/kasan.h:86:16: warning: cast to pointer from integer of different size [-Wint-to-pointer-cast]
86 | return (void *)addr;
| ^
Re: [PATCH v5 04/19] x86: Add arch specific kasan functions
Quoted 27 lines · Andrey Ryabinin
arch/x86/include/asm/kasan.h line 53
Quoted 10 lines · Maciej Wieczor-Retman
arch/x86/include/asm/kasan.h +10
-0+static inline void *__tag_set (const void *__addr,u8 tag) +{ +u64 addr = (u64 )__addr; + + addr &= ~__tag_shifted (KASAN_TAG_MASK ); + addr |=__tag_shifted (tag); + +return (void *)addr; +} +This requires some ifdef magic to avoid getting this into vdso32 image build process, otherwise we'll get this warning:
CC
arch/x86/entry/vdso/vdso32/vclock_gettime.o
In file included from../arch/x86/include/asm/page.h:10,from ../arch/x86/include/asm/processor.h:20, from ../arch/x86/include/asm/timex.h:5, from ../include/linux/timex.h:67, from ../include/linux/time32.h:13, from ../include/linux/time.h:60, from ../arch/x86/entry/vdso/vdso32/../vclock_gettime.c:11, from ../arch/x86/entry/vdso/vdso32/vclock_gettime.c:4:
Quoted 8 lines
../arch/x86/include/asm/kasan.h: In function ‘__tag_set’:../arch/x86/include/asm/kasan.h:81:20: warning: cast from pointer to integer of different size [-Wpointer-to-int-cast]
81 | u64 addr = (u64)__addr;
| ^../arch/x86/include/asm/kasan.h:86:16: warning: cast to pointer from integer of different size [-Wint-to-pointer-cast]
86 | return (void *)addr;
| ^
Thanks for noticing that, I'll fix it :)
Signature · 2 lines
Kind regards Maciej Wieczór-Retman
PATCH v5 5/19 kasan: arm64: x86: Make special tags arch specific
In reply to Maciej Wieczor-RetmanKASAN's tag-based mode defines multiple special tag values. They're reserved for:
- Native kernel value. On arm64 it's 0xFF and it causes an early return in the tag checking function.
- Invalid value. 0xFE marks an area as freed / unallocated. It's also the value that is used to initialize regions of shadow memory.
- Max value. 0xFD is the highest value that can be randomly generated for a new tag.
Metadata macro is also defined:
- Tag width equal to 8.
Tag-based mode on x86 is going to use 4 bit wide tags so all the above values need to be changed accordingly.
Make native kernel tag arch specific for x86 and arm64.
Replace hardcoded kernel tag value and tag width with macros in KASAN's non-arch specific code.
---
Changelog v5:
- Move KASAN_TAG_MIN to the arm64 kasan-tags.h for the hardware KASAN mode case.
Changelog v4:
- Move KASAN_TAG_MASK to kasan-tags.h.
Changelog v2:
- Remove risc-v from the patch.
MAINTAINERS | 2 +- arch/arm64/include/asm/kasan-tags.h | 13 +++++++++++++ arch/arm64/include/asm/kasan.h | 4 ---- arch/x86/include/asm/kasan-tags.h | 9 +++++++++ include/linux/kasan-tags.h | 10 +++++++++- include/linux/kasan.h | 4 +++- include/linux/mm.h | 6 +++--- include/linux/mmzone.h | 1 - include/linux/page-flags-layout.h | 9 +-------- 9 files changed, 39 insertions(+), 19 deletions(-) create mode 100644 arch/arm64/include/asm/kasan-tags.h create mode 100644 arch/x86/include/asm/kasan-tags.h
MAINTAINERS +1 -1
@@ -13176,7 +13176,7 @@ L: [email protected] S: Maintained B: https://bugzilla.kernel.org/buglist.cgi?component=Sanitizers&product=Memory%20Management F: Documentation/dev-tools/kasan.rst -F: arch/*/include/asm/*kasan.h +F: arch/*/include/asm/*kasan*.h F: arch/*/mm/kasan_init* F: include/linux/kasan*.h F: lib/Kconfig.kasan
arch/arm64/include/asm/kasan-tags.h new file +13 -0
@@ -0,0 +1,13 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +#ifndef __ASM_KASAN_TAGS_H +#define __ASM_KASAN_TAGS_H + +#define KASAN_TAG_KERNEL 0xFF/* native kernel pointers tag */ + +#define KASAN_TAG_WIDTH 8 + +#ifdef CONFIG_KASAN_HW_TAGS +#define KASAN_TAG_MIN 0xF0/* minimum value for random tags */ +#endif + +#endif /* ASM_KASAN_TAGS_H */
arch/arm64/include/asm/kasan.h +0 -4
@@ -7,10 +7,6 @@#include <linux/linkage.h> #include <asm/memory.h> -#ifdef CONFIG_KASAN_HW_TAGS -#define KASAN_TAG_MIN 0xF0/* minimum value for random tags */ -#endif -#define arch_kasan_set_tag (addr, tag) __tag_set(addr, tag)#define arch_kasan_reset_tag (addr) __tag_reset(addr)#define arch_kasan_get_tag (addr) __tag_get(addr)
arch/x86/include/asm/kasan-tags.h new file +9 -0
@@ -0,0 +1,9 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +#ifndef __ASM_KASAN_TAGS_H +#define __ASM_KASAN_TAGS_H + +#define KASAN_TAG_KERNEL 0xF/* native kernel pointers tag */ + +#define KASAN_TAG_WIDTH 4 + +#endif /* ASM_KASAN_TAGS_H */
include/linux/kasan-tags.h +9 -1
@@ -2,7 +2,15 @@#ifndef _LINUX_KASAN_TAGS_H#define _LINUX_KASAN_TAGS_H -#include <asm/kasan.h> +#if defined(CONFIG_KASAN_SW_TAGS ) || defined(CONFIG_KASAN_HW_TAGS ) +#include <asm/kasan-tags.h> +#endif + +#ifndef KASAN_TAG_WIDTH +#define KASAN_TAG_WIDTH 0 +#endif + +#define KASAN_TAG_MASK ((1UL << KASAN_TAG_WIDTH) - 1)#ifndef KASAN_TAG_KERNEL #define KASAN_TAG_KERNEL 0xFF/* native kernel pointers tag */
include/linux/kasan.h +3 -1
@@ -40,7 +40,9 @@ typedef unsigned int __bitwise kasan_vmalloc_flags_t;#ifdef CONFIG_KASAN_SW_TAGS /* This matches KASAN_TAG_INVALID. */ -#define KASAN_SHADOW_INIT 0xFE +#ifndef KASAN_SHADOW_INIT +#define KASAN_SHADOW_INIT KASAN_TAG_INVALID +#endif #else #define KASAN_SHADOW_INIT 0#endif
include/linux/mm.h +3 -3
@@ -1692,7 +1692,7 @@ static inline u8 page_kasan_tag(const struct page *page)if (kasan_enabled ()) { tag = (page->flags >>KASAN_TAG_PGSHIFT ) &KASAN_TAG_MASK ; - tag ^=0xff ; + tag ^=KASAN_TAG_KERNEL ; }return tag;
@@ -1705,7 +1705,7 @@ static inline void page_kasan_tag_set(struct page *page, u8 tag)if (!kasan_enabled ())return ; - tag ^=0xff ; + tag ^=KASAN_TAG_KERNEL ; old_flags =READ_ONCE (page->flags); do { flags = old_flags;
@@ -1724,7 +1724,7 @@ static inline void page_kasan_tag_reset(struct page *page)static inline u8 page_kasan_tag (const struct page *page) { -return 0xff ; +return KASAN_TAG_KERNEL ; }static inline void page_kasan_tag_set (struct page *page,u8 tag) { }
include/linux/mmzone.h +0 -1
@@ -1166,7 +1166,6 @@ static inline bool zone_is_empty(struct zone *zone)#define NODES_MASK ((1UL << NODES_WIDTH) - 1)#define SECTIONS_MASK ((1UL << SECTIONS_WIDTH) - 1)#define LAST_CPUPID_MASK ((1UL << LAST_CPUPID_SHIFT) - 1) -#define KASAN_TAG_MASK ((1UL << KASAN_TAG_WIDTH) - 1)#define ZONEID_MASK ((1UL << ZONEID_SHIFT) - 1)static inline enum zone_type page_zonenum (const struct page *page)
include/linux/page-flags-layout.h +1 -8
@@ -3,6 +3,7 @@#define PAGE_FLAGS_LAYOUT_H #include <linux/numa.h> +#include <linux/kasan-tags.h> #include <generated/bounds.h> /*
@@ -72,14 +73,6 @@#define NODE_NOT_IN_PAGE_FLAGS 1#endif -#if defined(CONFIG_KASAN_SW_TAGS ) -#define KASAN_TAG_WIDTH 8 -#elif defined(CONFIG_KASAN_HW_TAGS ) -#define KASAN_TAG_WIDTH 4 -#else -#define KASAN_TAG_WIDTH 0 -#endif -#ifdef CONFIG_NUMA_BALANCING #define LAST__PID_SHIFT 8#define LAST__PID_MASK ((1 << LAST__PID_SHIFT)-1)
Signature · 1 line
2.50.1
Andrey Konovalov
reviewerRe: [PATCH v5 05/19] kasan: arm64: x86: Make special tags arch specific
Quoted 134 lines · Maciej Wieczor-Retman
KASAN's tag-based mode defines multiple special tag values. They're reserved for:
- Native kernel value. On arm64 it's0xFFand it causes an early return in the tag checking function.
- Invalid value.0xFEmarks an area as freed / unallocated. It's also the value that is used to initialize regions of shadow memory.
- Max value.0xFDis the highest value that can be randomly generated for a new tag.Metadata macro is also defined:
- Tag width equal to 8.Tag-based mode on x86 is going to use 4 bit wide tags so all the above values need to be changed accordingly.
Make native kernel tag arch specific for x86 and arm64.
Replace hardcoded kernel tag value and tag width with macros in KASAN's non-arch specific code.
Signed-off-by Maciej Wieczor-Retman <[email protected]>---
Changelog v5:
- MoveKASAN_TAG_MINto the arm64kasan-tags.hfor the hardware KASAN mode case.Changelog v4:
- MoveKASAN_TAG_MASKtokasan-tags.h.Changelog v2:
- Remove risc-v from the patch.MAINTAINERS | 2 +- arch/arm64/include/asm/kasan-tags.h | 13 +++++++++++++ arch/arm64/include/asm/kasan.h | 4 ---- arch/x86/include/asm/kasan-tags.h | 9 +++++++++ include/linux/kasan-tags.h | 10 +++++++++- include/linux/kasan.h | 4 +++- include/linux/mm.h | 6 +++--- include/linux/mmzone.h | 1 - include/linux/page-flags-layout.h | 9 +-------- 9 files changed, 39 insertions(+), 19 deletions(-) create mode 100644 arch/arm64/include/asm/kasan-tags.h create mode 100644 arch/x86/include/asm/kasan-tags.hMAINTAINERS +1
-1@@ -13176,7 +13176,7 @@ L: [email protected] S: Maintained B: https://bugzilla.kernel.org/buglist.cgi?component=Sanitizers&product=Memory%20Management F: Documentation/dev-tools/kasan.rst -F: arch/*/include/asm/*kasan.h +F: arch/*/include/asm/*kasan*.h F: arch/*/mm/kasan_init* F: include/linux/kasan*.h F: lib/Kconfig.kasanarch/arm64/include/asm/kasan-tags.h new file +13
-0@@ -0,0 +1,13 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +#ifndef __ASM_KASAN_TAGS_H +#define __ASM_KASAN_TAGS_H + +#define KASAN_TAG_KERNEL 0xFF/* native kernel pointers tag */ + +#define KASAN_TAG_WIDTH 8 + +#ifdef CONFIG_KASAN_HW_TAGS +#define KASAN_TAG_MIN 0xF0/* minimum value for random tags */ +#endif + +#endif /* ASM_KASAN_TAGS_H */ arch/arm64/include/asm/kasan.h +0
-4@@ -7,10 +7,6 @@#include <linux/linkage.h> #include <asm/memory.h> -#ifdef CONFIG_KASAN_HW_TAGS -#define KASAN_TAG_MIN 0xF0/* minimum value for random tags */ -#endif -#define arch_kasan_set_tag (addr, tag) __tag_set(addr, tag)#define arch_kasan_reset_tag (addr) __tag_reset(addr)#define arch_kasan_get_tag (addr) __tag_get(addr)arch/x86/include/asm/kasan-tags.h new file +9
-0@@ -0,0 +1,9 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +#ifndef __ASM_KASAN_TAGS_H +#define __ASM_KASAN_TAGS_H + +#define KASAN_TAG_KERNEL 0xF/* native kernel pointers tag */ + +#define KASAN_TAG_WIDTH 4 + +#endif /* ASM_KASAN_TAGS_H */ include/linux/kasan-tags.h +9
-1@@ -2,7 +2,15 @@#ifndef _LINUX_KASAN_TAGS_H#define _LINUX_KASAN_TAGS_H -#include <asm/kasan.h> +#if defined(CONFIG_KASAN_SW_TAGS ) || defined(CONFIG_KASAN_HW_TAGS ) +#include <asm/kasan-tags.h> +#endif + +#ifndef KASAN_TAG_WIDTH +#define KASAN_TAG_WIDTH 0 +#endif + +#define KASAN_TAG_MASK ((1UL << KASAN_TAG_WIDTH) - 1)#ifndef KASAN_TAG_KERNEL #define KASAN_TAG_KERNEL 0xFF/* native kernel pointers tag */ include/linux/kasan.h +0
-0
include/linux/kasan.h typedef unsigned int __bitwise kasan_vmalloc_flags_t; · line 43
Quoted 5 lines
include/linux/kasan.h +1
-1@@ -40,7 +40,9 @@ typedef unsigned int __bitwise kasan_vmalloc_flags_t;#ifdef CONFIG_KASAN_SW_TAGS /* This matches KASAN_TAG_INVALID. */ -#define KASAN_SHADOW_INIT 0xFE +#ifndef KASAN_SHADOW_INIT
Do we need this ifndef?
Quoted 41 lines
include/linux/kasan.h +2
-0+#define KASAN_SHADOW_INIT KASAN_TAG_INVALID +#endif #else #define KASAN_SHADOW_INIT 0#endif include/linux/mm.h +3
-3@@ -1692,7 +1692,7 @@ static inline u8 page_kasan_tag(const struct page *page)if (kasan_enabled ()) { tag = (page->flags >>KASAN_TAG_PGSHIFT ) &KASAN_TAG_MASK ; - tag ^=0xff ; + tag ^=KASAN_TAG_KERNEL ; }return tag;@@ -1705,7 +1705,7 @@ static inline void page_kasan_tag_set(struct page *page, u8 tag)if (!kasan_enabled ())return ; - tag ^=0xff ; + tag ^=KASAN_TAG_KERNEL ; old_flags =READ_ONCE (page->flags); do { flags = old_flags;@@ -1724,7 +1724,7 @@ static inline void page_kasan_tag_reset(struct page *page)static inline u8 page_kasan_tag (const struct page *page) { -return 0xff ; +return KASAN_TAG_KERNEL ; }static inline void page_kasan_tag_set (struct page *page,u8 tag) { }include/linux/mmzone.h +0
-0
include/linux/mmzone.h static inline bool zone_is_empty(struct zone *zone) · line 1169
Quoted 4 lines
include/linux/mmzone.h +0
-1@@ -1166,7 +1166,6 @@ static inline bool zone_is_empty(struct zone *zone)#define NODES_MASK ((1UL << NODES_WIDTH) - 1)#define SECTIONS_MASK ((1UL << SECTIONS_WIDTH) - 1)#define LAST_CPUPID_MASK ((1UL << LAST_CPUPID_SHIFT) - 1) -#define KASAN_TAG_MASK ((1UL << KASAN_TAG_WIDTH) - 1)
So we cannot define this here because of include dependencies? Having this value defined here would look cleaner.
Otherwise, let's add a comment here with a reference to where this value is defined.
Quoted 16 lines
include/linux/mmzone.h +0
-0#define ZONEID_MASK ((1UL << ZONEID_SHIFT) - 1)static inline enum zone_type page_zonenum (const struct page *page)include/linux/page-flags-layout.h +1
-0@@ -3,6 +3,7 @@#define PAGE_FLAGS_LAYOUT_H #include <linux/numa.h> +#include <linux/kasan-tags.h> #include <generated/bounds.h> /*
include/linux/page-flags-layout.h line 78
Quoted 7 lines
include/linux/page-flags-layout.h +0
-4@@ -72,14 +73,6 @@#define NODE_NOT_IN_PAGE_FLAGS 1#endif -#if defined(CONFIG_KASAN_SW_TAGS ) -#define KASAN_TAG_WIDTH 8 -#elif defined(CONFIG_KASAN_HW_TAGS ) -#define KASAN_TAG_WIDTH 4
This case is removed here but not added to arch/arm64/include/asm/kasan-tags.h.
Quoted 10 lines
include/linux/page-flags-layout.h +0
-5-#else -#define KASAN_TAG_WIDTH 0 -#endif -#ifdef CONFIG_NUMA_BALANCING #define LAST__PID_SHIFT 8#define LAST__PID_MASK ((1 << LAST__PID_SHIFT)-1) --2.50.1
Re: [PATCH v5 05/19] kasan: arm64: x86: Make special tags arch specific
Quoted 137 lines · Andrey Konovalov
Quoted 135 lines · Maciej Wieczor-Retman
KASAN's tag-based mode defines multiple special tag values. They're reserved for:
- Native kernel value. On arm64 it's0xFFand it causes an early return in the tag checking function.
- Invalid value.0xFEmarks an area as freed / unallocated. It's also the value that is used to initialize regions of shadow memory.
- Max value.0xFDis the highest value that can be randomly generated for a new tag.Metadata macro is also defined:
- Tag width equal to 8.Tag-based mode on x86 is going to use 4 bit wide tags so all the above values need to be changed accordingly.
Make native kernel tag arch specific for x86 and arm64.
Replace hardcoded kernel tag value and tag width with macros in KASAN's non-arch specific code.
Signed-off-by Maciej Wieczor-Retman <[email protected]>---
Changelog v5:
- MoveKASAN_TAG_MINto the arm64kasan-tags.hfor the hardware KASAN mode case.Changelog v4:
- MoveKASAN_TAG_MASKtokasan-tags.h.Changelog v2:
- Remove risc-v from the patch.MAINTAINERS | 2 +- arch/arm64/include/asm/kasan-tags.h | 13 +++++++++++++ arch/arm64/include/asm/kasan.h | 4 ---- arch/x86/include/asm/kasan-tags.h | 9 +++++++++ include/linux/kasan-tags.h | 10 +++++++++- include/linux/kasan.h | 4 +++- include/linux/mm.h | 6 +++--- include/linux/mmzone.h | 1 - include/linux/page-flags-layout.h | 9 +-------- 9 files changed, 39 insertions(+), 19 deletions(-) create mode 100644 arch/arm64/include/asm/kasan-tags.h create mode 100644 arch/x86/include/asm/kasan-tags.hMAINTAINERS +1
-1@@ -13176,7 +13176,7 @@ L: [email protected] S: Maintained B: https://bugzilla.kernel.org/buglist.cgi?component=Sanitizers&product=Memory%20Management F: Documentation/dev-tools/kasan.rst -F: arch/*/include/asm/*kasan.h +F: arch/*/include/asm/*kasan*.h F: arch/*/mm/kasan_init* F: include/linux/kasan*.h F: lib/Kconfig.kasanarch/arm64/include/asm/kasan-tags.h new file +13
-0@@ -0,0 +1,13 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +#ifndef __ASM_KASAN_TAGS_H +#define __ASM_KASAN_TAGS_H + +#define KASAN_TAG_KERNEL 0xFF/* native kernel pointers tag */ + +#define KASAN_TAG_WIDTH 8 + +#ifdef CONFIG_KASAN_HW_TAGS +#define KASAN_TAG_MIN 0xF0/* minimum value for random tags */ +#endif + +#endif /* ASM_KASAN_TAGS_H */ arch/arm64/include/asm/kasan.h +0
-4@@ -7,10 +7,6 @@#include <linux/linkage.h> #include <asm/memory.h> -#ifdef CONFIG_KASAN_HW_TAGS -#define KASAN_TAG_MIN 0xF0/* minimum value for random tags */ -#endif -#define arch_kasan_set_tag (addr, tag) __tag_set(addr, tag)#define arch_kasan_reset_tag (addr) __tag_reset(addr)#define arch_kasan_get_tag (addr) __tag_get(addr)arch/x86/include/asm/kasan-tags.h new file +9
-0@@ -0,0 +1,9 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +#ifndef __ASM_KASAN_TAGS_H +#define __ASM_KASAN_TAGS_H + +#define KASAN_TAG_KERNEL 0xF/* native kernel pointers tag */ + +#define KASAN_TAG_WIDTH 4 + +#endif /* ASM_KASAN_TAGS_H */ include/linux/kasan-tags.h +9
-1@@ -2,7 +2,15 @@#ifndef _LINUX_KASAN_TAGS_H#define _LINUX_KASAN_TAGS_H -#include <asm/kasan.h> +#if defined(CONFIG_KASAN_SW_TAGS ) || defined(CONFIG_KASAN_HW_TAGS ) +#include <asm/kasan-tags.h> +#endif + +#ifndef KASAN_TAG_WIDTH +#define KASAN_TAG_WIDTH 0 +#endif + +#define KASAN_TAG_MASK ((1UL << KASAN_TAG_WIDTH) - 1)#ifndef KASAN_TAG_KERNEL #define KASAN_TAG_KERNEL 0xFF/* native kernel pointers tag */ include/linux/kasan.h +0
-0@@ -40,7 +40,9 @@ typedef unsigned int __bitwise kasan_vmalloc_flags_t;
Quoted 6 lines
include/linux/kasan.h typedef unsigned int __bitwise kasan_vmalloc_flags_t; · line 43
Quoted 4 lines
include/linux/kasan.h +1
-1#ifdef CONFIG_KASAN_SW_TAGS /* This matches KASAN_TAG_INVALID. */ -#define KASAN_SHADOW_INIT 0xFE +#ifndef KASAN_SHADOW_INIT Do we need this ifndef?
I just checked and you're right, it's not needed. I think it might have been a leftover of my dense mode code.
Quoted 50 lines
Quoted 42 lines
include/linux/kasan.h +2
-0+#define KASAN_SHADOW_INIT KASAN_TAG_INVALID +#endif #else #define KASAN_SHADOW_INIT 0#endif include/linux/mm.h +3
-3@@ -1692,7 +1692,7 @@ static inline u8 page_kasan_tag(const struct page *page)if (kasan_enabled ()) { tag = (page->flags >>KASAN_TAG_PGSHIFT ) &KASAN_TAG_MASK ; - tag ^=0xff ; + tag ^=KASAN_TAG_KERNEL ; }return tag;@@ -1705,7 +1705,7 @@ static inline void page_kasan_tag_set(struct page *page, u8 tag)if (!kasan_enabled ())return ; - tag ^=0xff ; + tag ^=KASAN_TAG_KERNEL ; old_flags =READ_ONCE (page->flags); do { flags = old_flags;@@ -1724,7 +1724,7 @@ static inline void page_kasan_tag_reset(struct page *page)static inline u8 page_kasan_tag (const struct page *page) { -return 0xff ; +return KASAN_TAG_KERNEL ; }static inline void page_kasan_tag_set (struct page *page,u8 tag) { }include/linux/mmzone.h +0
-0include/linux/mmzone.h static inline bool zone_is_empty(struct zone *zone) · line 1169
Quoted 4 lines
include/linux/mmzone.h +0
-1@@ -1166,7 +1166,6 @@ static inline bool zone_is_empty(struct zone *zone)#define NODES_MASK ((1UL << NODES_WIDTH) - 1)#define SECTIONS_MASK ((1UL << SECTIONS_WIDTH) - 1)#define LAST_CPUPID_MASK ((1UL << LAST_CPUPID_SHIFT) - 1) -#define KASAN_TAG_MASK ((1UL << KASAN_TAG_WIDTH) - 1)So we cannot define this here because of include dependencies? Having this value defined here would look cleaner.
Quoted 2 lines
Otherwise, let's add a comment here with a reference to where this value is defined.
I'll retest with a couple of configs but I removed this change and everything compile fine. Thanks for noticing that
Quoted 20 lines
Quoted 20 lines
include/linux/mmzone.h +0
-0#define ZONEID_MASK ((1UL << ZONEID_SHIFT) - 1)static inline enum zone_type page_zonenum (const struct page *page)include/linux/page-flags-layout.h +1
-0@@ -3,6 +3,7 @@#define PAGE_FLAGS_LAYOUT_H #include <linux/numa.h> +#include <linux/kasan-tags.h> #include <generated/bounds.h> /* @@ -72,14 +73,6 @@#define NODE_NOT_IN_PAGE_FLAGS 1#endif
Quoted 6 lines
include/linux/page-flags-layout.h line 78
Quoted 4 lines
include/linux/page-flags-layout.h +0
-4-#if defined(CONFIG_KASAN_SW_TAGS ) -#define KASAN_TAG_WIDTH 8 -#elif defined(CONFIG_KASAN_HW_TAGS ) -#define KASAN_TAG_WIDTH 4This case is removed here but not added to
arch/arm64/include/asm/kasan-tags.h.
Right, I'll correct that.
Quoted 12 lines
Quoted 12 lines
include/linux/page-flags-layout.h +0
-5-#else -#define KASAN_TAG_WIDTH 0 -#endif -#ifdef CONFIG_NUMA_BALANCING #define LAST__PID_SHIFT 8#define LAST__PID_MASK ((1 << LAST__PID_SHIFT)-1) --2.50.1
Signature · 2 lines
Kind regards Maciej Wieczór-Retman
PATCH v5 6/19 x86: Reset tag for virtual to physical address conversions
In reply to Maciej Wieczor-RetmanAny place where pointer arithmetic is used to convert a virtual address into a physical one can raise errors if the virtual address is tagged.
Reset the pointer's tag by sign extending the tag bits in macros that do pointer arithmetic in address conversions. There will be no change in compiled code with KASAN disabled since the compiler will optimize the __tag_reset() out.
---
Changelog v5:
- Move __tag_reset() calls into __phys_addr_nodebug() and __virt_addr_valid() instead of calling it on the arguments of higher level functions.
Changelog v4:
- Simplify page_to_virt() by removing pointless casts.
- Remove change in __is_canonical_address() because it's taken care of in a later patch due to a LAM compatible definition of canonical.
arch/x86/include/asm/page.h | 8 ++++++++ arch/x86/include/asm/page_64.h | 1 + arch/x86/mm/physaddr.c | 2 ++ 3 files changed, 11 insertions(+)
arch/x86/include/asm/page.h +8 -0
@@ -7,6 +7,7 @@#ifdef __KERNEL__#include <asm/page_types.h> +#include <asm/kasan.h> #ifdef CONFIG_X86_64 #include <asm/page_64.h>
@@ -65,6 +66,13 @@ static inline void copy_user_page(void *to, void *from, unsigned long vaddr,* virt_to_page(kaddr) returns a valid pointer if and only if * virt_addr_valid(kaddr) returns true. */ + +#ifdef CONFIG_KASAN_SW_TAGS +#define page_to_virt (x) ({ \ + void *__addr = __va(page_to_pfn((struct page *)x) << PAGE_SHIFT); \ + __tag_set(__addr, page_kasan_tag(x)); \ +}) +#endif #define virt_to_page (kaddr) pfn_to_page(__pa(kaddr) >> PAGE_SHIFT)extern bool __virt_addr_valid (unsigned long kaddr);#define virt_addr_valid (kaddr) __virt_addr_valid((unsigned long) (kaddr))
arch/x86/include/asm/page_64.h +1 -0
@@ -21,6 +21,7 @@ extern unsigned long direct_map_physmem_end;static __always_inline unsigned long __phys_addr_nodebug (unsigned long x) { + x =__tag_reset (x);unsigned long y = x - __START_KERNEL_map;/* use the carry flag to determine if x was < __START_KERNEL_map */
arch/x86/mm/physaddr.c +2 -0
@@ -14,6 +14,7 @@#ifdef CONFIG_DEBUG_VIRTUAL unsigned long __phys_addr (unsigned long x) { + x =__tag_reset (x);unsigned long y = x - __START_KERNEL_map;/* use the carry flag to determine if x was < __START_KERNEL_map */
@@ -46,6 +47,7 @@ EXPORT_SYMBOL(__phys_addr_symbol);bool __virt_addr_valid (unsigned long x) { + x =__tag_reset (x);unsigned long y = x - __START_KERNEL_map;/* use the carry flag to determine if x was < __START_KERNEL_map */
Signature · 1 line
2.50.1
PATCH v5 7/19 mm: x86: Untag addresses in EXECMEM_ROX related pointer arithmetic
In reply to Maciej Wieczor-RetmanARCH_HAS_EXECMEM_ROX was re-enabled in x86 at Linux 6.14 release. Related code has multiple spots where page virtual addresses end up used as arguments in arithmetic operations. Combined with enabled tag-based KASAN it can result in pointers that don't point where they should or logical operations not giving expected results.
vm_reset_perms() calculates range's start and end addresses using min() and max() functions. To do that it compares pointers but some are not tagged - addr variable is, start and end variables aren't.
within() and within_range() can receive tagged addresses which get compared to untagged start and end variables.
Reset tags in addresses used as function arguments in min(), max(), within().
execmem_cache_add() adds tagged pointers to a maple tree structure, which then are incorrectly compared when walking the tree. That results in different pointers being returned later and page permission violation errors panicking the kernel.
Reset tag of the address range inserted into the maple tree inside execmem_cache_add().
---
Changelog v5:
- Remove the within_range() change.
- arch_kasan_reset_tag -> kasan_reset_tag.
Changelog v4:
- Add patch to the series.
mm/execmem.c | 2 +- mm/vmalloc.c | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-)
mm/execmem.c +1 -1
@@ -186,7 +186,7 @@ static DECLARE_WORK(execmem_cache_clean_work, execmem_cache_clean);static int execmem_cache_add_locked (void *ptr,size_t size,gfp_t gfp_mask) {struct maple_tree *free_areas = &execmem_cache.free_areas; -unsigned long addr = (unsigned long )ptr; +unsigned long addr = (unsigned long )kasan_reset_tag (ptr);MA_STATE (mas, free_areas, addr -1 , addr +1 );unsigned long lower, upper;void *area =NULL ;
mm/vmalloc.c +1 -1
@@ -3322,7 +3322,7 @@ static void vm_reset_perms(struct vm_struct *area)* the vm_unmap_aliases() flush includes the direct map. */ for (i =0 ; i < area->nr_pages; i +=1U << page_order) { -unsigned long addr = (unsigned long )page_address (area->pages[i]); +unsigned long addr = (unsigned long )kasan_reset_tag (page_address (area->pages[i]));if (addr) {unsigned long page_size;
Signature · 1 line
2.50.1
Mike Rapoport
maintainerRe: [PATCH v5 07/19] mm: x86: Untag addresses in EXECMEM_ROX related pointer arithmetic
Quoted 42 lines · Maciej Wieczor-Retman
ARCH_HAS_EXECMEM_ROXwas re-enabled in x86 at Linux 6.14 release. Related code has multiple spots where page virtual addresses end up used as arguments in arithmetic operations. Combined with enabled tag-based KASAN it can result in pointers that don't point where they should or logical operations not giving expected results.
vm_reset_perms()calculates range's start and end addresses usingmin()andmax()functions. To do that it compares pointers but some are not tagged - addr variable is, start and end variables aren't.
within()andwithin_range()can receive tagged addresses which get compared to untagged start and end variables.Reset tags in addresses used as function arguments in
min(),max(),within().
execmem_cache_add()adds tagged pointers to a maple tree structure, which then are incorrectly compared when walking the tree. That results in different pointers being returned later and page permission violation errors panicking the kernel.Reset tag of the address range inserted into the maple tree inside
execmem_cache_add().Signed-off-by Maciej Wieczor-Retman <[email protected]>---
Changelog v5:
- Remove thewithin_range()change.
-arch_kasan_reset_tag->kasan_reset_tag.Changelog v4:
- Add patch to the series.mm/execmem.c | 2 +- mm/vmalloc.c | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-)mm/execmem.c +0
-0
mm/execmem.c static DECLARE_WORK(execmem_cache_clean_work, execmem_cache_clean); · line 189
Quoted 5 lines
mm/execmem.c +1
-1@@ -186,7 +186,7 @@ static DECLARE_WORK(execmem_cache_clean_work, execmem_cache_clean);static int execmem_cache_add_locked (void *ptr,size_t size,gfp_t gfp_mask) {struct maple_tree *free_areas = &execmem_cache.free_areas; -unsigned long addr = (unsigned long )ptr; +unsigned long addr = (unsigned long )kasan_reset_tag (ptr);
Thinking more about it, we anyway reset tag in execmem_alloc() and return untagged pointer to the caller. Let's just move kasan_reset_tag() to execmem_vmalloc() so that we always use untagged pointers. Seems more robust to me.
mm/vmalloc.c static void vm_reset_perms(struct vm_struct *area) · line 3325
Quoted 13 lines
mm/execmem.c +0
-0MA_STATE (mas, free_areas, addr -1 , addr +1 );unsigned long lower, upper;void *area =NULL ;mm/vmalloc.c +1
-1@@ -3322,7 +3322,7 @@ static void vm_reset_perms(struct vm_struct *area)* the vm_unmap_aliases() flush includes the direct map. */ for (i =0 ; i < area->nr_pages; i +=1U << page_order) { -unsigned long addr = (unsigned long )page_address (area->pages[i]); +unsigned long addr = (unsigned long )kasan_reset_tag (page_address (area->pages[i]));
This is not strictly related to execemem, there may other users of VM_FLUSH_RESET_PERMS.
Regardless, I wonder how this works on arm64 with tags enabled?
Also, it's not the only place in the kernel that does (unsigned long)page_address(page). Do other sites need to reset the tag as well?
Quoted 6 lines
mm/vmalloc.c +0
-0if (addr) {unsigned long page_size;Signature · 1 line
2.50.1
Signature · 2 lines
Sincerely yours, Mike.
Re: [PATCH v5 07/19] mm: x86: Untag addresses in EXECMEM_ROX related pointer arithmetic
Quoted 38 lines · Mike Rapoport
Quoted 37 lines · Maciej Wieczor-Retman
ARCH_HAS_EXECMEM_ROXwas re-enabled in x86 at Linux 6.14 release. Related code has multiple spots where page virtual addresses end up used as arguments in arithmetic operations. Combined with enabled tag-based KASAN it can result in pointers that don't point where they should or logical operations not giving expected results.
vm_reset_perms()calculates range's start and end addresses usingmin()andmax()functions. To do that it compares pointers but some are not tagged - addr variable is, start and end variables aren't.
within()andwithin_range()can receive tagged addresses which get compared to untagged start and end variables.Reset tags in addresses used as function arguments in
min(),max(),within().
execmem_cache_add()adds tagged pointers to a maple tree structure, which then are incorrectly compared when walking the tree. That results in different pointers being returned later and page permission violation errors panicking the kernel.Reset tag of the address range inserted into the maple tree inside
execmem_cache_add().Signed-off-by Maciej Wieczor-Retman <[email protected]>---
Changelog v5:
- Remove thewithin_range()change.
-arch_kasan_reset_tag->kasan_reset_tag.Changelog v4:
- Add patch to the series.mm/execmem.c | 2 +- mm/vmalloc.c | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-)
Quoted 15 lines
mm/execmem.c static DECLARE_WORK(execmem_cache_clean_work, execmem_cache_clean); · line 189
Quoted 10 lines
mm/execmem.c +1
-1@@ -186,7 +186,7 @@ static DECLARE_WORK(execmem_cache_clean_work, execmem_cache_clean);static int execmem_cache_add_locked (void *ptr,size_t size,gfp_t gfp_mask) {struct maple_tree *free_areas = &execmem_cache.free_areas; -unsigned long addr = (unsigned long )ptr; +unsigned long addr = (unsigned long )kasan_reset_tag (ptr);Thinking more about it, we anyway reset tag in
execmem_alloc()and return untagged pointer to the caller. Let's just movekasan_reset_tag()toexecmem_vmalloc()so that we always use untagged pointers. Seems more robust to me.
Sure, I'll test if it works and change it :)
Quoted 18 lines
Quoted 9 lines
mm/execmem.c +0
-0MA_STATE (mas, free_areas, addr -1 , addr +1 );unsigned long lower, upper;void *area =NULL ;mm/vmalloc.c +0
-0mm/vmalloc.c static void vm_reset_perms(struct vm_struct *area) · line 3325
Quoted 5 lines
mm/vmalloc.c +1
-1@@ -3322,7 +3322,7 @@ static void vm_reset_perms(struct vm_struct *area)* the vm_unmap_aliases() flush includes the direct map. */ for (i =0 ; i < area->nr_pages; i +=1U << page_order) { -unsigned long addr = (unsigned long )page_address (area->pages[i]); +unsigned long addr = (unsigned long )kasan_reset_tag (page_address (area->pages[i]));This is not strictly related to execemem, there may other users of
VM_FLUSH_RESET_PERMS.
Quoted 1 line
Regardless, I wonder how this works on arm64 with tags enabled?
Hmm, good point, I'll check it out in qemu if this function is called on arm64.
However this issue didn't pop up for me before 6.14 when EXECMEM_ROX was enabled, so maybe it just didn't hit tagged pages before? I'll try to recheck that on x86 too.
Quoted 2 lines
Also, it's not the only place in the kernel that does (unsigned long)page_address(page). Do other sites need to reset the tag as well?
This place is special in the sense that it does "start = min(addr, start)" and "end = max(addr, end)" just a few lines later. And start and end seem to always be untagged, while addr sometimes gets tagged. So with software KASAN and vmalloc support enabled it would get the final start and end values wrong and then a page permission error would happen someplace else.
I don't think all other page_address(page) sites need resetting, but I'll double check if there is any pointer arithmetic there.
Quoted 11 lines
Quoted 7 lines
mm/vmalloc.c +0
-0if (addr) {unsigned long page_size;Signature · 1 line
2.50.1Signature · 2 lines
Sincerely yours, Mike.
Signature · 2 lines
Kind regards Maciej Wieczór-Retman
PATCH v5 8/19 x86: Physical address comparisons in fill_p*d/pte
In reply to Maciej Wieczor-RetmanCalculating page offset returns a pointer without a tag. When comparing the calculated offset to a tagged page pointer an error is raised because they are not equal.
Change pointer comparisons to physical address comparisons as to avoid issues with tagged pointers that pointer arithmetic would create. Open code pte_offset_kernel(), pmd_offset(), pud_offset() and p4d_offset(). Because one parameter is always zero and the rest of the function insides are enclosed inside __va(), removing that layer lowers the complexity of final assembly.
---
Changelog v2:
- Open code *_offset() to avoid it's internal __va().
arch/x86/mm/init_64.c | 11 +++++++---- 1 file changed, 7 insertions(+), 4 deletions(-)
arch/x86/mm/init_64.c +7 -4
@@ -251,7 +251,10 @@ static p4d_t *fill_p4d(pgd_t *pgd, unsigned long vaddr)if (pgd_none (*pgd)) {p4d_t *p4d = (p4d_t *)spp_getpage ();pgd_populate (&init_mm, pgd, p4d); -if (p4d !=p4d_offset (pgd,0 )) + +if (__pa (p4d) != (pgtable_l5_enabled () ? +__pa (pgd) : + (unsigned long )pgd_val (*pgd) &PTE_PFN_MASK ))printk (KERN_ERR "PAGETABLE BUG #00! %p <-> %p\n" , p4d,p4d_offset (pgd,0 )); }
@@ -263,7 +266,7 @@ static pud_t *fill_pud(p4d_t *p4d, unsigned long vaddr)if (p4d_none (*p4d)) {pud_t *pud = (pud_t *)spp_getpage ();p4d_populate (&init_mm, p4d, pud); -if (pud !=pud_offset (p4d,0 )) +if (__pa (pud) != (p4d_val (*p4d) &p4d_pfn_mask (*p4d)))printk (KERN_ERR "PAGETABLE BUG #01! %p <-> %p\n" , pud,pud_offset (p4d,0 )); }
@@ -275,7 +278,7 @@ static pmd_t *fill_pmd(pud_t *pud, unsigned long vaddr)if (pud_none (*pud)) {pmd_t *pmd = (pmd_t *)spp_getpage ();pud_populate (&init_mm, pud, pmd); -if (pmd !=pmd_offset (pud,0 )) +if (__pa (pmd) != (pud_val (*pud) &pud_pfn_mask (*pud)))printk (KERN_ERR "PAGETABLE BUG #02! %p <-> %p\n" , pmd,pmd_offset (pud,0 )); }
@@ -287,7 +290,7 @@ static pte_t *fill_pte(pmd_t *pmd, unsigned long vaddr)if (pmd_none (*pmd)) {pte_t *pte = (pte_t *)spp_getpage ();pmd_populate_kernel (&init_mm, pmd, pte); -if (pte !=pte_offset_kernel (pmd,0 )) +if (__pa (pte) != (pmd_val (*pmd) &pmd_pfn_mask (*pmd)))printk (KERN_ERR "PAGETABLE BUG #03!\n" ); }return pte_offset_kernel (pmd, vaddr);
Signature · 1 line
2.50.1
PATCH v5 9/19 x86: KASAN raw shadow memory PTE init
In reply to Maciej Wieczor-RetmanIn KASAN's generic mode the default value in shadow memory is zero. During initialization of shadow memory pages they are allocated and zeroed.
In KASAN's tag-based mode the default tag for the arm64 architecture is 0xFE which corresponds to any memory that should not be accessed. On x86 (where tags are 4-bit wide instead of 8-bit wide) that tag is 0xE so during the initializations all the bytes in shadow memory pages should be filled with it.
Use memblock_alloc_try_nid_raw() instead of memblock_alloc_try_nid() to avoid zeroing out the memory so it can be set with the KASAN invalid tag.
---
Changelog v2:
- Remove dense mode references, use memset() instead of kasan_poison().
arch/x86/mm/kasan_init_64.c | 19 ++++++++++++++++--- 1 file changed, 16 insertions(+), 3 deletions(-)
arch/x86/mm/kasan_init_64.c +16 -3
@@ -34,6 +34,18 @@ static __init void *early_alloc(size_t size, int nid, bool should_panic)return ptr; } +static __init void *early_raw_alloc (size_t size,int nid,bool should_panic) +{ +void *ptr =memblock_alloc_try_nid_raw (size, size, +__pa (MAX_DMA_ADDRESS ),MEMBLOCK_ALLOC_ACCESSIBLE , nid); + +if (!ptr && should_panic) +panic ("%pS: Failed to allocate page, nid=%d from=%lx\n" , + (void *)_RET_IP_, nid,__pa (MAX_DMA_ADDRESS )); + +return ptr; +} +static void __init kasan_populate_pmd (pmd_t *pmd,unsigned long addr,unsigned long end,int nid) {
@@ -63,8 +75,9 @@ static void __init kasan_populate_pmd(pmd_t *pmd, unsigned long addr,if (!pte_none (*pte))continue ; - p =early_alloc (PAGE_SIZE , nid, true); - entry =pfn_pte (PFN_DOWN (__pa (p)),PAGE_KERNEL ); + p =early_raw_alloc (PAGE_SIZE , nid, true); +memset (p,PAGE_SIZE ,KASAN_SHADOW_INIT ); + entry =pfn_pte (PFN_DOWN (__pa_nodebug (p)),PAGE_KERNEL );set_pte_at (&init_mm, addr, pte, entry); }while (pte++, addr +=PAGE_SIZE , addr != end); }
@@ -436,7 +449,7 @@ void __init kasan_init(void)* it may contain some garbage. Now we can clear and write protect it, * since after the TLB flush no one should write to it. */ -memset (kasan_early_shadow_page,0 ,PAGE_SIZE ); +memset (kasan_early_shadow_page,KASAN_SHADOW_INIT ,PAGE_SIZE );for (i =0 ; i <PTRS_PER_PTE ; i++) {pte_t pte;pgprot_t prot;
Signature · 1 line
2.50.1
PATCH v5 10/19 x86: LAM compatible non-canonical definition
In reply to Maciej Wieczor-RetmanFor an address to be canonical it has to have its top bits equal to each other. The number of bits depends on the paging level and whether they're supposed to be ones or zeroes depends on whether the address points to kernel or user space.
With Linear Address Masking (LAM) enabled, the definition of linear address canonicality is modified. Not all of the previously required bits need to be equal, only the first and last from the previously equal bitmask. So for example a 5-level paging kernel address needs to have bits [63] and [56] set.
Add separate __canonical_address() implementation for CONFIG_KASAN_SW_TAGS since it's the only thing right now that enables LAM for kernel addresses (LAM_SUP bit in CR4).
---
Changelog v4:
- Add patch to the series.
arch/x86/include/asm/page.h | 10 ++++++++++ 1 file changed, 10 insertions(+)
arch/x86/include/asm/page.h +10 -0
@@ -82,10 +82,20 @@ static __always_inline void *pfn_to_kaddr(unsigned long pfn)return __va (pfn <<PAGE_SHIFT ); } +/* +* CONFIG_KASAN_SW_TAGS requires LAM which changes the canonicality checks. +*/ +#ifdef CONFIG_KASAN_SW_TAGS +static __always_inline u64 __canonical_address(u64 vaddr, u8 vaddr_bits) +{ +return (vaddr |BIT_ULL (63 ) |BIT_ULL (vaddr_bits -1 )); +} +#else static __always_inline u64 __canonical_address(u64 vaddr, u8 vaddr_bits) {return ((s64 )vaddr << (64 - vaddr_bits)) >> (64 - vaddr_bits); } +#endif static __always_inline u64 __is_canonical_address(u64 vaddr, u8 vaddr_bits) {
Signature · 1 line
2.50.1
Re: [PATCH v5 10/19] x86: LAM compatible non-canonical definition
Hi Maciej,
Quoted 35 lines · Maciej Wieczor-Retman
For an address to be canonical it has to have its top bits equal to each other. The number of bits depends on the paging level and whether they're supposed to be ones or zeroes depends on whether the address points to kernel or user space.
With Linear Address Masking (LAM) enabled, the definition of linear address canonicality is modified. Not all of the previously required bits need to be equal, only the first and last from the previously equal bitmask. So for example a 5-level paging kernel address needs to have bits [63] and [56] set.
Add separate
__canonical_address()implementation forCONFIG_KASAN_SW_TAGSsince it's the only thing right now that enables LAM for kernel addresses (LAM_SUPbit in CR4).Signed-off-by Maciej Wieczor-Retman <[email protected]>---
Changelog v4:
- Add patch to the series.arch/x86/include/asm/page.h | 10 ++++++++++ 1 file changed, 10 insertions(+)arch/x86/include/asm/page.h +4
-0@@ -82,10 +82,20 @@ static __always_inline void *pfn_to_kaddr(unsigned long pfn)return __va (pfn <<PAGE_SHIFT ); } +/* +* CONFIG_KASAN_SW_TAGS requires LAM which changes the canonicality checks. +*/ +#ifdef CONFIG_KASAN_SW_TAGS
arch/x86/include/asm/page.h static __always_inline void *pfn_to_kaddr(unsigned long pfn) · line 98
Quoted 10 lines
arch/x86/include/asm/page.h +6
-0+static __always_inline u64 __canonical_address(u64 vaddr, u8 vaddr_bits) +{ +return (vaddr |BIT_ULL (63 ) |BIT_ULL (vaddr_bits -1 )); +} +#else static __always_inline u64 __canonical_address(u64 vaddr, u8 vaddr_bits) {return ((s64 )vaddr << (64 - vaddr_bits)) >> (64 - vaddr_bits); } +#endif
These two implementations have different semantics. The new function works only on kernel addresses, whereas the existing one works on user addresses as well. It looks like at least KVM's use of __is_canonical_address() expects the function to work with user addresses.
Regards,
Samuel
Quoted 3 lines
arch/x86/include/asm/page.h +0
-0static __always_inline u64 __is_canonical_address(u64 vaddr, u8 vaddr_bits) {
Re: [PATCH v5 10/19] x86: LAM compatible non-canonical definition
Quoted 45 lines · Samuel Holland
Hi Maciej,
Quoted 42 lines · Maciej Wieczor-Retman
For an address to be canonical it has to have its top bits equal to each other. The number of bits depends on the paging level and whether they're supposed to be ones or zeroes depends on whether the address points to kernel or user space.
With Linear Address Masking (LAM) enabled, the definition of linear address canonicality is modified. Not all of the previously required bits need to be equal, only the first and last from the previously equal bitmask. So for example a 5-level paging kernel address needs to have bits [63] and [56] set.
Add separate
__canonical_address()implementation forCONFIG_KASAN_SW_TAGSsince it's the only thing right now that enables LAM for kernel addresses (LAM_SUPbit in CR4).Signed-off-by Maciej Wieczor-Retman <[email protected]>---
Changelog v4:
- Add patch to the series.arch/x86/include/asm/page.h | 10 ++++++++++ 1 file changed, 10 insertions(+)arch/x86/include/asm/page.h +9
-0@@ -82,10 +82,20 @@ static __always_inline void *pfn_to_kaddr(unsigned long pfn)return __va (pfn <<PAGE_SHIFT ); } +/* +* CONFIG_KASAN_SW_TAGS requires LAM which changes the canonicality checks. +*/ +#ifdef CONFIG_KASAN_SW_TAGS +static __always_inline u64 __canonical_address(u64 vaddr, u8 vaddr_bits) +{ +return (vaddr |BIT_ULL (63 ) |BIT_ULL (vaddr_bits -1 )); +} +#else static __always_inline u64 __canonical_address(u64 vaddr, u8 vaddr_bits) {
Quoted 8 lines
arch/x86/include/asm/page.h static __always_inline void *pfn_to_kaddr(unsigned long pfn) · line 98
Quoted 3 lines
arch/x86/include/asm/page.h +1
-0return ((s64)vaddr << (64 - vaddr_bits)) >> (64 - vaddr_bits); } +#endif These two implementations have different semantics. The new function works only on kernel addresses, whereas the existing one works on user addresses as well. It looks like at least KVM's use of
__is_canonical_address()expects the function to work with user addresses.
Thanks for noticing that, I'll think of a way to make it work for user addresses too :)
Quoted 8 lines
Regards,
SamuelQuoted 3 lines
arch/x86/include/asm/page.h +0
-0static __always_inline u64 __is_canonical_address(u64 vaddr, u8 vaddr_bits) {
Signature · 2 lines
Kind regards Maciej Wieczór-Retman
Re: [PATCH v5 10/19] x86: LAM compatible non-canonical definition
In reply to Maciej Wieczor-RetmanQuoted 4 lines · Maciej Wieczor-Retman
arch/x86/include/asm/page.h +4
-0+/* +* CONFIG_KASAN_SW_TAGS requires LAM which changes the canonicality checks. +*/ +#ifdef CONFIG_KASAN_SW_TAGS
arch/x86/include/asm/page.h static __always_inline void *pfn_to_kaddr(unsigned long pfn) · line 98
Quoted 10 lines
arch/x86/include/asm/page.h +6
-0+static __always_inline u64 __canonical_address(u64 vaddr, u8 vaddr_bits) +{ +return (vaddr |BIT_ULL (63 ) |BIT_ULL (vaddr_bits -1 )); +} +#else static __always_inline u64 __canonical_address(u64 vaddr, u8 vaddr_bits) {return ((s64 )vaddr << (64 - vaddr_bits)) >> (64 - vaddr_bits); } +#endif
This is the kind of thing that's bound to break. Could we distill it down to something simpler, perhaps?
In the end, the canonical enforcement mask is the thing that's changing. So perhaps it should be all common code except for the mask definition:
#ifdef CONFIG_KASAN_SW_TAGS
#define CANONICAL_MASK(vaddr_bits) (BIT_ULL(63) | BIT_ULL(vaddr_bits-1)) #else
#define CANONICAL_MASK(vaddr_bits) GENMASK_UL(63, vaddr_bits) #endif
(modulo off-by-one bugs ;)
Then the canonical check itself becomes something like:
unsigned long cmask = CANONICAL_MASK(vaddr_bits); return (vaddr & mask) == mask;
That, to me, is the most straightforward way to do it.
I don't see it addressed in the cover letter, but what happens when a CONFIG_KASAN_SW_TAGS=y kernel is booted on non-LAM hardware?
Re: [PATCH v5 10/19] x86: LAM compatible non-canonical definition
Quoted 28 lines · Dave Hansen
Quoted 4 lines · Maciej Wieczor-Retman
arch/x86/include/asm/page.h +4
-0+/* +* CONFIG_KASAN_SW_TAGS requires LAM which changes the canonicality checks. +*/ +#ifdef CONFIG_KASAN_SW_TAGS arch/x86/include/asm/page.h static __always_inline void *pfn_to_kaddr(unsigned long pfn) · line 98
Quoted 10 lines
arch/x86/include/asm/page.h +6
-0+static __always_inline u64 __canonical_address(u64 vaddr, u8 vaddr_bits) +{ +return (vaddr |BIT_ULL (63 ) |BIT_ULL (vaddr_bits -1 )); +} +#else static __always_inline u64 __canonical_address(u64 vaddr, u8 vaddr_bits) {return ((s64 )vaddr << (64 - vaddr_bits)) >> (64 - vaddr_bits); } +#endif This is the kind of thing that's bound to break. Could we distill it down to something simpler, perhaps?
In the end, the canonical enforcement mask is the thing that's changing. So perhaps it should be all common code except for the mask definition:
#ifdef
CONFIG_KASAN_SW_TAGS
#defineCANONICAL_MASK(vaddr_bits)(BIT_ULL(63)|BIT_ULL(vaddr_bits-1)) #else
#defineCANONICAL_MASK(vaddr_bits)GENMASK_UL(63, vaddr_bits)#endif
Quoted 8 lines
(modulo off-by-one bugs ;)
Then the canonical check itself becomes something like:
unsigned long cmask = CANONICAL_MASK(vaddr_bits); return (vaddr & mask) == mask;That, to me, is the most straightforward way to do it.
Thanks, I'll try something like this. I will also have to investigate what Samuel brought up that KVM possibly wants to pass user addresses to this function as well.
Quoted 3 lines
I don't see it addressed in the cover letter, but what happens when a
CONFIG_KASAN_SW_TAGS=ykernel is booted on non-LAM hardware?
That's a good point, I need to add it to the cover letter. On non-LAM hardware the kernel just doesn't boot. Disabling KASAN in runtime on unsupported hardware isn't that difficult in outline mode, but I'm not sure it can work in inline mode (where checks into shadow memory are just pasted into code by the compiler).
Since for now there is no compiler support for the inline mode anyway, I'll try to disable KASAN on non-LAM hardware in runtime.
Signature · 2 lines
Kind regards Maciej Wieczór-Retman
Re: [PATCH v5 10/19] x86: LAM compatible non-canonical definition
Hi Maciej,
Quoted 43 lines · Maciej Wieczor-Retman
Quoted 28 lines · Dave Hansen
Quoted 4 lines · Maciej Wieczor-Retman
arch/x86/include/asm/page.h +4
-0+/* +* CONFIG_KASAN_SW_TAGS requires LAM which changes the canonicality checks. +*/ +#ifdef CONFIG_KASAN_SW_TAGS arch/x86/include/asm/page.h static __always_inline void *pfn_to_kaddr(unsigned long pfn) · line 98
Quoted 10 lines
arch/x86/include/asm/page.h +6
-0+static __always_inline u64 __canonical_address(u64 vaddr, u8 vaddr_bits) +{ +return (vaddr |BIT_ULL (63 ) |BIT_ULL (vaddr_bits -1 )); +} +#else static __always_inline u64 __canonical_address(u64 vaddr, u8 vaddr_bits) {return ((s64 )vaddr << (64 - vaddr_bits)) >> (64 - vaddr_bits); } +#endif This is the kind of thing that's bound to break. Could we distill it down to something simpler, perhaps?
In the end, the canonical enforcement mask is the thing that's changing. So perhaps it should be all common code except for the mask definition:
#ifdef
CONFIG_KASAN_SW_TAGS
#defineCANONICAL_MASK(vaddr_bits)(BIT_ULL(63)|BIT_ULL(vaddr_bits-1)) #else
#defineCANONICAL_MASK(vaddr_bits)GENMASK_UL(63, vaddr_bits)#endifQuoted 8 lines
(modulo off-by-one bugs ;)
Then the canonical check itself becomes something like:
unsigned long cmask = CANONICAL_MASK(vaddr_bits); return (vaddr & mask) == mask;That, to me, is the most straightforward way to do it.
Thanks, I'll try something like this. I will also have to investigate what Samuel brought up that KVM possibly wants to pass user addresses to this function as well.
Quoted 1 line
Quoted 8 lines
Quoted 2 lines
I don't see it addressed in the cover letter, but what happens when a
CONFIG_KASAN_SW_TAGS=ykernel is booted on non-LAM hardware?That's a good point, I need to add it to the cover letter. On non-LAM hardware the kernel just doesn't boot. Disabling KASAN in runtime on unsupported hardware isn't that difficult in outline mode, but I'm not sure it can work in inline mode (where checks into shadow memory are just pasted into code by the compiler).
On RISC-V at least, I was able to run inline mode with missing hardware support. The shadow memory is still allocated, so the inline tag checks do not fault. And with a patch to make kasan_enabled() return false[1], all pointers remain canonical (they match the MatchAllTag), so the inline tag checks all succeed.
[1]:
https://lore.kernel.org/linux-riscv/[email protected]/
Regards,
Samuel
Quoted 3 lines
Since for now there is no compiler support for the inline mode anyway, I'll try to disable KASAN on non-LAM hardware in runtime.
Re: [PATCH v5 10/19] x86: LAM compatible non-canonical definition
Quoted 55 lines · Samuel Holland
Hi Maciej,
Quoted 43 lines · Maciej Wieczor-Retman
Quoted 28 lines · Dave Hansen
Quoted 4 lines · Maciej Wieczor-Retman
arch/x86/include/asm/page.h +4
-0+/* +* CONFIG_KASAN_SW_TAGS requires LAM which changes the canonicality checks. +*/ +#ifdef CONFIG_KASAN_SW_TAGS arch/x86/include/asm/page.h static __always_inline void *pfn_to_kaddr(unsigned long pfn) · line 98
Quoted 10 lines
arch/x86/include/asm/page.h +6
-0+static __always_inline u64 __canonical_address(u64 vaddr, u8 vaddr_bits) +{ +return (vaddr |BIT_ULL (63 ) |BIT_ULL (vaddr_bits -1 )); +} +#else static __always_inline u64 __canonical_address(u64 vaddr, u8 vaddr_bits) {return ((s64 )vaddr << (64 - vaddr_bits)) >> (64 - vaddr_bits); } +#endif This is the kind of thing that's bound to break. Could we distill it down to something simpler, perhaps?
In the end, the canonical enforcement mask is the thing that's changing. So perhaps it should be all common code except for the mask definition:
#ifdef
CONFIG_KASAN_SW_TAGS
#defineCANONICAL_MASK(vaddr_bits)(BIT_ULL(63)|BIT_ULL(vaddr_bits-1)) #else
#defineCANONICAL_MASK(vaddr_bits)GENMASK_UL(63, vaddr_bits)#endifQuoted 8 lines
(modulo off-by-one bugs ;)
Then the canonical check itself becomes something like:
unsigned long cmask = CANONICAL_MASK(vaddr_bits); return (vaddr & mask) == mask;That, to me, is the most straightforward way to do it.
Thanks, I'll try something like this. I will also have to investigate what Samuel brought up that KVM possibly wants to pass user addresses to this function as well.
Quoted 1 line
Quoted 8 lines
Quoted 2 lines
I don't see it addressed in the cover letter, but what happens when a
CONFIG_KASAN_SW_TAGS=ykernel is booted on non-LAM hardware?That's a good point, I need to add it to the cover letter. On non-LAM hardware the kernel just doesn't boot. Disabling KASAN in runtime on unsupported hardware isn't that difficult in outline mode, but I'm not sure it can work in inline mode (where checks into shadow memory are just pasted into code by the compiler).
Quoted 7 lines
On RISC-V at least, I was able to run inline mode with missing hardware support. The shadow memory is still allocated, so the inline tag checks do not fault. And with a patch to make
kasan_enabled()return false[1], all pointers remain canonical (they match the MatchAllTag), so the inline tag checks all succeed.
Thanks, that should work :)
I'll test it and apply to the series.
Quoted 8 lines
Regards,
SamuelQuoted 3 lines
Since for now there is no compiler support for the inline mode anyway, I'll try to disable KASAN on non-LAM hardware in runtime.
Signature · 2 lines
Kind regards Maciej Wieczór-Retman
PATCH v5 11/19 x86: LAM initialization
In reply to Maciej Wieczor-RetmanTo make use of KASAN's tag based mode on x86, Linear Address Masking (LAM) needs to be enabled. To do that the 28th bit in CR4 has to be set.
Set the bit in early memory initialization.
When launching secondary CPUs the LAM bit gets lost. To avoid this add it in a mask in head_64.S. The bitmask permits some bits of CR4 to pass from the primary CPU to the secondary CPUs without being cleared.
--- arch/x86/kernel/head_64.S | 3 +++ arch/x86/mm/init.c | 3 +++ 2 files changed, 6 insertions(+)
arch/x86/kernel/head_64.S +3 -0
@@ -209,6 +209,9 @@ SYM_INNER_LABEL(common_startup_64, SYM_L_LOCAL) * there will be no global TLB entries after the execution." */ movl $(X86_CR4_PAE | X86_CR4_LA57), %edx +#ifdef CONFIG_ADDRESS_MASKING + orl $X86_CR4_LAM_SUP, %edx +#endif #ifdef CONFIG_X86_MCE /* * Preserve CR4.MCE if the kernel will enable #MC support.
arch/x86/mm/init.c +3 -0
@@ -763,6 +763,9 @@ void __init init_mem_mapping(void)probe_page_size_mask ();setup_pcid (); +if (boot_cpu_has (X86_FEATURE_LAM ) &&IS_ENABLED (CONFIG_KASAN_SW_TAGS )) +cr4_set_bits_and_update_boot (X86_CR4_LAM_SUP ); +#ifdef CONFIG_X86_64 end = max_pfn <<PAGE_SHIFT ;#else
Signature · 1 line
2.50.1
Borislav Petkov
maintainerRe: [PATCH v5 11/19] x86: LAM initialization
arch/x86/mm/init.c void __init init_mem_mapping(void) · line 766
Quoted 9 lines · Maciej Wieczor-Retman
arch/x86/mm/init.c +1
-0@@ -763,6 +763,9 @@ void __init init_mem_mapping(void)probe_page_size_mask ();setup_pcid (); +if (boot_cpu_has (X86_FEATURE_LAM ) &&IS_ENABLED (CONFIG_KASAN_SW_TAGS ))
cpu_feature_enabled()
arch/x86/mm/init.c void __init init_mem_mapping(void) · line 771
Quoted 6 lines
arch/x86/mm/init.c +2
-0+cr4_set_bits_and_update_boot (X86_CR4_LAM_SUP ); +#ifdef CONFIG_X86_64 end = max_pfn <<PAGE_SHIFT ;#else
Also, for all your patches' subjects and text:
Pls read
https://www.kernel.org/doc/html/latest/process/maintainer-tip.html#patch-subject
https://www.kernel.org/doc/html/latest/process/maintainer-tip.html#changelog
and fixup.
Thx.
Signature · 4 lines
Regards/Gruss,
Boris.
https://people.kernel.org/tglx/notes-about-netiquetteRe: [PATCH v5 11/19] x86: LAM initialization
Quoted 9 lines · Borislav Petkov
Quoted 8 lines · Maciej Wieczor-Retman
arch/x86/mm/init.c +0
-0@@ -763,6 +763,9 @@ void __init init_mem_mapping(void)probe_page_size_mask ();setup_pcid ();
Quoted 4 lines
arch/x86/mm/init.c void __init init_mem_mapping(void) · line 766
Quoted 1 line
arch/x86/mm/init.c +1
-0+if (boot_cpu_has (X86_FEATURE_LAM ) &&IS_ENABLED (CONFIG_KASAN_SW_TAGS ))
cpu_feature_enabled()
Thanks, I'll correct it.
Quoted 9 lines
arch/x86/mm/init.c void __init init_mem_mapping(void) · line 771
Quoted 6 lines
arch/x86/mm/init.c +2
-0+cr4_set_bits_and_update_boot (X86_CR4_LAM_SUP ); +#ifdef CONFIG_X86_64 end = max_pfn <<PAGE_SHIFT ;#else Also, for all your patches' subjects and text:
Quoted 6 lines
Pls read
https://www.kernel.org/doc/html/latest/process/maintainer-tip.html#patch-subject
https://www.kernel.org/doc/html/latest/process/maintainer-tip.html#changelogand fixup.
Thanks, I'll recheck all the patches with that in mind.
Quoted 8 lines
Thx.
Signature · 4 lines
Regards/Gruss, Boris. https://people.kernel.org/tglx/notes-about-netiquette
Signature · 2 lines
Kind regards Maciej Wieczór-Retman
PATCH v5 12/19 x86: Minimal SLAB alignment
In reply to Maciej Wieczor-Retman8 byte minimal SLAB alignment interferes with KASAN's granularity of 16 bytes. It causes a lot of out-of-bounds errors for unaligned 8 byte allocations.
Compared to a kernel with KASAN disabled, the memory footprint increases because all kmalloc-8 allocations now are realized as kmalloc-16, which has twice the object size. But more meaningfully, when compared to a kernel with generic KASAN enabled, there is no difference. Because of redzones in generic KASAN, kmalloc-8' and kmalloc-16' object size is the same (48 bytes). So changing the minimal SLAB alignment of the tag-based mode doesn't have any negative impact when compared to the other software KASAN mode.
Adjust x86 minimal SLAB alignment to match KASAN granularity size.
---
Changelog v4:
- Extend the patch message with some more context and impact information.
Changelog v3:
- Fix typo in patch message 4 -> 16.
- Change define location to arch/x86/include/asm/cache.c.
arch/x86/include/asm/cache.h | 4 ++++ 1 file changed, 4 insertions(+)
arch/x86/include/asm/cache.h +4 -0
@@ -21,4 +21,8 @@#endif #endif +#ifdef CONFIG_KASAN_SW_TAGS +#define ARCH_SLAB_MINALIGN (1ULL << KASAN_SHADOW_SCALE_SHIFT) +#endif +#endif /* _ASM_X86_CACHE_H */
Signature · 1 line
2.50.1
Re: [PATCH v5 12/19] x86: Minimal SLAB alignment
Quoted 43 lines · Maciej Wieczor-Retman
8 byte minimal SLAB alignment interferes with KASAN's granularity of 16 bytes. It causes a lot of out-of-bounds errors for unaligned 8 byte allocations.
Compared to a kernel with KASAN disabled, the memory footprint increases because all kmalloc-8 allocations now are realized as kmalloc-16, which has twice the object size. But more meaningfully, when compared to a kernel with generic KASAN enabled, there is no difference. Because of redzones in generic KASAN, kmalloc-8' and kmalloc-16' object size is the same (48 bytes). So changing the minimal SLAB alignment of the tag-based mode doesn't have any negative impact when compared to the other software KASAN mode.
Adjust x86 minimal SLAB alignment to match KASAN granularity size.
Signed-off-by Maciej Wieczor-Retman <[email protected]>---
Changelog v4:
- Extend the patch message with some more context and impact information.Changelog v3:
- Fix typo in patch message 4 -> 16.
- Change define location toarch/x86/include/asm/cache.c.arch/x86/include/asm/cache.h | 4 ++++ 1 file changed, 4 insertions(+)arch/x86/include/asm/cache.h +4
-1@@ -21,4 +21,8 @@#endif #endif +#ifdef CONFIG_KASAN_SW_TAGS +#define ARCH_SLAB_MINALIGN (1ULL << KASAN_SHADOW_SCALE_SHIFT) +#endif +#endif /* _ASM_X86_CACHE_H */ --
Quoted 2 lines
2.50.1
PATCH v5 13/19 kasan: x86: Handle int3 for inline KASAN reports
In reply to Maciej Wieczor-RetmanInline KASAN on x86 does tag mismatch reports by passing the faulty address and metadata through the INT3 instruction - scheme that's setup in the LLVM's compiler code (specifically HWAddressSanitizer.cpp).
Add a kasan hook to the INT3 handling function.
Disable KASAN in an INT3 core kernel selftest function since it can raise a false tag mismatch report and potentially panic the kernel.
Make part of that hook - which decides whether to die or recover from a tag mismatch - arch independent to avoid duplicating a long comment on both x86 and arm64 architectures.
---
Changelog v5:
- Add die to argument list of kasan_inline_recover() in arch/arm64/kernel/traps.c.
Changelog v4:
- Make kasan_handler() a stub in a header file. Remove #ifdef from traps.c.
- Consolidate the "recover" comment into one place.
- Make small changes to the patch message.
MAINTAINERS | 2 +- arch/x86/include/asm/kasan.h | 26 ++++++++++++++++++++++++++ arch/x86/kernel/alternative.c | 4 +++- arch/x86/kernel/traps.c | 4 ++++ arch/x86/mm/Makefile | 2 ++ arch/x86/mm/kasan_inline.c | 23 +++++++++++++++++++++++ include/linux/kasan.h | 24 ++++++++++++++++++++++++ 7 files changed, 83 insertions(+), 2 deletions(-) create mode 100644 arch/x86/mm/kasan_inline.c
MAINTAINERS +1 -1
@@ -13177,7 +13177,7 @@ S: Maintained B: https://bugzilla.kernel.org/buglist.cgi?component=Sanitizers&product=Memory%20Management F: Documentation/dev-tools/kasan.rst F: arch/*/include/asm/*kasan*.h -F: arch/*/mm/kasan_init* +F: arch/*/mm/kasan_* F: include/linux/kasan*.h F: lib/Kconfig.kasan F: mm/kasan/
arch/x86/include/asm/kasan.h +26 -0
@@ -6,7 +6,28 @@#include <linux/kasan-tags.h> #include <linux/types.h> #define KASAN_SHADOW_OFFSET _AC(CONFIG_KASAN_SHADOW_OFFSET, UL) +#ifdef CONFIG_KASAN_SW_TAGS + +/* +* LLVM ABI for reporting tag mismatches in inline KASAN mode. +* On x86 the INT3 instruction is used to carry metadata in RAX +* to the KASAN report. +* +* SIZE refers to how many bytes the faulty memory access +* requested. +* WRITE bit, when set, indicates the access was a write, otherwise +* it was a read. +* RECOVER bit, when set, should allow the kernel to carry on after +* a tag mismatch. Otherwise die() is called. +*/ +#define KASAN_RAX_RECOVER 0x20 +#define KASAN_RAX_WRITE 0x10 +#define KASAN_RAX_SIZE_MASK 0x0f +#define KASAN_RAX_SIZE (rax) (1 << ((rax) & KASAN_RAX_SIZE_MASK)) + +#else #define KASAN_SHADOW_SCALE_SHIFT 3 +#endif /* * Compiler uses shadow offset assuming that addresses start
@@ -35,10 +56,15 @@#define __tag_shifted (tag) FIELD_PREP(GENMASK_ULL(60, 57), tag)#define __tag_reset (addr) (sign_extend64((u64)(addr), 56))#define __tag_get (addr) ((u8)FIELD_GET(GENMASK_ULL(60, 57), (u64)addr)) +bool kasan_inline_handler (struct pt_regs *regs);#else #define __tag_shifted (tag) 0UL#define __tag_reset (addr) (addr)#define __tag_get (addr) 0 +static inline bool kasan_inline_handler (struct pt_regs *regs) +{ +return false; +}#endif /* CONFIG_KASAN_SW_TAGS */ static inline void *__tag_set (const void *__addr,u8 tag)
arch/x86/kernel/alternative.c +3 -1
@@ -2228,7 +2228,7 @@ int3_exception_notify(struct notifier_block *self, unsigned long val, void *data }/* Must be noinline to ensure uniqueness of int3_selftest_ip. */ -static noinline void __init int3_selftest(void) +static noinline __no_sanitize_address void __init int3_selftest(void) {static __initdata struct notifier_block int3_exception_nb = { .notifier_call = int3_exception_notify,
@@ -2236,6 +2236,7 @@ static noinline void __init int3_selftest(void) };unsigned int val =0 ; +kasan_disable_current ();BUG_ON (register_die_notifier (&int3_exception_nb));/*
@@ -2253,6 +2254,7 @@ static noinline void __init int3_selftest(void)BUG_ON (val !=1 ); +kasan_enable_current ();unregister_die_notifier (&int3_exception_nb); }
arch/x86/kernel/traps.c +4 -0
@@ -912,6 +912,10 @@ static bool do_int3(struct pt_regs *regs)if (kprobe_int3_handler (regs))return true;#endif + +if (kasan_inline_handler (regs)) +return true; + res =notify_die (DIE_INT3 ,"int3" , regs,0 ,X86_TRAP_BP ,SIGTRAP );return res ==NOTIFY_STOP ;
arch/x86/mm/Makefile +2 -0
@@ -36,7 +36,9 @@ obj-$(CONFIG_PTDUMP) += dump_pagetables.o obj-$(CONFIG_PTDUMP_DEBUGFS) += debug_pagetables.o KASAN_SANITIZE_kasan_init_$(BITS).o := n +KASAN_SANITIZE_kasan_inline.o := n obj-$(CONFIG_KASAN) += kasan_init_$(BITS).o +obj-$(CONFIG_KASAN_SW_TAGS) += kasan_inline.o KMSAN_SANITIZE_kmsan_shadow.o := n obj-$(CONFIG_KMSAN) += kmsan_shadow.o
arch/x86/mm/kasan_inline.c new file +23 -0
@@ -0,0 +1,23 @@ +// SPDX-License-Identifier: GPL-2.0 +#include <linux/kasan.h> +#include <linux/kdebug.h> + +bool kasan_inline_handler (struct pt_regs *regs) +{ +int metadata = regs->ax; +u64 addr = regs->di; +u64 pc = regs->ip; +bool recover = metadata &KASAN_RAX_RECOVER ; +bool write = metadata &KASAN_RAX_WRITE ; +size_t size =KASAN_RAX_SIZE (metadata); + +if (user_mode (regs)) +return false; + +if (!kasan_report ((void *)addr, size, write, pc)) +return false; + +kasan_inline_recover (recover,"Oops - KASAN" , regs, metadata, die); + +return true; +}
include/linux/kasan.h +24 -0
@@ -663,4 +663,28 @@ void kasan_non_canonical_hook(unsigned long addr);static inline void kasan_non_canonical_hook (unsigned long addr) { }#endif /* CONFIG_KASAN_GENERIC || CONFIG_KASAN_SW_TAGS */ +#ifdef CONFIG_KASAN_SW_TAGS +/* +* The instrumentation allows to control whether we can proceed after +* a crash was detected. This is done by passing the -recover flag to +* the compiler. Disabling recovery allows to generate more compact +* code. +* +* Unfortunately disabling recovery doesn't work for the kernel right +* now. KASAN reporting is disabled in some contexts (for example when +* the allocator accesses slab object metadata; this is controlled by +* current->kasan_depth). All these accesses are detected by the tool, +* even though the reports for them are not printed. +* +* This is something that might be fixed at some point in the future. +*/ +static inline void kasan_inline_recover ( +bool recover,char *msg,struct pt_regs *regs,unsigned long err, +void die_fn (const char *str,struct pt_regs *regs,long err)) +{ +if (!recover) +die_fn (msg, regs, err); +} +#endif +#endif /* LINUX_KASAN_H */
Signature · 1 line
2.50.1
Andrey Konovalov
reviewerRe: [PATCH v5 13/19] kasan: x86: Handle int3 for inline KASAN reports
Quoted 68 lines · Maciej Wieczor-Retman
Inline KASAN on x86 does tag mismatch reports by passing the faulty address and metadata through the INT3 instruction - scheme that's setup in the LLVM's compiler code (specifically HWAddressSanitizer.cpp).
Add a kasan hook to the INT3 handling function.
Disable KASAN in an INT3 core kernel selftest function since it can raise a false tag mismatch report and potentially panic the kernel.
Make part of that hook - which decides whether to die or recover from a tag mismatch - arch independent to avoid duplicating a long comment on both x86 and arm64 architectures.
Signed-off-by Maciej Wieczor-Retman <[email protected]>---
Changelog v5:
- Add die to argument list ofkasan_inline_recover()inarch/arm64/kernel/traps.c.Changelog v4:
- Makekasan_handler()a stub in a header file. Remove #ifdef fromtraps.c.
- Consolidate the "recover" comment into one place.
- Make small changes to the patch message.MAINTAINERS | 2 +- arch/x86/include/asm/kasan.h | 26 ++++++++++++++++++++++++++ arch/x86/kernel/alternative.c | 4 +++- arch/x86/kernel/traps.c | 4 ++++ arch/x86/mm/Makefile | 2 ++ arch/x86/mm/kasan_inline.c | 23 +++++++++++++++++++++++ include/linux/kasan.h | 24 ++++++++++++++++++++++++ 7 files changed, 83 insertions(+), 2 deletions(-) create mode 100644 arch/x86/mm/kasan_inline.cMAINTAINERS +1
-1@@ -13177,7 +13177,7 @@ S: Maintained B: https://bugzilla.kernel.org/buglist.cgi?component=Sanitizers&product=Memory%20Management F: Documentation/dev-tools/kasan.rst F: arch/*/include/asm/*kasan*.h -F: arch/*/mm/kasan_init* +F: arch/*/mm/kasan_* F: include/linux/kasan*.h F: lib/Kconfig.kasan F: mm/kasan/arch/x86/include/asm/kasan.h +11
-0@@ -6,7 +6,28 @@#include <linux/kasan-tags.h> #include <linux/types.h> #define KASAN_SHADOW_OFFSET _AC(CONFIG_KASAN_SHADOW_OFFSET, UL) +#ifdef CONFIG_KASAN_SW_TAGS + +/* +* LLVM ABI for reporting tag mismatches in inline KASAN mode. +* On x86 the INT3 instruction is used to carry metadata in RAX +* to the KASAN report. +* +* SIZE refers to how many bytes the faulty memory access +* requested. +* WRITE bit, when set, indicates the access was a write, otherwise +* it was a read.
arch/x86/include/asm/kasan.h line 29
Quoted 10 lines
arch/x86/include/asm/kasan.h +9
-0+* RECOVER bit, when set, should allow the kernel to carry on after +* a tag mismatch. Otherwise die() is called. +*/ +#define KASAN_RAX_RECOVER 0x20 +#define KASAN_RAX_WRITE 0x10 +#define KASAN_RAX_SIZE_MASK 0x0f +#define KASAN_RAX_SIZE (rax) (1 << ((rax) & KASAN_RAX_SIZE_MASK)) + +#else #define KASAN_SHADOW_SCALE_SHIFT 3
Putting this under else in this patch looks odd, we can move this part to "x86: Make software tag-based kasan available".
Quoted 92 lines
arch/x86/include/asm/kasan.h +6
-0+#endif /* * Compiler uses shadow offset assuming that addresses start @@ -35,10 +56,15 @@#define __tag_shifted (tag) FIELD_PREP(GENMASK_ULL(60, 57), tag)#define __tag_reset (addr) (sign_extend64((u64)(addr), 56))#define __tag_get (addr) ((u8)FIELD_GET(GENMASK_ULL(60, 57), (u64)addr)) +bool kasan_inline_handler (struct pt_regs *regs);#else #define __tag_shifted (tag) 0UL#define __tag_reset (addr) (addr)#define __tag_get (addr) 0 +static inline bool kasan_inline_handler (struct pt_regs *regs) +{ +return false; +}#endif /* CONFIG_KASAN_SW_TAGS */ static inline void *__tag_set (const void *__addr,u8 tag)arch/x86/kernel/alternative.c +3
-1@@ -2228,7 +2228,7 @@ int3_exception_notify(struct notifier_block *self, unsigned long val, void *data }/* Must be noinline to ensure uniqueness of int3_selftest_ip. */ -static noinline void __init int3_selftest(void) +static noinline __no_sanitize_address void __init int3_selftest(void) {static __initdata struct notifier_block int3_exception_nb = { .notifier_call = int3_exception_notify,@@ -2236,6 +2236,7 @@ static noinline void __init int3_selftest(void) };unsigned int val =0 ; +kasan_disable_current ();BUG_ON (register_die_notifier (&int3_exception_nb));/* @@ -2253,6 +2254,7 @@ static noinline void __init int3_selftest(void)BUG_ON (val !=1 ); +kasan_enable_current ();unregister_die_notifier (&int3_exception_nb); }arch/x86/kernel/traps.c +4
-0@@ -912,6 +912,10 @@ static bool do_int3(struct pt_regs *regs)if (kprobe_int3_handler (regs))return true;#endif + +if (kasan_inline_handler (regs)) +return true; + res =notify_die (DIE_INT3 ,"int3" , regs,0 ,X86_TRAP_BP ,SIGTRAP );return res ==NOTIFY_STOP ;arch/x86/mm/Makefile +2
-0@@ -36,7 +36,9 @@ obj-$(CONFIG_PTDUMP) += dump_pagetables.o obj-$(CONFIG_PTDUMP_DEBUGFS) += debug_pagetables.o KASAN_SANITIZE_kasan_init_$(BITS).o := n +KASAN_SANITIZE_kasan_inline.o := n obj-$(CONFIG_KASAN) += kasan_init_$(BITS).o +obj-$(CONFIG_KASAN_SW_TAGS) += kasan_inline.o KMSAN_SANITIZE_kmsan_shadow.o := n obj-$(CONFIG_KMSAN) += kmsan_shadow.oarch/x86/mm/kasan_inline.c new file +8
-0@@ -0,0 +1,23 @@ +// SPDX-License-Identifier: GPL-2.0 +#include <linux/kasan.h> +#include <linux/kdebug.h> + +bool kasan_inline_handler (struct pt_regs *regs) +{ +int metadata = regs->ax; +u64 addr = regs->di;
arch/x86/mm/kasan_inline.c line 18
Quoted 10 lines
arch/x86/mm/kasan_inline.c new file +10
-0+u64 pc = regs->ip; +bool recover = metadata &KASAN_RAX_RECOVER ; +bool write = metadata &KASAN_RAX_WRITE ; +size_t size =KASAN_RAX_SIZE (metadata); + +if (user_mode (regs)) +return false; + +if (!kasan_report ((void *)addr, size, write, pc)) +return false;
Hm, this part is different than on arm64: there, we don't check the return value.
Do I understand correctly that the return value from this function controls whether we skip over the int3 instruction and continue the execution? If so, we should return the same value regardless of whether the report is suppressed or not. And then you should not need to explicitly check for KASAN_BIT_MULTI_SHOT in the latter patch.
arch/x86/mm/kasan_inline.c line 20
Quoted 2 lines
arch/x86/mm/kasan_inline.c new file +2
-0+ +kasan_inline_recover (recover,"Oops - KASAN" , regs, metadata, die);
Maybe name this is as kasan_die_unless_recover()?
Quoted 39 lines
arch/x86/mm/kasan_inline.c new file +3
-0+ +return true; +}include/linux/kasan.h +24
-1@@ -663,4 +663,28 @@ void kasan_non_canonical_hook(unsigned long addr);static inline void kasan_non_canonical_hook (unsigned long addr) { }#endif /* CONFIG_KASAN_GENERIC || CONFIG_KASAN_SW_TAGS */ +#ifdef CONFIG_KASAN_SW_TAGS +/* +* The instrumentation allows to control whether we can proceed after +* a crash was detected. This is done by passing the -recover flag to +* the compiler. Disabling recovery allows to generate more compact +* code. +* +* Unfortunately disabling recovery doesn't work for the kernel right +* now. KASAN reporting is disabled in some contexts (for example when +* the allocator accesses slab object metadata; this is controlled by +* current->kasan_depth). All these accesses are detected by the tool, +* even though the reports for them are not printed. +* +* This is something that might be fixed at some point in the future. +*/ +static inline void kasan_inline_recover ( +bool recover,char *msg,struct pt_regs *regs,unsigned long err, +void die_fn (const char *str,struct pt_regs *regs,long err)) +{ +if (!recover) +die_fn (msg, regs, err); +} +#endif +#endif /* LINUX_KASAN_H */ --2.50.1
Re: [PATCH v5 13/19] kasan: x86: Handle int3 for inline KASAN reports
Quoted 75 lines · Andrey Konovalov
Quoted 73 lines · Maciej Wieczor-Retman
Inline KASAN on x86 does tag mismatch reports by passing the faulty address and metadata through the INT3 instruction - scheme that's setup in the LLVM's compiler code (specifically HWAddressSanitizer.cpp).
Add a kasan hook to the INT3 handling function.
Disable KASAN in an INT3 core kernel selftest function since it can raise a false tag mismatch report and potentially panic the kernel.
Make part of that hook - which decides whether to die or recover from a tag mismatch - arch independent to avoid duplicating a long comment on both x86 and arm64 architectures.
Signed-off-by Maciej Wieczor-Retman <[email protected]>---
Changelog v5:
- Add die to argument list ofkasan_inline_recover()inarch/arm64/kernel/traps.c.Changelog v4:
- Makekasan_handler()a stub in a header file. Remove #ifdef fromtraps.c.
- Consolidate the "recover" comment into one place.
- Make small changes to the patch message.MAINTAINERS | 2 +- arch/x86/include/asm/kasan.h | 26 ++++++++++++++++++++++++++ arch/x86/kernel/alternative.c | 4 +++- arch/x86/kernel/traps.c | 4 ++++ arch/x86/mm/Makefile | 2 ++ arch/x86/mm/kasan_inline.c | 23 +++++++++++++++++++++++ include/linux/kasan.h | 24 ++++++++++++++++++++++++ 7 files changed, 83 insertions(+), 2 deletions(-) create mode 100644 arch/x86/mm/kasan_inline.cMAINTAINERS +1
-1@@ -13177,7 +13177,7 @@ S: Maintained B: https://bugzilla.kernel.org/buglist.cgi?component=Sanitizers&product=Memory%20Management F: Documentation/dev-tools/kasan.rst F: arch/*/include/asm/*kasan*.h -F: arch/*/mm/kasan_init* +F: arch/*/mm/kasan_* F: include/linux/kasan*.h F: lib/Kconfig.kasan F: mm/kasan/arch/x86/include/asm/kasan.h +16
-0@@ -6,7 +6,28 @@#include <linux/kasan-tags.h> #include <linux/types.h> #define KASAN_SHADOW_OFFSET _AC(CONFIG_KASAN_SHADOW_OFFSET, UL) +#ifdef CONFIG_KASAN_SW_TAGS + +/* +* LLVM ABI for reporting tag mismatches in inline KASAN mode. +* On x86 the INT3 instruction is used to carry metadata in RAX +* to the KASAN report. +* +* SIZE refers to how many bytes the faulty memory access +* requested. +* WRITE bit, when set, indicates the access was a write, otherwise +* it was a read. +* RECOVER bit, when set, should allow the kernel to carry on after +* a tag mismatch. Otherwise die() is called. +*/ +#define KASAN_RAX_RECOVER 0x20 +#define KASAN_RAX_WRITE 0x10
Quoted 8 lines
arch/x86/include/asm/kasan.h line 29
Quoted 5 lines
arch/x86/include/asm/kasan.h +4
-0+#define KASAN_RAX_SIZE_MASK 0x0f +#define KASAN_RAX_SIZE (rax) (1 << ((rax) & KASAN_RAX_SIZE_MASK)) + +#else #define KASAN_SHADOW_SCALE_SHIFT 3Putting this under else in this patch looks odd, we can move this part to "x86: Make software tag-based kasan available".
Sure, will do!
Quoted 107 lines
Quoted 93 lines
arch/x86/include/asm/kasan.h +6
-0+#endif /* * Compiler uses shadow offset assuming that addresses start @@ -35,10 +56,15 @@#define __tag_shifted (tag) FIELD_PREP(GENMASK_ULL(60, 57), tag)#define __tag_reset (addr) (sign_extend64((u64)(addr), 56))#define __tag_get (addr) ((u8)FIELD_GET(GENMASK_ULL(60, 57), (u64)addr)) +bool kasan_inline_handler (struct pt_regs *regs);#else #define __tag_shifted (tag) 0UL#define __tag_reset (addr) (addr)#define __tag_get (addr) 0 +static inline bool kasan_inline_handler (struct pt_regs *regs) +{ +return false; +}#endif /* CONFIG_KASAN_SW_TAGS */ static inline void *__tag_set (const void *__addr,u8 tag)arch/x86/kernel/alternative.c +3
-1@@ -2228,7 +2228,7 @@ int3_exception_notify(struct notifier_block *self, unsigned long val, void *data }/* Must be noinline to ensure uniqueness of int3_selftest_ip. */ -static noinline void __init int3_selftest(void) +static noinline __no_sanitize_address void __init int3_selftest(void) {static __initdata struct notifier_block int3_exception_nb = { .notifier_call = int3_exception_notify,@@ -2236,6 +2236,7 @@ static noinline void __init int3_selftest(void) };unsigned int val =0 ; +kasan_disable_current ();BUG_ON (register_die_notifier (&int3_exception_nb));/* @@ -2253,6 +2254,7 @@ static noinline void __init int3_selftest(void)BUG_ON (val !=1 ); +kasan_enable_current ();unregister_die_notifier (&int3_exception_nb); }arch/x86/kernel/traps.c +4
-0@@ -912,6 +912,10 @@ static bool do_int3(struct pt_regs *regs)if (kprobe_int3_handler (regs))return true;#endif + +if (kasan_inline_handler (regs)) +return true; + res =notify_die (DIE_INT3 ,"int3" , regs,0 ,X86_TRAP_BP ,SIGTRAP );return res ==NOTIFY_STOP ;arch/x86/mm/Makefile +2
-0@@ -36,7 +36,9 @@ obj-$(CONFIG_PTDUMP) += dump_pagetables.o obj-$(CONFIG_PTDUMP_DEBUGFS) += debug_pagetables.o KASAN_SANITIZE_kasan_init_$(BITS).o := n +KASAN_SANITIZE_kasan_inline.o := n obj-$(CONFIG_KASAN) += kasan_init_$(BITS).o +obj-$(CONFIG_KASAN_SW_TAGS) += kasan_inline.o KMSAN_SANITIZE_kmsan_shadow.o := n obj-$(CONFIG_KMSAN) += kmsan_shadow.oarch/x86/mm/kasan_inline.c new file +8
-0@@ -0,0 +1,23 @@ +// SPDX-License-Identifier: GPL-2.0 +#include <linux/kasan.h> +#include <linux/kdebug.h> + +bool kasan_inline_handler (struct pt_regs *regs) +{ +int metadata = regs->ax; +u64 addr = regs->di;arch/x86/mm/kasan_inline.c line 18
Quoted 10 lines
arch/x86/mm/kasan_inline.c new file +10
-0+u64 pc = regs->ip; +bool recover = metadata &KASAN_RAX_RECOVER ; +bool write = metadata &KASAN_RAX_WRITE ; +size_t size =KASAN_RAX_SIZE (metadata); + +if (user_mode (regs)) +return false; + +if (!kasan_report ((void *)addr, size, write, pc)) +return false;Hm, this part is different than on arm64: there, we don't check the return value.
Quoted 5 lines
Do I understand correctly that the return value from this function controls whether we skip over the int3 instruction and continue the execution? If so, we should return the same value regardless of whether the report is suppressed or not. And then you should not need to explicitly check for
KASAN_BIT_MULTI_SHOTin the latter patch.
I recall there were some corner cases where this code path got called in outline mode, didn't have a mismatch but still died due to the die() below. But I'll recheck and either apply what you wrote above or get add a better explanation to the patch message.
Quoted 5 lines
arch/x86/mm/kasan_inline.c line 20
Quoted 3 lines
arch/x86/mm/kasan_inline.c new file +2
-0+ +kasan_inline_recover (recover,"Oops - KASAN" , regs, metadata, die);Maybe name this is as
kasan_die_unless_recover()?
Sure, sounds good
Quoted 41 lines
Quoted 41 lines
arch/x86/mm/kasan_inline.c new file +3
-0+ +return true; +}include/linux/kasan.h +24
-1@@ -663,4 +663,28 @@ void kasan_non_canonical_hook(unsigned long addr);static inline void kasan_non_canonical_hook (unsigned long addr) { }#endif /* CONFIG_KASAN_GENERIC || CONFIG_KASAN_SW_TAGS */ +#ifdef CONFIG_KASAN_SW_TAGS +/* +* The instrumentation allows to control whether we can proceed after +* a crash was detected. This is done by passing the -recover flag to +* the compiler. Disabling recovery allows to generate more compact +* code. +* +* Unfortunately disabling recovery doesn't work for the kernel right +* now. KASAN reporting is disabled in some contexts (for example when +* the allocator accesses slab object metadata; this is controlled by +* current->kasan_depth). All these accesses are detected by the tool, +* even though the reports for them are not printed. +* +* This is something that might be fixed at some point in the future. +*/ +static inline void kasan_inline_recover ( +bool recover,char *msg,struct pt_regs *regs,unsigned long err, +void die_fn (const char *str,struct pt_regs *regs,long err)) +{ +if (!recover) +die_fn (msg, regs, err); +} +#endif +#endif /* LINUX_KASAN_H */ --2.50.1
Signature · 2 lines
Kind regards Maciej Wieczór-Retman
Re: [PATCH v5 13/19] kasan: x86: Handle int3 for inline KASAN reports
Quoted 36 lines · Maciej Wieczor-Retman
Quoted 29 lines · Andrey Konovalov
On Mon, Aug 25, 2025 at 10:30 PM Maciej Wieczor-Retman
Quoted 14 lines
arch/x86/mm/kasan_inline.c new file +8
-0@@ -0,0 +1,23 @@ +// SPDX-License-Identifier: GPL-2.0 +#include <linux/kasan.h> +#include <linux/kdebug.h> + +bool kasan_inline_handler (struct pt_regs *regs) +{ +int metadata = regs->ax; +u64 addr = regs->di;arch/x86/mm/kasan_inline.c line 18
Quoted 10 lines
arch/x86/mm/kasan_inline.c new file +10
-0+u64 pc = regs->ip; +bool recover = metadata &KASAN_RAX_RECOVER ; +bool write = metadata &KASAN_RAX_WRITE ; +size_t size =KASAN_RAX_SIZE (metadata); + +if (user_mode (regs)) +return false; + +if (!kasan_report ((void *)addr, size, write, pc)) +return false;Hm, this part is different than on arm64: there, we don't check the return value.
Quoted 5 lines
Do I understand correctly that the return value from this function controls whether we skip over the int3 instruction and continue the execution? If so, we should return the same value regardless of whether the report is suppressed or not. And then you should not need to explicitly check for
KASAN_BIT_MULTI_SHOTin the latter patch.
Quoted 4 lines
I recall there were some corner cases where this code path got called in outline mode, didn't have a mismatch but still died due to the
die()below. But I'll recheck and either apply what you wrote above or get add a better explanation to the patch message.
Okay, so the int3_selftest_ip() is causing a problem in outline mode.
I tried disabling kasan with kasan_disable_current() but thinking of it now it won't work because int3 handler will still be called and die() will happen.
What did you mean by "return the same value regardless of kasan_report()"? Then it will never reach the kasan_inline_recover() which I assume is needed for inline mode (once recover will work).
Signature · 2 lines
Kind regards Maciej Wieczór-Retman
Re: [PATCH v5 13/19] kasan: x86: Handle int3 for inline KASAN reports
Quoted 42 lines · Maciej Wieczor-Retman
Quoted 36 lines · Maciej Wieczor-Retman
Quoted 29 lines · Andrey Konovalov
On Mon, Aug 25, 2025 at 10:30 PM Maciej Wieczor-Retman
Quoted 14 lines
arch/x86/mm/kasan_inline.c new file +8
-0@@ -0,0 +1,23 @@ +// SPDX-License-Identifier: GPL-2.0 +#include <linux/kasan.h> +#include <linux/kdebug.h> + +bool kasan_inline_handler (struct pt_regs *regs) +{ +int metadata = regs->ax; +u64 addr = regs->di;arch/x86/mm/kasan_inline.c line 18
Quoted 10 lines
arch/x86/mm/kasan_inline.c new file +10
-0+u64 pc = regs->ip; +bool recover = metadata &KASAN_RAX_RECOVER ; +bool write = metadata &KASAN_RAX_WRITE ; +size_t size =KASAN_RAX_SIZE (metadata); + +if (user_mode (regs)) +return false; + +if (!kasan_report ((void *)addr, size, write, pc)) +return false;Hm, this part is different than on arm64: there, we don't check the return value.
Quoted 5 lines
Do I understand correctly that the return value from this function controls whether we skip over the int3 instruction and continue the execution? If so, we should return the same value regardless of whether the report is suppressed or not. And then you should not need to explicitly check for
KASAN_BIT_MULTI_SHOTin the latter patch.Quoted 4 lines
I recall there were some corner cases where this code path got called in outline mode, didn't have a mismatch but still died due to the
die()below. But I'll recheck and either apply what you wrote above or get add a better explanation to the patch message.
Quoted 4 lines
Okay, so the
int3_selftest_ip()is causing a problem in outline mode.I tried disabling kasan with
kasan_disable_current()but thinking of it now it won't work because int3 handler will still be called anddie()will happen.
Sorry, I meant to write that kasan_disable_current() works together with if(!kasan_report()). Because without checking kasan_report()' return value, if kasan is disabled through kasan_disable_current() it will have no effect in both inline mode, and if int3 is called in outline mode - the kasan_inline_handler will lead to die().
Quoted 8 lines
What did you mean by "return the same value regardless of
kasan_report()"? Then it will never reach thekasan_inline_recover()which I assume is needed for inline mode (once recover will work).Signature · 2 lines
Kind regards Maciej Wieczór-Retman
Signature · 2 lines
Kind regards Maciej Wieczór-Retman
Andrey Konovalov
reviewerRe: [PATCH v5 13/19] kasan: x86: Handle int3 for inline KASAN reports
Quoted 8 lines · Maciej Wieczor-Retman
Quoted 8 lines
Quoted 5 lines
I recall there were some corner cases where this code path got called in outline mode, didn't have a mismatch but still died due to the
die()below. But I'll recheck and either apply what you wrote above or get add a better explanation to the patch message.Okay, so the
int3_selftest_ip()is causing a problem in outline mode.
Quoted 8 lines
Quoted 2 lines
I tried disabling kasan with
kasan_disable_current()but thinking of it now it won't work because int3 handler will still be called anddie()will happen.Sorry, I meant to write that
kasan_disable_current()works together with if(!kasan_report()). Because without checkingkasan_report()' return value, if kasan is disabled throughkasan_disable_current()it will have no effect in both inline mode, and if int3 is called in outline mode - thekasan_inline_handlerwill lead todie().
So do I understand correctly, that we have no way to distinguish whether the int3 was inserted by the KASAN instrumentation or natively called (like in int3_selftest_ip())?
If so, I think that we need to fix/change the compiler first so that we can distinguish these cases. And only then introduce kasan_inline_handler(). (Without kasan_inline_handler(), the outline instrumentation would then just work, right?)
If we can distinguish them, then we should only call kasan_inline_handler() for the KASAN-inserted int3's. This is what we do on arm64 (via brk and KASAN_BRK_IMM). And then int3_selftest_ip() should not be affected.
Quoted 4 lines
Quoted 4 lines
What did you mean by "return the same value regardless of
kasan_report()"? Then it will never reach thekasan_inline_recover()which I assume is needed for inline mode (once recover will work).
I meant that with the recovery always enabled, it should not matter whether the report is suppressed (kasan_report() returns false) or printed (returns true). We should always skip over the int3 instruction and continue the execution.
Re: [PATCH v5 13/19] kasan: x86: Handle int3 for inline KASAN reports
Quoted 28 lines · Andrey Konovalov
Quoted 8 lines · Maciej Wieczor-Retman
Quoted 8 lines
Quoted 5 lines
I recall there were some corner cases where this code path got called in outline mode, didn't have a mismatch but still died due to the
die()below. But I'll recheck and either apply what you wrote above or get add a better explanation to the patch message.Okay, so the
int3_selftest_ip()is causing a problem in outline mode.Quoted 8 lines
Quoted 2 lines
I tried disabling kasan with
kasan_disable_current()but thinking of it now it won't work because int3 handler will still be called anddie()will happen.Sorry, I meant to write that
kasan_disable_current()works together with if(!kasan_report()). Because without checkingkasan_report()' return value, if kasan is disabled throughkasan_disable_current()it will have no effect in both inline mode, and if int3 is called in outline mode - thekasan_inline_handlerwill lead todie().So do I understand correctly, that we have no way to distinguish whether the int3 was inserted by the KASAN instrumentation or natively called (like in
int3_selftest_ip())?If so, I think that we need to fix/change the compiler first so that we can distinguish these cases. And only then introduce
kasan_inline_handler(). (Withoutkasan_inline_handler(), the outline instrumentation would then just work, right?)
Quoted 4 lines
If we can distinguish them, then we should only call
kasan_inline_handler()for the KASAN-inserted int3's. This is what we do on arm64 (via brk andKASAN_BRK_IMM). And thenint3_selftest_ip()should not be affected.
Looking at it again I suppose LLVM does pass a number along metadata to the int3. I didn't notice because no other function checks anything in the x86 int3 handler, compared to how it's done on arm64 with brk.
So right, thanks, after fixing it up it shouldn't affect the int3_selftest_ip().
Quoted 10 lines
Quoted 5 lines
Quoted 5 lines
What did you mean by "return the same value regardless of
kasan_report()"? Then it will never reach thekasan_inline_recover()which I assume is needed for inline mode (once recover will work).I meant that with the recovery always enabled, it should not matter whether the report is suppressed (
kasan_report()returns false) or printed (returns true). We should always skip over the int3 instruction and continue the execution.
Signature · 2 lines
Kind regards Maciej Wieczór-Retman
Peter Zijlstra
maintainerRe: [PATCH v5 13/19] kasan: x86: Handle int3 for inline KASAN reports
Quoted 34 lines · Maciej Wieczor-Retman
Quoted 28 lines · Andrey Konovalov
Quoted 8 lines · Maciej Wieczor-Retman
Quoted 8 lines
Quoted 5 lines
I recall there were some corner cases where this code path got called in outline mode, didn't have a mismatch but still died due to the
die()below. But I'll recheck and either apply what you wrote above or get add a better explanation to the patch message.Okay, so the
int3_selftest_ip()is causing a problem in outline mode.Quoted 8 lines
Quoted 2 lines
I tried disabling kasan with
kasan_disable_current()but thinking of it now it won't work because int3 handler will still be called anddie()will happen.Sorry, I meant to write that
kasan_disable_current()works together with if(!kasan_report()). Because without checkingkasan_report()' return value, if kasan is disabled throughkasan_disable_current()it will have no effect in both inline mode, and if int3 is called in outline mode - thekasan_inline_handlerwill lead todie().So do I understand correctly, that we have no way to distinguish whether the int3 was inserted by the KASAN instrumentation or natively called (like in
int3_selftest_ip())?If so, I think that we need to fix/change the compiler first so that we can distinguish these cases. And only then introduce
kasan_inline_handler(). (Withoutkasan_inline_handler(), the outline instrumentation would then just work, right?)Quoted 4 lines
If we can distinguish them, then we should only call
kasan_inline_handler()for the KASAN-inserted int3's. This is what we do on arm64 (via brk andKASAN_BRK_IMM). And thenint3_selftest_ip()should not be affected.
Quoted 5 lines
Looking at it again I suppose LLVM does pass a number along metadata to the int3. I didn't notice because no other function checks anything in the x86 int3 handler, compared to how it's done on arm64 with brk.
So right, thanks, after fixing it up it shouldn't affect the
int3_selftest_ip().
Seriously guys, stop using int3 for this. UBSAN uses UD1, why the heck would KASAN not do the same?
Peter Zijlstra
maintainerRe: [PATCH v5 13/19] kasan: x86: Handle int3 for inline KASAN reports
Quoted 35 lines · Peter Zijlstra
Quoted 34 lines · Maciej Wieczor-Retman
Quoted 32 lines · Andrey Konovalov
Quoted 8 lines · Maciej Wieczor-Retman
Quoted 8 lines
Quoted 5 lines
I recall there were some corner cases where this code path got called in outline mode, didn't have a mismatch but still died due to the
die()below. But I'll recheck and either apply what you wrote above or get add a better explanation to the patch message.Okay, so the
int3_selftest_ip()is causing a problem in outline mode.Quoted 8 lines
Quoted 2 lines
I tried disabling kasan with
kasan_disable_current()but thinking of it now it won't work because int3 handler will still be called anddie()will happen.Sorry, I meant to write that
kasan_disable_current()works together with if(!kasan_report()). Because without checkingkasan_report()' return value, if kasan is disabled throughkasan_disable_current()it will have no effect in both inline mode, and if int3 is called in outline mode - thekasan_inline_handlerwill lead todie().So do I understand correctly, that we have no way to distinguish whether the int3 was inserted by the KASAN instrumentation or natively called (like in
int3_selftest_ip())?If so, I think that we need to fix/change the compiler first so that we can distinguish these cases. And only then introduce
kasan_inline_handler(). (Withoutkasan_inline_handler(), the outline instrumentation would then just work, right?)If we can distinguish them, then we should only call
kasan_inline_handler()for the KASAN-inserted int3's. This is what we do on arm64 (via brk andKASAN_BRK_IMM). And thenint3_selftest_ip()should not be affected.
Quoted 8 lines
Quoted 5 lines
Looking at it again I suppose LLVM does pass a number along metadata to the int3. I didn't notice because no other function checks anything in the x86 int3 handler, compared to how it's done on arm64 with brk.
So right, thanks, after fixing it up it shouldn't affect the
int3_selftest_ip().Seriously guys, stop using int3 for this. UBSAN uses UD1, why the heck would KASAN not do the same?
Specifically, look at arch/x86/kernel/traps.h:decode_bug(), UBSan uses UD1 /0, I would suggest KASAN to use UD1 /1.
Re: [PATCH v5 13/19] kasan: x86: Handle int3 for inline KASAN reports
Quoted 40 lines · Peter Zijlstra
Quoted 39 lines · Peter Zijlstra
Quoted 38 lines · Maciej Wieczor-Retman
Quoted 28 lines · Andrey Konovalov
Quoted 8 lines · Maciej Wieczor-Retman
Quoted 8 lines
Quoted 5 lines
I recall there were some corner cases where this code path got called in outline mode, didn't have a mismatch but still died due to the
die()below. But I'll recheck and either apply what you wrote above or get add a better explanation to the patch message.Okay, so the
int3_selftest_ip()is causing a problem in outline mode.Quoted 8 lines
Quoted 2 lines
I tried disabling kasan with
kasan_disable_current()but thinking of it now it won't work because int3 handler will still be called anddie()will happen.Sorry, I meant to write that
kasan_disable_current()works together with if(!kasan_report()). Because without checkingkasan_report()' return value, if kasan is disabled throughkasan_disable_current()it will have no effect in both inline mode, and if int3 is called in outline mode - thekasan_inline_handlerwill lead todie().So do I understand correctly, that we have no way to distinguish whether the int3 was inserted by the KASAN instrumentation or natively called (like in
int3_selftest_ip())?If so, I think that we need to fix/change the compiler first so that we can distinguish these cases. And only then introduce
kasan_inline_handler(). (Withoutkasan_inline_handler(), the outline instrumentation would then just work, right?)Quoted 4 lines
If we can distinguish them, then we should only call
kasan_inline_handler()for the KASAN-inserted int3's. This is what we do on arm64 (via brk andKASAN_BRK_IMM). And thenint3_selftest_ip()should not be affected.Looking at it again I suppose LLVM does pass a number along metadata to the int3. I didn't notice because no other function checks anything in the x86 int3 handler, compared to how it's done on arm64 with brk.
Quoted 7 lines
Quoted 4 lines
Quoted 1 line
So right, thanks, after fixing it up it shouldn't affect the
int3_selftest_ip().Seriously guys, stop using int3 for this. UBSAN uses UD1, why the heck would KASAN not do the same?
Specifically, look at arch/x86/kernel/traps.h:decode_bug(), UBSan uses UD1 /0, I would suggest KASAN to use UD1 /1.
Okay, that sounds great, I'll change it in this patchset and write the LLVM patch later.
Signature · 2 lines
Kind regards Maciej Wieczór-Retman
Peter Zijlstra
maintainerRe: [PATCH v5 13/19] kasan: x86: Handle int3 for inline KASAN reports
Quoted 5 lines · Maciej Wieczor-Retman
Quoted 2 lines
Specifically, look at arch/x86/kernel/traps.h:decode_bug(), UBSan uses UD1 /0, I would suggest KASAN to use UD1 /1.
Okay, that sounds great, I'll change it in this patchset and write the LLVM patch later.
Thanks! Also note how UBSAN encodes an immediate in the UD1 instruction. You can use that same to pass through your meta-data thing.
MOD=1 gives you a single byte immediate, and MOD=2 gives you 4 bytes, eg:
0f b9 49 xx -- ud1 xx(%rcx), %rcx
When poking at LLVM, try and convince the thing to not emit that 'operand address size prefix' byte like UBSAN does, that's just a waste of bytes.
Re: [PATCH v5 13/19] kasan: x86: Handle int3 for inline KASAN reports
Quoted 11 lines · Peter Zijlstra
Quoted 5 lines · Maciej Wieczor-Retman
Quoted 2 lines
Specifically, look at arch/x86/kernel/traps.h:decode_bug(), UBSan uses UD1 /0, I would suggest KASAN to use UD1 /1.
Okay, that sounds great, I'll change it in this patchset and write the LLVM patch later.
Thanks! Also note how UBSAN encodes an immediate in the UD1 instruction. You can use that same to pass through your meta-data thing.
Quoted 8 lines
MOD=1 gives you a single byte immediate, and MOD=2 gives you 4 bytes, eg:
0f b9 49 xx -- ud1 xx(%rcx), %rcx
When poking at LLVM, try and convince the thing to not emit that 'operand address size prefix' byte like UBSAN does, that's just a waste of bytes.
Thanks, that's good tip :)
Signature · 2 lines
Kind regards Maciej Wieczór-Retman
Re: [PATCH v5 13/19] kasan: x86: Handle int3 for inline KASAN reports
In reply to Maciej Wieczor-RetmanQuoted 34 lines · Maciej Wieczor-Retman
Quoted 28 lines · Andrey Konovalov
Quoted 8 lines · Maciej Wieczor-Retman
Quoted 8 lines
Quoted 5 lines
I recall there were some corner cases where this code path got called in outline mode, didn't have a mismatch but still died due to the
die()below. But I'll recheck and either apply what you wrote above or get add a better explanation to the patch message.Okay, so the
int3_selftest_ip()is causing a problem in outline mode.Quoted 8 lines
Quoted 2 lines
I tried disabling kasan with
kasan_disable_current()but thinking of it now it won't work because int3 handler will still be called anddie()will happen.Sorry, I meant to write that
kasan_disable_current()works together with if(!kasan_report()). Because without checkingkasan_report()' return value, if kasan is disabled throughkasan_disable_current()it will have no effect in both inline mode, and if int3 is called in outline mode - thekasan_inline_handlerwill lead todie().So do I understand correctly, that we have no way to distinguish whether the int3 was inserted by the KASAN instrumentation or natively called (like in
int3_selftest_ip())?If so, I think that we need to fix/change the compiler first so that we can distinguish these cases. And only then introduce
kasan_inline_handler(). (Withoutkasan_inline_handler(), the outline instrumentation would then just work, right?)Quoted 4 lines
If we can distinguish them, then we should only call
kasan_inline_handler()for the KASAN-inserted int3's. This is what we do on arm64 (via brk andKASAN_BRK_IMM). And thenint3_selftest_ip()should not be affected.
Quoted 5 lines
Looking at it again I suppose LLVM does pass a number along metadata to the int3. I didn't notice because no other function checks anything in the x86 int3 handler, compared to how it's done on arm64 with brk.
So right, thanks, after fixing it up it shouldn't affect the
int3_selftest_ip().
But as Peter Zijlstra noticed, x86 already uses the #UD instruction similarly to BRK on arm64. So I think I'll use this one here, and then change INT3 to UD in the LLVM patch.
Signature · 2 lines
Kind regards Maciej Wieczór-Retman
Andrey Konovalov
reviewerRe: [PATCH v5 13/19] kasan: x86: Handle int3 for inline KASAN reports
Quoted 4 lines · Maciej Wieczor-Retman
But as Peter Zijlstra noticed, x86 already uses the #UD instruction similarly to BRK on arm64. So I think I'll use this one here, and then change INT3 to UD in the LLVM patch.
Sound good, thanks!
PATCH v5 14/19 arm64: Unify software tag-based KASAN inline recovery path
In reply to Maciej Wieczor-RetmanTo avoid having a copy of a long comment explaining the intricacies of the inline KASAN recovery system and issues for every architecture that uses the software tag-based mode, a unified kasan_inline_recover() function was added.
Use kasan_inline_recover() in the kasan brk handler to cleanup the long comment, that's kept in the non-arch KASAN code.
---
Changelog v5:
- Split arm64 portion of patch 13/18 into this one. (Peter Zijlstra)
arch/arm64/kernel/traps.c | 17 +---------------- 1 file changed, 1 insertion(+), 16 deletions(-)
arch/arm64/kernel/traps.c +1 -16
@@ -1068,22 +1068,7 @@ int kasan_brk_handler(struct pt_regs *regs, unsigned long esr)kasan_report (addr, size, write, pc); -/* -* The instrumentation allows to control whether we can proceed after -* a crash was detected. This is done by passing the -recover flag to -* the compiler. Disabling recovery allows to generate more compact -* code. -* -* Unfortunately disabling recovery doesn't work for the kernel right -* now. KASAN reporting is disabled in some contexts (for example when -* the allocator accesses slab object metadata; this is controlled by -* current->kasan_depth). All these accesses are detected by the tool, -* even though the reports for them are not printed. -* -* This is something that might be fixed at some point in the future. -*/ -if (!recover) -die ("Oops - KASAN" , regs, esr); +kasan_inline_recover (recover,"Oops - KASAN" , regs, esr, die);/* If thread survives, skip over the brk instruction and continue: */ arm64_skip_faulting_instruction (regs,AARCH64_INSN_SIZE );
Signature · 1 line
2.50.1
Re: [PATCH v5 14/19] arm64: Unify software tag-based KASAN inline recovery path
Quoted 9 lines · Maciej Wieczor-Retman
To avoid having a copy of a long comment explaining the intricacies of the inline KASAN recovery system and issues for every architecture that uses the software tag-based mode, a unified
kasan_inline_recover()function was added.Use
kasan_inline_recover()in the kasan brk handler to cleanup the long comment, that's kept in the non-arch KASAN code.Signed-off-by Maciej Wieczor-Retman <[email protected]>
PATCH v5 15/19 kasan: x86: Apply multishot to the inline report handler
In reply to Maciej Wieczor-RetmanKASAN by default reports only one tag mismatch and based on other command line parameters either keeps going or panics. The multishot mechanism - enabled either through a command line parameter or by inline enable/disable function calls - lifts that restriction and allows an infinite number of tag mismatch reports to be shown.
Inline KASAN uses the INT3 instruction to pass metadata to the report handling function. Currently the "recover" field in that metadata is broken in the compiler layer and causes every inline tag mismatch to panic the kernel.
Check the multishot state in the KASAN hook called inside the INT3 handling function.
---
Changelog v4:
- Add this patch to the series.
arch/x86/mm/kasan_inline.c | 3 +++ include/linux/kasan.h | 3 +++ mm/kasan/report.c | 8 +++++++- 3 files changed, 13 insertions(+), 1 deletion(-)
arch/x86/mm/kasan_inline.c +3 -0
@@ -17,6 +17,9 @@ bool kasan_inline_handler(struct pt_regs *regs)if (!kasan_report ((void *)addr, size, write, pc))return false; +if (kasan_multi_shot_enabled ()) +return true; +kasan_inline_recover (recover,"Oops - KASAN" , regs, metadata, die);return true;
include/linux/kasan.h +3 -0
@@ -663,7 +663,10 @@ void kasan_non_canonical_hook(unsigned long addr);static inline void kasan_non_canonical_hook (unsigned long addr) { }#endif /* CONFIG_KASAN_GENERIC || CONFIG_KASAN_SW_TAGS */ +bool kasan_multi_shot_enabled (void ); +#ifdef CONFIG_KASAN_SW_TAGS +/* * The instrumentation allows to control whether we can proceed after * a crash was detected. This is done by passing the -recover flag to
mm/kasan/report.c +7 -1
@@ -121,6 +121,12 @@ static void report_suppress_stop(void)#endif } +bool kasan_multi_shot_enabled (void ) +{ +return test_bit (KASAN_BIT_MULTI_SHOT , &kasan_flags); +} +EXPORT_SYMBOL (kasan_multi_shot_enabled); +/* * Used to avoid reporting more than one KASAN bug unless kasan_multi_shot * is enabled. Note that KASAN tests effectively enable kasan_multi_shot
@@ -128,7 +134,7 @@ static void report_suppress_stop(void)*/ static bool report_enabled (void ) { -if (test_bit (KASAN_BIT_MULTI_SHOT , &kasan_flags)) +if (kasan_multi_shot_enabled ())return true;return !test_and_set_bit (KASAN_BIT_REPORTED , &kasan_flags); }
Signature · 1 line
2.50.1
Andrey Konovalov
reviewerRe: [PATCH v5 15/19] kasan: x86: Apply multishot to the inline report handler
Quoted 30 lines · Maciej Wieczor-Retman
KASAN by default reports only one tag mismatch and based on other command line parameters either keeps going or panics. The multishot mechanism - enabled either through a command line parameter or by inline enable/disable function calls - lifts that restriction and allows an infinite number of tag mismatch reports to be shown.
Inline KASAN uses the INT3 instruction to pass metadata to the report handling function. Currently the "recover" field in that metadata is broken in the compiler layer and causes every inline tag mismatch to panic the kernel.
Check the multishot state in the KASAN hook called inside the INT3 handling function.
Signed-off-by Maciej Wieczor-Retman <[email protected]>---
Changelog v4:
- Add this patch to the series.arch/x86/mm/kasan_inline.c | 3 +++ include/linux/kasan.h | 3 +++ mm/kasan/report.c | 8 +++++++- 3 files changed, 13 insertions(+), 1 deletion(-)arch/x86/mm/kasan_inline.c +0
-0
arch/x86/mm/kasan_inline.c bool kasan_inline_handler(struct pt_regs *regs) · line 21
Quoted 5 lines
arch/x86/mm/kasan_inline.c +2
-0@@ -17,6 +17,9 @@ bool kasan_inline_handler(struct pt_regs *regs)if (!kasan_report ((void *)addr, size, write, pc))return false; +if (kasan_multi_shot_enabled ()) +return true;
It's odd this this is required on x86 but not on arm64, see my comment on the patch that adds kasan_inline_handler().
Quoted 48 lines
arch/x86/mm/kasan_inline.c +1
-0+kasan_inline_recover (recover,"Oops - KASAN" , regs, metadata, die);return true;include/linux/kasan.h +3
-0@@ -663,7 +663,10 @@ void kasan_non_canonical_hook(unsigned long addr);static inline void kasan_non_canonical_hook (unsigned long addr) { }#endif /* CONFIG_KASAN_GENERIC || CONFIG_KASAN_SW_TAGS */ +bool kasan_multi_shot_enabled (void ); +#ifdef CONFIG_KASAN_SW_TAGS +/* * The instrumentation allows to control whether we can proceed after * a crash was detected. This is done by passing the -recover flag to mm/kasan/report.c +7
-2@@ -121,6 +121,12 @@ static void report_suppress_stop(void)#endif } +bool kasan_multi_shot_enabled (void ) +{ +return test_bit (KASAN_BIT_MULTI_SHOT , &kasan_flags); +} +EXPORT_SYMBOL (kasan_multi_shot_enabled); +/* * Used to avoid reporting more than one KASAN bug unless kasan_multi_shot * is enabled. Note that KASAN tests effectively enable kasan_multi_shot @@ -128,7 +134,7 @@ static void report_suppress_stop(void)*/ static bool report_enabled (void ) { -if (test_bit (KASAN_BIT_MULTI_SHOT , &kasan_flags)) +if (kasan_multi_shot_enabled ())return true;return !test_and_set_bit (KASAN_BIT_REPORTED , &kasan_flags); } --2.50.1
Re: [PATCH v5 15/19] kasan: x86: Apply multishot to the inline report handler
Quoted 35 lines · Andrey Konovalov
Quoted 33 lines · Maciej Wieczor-Retman
KASAN by default reports only one tag mismatch and based on other command line parameters either keeps going or panics. The multishot mechanism - enabled either through a command line parameter or by inline enable/disable function calls - lifts that restriction and allows an infinite number of tag mismatch reports to be shown.
Inline KASAN uses the INT3 instruction to pass metadata to the report handling function. Currently the "recover" field in that metadata is broken in the compiler layer and causes every inline tag mismatch to panic the kernel.
Check the multishot state in the KASAN hook called inside the INT3 handling function.
Signed-off-by Maciej Wieczor-Retman <[email protected]>---
Changelog v4:
- Add this patch to the series.arch/x86/mm/kasan_inline.c | 3 +++ include/linux/kasan.h | 3 +++ mm/kasan/report.c | 8 +++++++- 3 files changed, 13 insertions(+), 1 deletion(-)arch/x86/mm/kasan_inline.c +0
-0@@ -17,6 +17,9 @@ bool kasan_inline_handler(struct pt_regs *regs)if (!kasan_report ((void *)addr, size, write, pc))return false;
Quoted 6 lines
arch/x86/mm/kasan_inline.c bool kasan_inline_handler(struct pt_regs *regs) · line 21
Quoted 2 lines
arch/x86/mm/kasan_inline.c +2
-0+if (kasan_multi_shot_enabled ()) +return true;It's odd this this is required on x86 but not on arm64, see my comment on the patch that adds
kasan_inline_handler().
I think this is needed if we want to keep the kasan_inline_recover below. Because without this patch, kasan_report() will report a mismatch, an then die() will be called. So the multishot gets ignored.
I'll check what happens on arm64 when a mismatch happens with inline mode + multishot.
Quoted 49 lines
Quoted 49 lines
arch/x86/mm/kasan_inline.c +1
-0+kasan_inline_recover (recover,"Oops - KASAN" , regs, metadata, die);return true;include/linux/kasan.h +3
-0@@ -663,7 +663,10 @@ void kasan_non_canonical_hook(unsigned long addr);static inline void kasan_non_canonical_hook (unsigned long addr) { }#endif /* CONFIG_KASAN_GENERIC || CONFIG_KASAN_SW_TAGS */ +bool kasan_multi_shot_enabled (void ); +#ifdef CONFIG_KASAN_SW_TAGS +/* * The instrumentation allows to control whether we can proceed after * a crash was detected. This is done by passing the -recover flag to mm/kasan/report.c +7
-2@@ -121,6 +121,12 @@ static void report_suppress_stop(void)#endif } +bool kasan_multi_shot_enabled (void ) +{ +return test_bit (KASAN_BIT_MULTI_SHOT , &kasan_flags); +} +EXPORT_SYMBOL (kasan_multi_shot_enabled); +/* * Used to avoid reporting more than one KASAN bug unless kasan_multi_shot * is enabled. Note that KASAN tests effectively enable kasan_multi_shot @@ -128,7 +134,7 @@ static void report_suppress_stop(void)*/ static bool report_enabled (void ) { -if (test_bit (KASAN_BIT_MULTI_SHOT , &kasan_flags)) +if (kasan_multi_shot_enabled ())return true;return !test_and_set_bit (KASAN_BIT_REPORTED , &kasan_flags); } --2.50.1
Signature · 2 lines
Kind regards Maciej Wieczór-Retman
Andrey Konovalov
reviewerRe: [PATCH v5 15/19] kasan: x86: Apply multishot to the inline report handler
Quoted 11 lines · Maciej Wieczor-Retman
Quoted 7 lines
arch/x86/mm/kasan_inline.c bool kasan_inline_handler(struct pt_regs *regs) · line 21
Quoted 3 lines
arch/x86/mm/kasan_inline.c +2
-0+if (kasan_multi_shot_enabled ()) +return true;It's odd this this is required on x86 but not on arm64, see my comment on the patch that adds
kasan_inline_handler().I think this is needed if we want to keep the
kasan_inline_recoverbelow. Because without this patch,kasan_report()will report a mismatch, an thendie()will be called. So the multishot gets ignored.
But die() should be called only when recovery is disabled. And recovery should always be enabled.
But maybe this is the problem with when kasan_inline_handler(), see my comment on the the patch #13.
Re: [PATCH v5 15/19] kasan: x86: Apply multishot to the inline report handler
Quoted 10 lines · Andrey Konovalov
Quoted 8 lines · Maciej Wieczor-Retman
Quoted 7 lines
arch/x86/mm/kasan_inline.c bool kasan_inline_handler(struct pt_regs *regs) · line 21
Quoted 3 lines
arch/x86/mm/kasan_inline.c +2
-0+if (kasan_multi_shot_enabled ()) +return true;It's odd this this is required on x86 but not on arm64, see my comment on the patch that adds
kasan_inline_handler().
Quoted 6 lines
Quoted 3 lines
I think this is needed if we want to keep the
kasan_inline_recoverbelow. Because without this patch,kasan_report()will report a mismatch, an thendie()will be called. So the multishot gets ignored.But
die()should be called only when recovery is disabled. And recovery should always be enabled.
Hmm I thought when I was testing inline mode last time, that recovery was always disabled. I'll recheck later.
But just looking at llvm code, hwasan-recover has init(false). And the kernel doesn't do anything to this value in Makefile.kasan. Perhaps it just needs to be corrected in the Makefile.kasan?
Quoted 2 lines
But maybe this is the problem with when
kasan_inline_handler(), see my comment on the the patch #13.
Signature · 2 lines
Kind regards Maciej Wieczór-Retman
Andrey Konovalov
reviewerRe: [PATCH v5 15/19] kasan: x86: Apply multishot to the inline report handler
Quoted 19 lines · Maciej Wieczor-Retman
Quoted 10 lines · Andrey Konovalov
Quoted 8 lines · Maciej Wieczor-Retman
Quoted 7 lines
arch/x86/mm/kasan_inline.c bool kasan_inline_handler(struct pt_regs *regs) · line 21
Quoted 3 lines
arch/x86/mm/kasan_inline.c +2
-0+if (kasan_multi_shot_enabled ()) +return true;It's odd this this is required on x86 but not on arm64, see my comment on the patch that adds
kasan_inline_handler().Quoted 6 lines
Quoted 3 lines
I think this is needed if we want to keep the
kasan_inline_recoverbelow. Because without this patch,kasan_report()will report a mismatch, an thendie()will be called. So the multishot gets ignored.But
die()should be called only when recovery is disabled. And recovery should always be enabled.
Quoted 6 lines
Hmm I thought when I was testing inline mode last time, that recovery was always disabled. I'll recheck later.
But just looking at llvm code, hwasan-recover has init(false). And the kernel doesn't do anything to this value in Makefile.kasan. Perhaps it just needs to be corrected in the Makefile.kasan?
Recovery should be disabled as the default when -fsanitize=kernel-hwaddress is used (unless something was broken/changed); see this patch:
https://github.com/llvm/llvm-project/commit/1ba9d9c6ca1ffeef7e833261ebca463a92adf82f
Andrey Konovalov
reviewerRe: [PATCH v5 15/19] kasan: x86: Apply multishot to the inline report handler
Quoted 22 lines · Andrey Konovalov
Quoted 19 lines · Maciej Wieczor-Retman
Quoted 16 lines · Andrey Konovalov
Quoted 11 lines · Maciej Wieczor-Retman
Quoted 7 lines
arch/x86/mm/kasan_inline.c bool kasan_inline_handler(struct pt_regs *regs) · line 21
Quoted 3 lines
arch/x86/mm/kasan_inline.c +2
-0+if (kasan_multi_shot_enabled ()) +return true;It's odd this this is required on x86 but not on arm64, see my comment on the patch that adds
kasan_inline_handler().I think this is needed if we want to keep the
kasan_inline_recoverbelow. Because without this patch,kasan_report()will report a mismatch, an thendie()will be called. So the multishot gets ignored.But
die()should be called only when recovery is disabled. And recovery should always be enabled.
Quoted 8 lines
Quoted 6 lines
Hmm I thought when I was testing inline mode last time, that recovery was always disabled. I'll recheck later.
But just looking at llvm code, hwasan-recover has init(false). And the kernel doesn't do anything to this value in Makefile.kasan. Perhaps it just needs to be corrected in the Makefile.kasan?
Recovery should be disabled as the default when
Eh, enabled, not disabled.
Quoted 4 lines
-fsanitize=kernel-hwaddress is used (unless something was broken/changed); see this patch:
https://github.com/llvm/llvm-project/commit/1ba9d9c6ca1ffeef7e833261ebca463a92adf82f
PATCH v5 16/19 kasan: x86: Logical bit shift for kasan_mem_to_shadow
In reply to Maciej Wieczor-RetmanWhile generally tag-based KASAN adopts an arithemitc bit shift to convert a memory address to a shadow memory address, it doesn't work for all cases on x86. Testing different shadow memory offsets proved that either 4 or 5 level paging didn't work correctly or inline mode ran into issues. Thus the best working scheme is the logical bit shift and non-canonical shadow offset that x86 uses for generic KASAN, of course adjusted for the increased granularity from 8 to 16 bytes.
Add an arch specific implementation of kasan_mem_to_shadow() that uses the logical bit shift.
The non-canonical hook tries to calculate whether an address came from kasan_mem_to_shadow(). First it checks whether this address fits into the legal set of values possible to output from the mem to shadow function.
Tie both generic and tag-based x86 KASAN modes to the address range check associated with generic KASAN.
---
Changelog v4:
- Add this patch to the series.
arch/x86/include/asm/kasan.h | 8 ++++++++ mm/kasan/report.c | 5 +++-- 2 files changed, 11 insertions(+), 2 deletions(-)
arch/x86/include/asm/kasan.h +8 -0
@@ -53,6 +53,14 @@#ifdef CONFIG_KASAN_SW_TAGS +static inline void *__kasan_mem_to_shadow (const void *addr) +{ +return (void *)((unsigned long )addr >>KASAN_SHADOW_SCALE_SHIFT ) + +KASAN_SHADOW_OFFSET ; +} + +#define kasan_mem_to_shadow (addr) __kasan_mem_to_shadow(addr) +#define __tag_shifted (tag) FIELD_PREP(GENMASK_ULL(60, 57), tag)#define __tag_reset (addr) (sign_extend64((u64)(addr), 56))#define __tag_get (addr) ((u8)FIELD_GET(GENMASK_ULL(60, 57), (u64)addr))
mm/kasan/report.c +3 -2
@@ -648,13 +648,14 @@ void kasan_non_canonical_hook(unsigned long addr)const char *bug_type;/* -* For Generic KASAN, kasan_mem_to_shadow() uses the logical right shift +* For Generic KASAN and Software Tag-Based mode on the x86 +* architecture, kasan_mem_to_shadow() uses the logical right shift * and never overflows with the chosen KASAN_SHADOW_OFFSET values (on * both x86 and arm64). Thus, the possible shadow addresses (even for * bogus pointers) belong to a single contiguous region that is the * result of kasan_mem_to_shadow() applied to the whole address space. */ -if (IS_ENABLED (CONFIG_KASAN_GENERIC )) { +if (IS_ENABLED (CONFIG_KASAN_GENERIC ) ||IS_ENABLED (CONFIG_X86_64 )) {if (addr < (unsigned long )kasan_mem_to_shadow ((void *)(0UL )) || addr > (unsigned long )kasan_mem_to_shadow ((void *)(~0UL )))return ;
Signature · 1 line
2.50.1
PATCH v5 17/19 mm: Unpoison pcpu chunks with base address tag
In reply to Maciej Wieczor-RetmanThe problem presented here is related to NUMA systems and tag-based KASAN mode. It can be explained in the following points:
1. There can be more than one virtual memory chunk.
2. Chunk's base address has a tag.
3. The base address points at the first chunk and thus inherits the tag of the first chunk.
4. The subsequent chunks will be accessed with the tag from the first chunk.
5. Thus, the subsequent chunks need to have their tag set to match that of the first chunk.
Refactor code by moving it into a helper in preparation for the actual fix.
---
Changelog v4:
- Redo the patch message numbered list.
- Do the refactoring in this patch and move additions to the next new one.
Changelog v3:
- Remove last version of this patch that just resets the tag on base_addr and add this patch that unpoisons all areas with the same tag instead.
include/linux/kasan.h | 10 ++++++++++ mm/kasan/hw_tags.c | 11 +++++++++++ mm/kasan/shadow.c | 10 ++++++++++ mm/vmalloc.c | 4 +--- 4 files changed, 32 insertions(+), 3 deletions(-)
include/linux/kasan.h +10 -0
@@ -613,6 +613,13 @@ static __always_inline void kasan_poison_vmalloc(const void *start,__kasan_poison_vmalloc (start, size); } +void __kasan_unpoison_vmap_areas (struct vm_struct **vms,int nr_vms); +static __always_inline void kasan_unpoison_vmap_areas(struct vm_struct **vms, int nr_vms) +{ +if (kasan_enabled ()) +__kasan_unpoison_vmap_areas (vms, nr_vms); +} +#else /* CONFIG_KASAN_VMALLOC */ static inline void kasan_populate_early_vm_area_shadow(void *start,
@@ -637,6 +644,9 @@ static inline void *kasan_unpoison_vmalloc(const void *start,static inline void kasan_poison_vmalloc (const void *start,unsigned long size) { } +static inline void kasan_unpoison_vmap_areas (struct vm_struct **vms,int nr_vms) +{ } +#endif /* CONFIG_KASAN_VMALLOC */ #if (defined(CONFIG_KASAN_GENERIC) || defined(CONFIG_KASAN_SW_TAGS)) && \
mm/kasan/hw_tags.c +11 -0
@@ -382,6 +382,17 @@ void __kasan_poison_vmalloc(const void *start, unsigned long size) */ } +void __kasan_unpoison_vmap_areas (struct vm_struct **vms,int nr_vms) +{ +int area; + +for (area =0 ; area < nr_vms ; area++) { + vms[area]->addr =__kasan_unpoison_vmalloc ( + vms[area]->addr, vms[area]->size, +KASAN_VMALLOC_PROT_NORMAL ); + } +} +#endif void kasan_enable_hw_tags (void )
mm/kasan/shadow.c +10 -0
@@ -646,6 +646,16 @@ void __kasan_poison_vmalloc(const void *start, unsigned long size) kasan_poison(start, size, KASAN_VMALLOC_INVALID, false); } +void __kasan_unpoison_vmap_areas (struct vm_struct **vms,int nr_vms) +{ +int area; + +for (area =0 ; area < nr_vms ; area++) { +kasan_poison (vms[area]->addr, vms[area]->size, +arch_kasan_get_tag (vms[area]->addr), false); + } +} +#else /* CONFIG_KASAN_VMALLOC */ int kasan_alloc_module_shadow (void *addr,size_t size,gfp_t gfp_mask)
mm/vmalloc.c +1 -3
@@ -4847,9 +4847,7 @@ struct vm_struct **pcpu_get_vm_areas(const unsigned long *offsets,* With hardware tag-based KASAN, marking is skipped for * non-VM_ALLOC mappings, see __kasan_unpoison_vmalloc(). */ -for (area =0 ; area < nr_vms; area++) - vms[area]->addr =kasan_unpoison_vmalloc (vms[area]->addr, - vms[area]->size,KASAN_VMALLOC_PROT_NORMAL ); +kasan_unpoison_vmap_areas (vms, nr_vms);kfree (vas);return vms;
Signature · 1 line
2.50.1
Re: [PATCH v5 17/19] mm: Unpoison pcpu chunks with base address tag
Hi ,
Quoted 12 lines · Maciej Wieczor-Retman
The problem presented here is related to NUMA systems and tag-based KASAN mode. It can be explained in the following points:
1. There can be more than one virtual memory chunk.
2. Chunk's base address has a tag.
3. The base address points at the first chunk and thus inherits the tag of the first chunk.
4. The subsequent chunks will be accessed with the tag from the first chunk.
5. Thus, the subsequent chunks need to have their tag set to match that of the first chunk.
Quoted 2 lines
Refactor code by moving it into a helper in preparation for the actual fix.
I got a boot breakage on a hpe-apollo arm64 system with sw_tags mode, and the boot breakage can be met stably. The detailed situation is reported in below link:
System is broken in KASAN sw_tags mode during bootup
https://lore.kernel.org/all/aKMLgHdTOEf9B92E@MiWiFi-R3L-srv/T/#u
After applying this patch 17 and patch 18 in this series, I can confirm the breakage is gone. Thanks for the great fix, and please feel free to add:
========================CONFIG_KASAN=yCONFIG_CC_HAS_KASAN_MEMINTRINSIC_PREFIX=y
# CONFIG_KASAN_GENERIC is not setCONFIG_KASAN_SW_TAGS=y
# CONFIG_KASAN_HW_TAGS is not set
# CONFIG_KASAN_OUTLINE is not setCONFIG_KASAN_INLINE=yCONFIG_KASAN_STACK=yCONFIG_KASAN_VMALLOC=yCONFIG_KASAN_KUNIT_TEST=m
================================
[ 100.907469] ================================================================== [ 100.907485] BUG: KASAN: invalid-access in pcpu_alloc_noprof+0x42c/0x9a8 [ 100.907509] Write of size 160 at addr 10fffd7fbdc00000 by task systemd/1 [ 100.907524] Pointer tag: [10], memory tag: [5b] [ 100.907532] [ 100.907544] CPU: 229 UID: 0 PID: 1 Comm: systemd Not tainted 6.16.0+ #2 PREEMPT(voluntary) [ 100.907562] Hardware name: HPE Apollo 70 /C01_APACHE_MB , BIOS L50_5.13_1.16 07/29/2020 [ 100.907571] Call trace: [ 100.907578] show_stack+0x30/0x98 (C) [ 100.907597] dump_stack_lvl+0x7c/0xa0 [ 100.907614] print_address_description.isra.0+0x90/0x2b8 [ 100.907635] print_report+0x120/0x208 [ 100.907651] kasan_report+0xc8/0x110 [ 100.907669] kasan_check_range+0x80/0xa0 [ 100.907685] __asan_memset+0x30/0x68 [ 100.907700] pcpu_alloc_noprof+0x42c/0x9a8 [ 100.907716] css_rstat_init+0x1bc/0x220 [ 100.907734] cgroup_create+0x188/0x540 [ 100.907749] cgroup_mkdir+0xb4/0x330 [ 100.907765] kernfs_iop_mkdir+0xb0/0x120 [ 100.907783] vfs_mkdir+0x250/0x380 [ 100.907800] do_mkdirat+0x254/0x298 [ 100.907815] __arm64_sys_mkdirat+0x80/0xc0 [ 100.907831] invoke_syscall.constprop.0+0x88/0x148 [ 100.907848] el0_svc_common.constprop.0+0x78/0x148 [ 100.907863] do_el0_svc+0x38/0x50 [ 100.907877] el0_svc+0x3c/0x160 [ 100.907895] el0t_64_sync_handler+0x10c/0x138 [ 100.907911] el0t_64_sync+0x1b0/0x1b8 [ 100.907925] [ 100.907931] The buggy address belongs to a 0-page vmalloc region starting at 0x5bfffd7fbdc00000 allocated at pcpu_get_vm_areas+0x0/0x1da8 [ 100.907963] The buggy address belongs to the physical page: [ 100.907970] page: refcount:1 mapcount:0 mapping:0000000000000000 index:0x0 pfn:0x8811a35 [ 100.907984] flags: 0xa6a00000000000(node=1|zone=2|kasantag=0x6a) [ 100.908006] raw: 00a6a00000000000 0000000000000000 dead000000000122 0000000000000000 [ 100.908019] raw: 0000000000000000 b4ff00878bce6400 00000001ffffffff 0000000000000000 [ 100.908029] raw: 00000000000fffff 0000000000000000 [ 100.908037] page dumped because: kasan: bad access detected [ 100.908044] [ 100.908048] Memory state around the buggy address: [ 100.908059] Unable to handle kernel paging request at virtual address ffff7fd7fbdbffe0 [ 100.908068] KASAN: probably wild-memory-access in range [0xfffffd7fbdbffe00-0xfffffd7fbdbffe0f] [ 100.908078] Mem abort info: [ 100.908083] ESR = 0x0000000096000007 [ 100.908089] EC = 0x25: DABT (current EL), IL = 32 bits [ 100.908098] SET = 0, FnV = 0 [ 100.908105] EA = 0, S1PTW = 0 [ 100.908111] FSC = 0x07: level 3 translation fault [ 100.908118] Data abort info: [ 100.908123] ISV = 0, ISS = 0x00000007, ISS2 = 0x00000000 [ 100.908130] CM = 0, WnR = 0, TnD = 0, TagAccess = 0 [ 100.908138] GCS = 0, Overlay = 0, DirtyBit = 0, Xs = 0 [ 100.908147] swapper pgtable: 4k pages, 48-bit VAs, pgdp=0000008ff8b76000 [ 100.908156] [ffff7fd7fbdbffe0] pgd=1000008ff0299403, p4d=1000008ff0299403, pud=1000008ff0298403, pmd=1000008811a17403, pte=0000000000000000 [ 100.908192] Internal error: Oops: 0000000096000007 [#1] SMP [ 101.185060] Modules linked in: i2c_dev [ 101.188820] CPU: 229 UID: 0 PID: 1 Comm: systemd Not tainted 6.16.0+ #2 PREEMPT(voluntary) [ 101.197175] Hardware name: HPE Apollo 70 /C01_APACHE_MB , BIOS L50_5.13_1.16 07/29/2020 [ 101.206912] pstate: 604000c9 (nZCv daIF +PAN -UAO -TCO -DIT -SSBS BTYPE=--) [ 101.213877] pc : __pi_memcpy_generic+0x24/0x230 [ 101.218418] lr : kasan_metadata_fetch_row+0x20/0x30 [ 101.223299] sp : ffff8000859b7700 [ 101.226610] x29: ffff8000859b7700 x28: 0000000000000100 x27: ffff008ec6291800 [ 101.233758] x26: 00000000000000a0 x25: 0000000000000000 x24: fffffd7fbdbfff00 [ 101.240904] x23: ffff8000826b1e58 x22: fffffd7fbdc00000 x21: 00000000fffffffe [ 101.248051] x20: ffff800082669d18 x19: fffffd7fbdbffe00 x18: 0000000000000000 [ 101.255196] x17: 3030303030303030 x16: 2066666666666666 x15: 6631303030303030 [ 101.262342] x14: 0000000000000001 x13: 0000000000000001 x12: 0000000000000001 [ 101.269487] x11: 687420646e756f72 x10: 0000000000000020 x9 : 0000000000000000 [ 101.276633] x8 : ffff78000859b76a x7 : 0000000000000000 x6 : 000000000000003a [ 101.283778] x5 : ffff8000859b7768 x4 : ffff7fd7fbdbfff0 x3 : efff800000000000 [ 101.290924] x2 : 0000000000000010 x1 : ffff7fd7fbdbffe0 x0 : ffff8000859b7758 [ 101.298070] Call trace: [ 101.300512] __pi_memcpy_generic+0x24/0x230 (P) [ 101.305051] print_report+0x180/0x208 [ 101.308719] kasan_report+0xc8/0x110 [ 101.312299] kasan_check_range+0x80/0xa0 [ 101.316227] __asan_memset+0x30/0x68 [ 101.319807] pcpu_alloc_noprof+0x42c/0x9a8 [ 101.323908] css_rstat_init+0x1bc/0x220 [ 101.327749] cgroup_create+0x188/0x540 [ 101.331502] cgroup_mkdir+0xb4/0x330 [ 101.335082] kernfs_iop_mkdir+0xb0/0x120 [ 101.339011] vfs_mkdir+0x250/0x380 [ 101.342416] do_mkdirat+0x254/0x298 [ 101.345908] __arm64_sys_mkdirat+0x80/0xc0 [ 101.350008] invoke_syscall.constprop.0+0x88/0x148 [ 101.354803] el0_svc_common.constprop.0+0x78/0x148 [ 101.359598] do_el0_svc+0x38/0x50 [ 101.362916] el0_svc+0x3c/0x160 [ 101.366061] el0t_64_sync_handler+0x10c/0x138 [ 101.370423] el0t_64_sync+0x1b0/0x1b8 [ 101.374095] Code: f100805f 540003c8 f100405f 540000c3 (a9401c26) [ 101.380187] ---[ end trace 0000000000000000 ]--- [ 101.384802] note: systemd[1] ex
** replaying previous printk message **
Quoted 109 lines
Signed-off-by Maciej Wieczor-Retman <[email protected]>---
Changelog v4:
- Redo the patch message numbered list.
- Do the refactoring in this patch and move additions to the next new one.Changelog v3:
- Remove last version of this patch that just resets the tag onbase_addrand add this patch that unpoisons all areas with the same tag instead.include/linux/kasan.h | 10 ++++++++++ mm/kasan/hw_tags.c | 11 +++++++++++ mm/kasan/shadow.c | 10 ++++++++++ mm/vmalloc.c | 4 +--- 4 files changed, 32 insertions(+), 3 deletions(-)include/linux/kasan.h +10
-0@@ -613,6 +613,13 @@ static __always_inline void kasan_poison_vmalloc(const void *start,__kasan_poison_vmalloc (start, size); } +void __kasan_unpoison_vmap_areas (struct vm_struct **vms,int nr_vms); +static __always_inline void kasan_unpoison_vmap_areas(struct vm_struct **vms, int nr_vms) +{ +if (kasan_enabled ()) +__kasan_unpoison_vmap_areas (vms, nr_vms); +} +#else /* CONFIG_KASAN_VMALLOC */ static inline void kasan_populate_early_vm_area_shadow(void *start,@@ -637,6 +644,9 @@ static inline void *kasan_unpoison_vmalloc(const void *start,static inline void kasan_poison_vmalloc (const void *start,unsigned long size) { } +static inline void kasan_unpoison_vmap_areas (struct vm_struct **vms,int nr_vms) +{ } +#endif /* CONFIG_KASAN_VMALLOC */ #if (defined(CONFIG_KASAN_GENERIC) || defined(CONFIG_KASAN_SW_TAGS)) && \mm/kasan/hw_tags.c +11
-0@@ -382,6 +382,17 @@ void __kasan_poison_vmalloc(const void *start, unsigned long size) */ } +void __kasan_unpoison_vmap_areas (struct vm_struct **vms,int nr_vms) +{ +int area; + +for (area =0 ; area < nr_vms ; area++) { + vms[area]->addr =__kasan_unpoison_vmalloc ( + vms[area]->addr, vms[area]->size, +KASAN_VMALLOC_PROT_NORMAL ); + } +} +#endif void kasan_enable_hw_tags (void )mm/kasan/shadow.c +10
-0@@ -646,6 +646,16 @@ void __kasan_poison_vmalloc(const void *start, unsigned long size) kasan_poison(start, size, KASAN_VMALLOC_INVALID, false); } +void __kasan_unpoison_vmap_areas (struct vm_struct **vms,int nr_vms) +{ +int area; + +for (area =0 ; area < nr_vms ; area++) { +kasan_poison (vms[area]->addr, vms[area]->size, +arch_kasan_get_tag (vms[area]->addr), false); + } +} +#else /* CONFIG_KASAN_VMALLOC */ int kasan_alloc_module_shadow (void *addr,size_t size,gfp_t gfp_mask)mm/vmalloc.c +1
-3@@ -4847,9 +4847,7 @@ struct vm_struct **pcpu_get_vm_areas(const unsigned long *offsets,* With hardware tag-based KASAN, marking is skipped for * non-VM_ALLOC mappings, see __kasan_unpoison_vmalloc(). */ -for (area =0 ; area < nr_vms; area++) - vms[area]->addr =kasan_unpoison_vmalloc (vms[area]->addr, - vms[area]->size,KASAN_VMALLOC_PROT_NORMAL ); +kasan_unpoison_vmap_areas (vms, nr_vms);kfree (vas);return vms;Signature · 1 line
2.50.1
Re: [PATCH v5 17/19] mm: Unpoison pcpu chunks with base address tag
Thanks for testing the series! And I'm happy the fix was useful :)
Quoted 252 lines · Baoquan He
Hi ,
Quoted 12 lines · Maciej Wieczor-Retman
The problem presented here is related to NUMA systems and tag-based KASAN mode. It can be explained in the following points:
1. There can be more than one virtual memory chunk.
2. Chunk's base address has a tag.
3. The base address points at the first chunk and thus inherits the tag of the first chunk.
4. The subsequent chunks will be accessed with the tag from the first chunk.
5. Thus, the subsequent chunks need to have their tag set to match that of the first chunk.Quoted 2 lines
Refactor code by moving it into a helper in preparation for the actual fix.
I got a boot breakage on a hpe-apollo arm64 system with
sw_tagsmode, and the boot breakage can be met stably. The detailed situation is reported in below link:System is broken in KASAN
sw_tagsmode during bootup
https://lore.kernel.org/all/aKMLgHdTOEf9B92E@MiWiFi-R3L-srv/T/#uAfter applying this patch 17 and patch 18 in this series, I can confirm the breakage is gone. Thanks for the great fix, and please feel free to add:
Tested-by Baoquan He <[email protected]>========================
CONFIG_KASAN=yCONFIG_CC_HAS_KASAN_MEMINTRINSIC_PREFIX=y
#CONFIG_KASAN_GENERICis not setCONFIG_KASAN_SW_TAGS=y
#CONFIG_KASAN_HW_TAGSis not set
#CONFIG_KASAN_OUTLINEis not setCONFIG_KASAN_INLINE=yCONFIG_KASAN_STACK=yCONFIG_KASAN_VMALLOC=yCONFIG_KASAN_KUNIT_TEST=m
================================[ 100.907469] ================================================================== [ 100.907485] BUG: KASAN: invalid-access in pcpu_alloc_noprof+0x42c/0x9a8 [ 100.907509] Write of size 160 at addr 10fffd7fbdc00000 by task systemd/1 [ 100.907524] Pointer tag: [10], memory tag: [5b] [ 100.907532] [ 100.907544] CPU: 229 UID: 0 PID: 1 Comm: systemd Not tainted 6.16.0+ #2 PREEMPT(voluntary) [ 100.907562] Hardware name: HPE Apollo 70 /C01_APACHE_MB , BIOS L50_5.13_1.16 07/29/2020 [ 100.907571] Call trace: [ 100.907578] show_stack+0x30/0x98 (C) [ 100.907597] dump_stack_lvl+0x7c/0xa0 [ 100.907614] print_address_description.isra.0+0x90/0x2b8 [ 100.907635] print_report+0x120/0x208 [ 100.907651] kasan_report+0xc8/0x110 [ 100.907669] kasan_check_range+0x80/0xa0 [ 100.907685] __asan_memset+0x30/0x68 [ 100.907700] pcpu_alloc_noprof+0x42c/0x9a8 [ 100.907716] css_rstat_init+0x1bc/0x220 [ 100.907734] cgroup_create+0x188/0x540 [ 100.907749] cgroup_mkdir+0xb4/0x330 [ 100.907765] kernfs_iop_mkdir+0xb0/0x120 [ 100.907783] vfs_mkdir+0x250/0x380 [ 100.907800] do_mkdirat+0x254/0x298 [ 100.907815] __arm64_sys_mkdirat+0x80/0xc0 [ 100.907831] invoke_syscall.constprop.0+0x88/0x148 [ 100.907848] el0_svc_common.constprop.0+0x78/0x148 [ 100.907863] do_el0_svc+0x38/0x50 [ 100.907877] el0_svc+0x3c/0x160 [ 100.907895] el0t_64_sync_handler+0x10c/0x138 [ 100.907911] el0t_64_sync+0x1b0/0x1b8 [ 100.907925] [ 100.907931] The buggy address belongs to a 0-page vmalloc region starting at 0x5bfffd7fbdc00000 allocated at pcpu_get_vm_areas+0x0/0x1da8 [ 100.907963] The buggy address belongs to the physical page: [ 100.907970] page: refcount:1 mapcount:0 mapping:0000000000000000 index:0x0 pfn:0x8811a35 [ 100.907984] flags: 0xa6a00000000000(node=1|zone=2|kasantag=0x6a) [ 100.908006] raw: 00a6a00000000000 0000000000000000 dead000000000122 0000000000000000 [ 100.908019] raw: 0000000000000000 b4ff00878bce6400 00000001ffffffff 0000000000000000 [ 100.908029] raw: 00000000000fffff 0000000000000000 [ 100.908037] page dumped because: kasan: bad access detected [ 100.908044] [ 100.908048] Memory state around the buggy address: [ 100.908059] Unable to handle kernel paging request at virtual address ffff7fd7fbdbffe0 [ 100.908068] KASAN: probably wild-memory-access in range [0xfffffd7fbdbffe00-0xfffffd7fbdbffe0f] [ 100.908078] Mem abort info: [ 100.908083] ESR = 0x0000000096000007 [ 100.908089] EC = 0x25: DABT (current EL), IL = 32 bits [ 100.908098] SET = 0, FnV = 0 [ 100.908105] EA = 0, S1PTW = 0 [ 100.908111] FSC = 0x07: level 3 translation fault [ 100.908118] Data abort info: [ 100.908123] ISV = 0, ISS = 0x00000007, ISS2 = 0x00000000 [ 100.908130] CM = 0, WnR = 0, TnD = 0, TagAccess = 0 [ 100.908138] GCS = 0, Overlay = 0, DirtyBit = 0, Xs = 0 [ 100.908147] swapper pgtable: 4k pages, 48-bit VAs, pgdp=0000008ff8b76000 [ 100.908156] [ffff7fd7fbdbffe0] pgd=1000008ff0299403, p4d=1000008ff0299403, pud=1000008ff0298403, pmd=1000008811a17403, pte=0000000000000000 [ 100.908192] Internal error: Oops: 0000000096000007 [#1] SMP [ 101.185060] Modules linked in: i2c_dev [ 101.188820] CPU: 229 UID: 0 PID: 1 Comm: systemd Not tainted 6.16.0+ #2 PREEMPT(voluntary) [ 101.197175] Hardware name: HPE Apollo 70 /C01_APACHE_MB , BIOS L50_5.13_1.16 07/29/2020 [ 101.206912] pstate: 604000c9 (nZCv daIF +PAN -UAO -TCO -DIT -SSBS BTYPE=--) [ 101.213877] pc : __pi_memcpy_generic+0x24/0x230 [ 101.218418] lr : kasan_metadata_fetch_row+0x20/0x30 [ 101.223299] sp : ffff8000859b7700 [ 101.226610] x29: ffff8000859b7700 x28: 0000000000000100 x27: ffff008ec6291800 [ 101.233758] x26: 00000000000000a0 x25: 0000000000000000 x24: fffffd7fbdbfff00 [ 101.240904] x23: ffff8000826b1e58 x22: fffffd7fbdc00000 x21: 00000000fffffffe [ 101.248051] x20: ffff800082669d18 x19: fffffd7fbdbffe00 x18: 0000000000000000 [ 101.255196] x17: 3030303030303030 x16: 2066666666666666 x15: 6631303030303030 [ 101.262342] x14: 0000000000000001 x13: 0000000000000001 x12: 0000000000000001 [ 101.269487] x11: 687420646e756f72 x10: 0000000000000020 x9 : 0000000000000000 [ 101.276633] x8 : ffff78000859b76a x7 : 0000000000000000 x6 : 000000000000003a [ 101.283778] x5 : ffff8000859b7768 x4 : ffff7fd7fbdbfff0 x3 : efff800000000000 [ 101.290924] x2 : 0000000000000010 x1 : ffff7fd7fbdbffe0 x0 : ffff8000859b7758 [ 101.298070] Call trace: [ 101.300512] __pi_memcpy_generic+0x24/0x230 (P) [ 101.305051] print_report+0x180/0x208 [ 101.308719] kasan_report+0xc8/0x110 [ 101.312299] kasan_check_range+0x80/0xa0 [ 101.316227] __asan_memset+0x30/0x68 [ 101.319807] pcpu_alloc_noprof+0x42c/0x9a8 [ 101.323908] css_rstat_init+0x1bc/0x220 [ 101.327749] cgroup_create+0x188/0x540 [ 101.331502] cgroup_mkdir+0xb4/0x330 [ 101.335082] kernfs_iop_mkdir+0xb0/0x120 [ 101.339011] vfs_mkdir+0x250/0x380 [ 101.342416] do_mkdirat+0x254/0x298 [ 101.345908] __arm64_sys_mkdirat+0x80/0xc0 [ 101.350008] invoke_syscall.constprop.0+0x88/0x148 [ 101.354803] el0_svc_common.constprop.0+0x78/0x148 [ 101.359598] do_el0_svc+0x38/0x50 [ 101.362916] el0_svc+0x3c/0x160 [ 101.366061] el0t_64_sync_handler+0x10c/0x138 [ 101.370423] el0t_64_sync+0x1b0/0x1b8 [ 101.374095] Code: f100805f 540003c8 f100405f 540000c3 (a9401c26) [ 101.380187] ---[ end trace 0000000000000000 ]--- [ 101.384802] note: systemd[1] ex** replaying previous printk message **
Quoted 109 lines
Signed-off-by Maciej Wieczor-Retman <[email protected]>---
Changelog v4:
- Redo the patch message numbered list.
- Do the refactoring in this patch and move additions to the next new one.Changelog v3:
- Remove last version of this patch that just resets the tag onbase_addrand add this patch that unpoisons all areas with the same tag instead.include/linux/kasan.h | 10 ++++++++++ mm/kasan/hw_tags.c | 11 +++++++++++ mm/kasan/shadow.c | 10 ++++++++++ mm/vmalloc.c | 4 +--- 4 files changed, 32 insertions(+), 3 deletions(-)include/linux/kasan.h +10
-0@@ -613,6 +613,13 @@ static __always_inline void kasan_poison_vmalloc(const void *start,__kasan_poison_vmalloc (start, size); } +void __kasan_unpoison_vmap_areas (struct vm_struct **vms,int nr_vms); +static __always_inline void kasan_unpoison_vmap_areas(struct vm_struct **vms, int nr_vms) +{ +if (kasan_enabled ()) +__kasan_unpoison_vmap_areas (vms, nr_vms); +} +#else /* CONFIG_KASAN_VMALLOC */ static inline void kasan_populate_early_vm_area_shadow(void *start,@@ -637,6 +644,9 @@ static inline void *kasan_unpoison_vmalloc(const void *start,static inline void kasan_poison_vmalloc (const void *start,unsigned long size) { } +static inline void kasan_unpoison_vmap_areas (struct vm_struct **vms,int nr_vms) +{ } +#endif /* CONFIG_KASAN_VMALLOC */ #if (defined(CONFIG_KASAN_GENERIC) || defined(CONFIG_KASAN_SW_TAGS)) && \mm/kasan/hw_tags.c +11
-0@@ -382,6 +382,17 @@ void __kasan_poison_vmalloc(const void *start, unsigned long size) */ } +void __kasan_unpoison_vmap_areas (struct vm_struct **vms,int nr_vms) +{ +int area; + +for (area =0 ; area < nr_vms ; area++) { + vms[area]->addr =__kasan_unpoison_vmalloc ( + vms[area]->addr, vms[area]->size, +KASAN_VMALLOC_PROT_NORMAL ); + } +} +#endif void kasan_enable_hw_tags (void )mm/kasan/shadow.c +10
-0@@ -646,6 +646,16 @@ void __kasan_poison_vmalloc(const void *start, unsigned long size) kasan_poison(start, size, KASAN_VMALLOC_INVALID, false); } +void __kasan_unpoison_vmap_areas (struct vm_struct **vms,int nr_vms) +{ +int area; + +for (area =0 ; area < nr_vms ; area++) { +kasan_poison (vms[area]->addr, vms[area]->size, +arch_kasan_get_tag (vms[area]->addr), false); + } +} +#else /* CONFIG_KASAN_VMALLOC */ int kasan_alloc_module_shadow (void *addr,size_t size,gfp_t gfp_mask)mm/vmalloc.c +1
-3@@ -4847,9 +4847,7 @@ struct vm_struct **pcpu_get_vm_areas(const unsigned long *offsets,* With hardware tag-based KASAN, marking is skipped for * non-VM_ALLOC mappings, see __kasan_unpoison_vmalloc(). */ -for (area =0 ; area < nr_vms; area++) - vms[area]->addr =kasan_unpoison_vmalloc (vms[area]->addr, - vms[area]->size,KASAN_VMALLOC_PROT_NORMAL ); +kasan_unpoison_vmap_areas (vms, nr_vms);kfree (vas);return vms;Signature · 1 line
2.50.1
Signature · 2 lines
Kind regards Maciej Wieczór-Retman
PATCH v5 18/19 mm: Unpoison vms[area] addresses with a common tag
In reply to Maciej Wieczor-RetmanThe problem presented here is related to NUMA systems and tag-based KASAN mode. It can be explained in the following points:
1. There can be more than one virtual memory chunk.
2. Chunk's base address has a tag.
3. The base address points at the first chunk and thus inherits the tag of the first chunk.
4. The subsequent chunks will be accessed with the tag from the first chunk.
5. Thus, the subsequent chunks need to have their tag set to match that of the first chunk.
Unpoison all vms[]->addr memory and pointers with the same tag to resolve the mismatch.
---
Changelog v4:
- Move tagging the vms[]->addr to this new patch and leave refactoring there.
- Comment the fix to provide some context.
mm/kasan/shadow.c | 10 +++++++++- 1 file changed, 9 insertions(+), 1 deletion(-)
mm/kasan/shadow.c +9 -1
@@ -646,13 +646,21 @@ void __kasan_poison_vmalloc(const void *start, unsigned long size)kasan_poison (start, size,KASAN_VMALLOC_INVALID , false); } +/* +* A tag mismatch happens when calculating per-cpu chunk addresses, because +* they all inherit the tag from vms[0]->addr, even when nr_vms is bigger +* than 1. This is a problem because all the vms[]->addr come from separate +* allocations and have different tags so while the calculated address is +* correct the tag isn't. +*/ void __kasan_unpoison_vmap_areas (struct vm_struct **vms,int nr_vms) {int area;for (area =0 ; area < nr_vms ; area++) {kasan_poison (vms[area]->addr, vms[area]->size, -arch_kasan_get_tag (vms[area]->addr), false); +arch_kasan_get_tag (vms[0 ]->addr), false); +arch_kasan_set_tag (vms[area]->addr,arch_kasan_get_tag (vms[0 ]->addr)); } }
Signature · 1 line
2.50.1
Andrey Konovalov
reviewerRe: [PATCH v5 18/19] mm: Unpoison vms[area] addresses with a common tag
Quoted 54 lines · Maciej Wieczor-Retman
The problem presented here is related to NUMA systems and tag-based KASAN mode. It can be explained in the following points:
1. There can be more than one virtual memory chunk.
2. Chunk's base address has a tag.
3. The base address points at the first chunk and thus inherits the tag of the first chunk.
4. The subsequent chunks will be accessed with the tag from the first chunk.
5. Thus, the subsequent chunks need to have their tag set to match that of the first chunk.Unpoison all vms[]->addr memory and pointers with the same tag to resolve the mismatch.
Signed-off-by Maciej Wieczor-Retman <[email protected]>---
Changelog v4:
- Move tagging the vms[]->addr to this new patch and leave refactoring there.
- Comment the fix to provide some context.mm/kasan/shadow.c | 10 +++++++++- 1 file changed, 9 insertions(+), 1 deletion(-)mm/kasan/shadow.c +9
-2@@ -646,13 +646,21 @@ void __kasan_poison_vmalloc(const void *start, unsigned long size)kasan_poison (start, size,KASAN_VMALLOC_INVALID , false); } +/* +* A tag mismatch happens when calculating per-cpu chunk addresses, because +* they all inherit the tag from vms[0]->addr, even when nr_vms is bigger +* than 1. This is a problem because all the vms[]->addr come from separate +* allocations and have different tags so while the calculated address is +* correct the tag isn't. +*/ void __kasan_unpoison_vmap_areas (struct vm_struct **vms,int nr_vms) {int area;for (area =0 ; area < nr_vms ; area++) {kasan_poison (vms[area]->addr, vms[area]->size, -arch_kasan_get_tag (vms[area]->addr), false); +arch_kasan_get_tag (vms[0 ]->addr), false); +arch_kasan_set_tag (vms[area]->addr,arch_kasan_get_tag (vms[0 ]->addr)); } } --
Quoted 2 lines
2.50.1
Do we need this fix for the HW_TAGS mode too?
Re: [PATCH v5 18/19] mm: Unpoison vms[area] addresses with a common tag
Quoted 55 lines · Andrey Konovalov
Quoted 53 lines · Maciej Wieczor-Retman
The problem presented here is related to NUMA systems and tag-based KASAN mode. It can be explained in the following points:
1. There can be more than one virtual memory chunk.
2. Chunk's base address has a tag.
3. The base address points at the first chunk and thus inherits the tag of the first chunk.
4. The subsequent chunks will be accessed with the tag from the first chunk.
5. Thus, the subsequent chunks need to have their tag set to match that of the first chunk.Unpoison all vms[]->addr memory and pointers with the same tag to resolve the mismatch.
Signed-off-by Maciej Wieczor-Retman <[email protected]>---
Changelog v4:
- Move tagging the vms[]->addr to this new patch and leave refactoring there.
- Comment the fix to provide some context.mm/kasan/shadow.c | 10 +++++++++- 1 file changed, 9 insertions(+), 1 deletion(-)mm/kasan/shadow.c +9
-1@@ -646,13 +646,21 @@ void __kasan_poison_vmalloc(const void *start, unsigned long size)kasan_poison (start, size,KASAN_VMALLOC_INVALID , false); } +/* +* A tag mismatch happens when calculating per-cpu chunk addresses, because +* they all inherit the tag from vms[0]->addr, even when nr_vms is bigger +* than 1. This is a problem because all the vms[]->addr come from separate +* allocations and have different tags so while the calculated address is +* correct the tag isn't. +*/ void __kasan_unpoison_vmap_areas (struct vm_struct **vms,int nr_vms) {int area;for (area =0 ; area < nr_vms ; area++) {kasan_poison (vms[area]->addr, vms[area]->size, -arch_kasan_get_tag (vms[area]->addr), false); +arch_kasan_get_tag (vms[0 ]->addr), false); +arch_kasan_set_tag (vms[area]->addr,arch_kasan_get_tag (vms[0 ]->addr)); } }
Quoted 5 lines
Quoted 3 lines
--
2.50.1Do we need this fix for the
HW_TAGSmode too?
Oh, I suppose it could also affect the hardware mode since this is related to tagged pointers and NUMA nodes. I'll try to also make it work for HW_TAGS.
Signature · 2 lines
Kind regards Maciej Wieczór-Retman
Andrey Konovalov
reviewerRe: [PATCH v5 18/19] mm: Unpoison vms[area] addresses with a common tag
Quoted 5 lines · Maciej Wieczor-Retman
Quoted 2 lines
Do we need this fix for the
HW_TAGSmode too?Oh, I suppose it could also affect the hardware mode since this is related to tagged pointers and NUMA nodes. I'll try to also make it work for
HW_TAGS.
Ack. I suspect you won't need to provide two separate implementations for this then.
Also, could you split out this fix into a separate patch with the Fixes and CC stable tags (or put the fix first in the series)?
Re: [PATCH v5 18/19] mm: Unpoison vms[area] addresses with a common tag
Quoted 5 lines · Andrey Konovalov
Quoted 3 lines · Maciej Wieczor-Retman
Quoted 2 lines
Do we need this fix for the
HW_TAGSmode too?
Quoted 8 lines
Quoted 2 lines
Oh, I suppose it could also affect the hardware mode since this is related to tagged pointers and NUMA nodes. I'll try to also make it work for
HW_TAGS.Ack. I suspect you won't need to provide two separate implementations for this then.
Also, could you split out this fix into a separate patch with the Fixes and CC stable tags (or put the fix first in the series)?
Sure, I'll move it to the beginning of the series.
Signature · 2 lines
Kind regards Maciej Wieczór-Retman
PATCH v5 19/19 x86: Make software tag-based kasan available
In reply to Maciej Wieczor-RetmanMake CONFIG_KASAN_SW_TAGS available for x86 machines if they have ADDRESS_MASKING enabled (LAM) as that works similarly to Top-Byte Ignore (TBI) that allows the software tag-based mode on arm64 platform.
Set scale macro based on KASAN mode: in software tag-based mode 16 bytes of memory map to one shadow byte and 8 in generic mode.
Disable CONFIG_KASAN_INLINE and CONFIG_KASAN_STACK when CONFIG_KASAN_SW_TAGS is enabled on x86 until the appropriate compiler support is available.
---
Changelog v4:
- Add x86 specific kasan_mem_to_shadow().
- Revert x86 to the older unsigned KASAN_SHADOW_OFFSET. Do the same to KASAN_SHADOW_START/END.
- Modify scripts/gdb/linux/kasan.py to keep x86 using unsigned offset.
- Disable inline and stack support when software tags are enabled on x86.
Changelog v3:
- Remove runtime_const from previous patch and merge the rest here.
- Move scale shift definition back to header file.
- Add new kasan offset for software tag based mode.
- Fix patch message typo 32 -> 16, and 16 -> 8.
- Update lib/Kconfig.kasan with x86 now having software tag-based support.
Changelog v2:
- Remove KASAN dense code.
Documentation/arch/x86/x86_64/mm.rst | 6 ++++-- arch/x86/Kconfig | 4 +++- arch/x86/boot/compressed/misc.h | 1 + arch/x86/include/asm/kasan.h | 1 + arch/x86/kernel/setup.c | 2 ++ lib/Kconfig.kasan | 3 ++- scripts/gdb/linux/kasan.py | 4 ++-- 7 files changed, 15 insertions(+), 6 deletions(-)
Documentation/arch/x86/x86_64/mm.rst +4 -2
@@ -60,7 +60,8 @@ Complete virtual memory map with 4-level page tables ffffe90000000000 | -23 TB | ffffe9ffffffffff | 1 TB | ... unused hole ffffea0000000000 | -22 TB | ffffeaffffffffff | 1 TB | virtual memory map (vmemmap_base) ffffeb0000000000 | -21 TB | ffffebffffffffff | 1 TB | ... unused hole - ffffec0000000000 | -20 TB | fffffbffffffffff | 16 TB | KASAN shadow memory + ffffec0000000000 | -20 TB | fffffbffffffffff | 16 TB | KASAN shadow memory (generic mode) + fffff40000000000 | -8 TB | fffffbffffffffff | 8 TB | KASAN shadow memory (software tag-based mode) __________________|____________|__________________|_________|____________________________________________________________ | | Identical layout to the 56-bit one from here on:
@@ -130,7 +131,8 @@ Complete virtual memory map with 5-level page tables ffd2000000000000 | -11.5 PB | ffd3ffffffffffff | 0.5 PB | ... unused hole ffd4000000000000 | -11 PB | ffd5ffffffffffff | 0.5 PB | virtual memory map (vmemmap_base) ffd6000000000000 | -10.5 PB | ffdeffffffffffff | 2.25 PB | ... unused hole - ffdf000000000000 | -8.25 PB | fffffbffffffffff | ~8 PB | KASAN shadow memory + ffdf000000000000 | -8.25 PB | fffffbffffffffff | ~8 PB | KASAN shadow memory (generic mode) + ffeffc0000000000 | -6 PB | fffffbffffffffff | 4 PB | KASAN shadow memory (software tag-based mode) __________________|____________|__________________|_________|____________________________________________________________ | | Identical layout to the 47-bit one from here on:
arch/x86/Kconfig +3 -1
@@ -69,6 +69,7 @@ config X86 select ARCH_CLOCKSOURCE_INIT select ARCH_CONFIGURES_CPU_MITIGATIONS select ARCH_CORRECT_STACKTRACE_ON_KRETPROBE + select ARCH_DISABLE_KASAN_INLINE if X86_64 && KASAN_SW_TAGS select ARCH_ENABLE_HUGEPAGE_MIGRATION if X86_64 && HUGETLB_PAGE && MIGRATION select ARCH_ENABLE_MEMORY_HOTPLUG if X86_64 select ARCH_ENABLE_MEMORY_HOTREMOVE if MEMORY_HOTPLUG
@@ -199,6 +200,7 @@ config X86 select HAVE_ARCH_JUMP_LABEL_RELATIVE select HAVE_ARCH_KASAN if X86_64 select HAVE_ARCH_KASAN_VMALLOC if X86_64 + select HAVE_ARCH_KASAN_SW_TAGS if ADDRESS_MASKING select HAVE_ARCH_KFENCE select HAVE_ARCH_KMSAN if X86_64 select HAVE_ARCH_KGDB
@@ -403,7 +405,7 @@ config AUDIT_ARCH config KASAN_SHADOW_OFFSET hex - depends on KASAN + default 0xeffffc0000000000 if KASAN_SW_TAGS default 0xdffffc0000000000 config HAVE_INTEL_TXT
arch/x86/boot/compressed/misc.h +1 -0
@@ -13,6 +13,7 @@#undef CONFIG_PARAVIRT_SPINLOCKS#undef CONFIG_KASAN#undef CONFIG_KASAN_GENERIC +#undef CONFIG_KASAN_SW_TAGS#define __NO_FORTIFY
arch/x86/include/asm/kasan.h +1 -0
@@ -7,6 +7,7 @@#include <linux/types.h> #define KASAN_SHADOW_OFFSET _AC(CONFIG_KASAN_SHADOW_OFFSET, UL)#ifdef CONFIG_KASAN_SW_TAGS +#define KASAN_SHADOW_SCALE_SHIFT 4/* * LLVM ABI for reporting tag mismatches in inline KASAN mode.
arch/x86/kernel/setup.c +2 -0
@@ -1207,6 +1207,8 @@ void __init setup_arch(char **cmdline_p)kasan_init (); +kasan_init_sw_tags (); +/* * Sync back kernel address range. *
lib/Kconfig.kasan +2 -1
@@ -100,7 +100,8 @@ config KASAN_SW_TAGS Requires GCC 11+ or Clang. - Supported only on arm64 CPUs and relies on Top Byte Ignore. + Supported on arm64 CPUs that support Top Byte Ignore and on x86 CPUs + that support Linear Address Masking. Consumes about 1/16th of available memory at kernel start and add an overhead of ~20% for dynamic allocations.
scripts/gdb/linux/kasan.py +2 -2
@@ -7,7 +7,7 @@ # import gdb -from linux import constants, mm +from linux import constants, utils, mm from ctypes import c_int64 as s64 def help():
@@ -40,7 +40,7 @@ class KasanMemToShadow(gdb.Command): else: help() def kasan_mem_to_shadow(self, addr): - if constants.CONFIG_KASAN_SW_TAGS: + if constants.CONFIG_KASAN_SW_TAGS and not utils.is_target_arch('x86'): addr = s64(addr) return (addr >> self.p_ops.KASAN_SHADOW_SCALE_SHIFT) + self.p_ops.KASAN_SHADOW_OFFSET
Signature · 1 line
2.50.1
Andrey Konovalov
reviewerRe: [PATCH v5 19/19] x86: Make software tag-based kasan available
Quoted 71 lines · Maciej Wieczor-Retman
Make
CONFIG_KASAN_SW_TAGSavailable for x86 machines if they haveADDRESS_MASKINGenabled (LAM) as that works similarly to Top-Byte Ignore (TBI) that allows the software tag-based mode on arm64 platform.Set scale macro based on KASAN mode: in software tag-based mode 16 bytes of memory map to one shadow byte and 8 in generic mode.
Disable
CONFIG_KASAN_INLINEandCONFIG_KASAN_STACKwhenCONFIG_KASAN_SW_TAGSis enabled on x86 until the appropriate compiler support is available.Signed-off-by Maciej Wieczor-Retman <[email protected]>---
Changelog v4:
- Add x86 specifickasan_mem_to_shadow().
- Revert x86 to the older unsignedKASAN_SHADOW_OFFSET. Do the same to KASAN_SHADOW_START/END.
- Modifyscripts/gdb/linux/kasan.pyto keep x86 using unsigned offset.
- Disable inline and stack support when software tags are enabled on x86.Changelog v3:
- Removeruntime_constfrom previous patch and merge the rest here.
- Move scale shift definition back to header file.
- Add new kasan offset for software tag based mode.
- Fix patch message typo 32 -> 16, and 16 -> 8.
- Updatelib/Kconfig.kasanwith x86 now having software tag-based support.Changelog v2:
- Remove KASAN dense code.Documentation/arch/x86/x86_64/mm.rst | 6 ++++-- arch/x86/Kconfig | 4 +++- arch/x86/boot/compressed/misc.h | 1 + arch/x86/include/asm/kasan.h | 1 + arch/x86/kernel/setup.c | 2 ++ lib/Kconfig.kasan | 3 ++- scripts/gdb/linux/kasan.py | 4 ++-- 7 files changed, 15 insertions(+), 6 deletions(-)Documentation/arch/x86/x86_64/mm.rst +4
-2@@ -60,7 +60,8 @@ Complete virtual memory map with 4-level page tables ffffe90000000000 | -23 TB | ffffe9ffffffffff | 1 TB | ... unused hole ffffea0000000000 | -22 TB | ffffeaffffffffff | 1 TB | virtual memory map (vmemmap_base) ffffeb0000000000 | -21 TB | ffffebffffffffff | 1 TB | ... unused hole - ffffec0000000000 | -20 TB | fffffbffffffffff | 16 TB | KASAN shadow memory + ffffec0000000000 | -20 TB | fffffbffffffffff | 16 TB | KASAN shadow memory (generic mode) + fffff40000000000 | -8 TB | fffffbffffffffff | 8 TB | KASAN shadow memory (software tag-based mode) __________________|____________|__________________|_________|____________________________________________________________ | | Identical layout to the 56-bit one from here on:@@ -130,7 +131,8 @@ Complete virtual memory map with 5-level page tables ffd2000000000000 | -11.5 PB | ffd3ffffffffffff | 0.5 PB | ... unused hole ffd4000000000000 | -11 PB | ffd5ffffffffffff | 0.5 PB | virtual memory map (vmemmap_base) ffd6000000000000 | -10.5 PB | ffdeffffffffffff | 2.25 PB | ... unused hole - ffdf000000000000 | -8.25 PB | fffffbffffffffff | ~8 PB | KASAN shadow memory + ffdf000000000000 | -8.25 PB | fffffbffffffffff | ~8 PB | KASAN shadow memory (generic mode) + ffeffc0000000000 | -6 PB | fffffbffffffffff | 4 PB | KASAN shadow memory (software tag-based mode) __________________|____________|__________________|_________|____________________________________________________________ | | Identical layout to the 47-bit one from here on:arch/x86/Kconfig +0
-0
arch/x86/Kconfig config X86 · line 72
Quoted 4 lines
arch/x86/Kconfig +1
-0@@ -69,6 +69,7 @@ config X86 select ARCH_CLOCKSOURCE_INIT select ARCH_CONFIGURES_CPU_MITIGATIONS select ARCH_CORRECT_STACKTRACE_ON_KRETPROBE + select ARCH_DISABLE_KASAN_INLINE if X86_64 && KASAN_SW_TAGS
Do you think it would make sense to drop the parts of the series that add int3 handling, since the inline instrumentation does not work yet anyway?
Quoted 12 lines
arch/x86/Kconfig +1
-0select ARCH_ENABLE_HUGEPAGE_MIGRATION if X86_64 && HUGETLB_PAGE && MIGRATION select ARCH_ENABLE_MEMORY_HOTPLUG if X86_64 select ARCH_ENABLE_MEMORY_HOTREMOVE if MEMORY_HOTPLUG@@ -199,6 +200,7 @@ config X86 select HAVE_ARCH_JUMP_LABEL_RELATIVE select HAVE_ARCH_KASAN if X86_64 select HAVE_ARCH_KASAN_VMALLOC if X86_64 + select HAVE_ARCH_KASAN_SW_TAGS if ADDRESS_MASKING select HAVE_ARCH_KFENCE select HAVE_ARCH_KMSAN if X86_64 select HAVE_ARCH_KGDB
arch/x86/Kconfig config AUDIT_ARCH · line 406
Quoted 4 lines
arch/x86/Kconfig +0
-1@@ -403,7 +405,7 @@ config AUDIT_ARCH config KASAN_SHADOW_OFFSET hex - depends on KASAN
Line accidentally removed?
Quoted 69 lines
arch/x86/Kconfig +1
-0+ default 0xeffffc0000000000 if KASAN_SW_TAGS default 0xdffffc0000000000 config HAVE_INTEL_TXTarch/x86/boot/compressed/misc.h +1
-0@@ -13,6 +13,7 @@#undef CONFIG_PARAVIRT_SPINLOCKS#undef CONFIG_KASAN#undef CONFIG_KASAN_GENERIC +#undef CONFIG_KASAN_SW_TAGS#define __NO_FORTIFYarch/x86/include/asm/kasan.h +1
-0@@ -7,6 +7,7 @@#include <linux/types.h> #define KASAN_SHADOW_OFFSET _AC(CONFIG_KASAN_SHADOW_OFFSET, UL)#ifdef CONFIG_KASAN_SW_TAGS +#define KASAN_SHADOW_SCALE_SHIFT 4/* * LLVM ABI for reporting tag mismatches in inline KASAN mode. arch/x86/kernel/setup.c +2
-0@@ -1207,6 +1207,8 @@ void __init setup_arch(char **cmdline_p)kasan_init (); +kasan_init_sw_tags (); +/* * Sync back kernel address range. * lib/Kconfig.kasan +2
-1@@ -100,7 +100,8 @@ config KASAN_SW_TAGS Requires GCC 11+ or Clang. - Supported only on arm64 CPUs and relies on Top Byte Ignore. + Supported on arm64 CPUs that support Top Byte Ignore and on x86 CPUs + that support Linear Address Masking. Consumes about 1/16th of available memory at kernel start and add an overhead of ~20% for dynamic allocations.scripts/gdb/linux/kasan.py +1
-1@@ -7,7 +7,7 @@ # import gdb -from linux import constants, mm +from linux import constants, utils, mm from ctypes import c_int64 as s64 def help():
scripts/gdb/linux/kasan.py class KasanMemToShadow(gdb.Command): · line 43
Quoted 5 lines
scripts/gdb/linux/kasan.py +1
-1@@ -40,7 +40,7 @@ class KasanMemToShadow(gdb.Command): else: help() def kasan_mem_to_shadow(self, addr): - if constants.CONFIG_KASAN_SW_TAGS: + if constants.CONFIG_KASAN_SW_TAGS and not utils.is_target_arch('x86'):
This change seems to belong to the patch that changes how the shadow memory address is calculated.
Quoted 6 lines
scripts/gdb/linux/kasan.py +0
-1addr = s64(addr) return (addr >> self.p_ops.KASAN_SHADOW_SCALE_SHIFT) + self.p_ops.KASAN_SHADOW_OFFSET --2.50.1
Re: [PATCH v5 19/19] x86: Make software tag-based kasan available
Quoted 73 lines · Andrey Konovalov
Quoted 71 lines · Maciej Wieczor-Retman
Make
CONFIG_KASAN_SW_TAGSavailable for x86 machines if they haveADDRESS_MASKINGenabled (LAM) as that works similarly to Top-Byte Ignore (TBI) that allows the software tag-based mode on arm64 platform.Set scale macro based on KASAN mode: in software tag-based mode 16 bytes of memory map to one shadow byte and 8 in generic mode.
Disable
CONFIG_KASAN_INLINEandCONFIG_KASAN_STACKwhenCONFIG_KASAN_SW_TAGSis enabled on x86 until the appropriate compiler support is available.Signed-off-by Maciej Wieczor-Retman <[email protected]>---
Changelog v4:
- Add x86 specifickasan_mem_to_shadow().
- Revert x86 to the older unsignedKASAN_SHADOW_OFFSET. Do the same to KASAN_SHADOW_START/END.
- Modifyscripts/gdb/linux/kasan.pyto keep x86 using unsigned offset.
- Disable inline and stack support when software tags are enabled on x86.Changelog v3:
- Removeruntime_constfrom previous patch and merge the rest here.
- Move scale shift definition back to header file.
- Add new kasan offset for software tag based mode.
- Fix patch message typo 32 -> 16, and 16 -> 8.
- Updatelib/Kconfig.kasanwith x86 now having software tag-based support.Changelog v2:
- Remove KASAN dense code.Documentation/arch/x86/x86_64/mm.rst | 6 ++++-- arch/x86/Kconfig | 4 +++- arch/x86/boot/compressed/misc.h | 1 + arch/x86/include/asm/kasan.h | 1 + arch/x86/kernel/setup.c | 2 ++ lib/Kconfig.kasan | 3 ++- scripts/gdb/linux/kasan.py | 4 ++-- 7 files changed, 15 insertions(+), 6 deletions(-)Documentation/arch/x86/x86_64/mm.rst +4
-2@@ -60,7 +60,8 @@ Complete virtual memory map with 4-level page tables ffffe90000000000 | -23 TB | ffffe9ffffffffff | 1 TB | ... unused hole ffffea0000000000 | -22 TB | ffffeaffffffffff | 1 TB | virtual memory map (vmemmap_base) ffffeb0000000000 | -21 TB | ffffebffffffffff | 1 TB | ... unused hole - ffffec0000000000 | -20 TB | fffffbffffffffff | 16 TB | KASAN shadow memory + ffffec0000000000 | -20 TB | fffffbffffffffff | 16 TB | KASAN shadow memory (generic mode) + fffff40000000000 | -8 TB | fffffbffffffffff | 8 TB | KASAN shadow memory (software tag-based mode) __________________|____________|__________________|_________|____________________________________________________________ | | Identical layout to the 56-bit one from here on:@@ -130,7 +131,8 @@ Complete virtual memory map with 5-level page tables ffd2000000000000 | -11.5 PB | ffd3ffffffffffff | 0.5 PB | ... unused hole ffd4000000000000 | -11 PB | ffd5ffffffffffff | 0.5 PB | virtual memory map (vmemmap_base) ffd6000000000000 | -10.5 PB | ffdeffffffffffff | 2.25 PB | ... unused hole - ffdf000000000000 | -8.25 PB | fffffbffffffffff | ~8 PB | KASAN shadow memory + ffdf000000000000 | -8.25 PB | fffffbffffffffff | ~8 PB | KASAN shadow memory (generic mode) + ffeffc0000000000 | -6 PB | fffffbffffffffff | 4 PB | KASAN shadow memory (software tag-based mode) __________________|____________|__________________|_________|____________________________________________________________ | | Identical layout to the 47-bit one from here on:arch/x86/Kconfig +0
-0@@ -69,6 +69,7 @@ config X86
Quoted 8 lines
arch/x86/Kconfig config X86 · line 72
Quoted 4 lines
arch/x86/Kconfig +1
-0select ARCH_CLOCKSOURCE_INIT select ARCH_CONFIGURES_CPU_MITIGATIONS select ARCH_CORRECT_STACKTRACE_ON_KRETPROBE + select ARCH_DISABLE_KASAN_INLINE if X86_64 && KASAN_SW_TAGSDo you think it would make sense to drop the parts of the series that add int3 handling, since the inline instrumentation does not work yet anyway?
I thought we might as well put it into the kernel, so once the compiler side gets upstreamed only the Kconfig needs to be modified.
But both options are okay, I thought itd be easy to argument changes to LLVM if this inline mode is already prepared in the kernel.
Quoted 14 lines
Quoted 14 lines
arch/x86/Kconfig +1
-0select ARCH_ENABLE_HUGEPAGE_MIGRATION if X86_64 && HUGETLB_PAGE && MIGRATION select ARCH_ENABLE_MEMORY_HOTPLUG if X86_64 select ARCH_ENABLE_MEMORY_HOTREMOVE if MEMORY_HOTPLUG@@ -199,6 +200,7 @@ config X86 select HAVE_ARCH_JUMP_LABEL_RELATIVE select HAVE_ARCH_KASAN if X86_64 select HAVE_ARCH_KASAN_VMALLOC if X86_64 + select HAVE_ARCH_KASAN_SW_TAGS if ADDRESS_MASKING select HAVE_ARCH_KFENCE select HAVE_ARCH_KMSAN if X86_64 select HAVE_ARCH_KGDB@@ -403,7 +405,7 @@ config AUDIT_ARCH
Quoted 5 lines
arch/x86/Kconfig config AUDIT_ARCH · line 406
Quoted 3 lines
arch/x86/Kconfig +0
-1config KASAN_SHADOW_OFFSET hex - depends on KASANLine accidentally removed?
Yes, sorry, I'll put it back in.
Quoted 68 lines
Quoted 68 lines
arch/x86/Kconfig +1
-0+ default 0xeffffc0000000000 if KASAN_SW_TAGS default 0xdffffc0000000000 config HAVE_INTEL_TXTarch/x86/boot/compressed/misc.h +1
-0@@ -13,6 +13,7 @@#undef CONFIG_PARAVIRT_SPINLOCKS#undef CONFIG_KASAN#undef CONFIG_KASAN_GENERIC +#undef CONFIG_KASAN_SW_TAGS#define __NO_FORTIFYarch/x86/include/asm/kasan.h +1
-0@@ -7,6 +7,7 @@#include <linux/types.h> #define KASAN_SHADOW_OFFSET _AC(CONFIG_KASAN_SHADOW_OFFSET, UL)#ifdef CONFIG_KASAN_SW_TAGS +#define KASAN_SHADOW_SCALE_SHIFT 4/* * LLVM ABI for reporting tag mismatches in inline KASAN mode. arch/x86/kernel/setup.c +2
-0@@ -1207,6 +1207,8 @@ void __init setup_arch(char **cmdline_p)kasan_init (); +kasan_init_sw_tags (); +/* * Sync back kernel address range. * lib/Kconfig.kasan +2
-1@@ -100,7 +100,8 @@ config KASAN_SW_TAGS Requires GCC 11+ or Clang. - Supported only on arm64 CPUs and relies on Top Byte Ignore. + Supported on arm64 CPUs that support Top Byte Ignore and on x86 CPUs + that support Linear Address Masking. Consumes about 1/16th of available memory at kernel start and add an overhead of ~20% for dynamic allocations.scripts/gdb/linux/kasan.py +1
-1@@ -7,7 +7,7 @@ # import gdb -from linux import constants, mm +from linux import constants, utils, mm from ctypes import c_int64 as s64
Quoted 10 lines
scripts/gdb/linux/kasan.py class KasanMemToShadow(gdb.Command): · line 43
Quoted 7 lines
scripts/gdb/linux/kasan.py +1
-1def help():@@ -40,7 +40,7 @@ class KasanMemToShadow(gdb.Command): else: help() def kasan_mem_to_shadow(self, addr): - if constants.CONFIG_KASAN_SW_TAGS: + if constants.CONFIG_KASAN_SW_TAGS and not utils.is_target_arch('x86'):This change seems to belong to the patch that changes how the shadow memory address is calculated.
Okay, I can move it there.
Quoted 7 lines
Quoted 7 lines
scripts/gdb/linux/kasan.py +0
-1addr = s64(addr) return (addr >> self.p_ops.KASAN_SHADOW_SCALE_SHIFT) + self.p_ops.KASAN_SHADOW_OFFSET --2.50.1
Signature · 2 lines
Kind regards Maciej Wieczór-Retman