khwasan: kernel hardware assisted address sanitizer
Subsystems (13)
v6 cover khwasan: kernel hardware assisted address sanitizer
This patchset adds a new mode to KASAN [1], which is called KHWASAN (Kernel HardWare assisted Address SANitizer).
The plan is to implement HWASan [2] for the kernel with the incentive, that it's going to have comparable to KASAN performance, but in the same time consume much less memory, trading that off for somewhat imprecise bug detection and being supported only for arm64.
The overall idea of the approach used by KHWASAN is the following:
1. By using the Top Byte Ignore arm64 CPU feature, we can store pointer tags in the top byte of each kernel pointer.
2. Using shadow memory, we can store memory tags for each chunk of kernel memory.
3. On each memory allocation, we can generate a random tag, embed it into the returned pointer and set the memory tags that correspond to this chunk of memory to the same value.
4. By using compiler instrumentation, before each memory access we can add a check that the pointer tag matches the tag of the memory that is being accessed.
5. On a tag mismatch we report an error.
[1] https://www.kernel.org/doc/html/latest/dev-tools/kasan.html
[2] http://clang.llvm.org/docs/HardwareAssistedAddressSanitizerDesign.html
====== Rationale
On mobile devices KASAN's memory usage is significant problem. One of the main reasons to have KWHASAN is to be able to perform a similar set of checks that KASAN does, but with lower memory requirements.
Comment from Vishwath Mohan <[email protected]>:
I don't have data on-hand, but anecdotally both ASAN and KASAN have proven problematic to enable for environments that don't tolerate the increased memory pressure well. This includes,
(a) Low-memory form factors - Wear, TV, Things, lower-tier phones like Go,
(c) Connected components like Pixel's visual core [1].
These are both places I'd love to have a low(er) memory footprint option at my disposal.
Comment from Evgenii Stepanov <[email protected]>:
Looking at a live Android device under load, slab (according to /proc/meminfo) + kernel stack take 8-10% available RAM (~350MB). KASAN's overhead of 2x - 3x on top of it is not insignificant.
Not having this overhead enables near-production use - ex. running KASAN/KHWASAN kernel on a personal, daily-use device to catch bugs that do not reproduce in test configuration. These are the ones that often cost the most engineering time to track down.
CPU overhead is bad, but generally tolerable. RAM is critical, in our experience. Once it gets low enough, OOM-killer makes your life miserable.
====== Technical details
KHWASAN is implemented in a very similar way to KASAN. This patchset essentially does the following:
1. TCR_TBI1 is set to enable Top Byte Ignore.
2. Shadow memory is used (with a different scale, 1:16, so each shadow byte corresponds to 16 bytes of kernel memory) to store memory tags.
3. All slab objects are aligned to shadow scale, which is 16 bytes.
4. All pointers returned from the slab allocator are tagged with a random tag and the corresponding shadow memory is poisoned with the same value.
5. Compiler instrumentation is used to insert tag checks. Either by calling callbacks or by inlining them (CONFIG_KASAN_OUTLINE and CONFIG_KASAN_INLINE flags are reused).
6. When a tag mismatch is detected in callback instrumentation mode KHWASAN simply prints a bug report. In case of inline instrumentation, clang inserts a brk instruction, and KHWASAN has it's own brk handler, which reports the bug.
7. The memory in between slab objects is marked with a reserved tag, and acts as a redzone.
8. When a slab object is freed it's marked with a reserved tag.
Bug detection is imprecise for two reasons:
1. We won't catch some small out-of-bounds accesses, that fall into the same shadow cell, as the last byte of a slab object.
2. We only have 1 byte to store tags, which means we have a 1/256 probability of a tag match for an incorrect access (actually even slightly less due to reserved tag values).
Despite that there's a particular type of bugs that KHWASAN can detect compared to KASAN: use-after-free after the object has been allocated by someone else.
====== Testing
Some kernel developers voiced a concern that changing the top byte of kernel pointers may lead to subtle bugs that are difficult to discover. To address this concern deliberate testing has been performed.
It doesn't seem feasible to do some kind of static checking to find potential issues with pointer tagging, so a dynamic approach was taken. All pointer comparisons/subtractions have been instrumented in an LLVM compiler pass and a kernel module that would print a bug report whenever two pointers with different tags are being compared/subtracted (ignoring comparisons with NULL pointers and with pointers obtained by casting an error code to a pointer type) has been used. Then the kernel has been booted in QEMU and on an Odroid C2 board and syzkaller has been run.
This yielded the following results.
The two places that look interesting are:
is_vmalloc_addr in include/linux/mm.his_kernel_rodata in mm/util.c
Here we compare a pointer with some fixed untagged values to make sure that the pointer lies in a particular part of the kernel address space. Since KWHASAN doesn't add tags to pointers that belong to rodata or vmalloc regions, this should work as is. To make sure debug checks to those two functions that check that the result doesn't change whether we operate on pointers with or without untagging has been added.
A few other cases that don't look that interesting:
Comparing pointers to achieve unique sorting order of pointee objects (e.g. sorting locks addresses before performing a double lock):
tty_ldisc_lock_pair_timeout in drivers/tty/tty_ldisc.c pipe_double_lock in fs/pipe.c unix_state_double_lock in net/unix/af_unix.c lock_two_nondirectories in fs/inode.c mutex_lock_double in kernel/events/core.c
ep_cmp_ffd in fs/eventpoll.cfsnotify_compare_groups fs/notify/mark.c
Nothing needs to be done here, since the tags embedded into pointers don't change, so the sorting order would still be unique.
Checks that a pointer belongs to some particular allocation:
is_sibling_entry in lib/radix-tree.c object_is_on_stack in include/linux/sched/task_stack.h
Nothing needs to be done here either, since two pointers can only belong to the same allocation if they have the same tag.
Overall, since the kernel boots and works, there are no critical bugs. As for the rest, the traditional kernel testing way (use until fails) is the only one that looks feasible.
Another point here is that KWHASAN is available under a separate config option that needs to be deliberately enabled. Even though it might be used in a "near-production" environment to find bugs that are not found during fuzzing or running tests, it is still a debug tool.
====== Benchmarks
The following numbers were collected on Odroid C2 board. Both KASAN and KHWASAN were used in inline instrumentation mode.
Boot time [1]:
* ~1.7 sec for clean kernel
* ~5.0 sec for KASAN
* ~5.0 sec for KHWASAN
Network performance [2]:
* 8.33 Gbits/sec for clean kernel
* 3.17 Gbits/sec for KASAN
* 2.85 Gbits/sec for KHWASAN
Slab memory usage after boot [3]:
* ~40 kb for clean kernel
* ~105 kb (~260% overhead) for KASAN
* ~47 kb (~20% overhead) for KHWASAN
KASAN memory overhead consists of three main parts:
1. Increased slab memory usage due to redzones.
2. Shadow memory (the whole reserved once during boot).
3. Quaratine (grows gradually until some preset limit; the more the limit, the more the chance to detect a use-after-free).
Comparing KWHASAN vs KASAN for each of these points:
1. 20% vs 260% overhead.
2. 1/16th vs 1/8th of physical memory.
3. KHWASAN doesn't require quarantine.
[1] Time before the ext4 driver is initialized.
[2] Measured as iperf -s & iperf -c 127.0.0.1 -t 30.
[3] Measured as cat /proc/meminfo | grep Slab.
====== Some notes
A few notes:
1. The patchset can be found here:
https://github.com/xairy/kasan-prototype/tree/khwasan
2. Building requires a recent LLVM version (r330044 or later).
3. Stack instrumentation is not supported yet and will be added later.
====== Changes
Changes in v6:
- Rebased onto 050cdc6c (4.19-rc1+).
- Added notes regarding patchset testing into the cover letter.
Changes in v5:
- Rebased onto 1ffaddd029 (4.18-rc8).
- Preassign tags for objects from caches with constructors and SLAB_TYPESAFE_BY_RCU caches.
- Fix SLAB allocator support by untagging page->s_mem in kasan_poison_slab().
- Performed dynamic testing to find potential places where pointer tagging might result in bugs [1].
- Clarified and fixed memory usage benchmarks in the cover letter.
- Added a rationale for having KHWASAN to the cover letter.
Changes in v4:
- Fixed SPDX comment style in mm/kasan/kasan.h.
- Fixed mm/kasan/kasan.h changes being included in a wrong patch.
- Swapped "khwasan, arm64: fix up fault handling logic" and "khwasan: add tag related helper functions" patches order.
- Rebased onto 6f0d349d (4.18-rc2+).
Changes in v3:
- Minor documentation fixes.
- Fixed CFLAGS variable name in KASAN makefile.
- Added a "SPDX-License-Identifier: GPL-2.0" line to all source files under mm/kasan.
- Rebased onto 81e97f013 (4.18-rc1+).
Changes in v2:
- Changed kmalloc_large_node_hook to return tagged pointer instead of using an output argument.
- Fix checking whether -fsanitize=hwaddress is supported by the compiler.
- Removed duplication of -fno-builtin for KASAN and KHWASAN.
- Removed {} block for one line for_each_possible_cpu loop.
- Made set_track() static inline as it is used only in common.c.
- Moved optimal_redzone() to common.c.
- Fixed using tagged pointer for shadow calculation in kasan_unpoison_shadow().
- Restored setting cache->align in kasan_cache_create(), which was accidentally lost.
- Simplified __kasan_slab_free(), kasan_alloc_pages() and kasan_kmalloc().
- Removed tagging from kasan_kmalloc_large().
- Added page_kasan_tag_reset() to kasan_poison_slab() and removed !PageSlab() check from page_to_virt.
- Reset pointer tag in _virt_addr_is_linear.
- Set page tag for each page when multiple pages are allocated or freed.
- Added a comment as to why we ignore cma allocated pages.
Changes in v1:
- Rebased onto 4.17-rc4.
- Updated benchmarking stats.
- Documented compiler version requirements, memory usage and slowdown.
- Dropped kvm patches, as clang + arm64 + kvm is completely broken [1].
Changes in RFC v3:
- Renamed CONFIG_KASAN_CLASSIC and CONFIG_KASAN_TAGS to CONFIG_KASAN_GENERIC and CONFIG_KASAN_HW respectively.
- Switch to -fsanitize=kernel-hwaddress instead of -fsanitize=hwaddress.
- Removed unnecessary excessive shadow initialization.
- Removed khwasan_enabled flag (it?s not needed since KHWASAN is initialized before any slab caches are used).
- Split out kasan_report.c and khwasan_report.c from report.c.
- Moved more common KASAN and KHWASAN functions to common.c.
- Added tagging to pagealloc.
- Rebased onto 4.17-rc1.
- Temporarily dropped patch that adds kvm support (arm64 + kvm + clang combo is broken right now [2]).
Changes in RFC v2:
- Removed explicit casts to u8 * for kasan_mem_to_shadow() calls.
- Introduced KASAN_TCR_FLAGS for setting the TCR_TBI1 flag.
- Added a comment regarding the non-atomic RMW sequence in khwasan_random_tag().
- Made all tag related functions accept const void *.
- Untagged pointers in __kimg_to_phys, which is used by virt_to_phys.
- Untagged pointers in show_ptr in fault handling logic.
- Untagged pointers passed to KVM.
- Added two reserved tag values: 0xFF and 0xFE.
- Used the reserved tag 0xFF to disable validity checking (to resolve the issue with pointer tag being lost after page_address + kmap usage).
- Used the reserved tag 0xFE to mark redzones and freed objects.
- Added mnemonics for esr manipulation in KHWASAN brk handler.
- Added a comment about the -recover flag.
- Some minor cleanups and fixes.
- Rebased onto 3215b9d5 (4.16-rc6+).
- Tested on real hardware (Odroid C2 board).
- Added better benchmarks.
[1] https://lkml.org/lkml/2018/7/18/765
[2] https://lkml.org/lkml/2018/4/19/775
Andrey Konovalov (18):
khwasan, mm: change kasan hooks signatures
khwasan: move common kasan and khwasan code to common.c
khwasan: add CONFIG_KASAN_GENERIC and CONFIG_KASAN_HW khwasan, arm64: adjust shadow size for CONFIG_KASAN_HW
khwasan: initialize shadow to 0xff
khwasan, arm64: untag virt address in __kimg_to_phys and
_virt_addr_is_linear
khwasan: add tag related helper functions
khwasan: preassign tags to objects with ctors or SLAB_TYPESAFE_BY_RCU khwasan, arm64: fix up fault handling logic
khwasan, arm64: enable top byte ignore for the kernel khwasan, mm: perform untagged pointers comparison in krealloc
khwasan: split out kasan_report.c from report.c
khwasan: add bug reporting routines
khwasan: add hooks implementation
khwasan, arm64: add brk handler for inline instrumentation khwasan, mm, arm64: tag non slab memory allocated via pagealloc
khwasan: update kasan documentation
kasan: add SPDX-License-Identifier mark to source files
Documentation/dev-tools/kasan.rst | 213 ++++---- arch/arm64/Kconfig | 1 + arch/arm64/Makefile | 2 +- arch/arm64/include/asm/brk-imm.h | 2 + arch/arm64/include/asm/memory.h | 41 +- arch/arm64/include/asm/pgtable-hwdef.h | 1 + arch/arm64/kernel/traps.c | 69 ++- arch/arm64/mm/fault.c | 3 + arch/arm64/mm/kasan_init.c | 18 +- arch/arm64/mm/proc.S | 8 +- include/linux/compiler-clang.h | 3 +- include/linux/compiler-gcc.h | 4 + include/linux/compiler.h | 3 +- include/linux/kasan.h | 90 +++- include/linux/mm.h | 29 ++ include/linux/page-flags-layout.h | 10 + lib/Kconfig.kasan | 77 ++- mm/cma.c | 11 + mm/kasan/Makefile | 9 +- mm/kasan/common.c | 663 +++++++++++++++++++++++++ mm/kasan/kasan.c | 565 +-------------------- mm/kasan/kasan.h | 85 +++- mm/kasan/kasan_init.c | 1 + mm/kasan/kasan_report.c | 156 ++++++ mm/kasan/khwasan.c | 181 +++++++ mm/kasan/khwasan_report.c | 61 +++ mm/kasan/quarantine.c | 1 + mm/kasan/report.c | 272 +++------- mm/page_alloc.c | 1 + mm/slab.c | 18 +- mm/slab.h | 2 +- mm/slab_common.c | 6 +- mm/slub.c | 21 +- scripts/Makefile.kasan | 27 +- 34 files changed, 1734 insertions(+), 920 deletions(-) create mode 100644 mm/kasan/common.c create mode 100644 mm/kasan/kasan_report.c create mode 100644 mm/kasan/khwasan.c create mode 100644 mm/kasan/khwasan_report.c
Signature · 1 line
2.19.0.rc0.228.g281dcd1b4d0-goog
PATCH v6 1/18 khwasan, mm: change kasan hooks signatures
KHWASAN will change the value of the top byte of pointers returned from the kernel allocation functions (such as kmalloc). This patch updates KASAN hooks signatures and their usage in SLAB and SLUB code to reflect that.
--- include/linux/kasan.h | 34 +++++++++++++++++++++++----------- mm/kasan/kasan.c | 24 ++++++++++++++---------- mm/slab.c | 12 ++++++------ mm/slab.h | 2 +- mm/slab_common.c | 4 ++-- mm/slub.c | 15 +++++++-------- 6 files changed, 53 insertions(+), 38 deletions(-)
include/linux/kasan.h +23 -11
@@ -53,14 +53,14 @@ void kasan_unpoison_object_data(struct kmem_cache *cache, void *object);void kasan_poison_object_data (struct kmem_cache *cache,void *object);void kasan_init_slab_obj (struct kmem_cache *cache,const void *object); -void kasan_kmalloc_large (const void *ptr,size_t size,gfp_t flags); +void *kasan_kmalloc_large (const void *ptr,size_t size,gfp_t flags);void kasan_kfree_large (void *ptr,unsigned long ip);void kasan_poison_kfree (void *ptr,unsigned long ip); -void kasan_kmalloc (struct kmem_cache *s,const void *object,size_t size, +void *kasan_kmalloc (struct kmem_cache *s,const void *object,size_t size,gfp_t flags); -void kasan_krealloc (const void *object,size_t new_size,gfp_t flags); +void *kasan_krealloc (const void *object,size_t new_size,gfp_t flags); -void kasan_slab_alloc (struct kmem_cache *s,void *object,gfp_t flags); +void *kasan_slab_alloc (struct kmem_cache *s,void *object,gfp_t flags);bool kasan_slab_free (struct kmem_cache *s,void *object,unsigned long ip);struct kasan_cache {
@@ -108,16 +108,28 @@ static inline void kasan_poison_object_data(struct kmem_cache *cache,static inline void kasan_init_slab_obj (struct kmem_cache *cache,const void *object) {} -static inline void kasan_kmalloc_large (void *ptr,size_t size,gfp_t flags) {} +static inline void *kasan_kmalloc_large (void *ptr,size_t size,gfp_t flags) +{ +return ptr; +}static inline void kasan_kfree_large (void *ptr,unsigned long ip) {}static inline void kasan_poison_kfree (void *ptr,unsigned long ip) {} -static inline void kasan_kmalloc (struct kmem_cache *s,const void *object, -size_t size,gfp_t flags) {} -static inline void kasan_krealloc (const void *object,size_t new_size, -gfp_t flags) {} +static inline void *kasan_kmalloc (struct kmem_cache *s,const void *object, +size_t size,gfp_t flags) +{ +return (void *)object; +} +static inline void *kasan_krealloc (const void *object,size_t new_size, +gfp_t flags) +{ +return (void *)object; +} -static inline void kasan_slab_alloc (struct kmem_cache *s,void *object, -gfp_t flags) {} +static inline void *kasan_slab_alloc (struct kmem_cache *s,void *object, +gfp_t flags) +{ +return object; +}static inline bool kasan_slab_free (struct kmem_cache *s,void *object,unsigned long ip) {
mm/kasan/kasan.c +14 -10
@@ -485,9 +485,9 @@ void kasan_init_slab_obj(struct kmem_cache *cache, const void *object)__memset (alloc_info,0 ,sizeof (*alloc_info)); } -void kasan_slab_alloc (struct kmem_cache *cache,void *object,gfp_t flags) +void *kasan_slab_alloc (struct kmem_cache *cache,void *object,gfp_t flags) { -kasan_kmalloc (cache, object, cache->object_size, flags); +return kasan_kmalloc (cache, object, cache->object_size, flags); } static bool __kasan_slab_free(struct kmem_cache *cache, void *object,
@@ -528,7 +528,7 @@ bool kasan_slab_free(struct kmem_cache *cache, void *object, unsigned long ip)return __kasan_slab_free (cache, object, ip, true); } -void kasan_kmalloc (struct kmem_cache *cache,const void *object,size_t size, +void *kasan_kmalloc (struct kmem_cache *cache,const void *object,size_t size,gfp_t flags) {unsigned long redzone_start;
@@ -538,7 +538,7 @@ void kasan_kmalloc(struct kmem_cache *cache, const void *object, size_t size,quarantine_reduce ();if (unlikely (object ==NULL )) -return ; +return NULL ; redzone_start =round_up ((unsigned long )(object + size),KASAN_SHADOW_SCALE_SIZE );
@@ -551,10 +551,12 @@ void kasan_kmalloc(struct kmem_cache *cache, const void *object, size_t size,if (cache->flags &SLAB_KASAN )set_track (&get_alloc_info (cache, object)->alloc_track, flags); + +return (void *)object; }EXPORT_SYMBOL (kasan_kmalloc); -void kasan_kmalloc_large (const void *ptr,size_t size,gfp_t flags) +void *kasan_kmalloc_large (const void *ptr,size_t size,gfp_t flags) {struct page *page;unsigned long redzone_start;
@@ -564,7 +566,7 @@ void kasan_kmalloc_large(const void *ptr, size_t size, gfp_t flags)quarantine_reduce ();if (unlikely (ptr ==NULL )) -return ; +return NULL ; page =virt_to_page (ptr); redzone_start = round_up((unsigned long)(ptr + size),
@@ -574,21 +576,23 @@ void kasan_kmalloc_large(const void *ptr, size_t size, gfp_t flags)kasan_unpoison_shadow (ptr, size);kasan_poison_shadow ((void *)redzone_start, redzone_end - redzone_start,KASAN_PAGE_REDZONE ); + +return (void *)ptr; } -void kasan_krealloc (const void *object,size_t size,gfp_t flags) +void *kasan_krealloc (const void *object,size_t size,gfp_t flags) {struct page *page;if (unlikely (object ==ZERO_SIZE_PTR )) -return ; +return ZERO_SIZE_PTR ; page =virt_to_head_page (object);if (unlikely (!PageSlab (page))) -kasan_kmalloc_large (object, size, flags); +return kasan_kmalloc_large (object, size, flags);else -kasan_kmalloc (page->slab_cache, object, size, flags); +return kasan_kmalloc (page->slab_cache, object, size, flags); }void kasan_poison_kfree (void *ptr,unsigned long ip)
mm/slab.c +6 -6
@@ -3551,7 +3551,7 @@ void *kmem_cache_alloc(struct kmem_cache *cachep, gfp_t flags) {void *ret =slab_alloc (cachep, flags, _RET_IP_); -kasan_slab_alloc (cachep, ret, flags); + ret =kasan_slab_alloc (cachep, ret, flags);trace_kmem_cache_alloc (_RET_IP_, ret, cachep->object_size, cachep->size, flags);
@@ -3617,7 +3617,7 @@ kmem_cache_alloc_trace(struct kmem_cache *cachep, gfp_t flags, size_t size) ret =slab_alloc (cachep, flags, _RET_IP_); -kasan_kmalloc (cachep, ret, size, flags); + ret =kasan_kmalloc (cachep, ret, size, flags);trace_kmalloc (_RET_IP_, ret, size, cachep->size, flags);return ret;
@@ -3641,7 +3641,7 @@ void *kmem_cache_alloc_node(struct kmem_cache *cachep, gfp_t flags, int nodeid) {void *ret =slab_alloc_node (cachep, flags, nodeid, _RET_IP_); -kasan_slab_alloc (cachep, ret, flags); + ret =kasan_slab_alloc (cachep, ret, flags);trace_kmem_cache_alloc_node (_RET_IP_, ret, cachep->object_size, cachep->size, flags, nodeid);
@@ -3660,7 +3660,7 @@ void *kmem_cache_alloc_node_trace(struct kmem_cache *cachep, ret =slab_alloc_node (cachep, flags, nodeid, _RET_IP_); -kasan_kmalloc (cachep, ret, size, flags); + ret =kasan_kmalloc (cachep, ret, size, flags);trace_kmalloc_node (_RET_IP_, ret, size, cachep->size, flags, nodeid);
@@ -3679,7 +3679,7 @@ __do_kmalloc_node(size_t size, gfp_t flags, int node, unsigned long caller)if (unlikely (ZERO_OR_NULL_PTR (cachep)))return cachep; ret =kmem_cache_alloc_node_trace (cachep, flags, node, size); -kasan_kmalloc (cachep, ret, size, flags); + ret =kasan_kmalloc (cachep, ret, size, flags);return ret; }
@@ -3715,7 +3715,7 @@ static __always_inline void *__do_kmalloc(size_t size, gfp_t flags,return cachep; ret =slab_alloc (cachep, flags, caller); -kasan_kmalloc (cachep, ret, size, flags); + ret =kasan_kmalloc (cachep, ret, size, flags);trace_kmalloc (caller, ret, size, cachep->size, flags);
mm/slab.h +1 -1
@@ -441,7 +441,7 @@ static inline void slab_post_alloc_hook(struct kmem_cache *s, gfp_t flags,kmemleak_alloc_recursive (object, s->object_size,1 , s->flags, flags); -kasan_slab_alloc (s, object, flags); + p[i] =kasan_slab_alloc (s, object, flags); }if (memcg_kmem_enabled ())
mm/slab_common.c +2 -2
@@ -1183,7 +1183,7 @@ void *kmalloc_order(size_t size, gfp_t flags, unsigned int order) page =alloc_pages (flags, order); ret = page ?page_address (page) :NULL ;kmemleak_alloc (ret, size,1 , flags); -kasan_kmalloc_large (ret, size, flags); + ret =kasan_kmalloc_large (ret, size, flags);return ret; }EXPORT_SYMBOL (kmalloc_order);
@@ -1461,7 +1461,7 @@ static __always_inline void *__do_krealloc(const void *p, size_t new_size, ks =ksize (p);if (ks >= new_size) { -kasan_krealloc ((void *)p, new_size, flags); + p =kasan_krealloc ((void *)p, new_size, flags);return (void *)p; }
mm/slub.c +7 -8
@@ -1335,10 +1335,10 @@ static inline void dec_slabs_node(struct kmem_cache *s, int node,* Hooks for other subsystems that check memory allocations. In a typical * production configuration these hooks all should produce no code at all. */ -static inline void kmalloc_large_node_hook (void *ptr,size_t size,gfp_t flags) +static inline void *kmalloc_large_node_hook (void *ptr,size_t size,gfp_t flags) {kmemleak_alloc (ptr, size,1 , flags); -kasan_kmalloc_large (ptr, size, flags); +return kasan_kmalloc_large (ptr, size, flags); } static __always_inline void kfree_hook(void *x)
@@ -2731,7 +2731,7 @@ void *kmem_cache_alloc_trace(struct kmem_cache *s, gfp_t gfpflags, size_t size) {void *ret =slab_alloc (s, gfpflags, _RET_IP_);trace_kmalloc (_RET_IP_, ret, size, s->size, gfpflags); -kasan_kmalloc (s, ret, size, gfpflags); + ret =kasan_kmalloc (s, ret, size, gfpflags);return ret; }EXPORT_SYMBOL (kmem_cache_alloc_trace);
@@ -2759,7 +2759,7 @@ void *kmem_cache_alloc_node_trace(struct kmem_cache *s,trace_kmalloc_node (_RET_IP_, ret, size, s->size, gfpflags, node); -kasan_kmalloc (s, ret, size, gfpflags); + ret =kasan_kmalloc (s, ret, size, gfpflags);return ret; }EXPORT_SYMBOL (kmem_cache_alloc_node_trace);
@@ -3749,7 +3749,7 @@ void *__kmalloc(size_t size, gfp_t flags)trace_kmalloc (_RET_IP_, ret, size, s->size, flags); -kasan_kmalloc (s, ret, size, flags); + ret =kasan_kmalloc (s, ret, size, flags);return ret; }
@@ -3766,8 +3766,7 @@ static void *kmalloc_large_node(size_t size, gfp_t flags, int node)if (page) ptr =page_address (page); -kmalloc_large_node_hook (ptr, size, flags); -return ptr; +return kmalloc_large_node_hook (ptr, size, flags); }void *__kmalloc_node (size_t size,gfp_t flags,int node)
@@ -3794,7 +3793,7 @@ void *__kmalloc_node(size_t size, gfp_t flags, int node)trace_kmalloc_node (_RET_IP_, ret, size, s->size, flags, node); -kasan_kmalloc (s, ret, size, flags); + ret =kasan_kmalloc (s, ret, size, flags);return ret; }
Signature · 1 line
2.19.0.rc0.228.g281dcd1b4d0-goog
PATCH v6 2/18 khwasan: move common kasan and khwasan code to common.c
In reply to Andrey KonovalovKHWASAN will reuse a significant part of KASAN code, so move the common parts to common.c without any functional changes.
--- mm/kasan/Makefile | 5 +- mm/kasan/common.c | 604 ++++++++++++++++++++++++++++++++++++++++++++++ mm/kasan/kasan.c | 568 +------------------------------------------ mm/kasan/kasan.h | 5 + 4 files changed, 615 insertions(+), 567 deletions(-) create mode 100644 mm/kasan/common.c
mm/kasan/Makefile +4 -1
@@ -1,11 +1,14 @@ # SPDX-License-Identifier: GPL-2.0 KASAN_SANITIZE := n +UBSAN_SANITIZE_common.o := n UBSAN_SANITIZE_kasan.o := n KCOV_INSTRUMENT := n CFLAGS_REMOVE_kasan.o = -pg # Function splitter causes unnecessary splits in __asan_load1/__asan_store1 # see: https://gcc.gnu.org/bugzilla/show_bug.cgi?id=63533 + +CFLAGS_common.o := $(call cc-option, -fno-conserve-stack -fno-stack-protector) CFLAGS_kasan.o := $(call cc-option, -fno-conserve-stack -fno-stack-protector) -obj-y := kasan.o report.o kasan_init.o quarantine.o +obj-y := common.o kasan.o report.o kasan_init.o quarantine.o
mm/kasan/common.c new file +604 -0
604 lines not on this page · raw message
mm/kasan/kasan.c +2 -566
598 lines not on this page · raw message
mm/kasan/kasan.h +5 -0
@@ -105,6 +105,11 @@ static inline const void *kasan_shadow_to_mem(const void *shadow_addr) << KASAN_SHADOW_SCALE_SHIFT); } +void kasan_poison_shadow (const void *address,size_t size,u8 value); + +void check_memory_region (unsigned long addr,size_t size,bool write, +unsigned long ret_ip); +void kasan_report (unsigned long addr,size_t size,bool is_write,unsigned long ip);void kasan_report_invalid_free (void *object,unsigned long ip);
Signature · 1 line
2.19.0.rc0.228.g281dcd1b4d0-goog
PATCH v6 3/18 khwasan: add CONFIG_KASAN_GENERIC and CONFIG_KASAN_HW
In reply to Andrey KonovalovThis commit splits the current CONFIG_KASAN config option into two:
1. CONFIG_KASAN_GENERIC, that enables the generic software-only KASAN version (the one that exists now);
2. CONFIG_KASAN_HW, that enables KHWASAN.
With CONFIG_KASAN_HW enabled, compiler options are changed to instrument kernel files wiht -fsantize=hwaddress (except the ones for which KASAN_SANITIZE := n is set).
Both CONFIG_KASAN_GENERIC and CONFIG_KASAN_HW support both CONFIG_KASAN_INLINE and CONFIG_KASAN_OUTLINE instrumentation modes.
This commit also adds empty placeholder (for now) implementation of KHWASAN specific hooks inserted by the compiler and adjusts common hooks implementation to compile correctly with each of the config options.
--- arch/arm64/Kconfig | 1 + include/linux/compiler-clang.h | 3 +- include/linux/compiler-gcc.h | 4 ++ include/linux/compiler.h | 3 +- include/linux/kasan.h | 16 +++++-- lib/Kconfig.kasan | 77 ++++++++++++++++++++++++++-------- mm/kasan/Makefile | 6 ++- mm/kasan/kasan.h | 3 +- mm/kasan/khwasan.c | 75 +++++++++++++++++++++++++++++++++ mm/slub.c | 2 +- scripts/Makefile.kasan | 27 +++++++++++- 11 files changed, 188 insertions(+), 29 deletions(-) create mode 100644 mm/kasan/khwasan.c
arch/arm64/Kconfig +1 -0
@@ -105,6 +105,7 @@ config ARM64 select HAVE_ARCH_HUGE_VMAP select HAVE_ARCH_JUMP_LABEL select HAVE_ARCH_KASAN if !(ARM64_16K_PAGES && ARM64_VA_BITS_48) + select HAVE_ARCH_KASAN_HW if !(ARM64_16K_PAGES && ARM64_VA_BITS_48) select HAVE_ARCH_KGDB select HAVE_ARCH_MMAP_RND_BITS select HAVE_ARCH_MMAP_RND_COMPAT_BITS if COMPAT
include/linux/compiler-clang.h +2 -1
@@ -17,11 +17,12 @@#define KASAN_ABI_VERSION 5/* emulate gcc's __SANITIZE_ADDRESS__ flag */ -#if __has_feature (address_sanitizer) +#if __has_feature (address_sanitizer) ||__has_feature (hwaddress_sanitizer)#define __SANITIZE_ADDRESS__#endif #define __no_sanitize_address __attribute__((no_sanitize("address"))) +#define __no_sanitize_hwaddress __attribute__((no_sanitize("hwaddress")))/* * Not all versions of clang implement the the type-generic versions
include/linux/compiler-gcc.h +4 -0
@@ -227,6 +227,10 @@#define __no_sanitize_address#endif +#if !defined(__no_sanitize_hwaddress) +#define __no_sanitize_hwaddress/* gcc doesn't support KHWASAN */ +#endif +/* * Turn individual warnings and errors on and off locally, depending * on version.
include/linux/compiler.h +2 -1
@@ -195,7 +195,8 @@ void __read_once_size(const volatile void *p, void *res, int size)* https://gcc.gnu.org/bugzilla/show_bug.cgi?id=67368 * '__maybe_unused' allows us to avoid defined-but-not-used warnings. */ -# define __no_kasan_or_inline __no_sanitize_address __maybe_unused +# define __no_kasan_or_inline __no_sanitize_address __no_sanitize_hwaddress \ + __maybe_unused#else # define __no_kasan_or_inline __always_inline#endif
include/linux/kasan.h +12 -4
@@ -45,8 +45,6 @@ void kasan_free_pages(struct page *page, unsigned int order);void kasan_cache_create (struct kmem_cache *cache,unsigned int *size,slab_flags_t *flags); -void kasan_cache_shrink (struct kmem_cache *cache); -void kasan_cache_shutdown (struct kmem_cache *cache);void kasan_poison_slab (struct page *page);void kasan_unpoison_object_data (struct kmem_cache *cache,void *object);
@@ -97,8 +95,6 @@ static inline void kasan_free_pages(struct page *page, unsigned int order) {}static inline void kasan_cache_create (struct kmem_cache *cache,unsigned int *size,slab_flags_t *flags) {} -static inline void kasan_cache_shrink (struct kmem_cache *cache) {} -static inline void kasan_cache_shutdown (struct kmem_cache *cache) {}static inline void kasan_poison_slab (struct page *page) {} static inline void kasan_unpoison_object_data(struct kmem_cache *cache,
@@ -152,4 +148,16 @@ static inline size_t kasan_metadata_size(struct kmem_cache *cache) { return 0; }#endif /* CONFIG_KASAN */ +#ifdef CONFIG_KASAN_GENERIC + +void kasan_cache_shrink (struct kmem_cache *cache); +void kasan_cache_shutdown (struct kmem_cache *cache); + +#else /* CONFIG_KASAN_GENERIC */ + +static inline void kasan_cache_shrink (struct kmem_cache *cache) {} +static inline void kasan_cache_shutdown (struct kmem_cache *cache) {} + +#endif /* CONFIG_KASAN_GENERIC */ +#endif /* LINUX_KASAN_H */
lib/Kconfig.kasan +59 -18
109 lines not on this page · raw message
mm/kasan/Makefile +5 -1
@@ -2,6 +2,7 @@ KASAN_SANITIZE := n UBSAN_SANITIZE_common.o := n UBSAN_SANITIZE_kasan.o := n +UBSAN_SANITIZE_khwasan.o := n KCOV_INSTRUMENT := n CFLAGS_REMOVE_kasan.o = -pg
@@ -10,5 +11,8 @@ CFLAGS_REMOVE_kasan.o = -pg CFLAGS_common.o := $(call cc-option, -fno-conserve-stack -fno-stack-protector) CFLAGS_kasan.o := $(call cc-option, -fno-conserve-stack -fno-stack-protector) +CFLAGS_khwasan.o := $(call cc-option, -fno-conserve-stack -fno-stack-protector) -obj-y := common.o kasan.o report.o kasan_init.o quarantine.o +obj-$(CONFIG_KASAN) := common.o kasan_init.o report.o +obj-$(CONFIG_KASAN_GENERIC) += kasan.o quarantine.o +obj-$(CONFIG_KASAN_HW) += khwasan.o
mm/kasan/kasan.h +2 -1
@@ -114,7 +114,8 @@ void kasan_report(unsigned long addr, size_t size, bool is_write, unsigned long ip);void kasan_report_invalid_free (void *object,unsigned long ip); -#if defined(CONFIG_SLAB ) || defined(CONFIG_SLUB ) +#if defined(CONFIG_KASAN_GENERIC ) && \ + (defined(CONFIG_SLAB ) || defined(CONFIG_SLUB ))void quarantine_put (struct kasan_free_meta *info,struct kmem_cache *cache);void quarantine_reduce (void );void quarantine_remove_cache (struct kmem_cache *cache);
mm/kasan/khwasan.c new file +75 -0
75 lines not on this page · raw message
mm/slub.c +1 -1
@@ -2955,7 +2955,7 @@ static __always_inline void slab_free(struct kmem_cache *s, struct page *page,do_slab_free (s, page, head, tail, cnt, addr); } -#ifdef CONFIG_KASAN +#ifdef CONFIG_KASAN_GENERIC void ___cache_free (struct kmem_cache *cache,void *x,unsigned long addr) {do_slab_free (cache,virt_to_head_page (x), x,NULL ,1 , addr);
scripts/Makefile.kasan +25 -2
@@ -1,5 +1,5 @@ # SPDX-License-Identifier: GPL-2.0 -ifdef CONFIG_KASAN +ifdef CONFIG_KASAN_GENERIC ifdef CONFIG_KASAN_INLINE call_threshold := 10000 else
@@ -42,6 +42,29 @@ ifdef CONFIG_KASAN_EXTRA CFLAGS_KASAN += $(call cc-option, -fsanitize-address-use-after-scope) endif -CFLAGS_KASAN_NOSANITIZE := -fno-builtin +endif + +ifdef CONFIG_KASAN_HW + +ifdef CONFIG_KASAN_INLINE + instrumentation_flags := -mllvm -hwasan-mapping-offset=$(KASAN_SHADOW_OFFSET) +else + instrumentation_flags := -mllvm -hwasan-instrument-with-calls=1 +endif +CFLAGS_KASAN := -fsanitize=kernel-hwaddress \ + -mllvm -hwasan-instrument-stack=0 \ + $(instrumentation_flags) + +ifeq ($(call cc-option, $(CFLAGS_KASAN) -Werror),) + ifneq ($(CONFIG_COMPILE_TEST),y) + $(warning Cannot use CONFIG_KASAN_HW: \ + -fsanitize=hwaddress is not supported by compiler) + endif +endif + +endif + +ifdef CONFIG_KASAN +CFLAGS_KASAN_NOSANITIZE := -fno-builtin endif
Signature · 1 line
2.19.0.rc0.228.g281dcd1b4d0-goog
Dmitry Vyukov
reviewerRe: [PATCH v6 03/18] khwasan: add CONFIG_KASAN_GENERIC and CONFIG_KASAN_HW
Quoted 49 lines · Andrey Konovalov
This commit splits the current
CONFIG_KASANconfig option into two:
1.CONFIG_KASAN_GENERIC, that enables the generic software-only KASAN version (the one that exists now);
2.CONFIG_KASAN_HW, that enables KHWASAN.With
CONFIG_KASAN_HWenabled, compiler options are changed to instrument kernel files wiht -fsantize=hwaddress (except the ones for whichKASAN_SANITIZE:= n is set).Both
CONFIG_KASAN_GENERICandCONFIG_KASAN_HWsupport bothCONFIG_KASAN_INLINEandCONFIG_KASAN_OUTLINEinstrumentation modes.This commit also adds empty placeholder (for now) implementation of KHWASAN specific hooks inserted by the compiler and adjusts common hooks implementation to compile correctly with each of the config options.
Signed-off-by Andrey Konovalov <[email protected]>--- arch/arm64/Kconfig | 1 + include/linux/compiler-clang.h | 3 +- include/linux/compiler-gcc.h | 4 ++ include/linux/compiler.h | 3 +- include/linux/kasan.h | 16 +++++-- lib/Kconfig.kasan | 77 ++++++++++++++++++++++++++-------- mm/kasan/Makefile | 6 ++- mm/kasan/kasan.h | 3 +- mm/kasan/khwasan.c | 75 +++++++++++++++++++++++++++++++++ mm/slub.c | 2 +- scripts/Makefile.kasan | 27 +++++++++++- 11 files changed, 188 insertions(+), 29 deletions(-) create mode 100644 mm/kasan/khwasan.carch/arm64/Kconfig +1
-0@@ -105,6 +105,7 @@ config ARM64 select HAVE_ARCH_HUGE_VMAP select HAVE_ARCH_JUMP_LABEL select HAVE_ARCH_KASAN if !(ARM64_16K_PAGES && ARM64_VA_BITS_48) + select HAVE_ARCH_KASAN_HW if !(ARM64_16K_PAGES && ARM64_VA_BITS_48) select HAVE_ARCH_KGDB select HAVE_ARCH_MMAP_RND_BITS select HAVE_ARCH_MMAP_RND_COMPAT_BITS if COMPATinclude/linux/compiler-clang.h +0
-0
include/linux/compiler-clang.h line 25
Quoted 10 lines
include/linux/compiler-clang.h +2
-1@@ -17,11 +17,12 @@#define KASAN_ABI_VERSION 5/* emulate gcc's __SANITIZE_ADDRESS__ flag */ -#if __has_feature (address_sanitizer) +#if __has_feature (address_sanitizer) ||__has_feature (hwaddress_sanitizer)#define __SANITIZE_ADDRESS__#endif #define __no_sanitize_address __attribute__((no_sanitize("address"))) +#define __no_sanitize_hwaddress __attribute__((no_sanitize("hwaddress")))
It seems that it would be better to have just 1 attribute for both types. Currently __no_sanitize_address is used just in a single place. But if it ever used more, people will need to always spell both which looks unnecessary, or, worse will only fix asan but forget about khwasan.
If we do just:
#define __no_sanitize_address __attribute__((no_sanitize("address", "hwaddress")))
Then we don't need any changes in compiler-gcc.h nor in compiler.h, and no chance or forgetting one of them.
Quoted 79 lines
include/linux/compiler-clang.h +0
-0/* * Not all versions of clang implement the the type-generic versions include/linux/compiler-gcc.h +4
-0@@ -227,6 +227,10 @@#define __no_sanitize_address#endif +#if !defined(__no_sanitize_hwaddress) +#define __no_sanitize_hwaddress/* gcc doesn't support KHWASAN */ +#endif +/* * Turn individual warnings and errors on and off locally, depending * on version. include/linux/compiler.h +2
-1@@ -195,7 +195,8 @@ void __read_once_size(const volatile void *p, void *res, int size)* https://gcc.gnu.org/bugzilla/show_bug.cgi?id=67368 * '__maybe_unused' allows us to avoid defined-but-not-used warnings. */ -# define __no_kasan_or_inline __no_sanitize_address __maybe_unused +# define __no_kasan_or_inline __no_sanitize_address __no_sanitize_hwaddress \ + __maybe_unused#else # define __no_kasan_or_inline __always_inline#endif include/linux/kasan.h +12
-4@@ -45,8 +45,6 @@ void kasan_free_pages(struct page *page, unsigned int order);void kasan_cache_create (struct kmem_cache *cache,unsigned int *size,slab_flags_t *flags); -void kasan_cache_shrink (struct kmem_cache *cache); -void kasan_cache_shutdown (struct kmem_cache *cache);void kasan_poison_slab (struct page *page);void kasan_unpoison_object_data (struct kmem_cache *cache,void *object);@@ -97,8 +95,6 @@ static inline void kasan_free_pages(struct page *page, unsigned int order) {}static inline void kasan_cache_create (struct kmem_cache *cache,unsigned int *size,slab_flags_t *flags) {} -static inline void kasan_cache_shrink (struct kmem_cache *cache) {} -static inline void kasan_cache_shutdown (struct kmem_cache *cache) {}static inline void kasan_poison_slab (struct page *page) {} static inline void kasan_unpoison_object_data(struct kmem_cache *cache,@@ -152,4 +148,16 @@ static inline size_t kasan_metadata_size(struct kmem_cache *cache) { return 0; }#endif /* CONFIG_KASAN */ +#ifdef CONFIG_KASAN_GENERIC + +void kasan_cache_shrink (struct kmem_cache *cache); +void kasan_cache_shutdown (struct kmem_cache *cache); + +#else /* CONFIG_KASAN_GENERIC */ + +static inline void kasan_cache_shrink (struct kmem_cache *cache) {} +static inline void kasan_cache_shutdown (struct kmem_cache *cache) {} + +#endif /* CONFIG_KASAN_GENERIC */ +#endif /* LINUX_KASAN_H */ lib/Kconfig.kasan +1
-0@@ -1,34 +1,75 @@ config HAVE_ARCH_KASAN bool +config HAVE_ARCH_KASAN_HW
lib/Kconfig.kasan line 13
Quoted 10 lines
lib/Kconfig.kasan +6
-1+ bool + if HAVE_ARCH_KASAN config KASAN - bool "KASan: runtime memory debugger" + bool "KASAN: runtime memory debugger" + help + Enables KASAN (KernelAddressSANitizer) - runtime memory debugger, + designed to find out-of-bounds accesses and use-after-free bugs.
Perhaps also give link to Documentation/dev-tools/kasan.rst while we are here.
lib/Kconfig.kasan line 20
Quoted 7 lines
lib/Kconfig.kasan +7
-0+ +choice + prompt "KASAN mode" + depends on KASAN + default KASAN_GENERIC + help + KASAN has two modes: KASAN (a classic version, similar to userspace
In these few sentences we call the old mode with 3 different terms:
"generic", "classic" and "KASAN" :)
This is somewhat confusing. Let's call it "generic" throughout (here and in the docs patch). "Generic" as in "supported on multiple arch and not-dependent on hardware features". "Classic" makes sense for people who knew KASAN before, but for future readers in won't make sense.
Quoted 21 lines
lib/Kconfig.kasan +9
-7+ ASan, enabled with CONFIG_KASAN_GENERIC) and KHWASAN (a version + based on pointer tagging, only for arm64, similar to userspace + HWASan, enabled with CONFIG_KASAN_HW). + +config KASAN_GENERIC + bool "KASAN: the generic mode" depends on (SLUB && SYSFS) || (SLAB && !DEBUG_SLAB) select SLUB_DEBUG if SLUB select CONSTRUCTORS select STACKDEPOT help - Enables kernel address sanitizer - runtime memory debugger, - designed to find out-of-bounds accesses and use-after-free bugs. - This is strictly a debugging feature and it requires a gcc version - of 4.9.2 or later. Detection of out of bounds accesses to stack or - global variables requires gcc 5.0 or later. - This feature consumes about 1/8 of available memory and brings about - ~x3 performance slowdown. + Enables the generic mode of KASAN. + This is strictly a debugging feature and it requires a GCC version + of 4.9.2 or later. Detection of out-of-bounds accesses to stack or
lib/Kconfig.kasan line 43
Quoted 10 lines
lib/Kconfig.kasan +6
-1+ global variables requires GCC 5.0 or later. + This mode consumes about 1/8 of available memory at kernel start + and introduces an overhead of ~x1.5 for the rest of the allocations. + The performance slowdown is ~x3. For better error detection enable CONFIG_STACKTRACE. - Currently CONFIG_KASAN doesn't work with CONFIG_DEBUG_SLAB + Currently CONFIG_KASAN_GENERIC doesn't work with CONFIG_DEBUG_SLAB (the resulting kernel does not boot). +if HAVE_ARCH_KASAN_HW
This choice looks somewhat weird on non-arm64. It's kinda a choice menu, but one can't really choose anything. Should we put the whole choice under HAVE_ARCH_KASAN_HW, and just select KASAN_GENERIC otherwise? I don't know what't the practice here. Andrey R?
Quoted 232 lines
lib/Kconfig.kasan +29
-9+config KASAN_HW + bool "KHWASAN: the hardware assisted mode" + depends on (SLUB && SYSFS) || (SLAB && !DEBUG_SLAB) + select SLUB_DEBUG if SLUB + select CONSTRUCTORS + select STACKDEPOT + help + Enabled KHWASAN (KASAN mode based on pointer tagging). + This mode requires Top Byte Ignore support by the CPU and therefore + only supported for arm64. + This feature requires clang revision 330044 or later. + This mode consumes about 1/16 of available memory at kernel start + and introduces an overhead of ~20% for the rest of the allocations. + For better error detection enable CONFIG_STACKTRACE. + Currently CONFIG_KASAN_HW doesn't work with CONFIG_DEBUG_SLAB + (the resulting kernel does not boot). + +endif + +endchoice + config KASAN_EXTRA - bool "KAsan: extra checks" - depends on KASAN && DEBUG_KERNEL && !COMPILE_TEST + bool "KASAN: extra checks" + depends on KASAN_GENERIC && DEBUG_KERNEL && !COMPILE_TEST help - This enables further checks in the kernel address sanitizer, for now - it only includes the address-use-after-scope check that can lead - to excessive kernel stack usage, frame size warnings and longer - compile time. + This enables further checks in KASAN, for now it only includes the + address-use-after-scope check that can lead to excessive kernel + stack usage, frame size warnings and longer compile time. https://gcc.gnu.org/bugzilla/show_bug.cgi?id=81715 has more@@ -53,16 +94,16 @@ config KASAN_INLINE memory accesses. This is faster than outline (in some workloads it gives about x2 boost over outline instrumentation), but make kernel's .text size much bigger. - This requires a gcc version of 5.0 or later. + For CONFIG_KASAN_GENERIC this requires GCC 5.0 or later. endchoice config TEST_KASAN - tristate "Module for testing kasan for bug detection" + tristate "Module for testing KASAN for bug detection" depends on m && KASAN help This is a test module doing various nasty things like out of bounds accesses, use after free. It is useful for testing - kernel debugging features like kernel address sanitizer. + kernel debugging features like KASAN. endifmm/kasan/Makefile +5
-1@@ -2,6 +2,7 @@ KASAN_SANITIZE := n UBSAN_SANITIZE_common.o := n UBSAN_SANITIZE_kasan.o := n +UBSAN_SANITIZE_khwasan.o := n KCOV_INSTRUMENT := n CFLAGS_REMOVE_kasan.o = -pg@@ -10,5 +11,8 @@ CFLAGS_REMOVE_kasan.o = -pg CFLAGS_common.o := $(call cc-option, -fno-conserve-stack -fno-stack-protector) CFLAGS_kasan.o := $(call cc-option, -fno-conserve-stack -fno-stack-protector) +CFLAGS_khwasan.o := $(call cc-option, -fno-conserve-stack -fno-stack-protector) -obj-y := common.o kasan.o report.o kasan_init.o quarantine.o +obj-$(CONFIG_KASAN) := common.o kasan_init.o report.o +obj-$(CONFIG_KASAN_GENERIC) += kasan.o quarantine.o +obj-$(CONFIG_KASAN_HW) += khwasan.omm/kasan/kasan.h +2
-1@@ -114,7 +114,8 @@ void kasan_report(unsigned long addr, size_t size, bool is_write, unsigned long ip);void kasan_report_invalid_free (void *object,unsigned long ip); -#if defined(CONFIG_SLAB ) || defined(CONFIG_SLUB ) +#if defined(CONFIG_KASAN_GENERIC ) && \ + (defined(CONFIG_SLAB ) || defined(CONFIG_SLUB ))void quarantine_put (struct kasan_free_meta *info,struct kmem_cache *cache);void quarantine_reduce (void );void quarantine_remove_cache (struct kmem_cache *cache);mm/kasan/khwasan.c new file +75
-0@@ -0,0 +1,75 @@ +/* +* This file contains core KHWASAN code. +* +* Copyright (c) 2018 Google, Inc. +* Author: Andrey Konovalov <[email protected]> +* +* This program is free software; you can redistribute it and/or modify +* it under the terms of the GNU General Public License version 2 as +* published by the Free Software Foundation. +* +*/ + +#define pr_fmt (fmt) KBUILD_MODNAME ": " fmt +#define DISABLE_BRANCH_PROFILING + +#include <linux/export.h> +#include <linux/interrupt.h> +#include <linux/init.h> +#include <linux/kasan.h> +#include <linux/kernel.h> +#include <linux/kmemleak.h> +#include <linux/linkage.h> +#include <linux/memblock.h> +#include <linux/memory.h> +#include <linux/mm.h> +#include <linux/module.h> +#include <linux/printk.h> +#include <linux/random.h> +#include <linux/sched.h> +#include <linux/sched/task_stack.h> +#include <linux/slab.h> +#include <linux/stacktrace.h> +#include <linux/string.h> +#include <linux/types.h> +#include <linux/vmalloc.h> +#include <linux/bug.h> + +#include "kasan.h" +#include "../slab.h" + +void check_memory_region (unsigned long addr,size_t size,bool write, +unsigned long ret_ip) +{ +} + +#define DEFINE_HWASAN_LOAD_STORE (size) \ + void __hwasan_load##size##_noabort(unsigned long addr) \ + { \ + } \ + EXPORT_SYMBOL(__hwasan_load##size##_noabort); \ + void __hwasan_store##size##_noabort(unsigned long addr) \ + { \ + } \ + EXPORT_SYMBOL(__hwasan_store##size##_noabort) + +DEFINE_HWASAN_LOAD_STORE (1 ); +DEFINE_HWASAN_LOAD_STORE (2 ); +DEFINE_HWASAN_LOAD_STORE (4 ); +DEFINE_HWASAN_LOAD_STORE (8 ); +DEFINE_HWASAN_LOAD_STORE (16 ); + +void __hwasan_loadN_noabort (unsigned long addr,unsigned long size) +{ +} +EXPORT_SYMBOL (__hwasan_loadN_noabort); + +void __hwasan_storeN_noabort (unsigned long addr,unsigned long size) +{ +} +EXPORT_SYMBOL (__hwasan_storeN_noabort); + +void __hwasan_tag_memory (unsigned long addr,u8 tag,unsigned long size) +{ +} +EXPORT_SYMBOL (__hwasan_tag_memory);mm/slub.c +1
-1@@ -2955,7 +2955,7 @@ static __always_inline void slab_free(struct kmem_cache *s, struct page *page,do_slab_free (s, page, head, tail, cnt, addr); } -#ifdef CONFIG_KASAN +#ifdef CONFIG_KASAN_GENERIC void ___cache_free (struct kmem_cache *cache,void *x,unsigned long addr) {do_slab_free (cache,virt_to_head_page (x), x,NULL ,1 , addr);scripts/Makefile.kasan +25
-3@@ -1,5 +1,5 @@ # SPDX-License-Identifier: GPL-2.0 -ifdef CONFIG_KASAN +ifdef CONFIG_KASAN_GENERIC ifdef CONFIG_KASAN_INLINE call_threshold := 10000 else@@ -42,6 +42,29 @@ ifdef CONFIG_KASAN_EXTRA CFLAGS_KASAN += $(call cc-option, -fsanitize-address-use-after-scope) endif -CFLAGS_KASAN_NOSANITIZE := -fno-builtin +endif + +ifdef CONFIG_KASAN_HW + +ifdef CONFIG_KASAN_INLINE + instrumentation_flags := -mllvm -hwasan-mapping-offset=$(KASAN_SHADOW_OFFSET) +else + instrumentation_flags := -mllvm -hwasan-instrument-with-calls=1 +endif +CFLAGS_KASAN := -fsanitize=kernel-hwaddress \ + -mllvm -hwasan-instrument-stack=0 \ + $(instrumentation_flags) + +ifeq ($(call cc-option, $(CFLAGS_KASAN) -Werror),) + ifneq ($(CONFIG_COMPILE_TEST),y) + $(warning Cannot use CONFIG_KASAN_HW: \ + -fsanitize=hwaddress is not supported by compiler) + endif +endif + +endif + +ifdef CONFIG_KASAN +CFLAGS_KASAN_NOSANITIZE := -fno-builtin endif --2.19.0.rc0.228.g281dcd1b4d0-goog
Dmitry Vyukov
reviewerRe: [PATCH v6 03/18] khwasan: add CONFIG_KASAN_GENERIC and CONFIG_KASAN_HW
Quoted 213 lines · Dmitry Vyukov
Quoted 49 lines · Andrey Konovalov
This commit splits the current
CONFIG_KASANconfig option into two:
1.CONFIG_KASAN_GENERIC, that enables the generic software-only KASAN version (the one that exists now);
2.CONFIG_KASAN_HW, that enables KHWASAN.With
CONFIG_KASAN_HWenabled, compiler options are changed to instrument kernel files wiht -fsantize=hwaddress (except the ones for whichKASAN_SANITIZE:= n is set).Both
CONFIG_KASAN_GENERICandCONFIG_KASAN_HWsupport bothCONFIG_KASAN_INLINEandCONFIG_KASAN_OUTLINEinstrumentation modes.This commit also adds empty placeholder (for now) implementation of KHWASAN specific hooks inserted by the compiler and adjusts common hooks implementation to compile correctly with each of the config options.
Signed-off-by Andrey Konovalov <[email protected]>--- arch/arm64/Kconfig | 1 + include/linux/compiler-clang.h | 3 +- include/linux/compiler-gcc.h | 4 ++ include/linux/compiler.h | 3 +- include/linux/kasan.h | 16 +++++-- lib/Kconfig.kasan | 77 ++++++++++++++++++++++++++-------- mm/kasan/Makefile | 6 ++- mm/kasan/kasan.h | 3 +- mm/kasan/khwasan.c | 75 +++++++++++++++++++++++++++++++++ mm/slub.c | 2 +- scripts/Makefile.kasan | 27 +++++++++++- 11 files changed, 188 insertions(+), 29 deletions(-) create mode 100644 mm/kasan/khwasan.carch/arm64/Kconfig +1
-0@@ -105,6 +105,7 @@ config ARM64 select HAVE_ARCH_HUGE_VMAP select HAVE_ARCH_JUMP_LABEL select HAVE_ARCH_KASAN if !(ARM64_16K_PAGES && ARM64_VA_BITS_48) + select HAVE_ARCH_KASAN_HW if !(ARM64_16K_PAGES && ARM64_VA_BITS_48) select HAVE_ARCH_KGDB select HAVE_ARCH_MMAP_RND_BITS select HAVE_ARCH_MMAP_RND_COMPAT_BITS if COMPATinclude/linux/compiler-clang.h +0
-0include/linux/compiler-clang.h line 25
Quoted 10 lines
include/linux/compiler-clang.h +2
-1@@ -17,11 +17,12 @@#define KASAN_ABI_VERSION 5/* emulate gcc's __SANITIZE_ADDRESS__ flag */ -#if __has_feature (address_sanitizer) +#if __has_feature (address_sanitizer) ||__has_feature (hwaddress_sanitizer)#define __SANITIZE_ADDRESS__#endif #define __no_sanitize_address __attribute__((no_sanitize("address"))) +#define __no_sanitize_hwaddress __attribute__((no_sanitize("hwaddress")))It seems that it would be better to have just 1 attribute for both types. Currently
__no_sanitize_addressis used just in a single place. But if it ever used more, people will need to always spell both which looks unnecessary, or, worse will only fix asan but forget about khwasan.If we do just:
#define
__no_sanitize_address__attribute__((no_sanitize("address", "hwaddress")))Then we don't need any changes in
compiler-gcc.hnor incompiler.h, and no chance or forgetting one of them.Quoted 79 lines
include/linux/compiler-clang.h +0
-0/* * Not all versions of clang implement the the type-generic versions include/linux/compiler-gcc.h +4
-0@@ -227,6 +227,10 @@#define __no_sanitize_address#endif +#if !defined(__no_sanitize_hwaddress) +#define __no_sanitize_hwaddress/* gcc doesn't support KHWASAN */ +#endif +/* * Turn individual warnings and errors on and off locally, depending * on version. include/linux/compiler.h +2
-1@@ -195,7 +195,8 @@ void __read_once_size(const volatile void *p, void *res, int size)* https://gcc.gnu.org/bugzilla/show_bug.cgi?id=67368 * '__maybe_unused' allows us to avoid defined-but-not-used warnings. */ -# define __no_kasan_or_inline __no_sanitize_address __maybe_unused +# define __no_kasan_or_inline __no_sanitize_address __no_sanitize_hwaddress \ + __maybe_unused#else # define __no_kasan_or_inline __always_inline#endif include/linux/kasan.h +12
-4@@ -45,8 +45,6 @@ void kasan_free_pages(struct page *page, unsigned int order);void kasan_cache_create (struct kmem_cache *cache,unsigned int *size,slab_flags_t *flags); -void kasan_cache_shrink (struct kmem_cache *cache); -void kasan_cache_shutdown (struct kmem_cache *cache);void kasan_poison_slab (struct page *page);void kasan_unpoison_object_data (struct kmem_cache *cache,void *object);@@ -97,8 +95,6 @@ static inline void kasan_free_pages(struct page *page, unsigned int order) {}static inline void kasan_cache_create (struct kmem_cache *cache,unsigned int *size,slab_flags_t *flags) {} -static inline void kasan_cache_shrink (struct kmem_cache *cache) {} -static inline void kasan_cache_shutdown (struct kmem_cache *cache) {}static inline void kasan_poison_slab (struct page *page) {} static inline void kasan_unpoison_object_data(struct kmem_cache *cache,@@ -152,4 +148,16 @@ static inline size_t kasan_metadata_size(struct kmem_cache *cache) { return 0; }#endif /* CONFIG_KASAN */ +#ifdef CONFIG_KASAN_GENERIC + +void kasan_cache_shrink (struct kmem_cache *cache); +void kasan_cache_shutdown (struct kmem_cache *cache); + +#else /* CONFIG_KASAN_GENERIC */ + +static inline void kasan_cache_shrink (struct kmem_cache *cache) {} +static inline void kasan_cache_shutdown (struct kmem_cache *cache) {} + +#endif /* CONFIG_KASAN_GENERIC */ +#endif /* LINUX_KASAN_H */ lib/Kconfig.kasan +1
-0@@ -1,34 +1,75 @@ config HAVE_ARCH_KASAN bool +config HAVE_ARCH_KASAN_HWlib/Kconfig.kasan line 13
Quoted 10 lines
lib/Kconfig.kasan +6
-1+ bool + if HAVE_ARCH_KASAN config KASAN - bool "KASan: runtime memory debugger" + bool "KASAN: runtime memory debugger" + help + Enables KASAN (KernelAddressSANitizer) - runtime memory debugger, + designed to find out-of-bounds accesses and use-after-free bugs.Perhaps also give link to
Documentation/dev-tools/kasan.rstwhile we are here.lib/Kconfig.kasan line 20
Quoted 7 lines
lib/Kconfig.kasan +7
-0+ +choice + prompt "KASAN mode" + depends on KASAN + default KASAN_GENERIC + help + KASAN has two modes: KASAN (a classic version, similar to userspaceIn these few sentences we call the old mode with 3 different terms:
"generic", "classic" and "KASAN" :)
This is somewhat confusing. Let's call it "generic" throughout (here and in the docs patch). "Generic" as in "supported on multiple arch and not-dependent on hardware features". "Classic" makes sense for people who knew KASAN before, but for future readers in won't make sense.Quoted 30 lines
lib/Kconfig.kasan +14
-8+ ASan, enabled with CONFIG_KASAN_GENERIC) and KHWASAN (a version + based on pointer tagging, only for arm64, similar to userspace + HWASan, enabled with CONFIG_KASAN_HW). + +config KASAN_GENERIC + bool "KASAN: the generic mode" depends on (SLUB && SYSFS) || (SLAB && !DEBUG_SLAB) select SLUB_DEBUG if SLUB select CONSTRUCTORS select STACKDEPOT help - Enables kernel address sanitizer - runtime memory debugger, - designed to find out-of-bounds accesses and use-after-free bugs. - This is strictly a debugging feature and it requires a gcc version - of 4.9.2 or later. Detection of out of bounds accesses to stack or - global variables requires gcc 5.0 or later. - This feature consumes about 1/8 of available memory and brings about - ~x3 performance slowdown. + Enables the generic mode of KASAN. + This is strictly a debugging feature and it requires a GCC version + of 4.9.2 or later. Detection of out-of-bounds accesses to stack or + global variables requires GCC 5.0 or later. + This mode consumes about 1/8 of available memory at kernel start + and introduces an overhead of ~x1.5 for the rest of the allocations. + The performance slowdown is ~x3. For better error detection enable CONFIG_STACKTRACE. - Currently CONFIG_KASAN doesn't work with CONFIG_DEBUG_SLAB + Currently CONFIG_KASAN_GENERIC doesn't work with CONFIG_DEBUG_SLAB (the resulting kernel does not boot).
Quoted 9 lines
lib/Kconfig.kasan line 43
Quoted 1 line
lib/Kconfig.kasan +1
-0+if HAVE_ARCH_KASAN_HWThis choice looks somewhat weird on non-arm64. It's kinda a choice menu, but one can't really choose anything. Should we put the whole choice under
HAVE_ARCH_KASAN_HW, and just selectKASAN_GENERICotherwise? I don't know what't the practice here. Andrey R?Quoted 2 lines
lib/Kconfig.kasan +2
-0+config KASAN_HW + bool "KHWASAN: the hardware assisted mode"
Do we need a hyphen here? hardware-assisted?
Quoted 230 lines
Quoted 230 lines
lib/Kconfig.kasan +27
-9+ depends on (SLUB && SYSFS) || (SLAB && !DEBUG_SLAB) + select SLUB_DEBUG if SLUB + select CONSTRUCTORS + select STACKDEPOT + help + Enabled KHWASAN (KASAN mode based on pointer tagging). + This mode requires Top Byte Ignore support by the CPU and therefore + only supported for arm64. + This feature requires clang revision 330044 or later. + This mode consumes about 1/16 of available memory at kernel start + and introduces an overhead of ~20% for the rest of the allocations. + For better error detection enable CONFIG_STACKTRACE. + Currently CONFIG_KASAN_HW doesn't work with CONFIG_DEBUG_SLAB + (the resulting kernel does not boot). + +endif + +endchoice + config KASAN_EXTRA - bool "KAsan: extra checks" - depends on KASAN && DEBUG_KERNEL && !COMPILE_TEST + bool "KASAN: extra checks" + depends on KASAN_GENERIC && DEBUG_KERNEL && !COMPILE_TEST help - This enables further checks in the kernel address sanitizer, for now - it only includes the address-use-after-scope check that can lead - to excessive kernel stack usage, frame size warnings and longer - compile time. + This enables further checks in KASAN, for now it only includes the + address-use-after-scope check that can lead to excessive kernel + stack usage, frame size warnings and longer compile time. https://gcc.gnu.org/bugzilla/show_bug.cgi?id=81715 has more@@ -53,16 +94,16 @@ config KASAN_INLINE memory accesses. This is faster than outline (in some workloads it gives about x2 boost over outline instrumentation), but make kernel's .text size much bigger. - This requires a gcc version of 5.0 or later. + For CONFIG_KASAN_GENERIC this requires GCC 5.0 or later. endchoice config TEST_KASAN - tristate "Module for testing kasan for bug detection" + tristate "Module for testing KASAN for bug detection" depends on m && KASAN help This is a test module doing various nasty things like out of bounds accesses, use after free. It is useful for testing - kernel debugging features like kernel address sanitizer. + kernel debugging features like KASAN. endifmm/kasan/Makefile +5
-1@@ -2,6 +2,7 @@ KASAN_SANITIZE := n UBSAN_SANITIZE_common.o := n UBSAN_SANITIZE_kasan.o := n +UBSAN_SANITIZE_khwasan.o := n KCOV_INSTRUMENT := n CFLAGS_REMOVE_kasan.o = -pg@@ -10,5 +11,8 @@ CFLAGS_REMOVE_kasan.o = -pg CFLAGS_common.o := $(call cc-option, -fno-conserve-stack -fno-stack-protector) CFLAGS_kasan.o := $(call cc-option, -fno-conserve-stack -fno-stack-protector) +CFLAGS_khwasan.o := $(call cc-option, -fno-conserve-stack -fno-stack-protector) -obj-y := common.o kasan.o report.o kasan_init.o quarantine.o +obj-$(CONFIG_KASAN) := common.o kasan_init.o report.o +obj-$(CONFIG_KASAN_GENERIC) += kasan.o quarantine.o +obj-$(CONFIG_KASAN_HW) += khwasan.omm/kasan/kasan.h +2
-1@@ -114,7 +114,8 @@ void kasan_report(unsigned long addr, size_t size, bool is_write, unsigned long ip);void kasan_report_invalid_free (void *object,unsigned long ip); -#if defined(CONFIG_SLAB ) || defined(CONFIG_SLUB ) +#if defined(CONFIG_KASAN_GENERIC ) && \ + (defined(CONFIG_SLAB ) || defined(CONFIG_SLUB ))void quarantine_put (struct kasan_free_meta *info,struct kmem_cache *cache);void quarantine_reduce (void );void quarantine_remove_cache (struct kmem_cache *cache);mm/kasan/khwasan.c new file +75
-0@@ -0,0 +1,75 @@ +/* +* This file contains core KHWASAN code. +* +* Copyright (c) 2018 Google, Inc. +* Author: Andrey Konovalov <[email protected]> +* +* This program is free software; you can redistribute it and/or modify +* it under the terms of the GNU General Public License version 2 as +* published by the Free Software Foundation. +* +*/ + +#define pr_fmt (fmt) KBUILD_MODNAME ": " fmt +#define DISABLE_BRANCH_PROFILING + +#include <linux/export.h> +#include <linux/interrupt.h> +#include <linux/init.h> +#include <linux/kasan.h> +#include <linux/kernel.h> +#include <linux/kmemleak.h> +#include <linux/linkage.h> +#include <linux/memblock.h> +#include <linux/memory.h> +#include <linux/mm.h> +#include <linux/module.h> +#include <linux/printk.h> +#include <linux/random.h> +#include <linux/sched.h> +#include <linux/sched/task_stack.h> +#include <linux/slab.h> +#include <linux/stacktrace.h> +#include <linux/string.h> +#include <linux/types.h> +#include <linux/vmalloc.h> +#include <linux/bug.h> + +#include "kasan.h" +#include "../slab.h" + +void check_memory_region (unsigned long addr,size_t size,bool write, +unsigned long ret_ip) +{ +} + +#define DEFINE_HWASAN_LOAD_STORE (size) \ + void __hwasan_load##size##_noabort(unsigned long addr) \ + { \ + } \ + EXPORT_SYMBOL(__hwasan_load##size##_noabort); \ + void __hwasan_store##size##_noabort(unsigned long addr) \ + { \ + } \ + EXPORT_SYMBOL(__hwasan_store##size##_noabort) + +DEFINE_HWASAN_LOAD_STORE (1 ); +DEFINE_HWASAN_LOAD_STORE (2 ); +DEFINE_HWASAN_LOAD_STORE (4 ); +DEFINE_HWASAN_LOAD_STORE (8 ); +DEFINE_HWASAN_LOAD_STORE (16 ); + +void __hwasan_loadN_noabort (unsigned long addr,unsigned long size) +{ +} +EXPORT_SYMBOL (__hwasan_loadN_noabort); + +void __hwasan_storeN_noabort (unsigned long addr,unsigned long size) +{ +} +EXPORT_SYMBOL (__hwasan_storeN_noabort); + +void __hwasan_tag_memory (unsigned long addr,u8 tag,unsigned long size) +{ +} +EXPORT_SYMBOL (__hwasan_tag_memory);mm/slub.c +1
-1@@ -2955,7 +2955,7 @@ static __always_inline void slab_free(struct kmem_cache *s, struct page *page,do_slab_free (s, page, head, tail, cnt, addr); } -#ifdef CONFIG_KASAN +#ifdef CONFIG_KASAN_GENERIC void ___cache_free (struct kmem_cache *cache,void *x,unsigned long addr) {do_slab_free (cache,virt_to_head_page (x), x,NULL ,1 , addr);scripts/Makefile.kasan +25
-3@@ -1,5 +1,5 @@ # SPDX-License-Identifier: GPL-2.0 -ifdef CONFIG_KASAN +ifdef CONFIG_KASAN_GENERIC ifdef CONFIG_KASAN_INLINE call_threshold := 10000 else@@ -42,6 +42,29 @@ ifdef CONFIG_KASAN_EXTRA CFLAGS_KASAN += $(call cc-option, -fsanitize-address-use-after-scope) endif -CFLAGS_KASAN_NOSANITIZE := -fno-builtin +endif + +ifdef CONFIG_KASAN_HW + +ifdef CONFIG_KASAN_INLINE + instrumentation_flags := -mllvm -hwasan-mapping-offset=$(KASAN_SHADOW_OFFSET) +else + instrumentation_flags := -mllvm -hwasan-instrument-with-calls=1 +endif +CFLAGS_KASAN := -fsanitize=kernel-hwaddress \ + -mllvm -hwasan-instrument-stack=0 \ + $(instrumentation_flags) + +ifeq ($(call cc-option, $(CFLAGS_KASAN) -Werror),) + ifneq ($(CONFIG_COMPILE_TEST),y) + $(warning Cannot use CONFIG_KASAN_HW: \ + -fsanitize=hwaddress is not supported by compiler) + endif +endif + +endif + +ifdef CONFIG_KASAN +CFLAGS_KASAN_NOSANITIZE := -fno-builtin endif --2.19.0.rc0.228.g281dcd1b4d0-goog
Re: [PATCH v6 03/18] khwasan: add CONFIG_KASAN_GENERIC and CONFIG_KASAN_HW
In reply to Dmitry VyukovQuoted 11 lines · Dmitry Vyukov
Quoted 3 lines · Andrey Konovalov
#define
__no_sanitize_address__attribute__((no_sanitize("address")))
+#define__no_sanitize_hwaddress__attribute__((no_sanitize("hwaddress")))It seems that it would be better to have just 1 attribute for both types. Currently
__no_sanitize_addressis used just in a single place. But if it ever used more, people will need to always spell both which looks unnecessary, or, worse will only fix asan but forget about khwasan.
Quoted 7 lines
If we do just:
#define
__no_sanitize_address__attribute__((no_sanitize("address", "hwaddress")))Then we don't need any changes in
compiler-gcc.hnor incompiler.h, and no chance or forgetting one of them.
Will do in v7.
Quoted 8 lines
lib/Kconfig.kasan line 13
Quoted 6 lines
config KASAN
lib/Kconfig.kasan +4
-1- bool "KASan: runtime memory debugger" + bool "KASAN: runtime memory debugger" + help + Enables KASAN (KernelAddressSANitizer) - runtime memory debugger, + designed to find out-of-bounds accesses and use-after-free bugs.Perhaps also give link to
Documentation/dev-tools/kasan.rstwhile we are here.
Will do in v7.
Quoted 16 lines
lib/Kconfig.kasan line 20
Quoted 8 lines
lib/Kconfig.kasan +7
-0+ +choice + prompt "KASAN mode" + depends on KASAN + default KASAN_GENERIC + help + KASAN has two modes: KASAN (a classic version, similar to userspaceIn these few sentences we call the old mode with 3 different terms:
"generic", "classic" and "KASAN" :)
This is somewhat confusing. Let's call it "generic" throughout (here and in the docs patch). "Generic" as in "supported on multiple arch and not-dependent on hardware features". "Classic" makes sense for people who knew KASAN before, but for future readers in won't make sense.
Will use "generic" in v7.
Quoted 7 lines
Quoted 2 lines
+if
HAVE_ARCH_KASAN_HWThis choice looks somewhat weird on non-arm64. It's kinda a choice menu, but one can't really choose anything. Should we put the whole choice under
HAVE_ARCH_KASAN_HW, and just selectKASAN_GENERICotherwise? I don't know what't the practice here. Andrey R?
I think having one option that is auto selected is fine.
Quoted 2 lines
lib/Kconfig.kasan line 46
Quoted 2 lines
lib/Kconfig.kasan +2
-0+config KASAN_HW + bool "KHWASAN: the hardware assisted mode"
Do we need a hyphen here? hardware-assisted?
Yes, will fix in v7.
PATCH v6 4/18 khwasan, arm64: adjust shadow size for CONFIG_KASAN_HW
In reply to Andrey KonovalovKWHASAN uses 1 shadow byte for 16 bytes of kernel memory, so it requires 1/16th of the kernel virtual address space for the shadow memory.
This commit sets KASAN_SHADOW_SCALE_SHIFT to 4 when KHWASAN is enabled.
--- arch/arm64/Makefile | 2 +- arch/arm64/include/asm/memory.h | 13 +++++++++---- 2 files changed, 10 insertions(+), 5 deletions(-)
arch/arm64/Makefile +1 -1
@@ -94,7 +94,7 @@ endif # KASAN_SHADOW_OFFSET = VA_START + (1 << (VA_BITS - KASAN_SHADOW_SCALE_SHIFT)) # - (1 << (64 - KASAN_SHADOW_SCALE_SHIFT)) # in 32-bit arithmetic -KASAN_SHADOW_SCALE_SHIFT := 3 +KASAN_SHADOW_SCALE_SHIFT := $(if $(CONFIG_KASAN_HW), 4, 3) KASAN_SHADOW_OFFSET := $(shell printf "0x%08x00000000\n" $$(( \ (0xffffffff & (-1 << ($(CONFIG_ARM64_VA_BITS) - 32))) \ + (1 << ($(CONFIG_ARM64_VA_BITS) - 32 - $(KASAN_SHADOW_SCALE_SHIFT))) \
arch/arm64/include/asm/memory.h +9 -4
@@ -74,12 +74,17 @@#define KERNEL_END _end/* -* KASAN requires 1/8th of the kernel virtual address space for the shadow -* region. KASAN can bloat the stack significantly, so double the (minimum) -* stack size when KASAN is in use. +* KASAN and KHWASAN require 1/8th and 1/16th of the kernel virtual address +* space for the shadow region respectively. They can bloat the stack +* significantly, so double the (minimum) stack size when they are in use. */ -#ifdef CONFIG_KASAN +#ifdef CONFIG_KASAN_GENERIC #define KASAN_SHADOW_SCALE_SHIFT 3 +#endif +#ifdef CONFIG_KASAN_HW +#define KASAN_SHADOW_SCALE_SHIFT 4 +#endif +#ifdef CONFIG_KASAN #define KASAN_SHADOW_SIZE (UL(1) << (VA_BITS - KASAN_SHADOW_SCALE_SHIFT))#define KASAN_THREAD_SHIFT 1#else
Signature · 1 line
2.19.0.rc0.228.g281dcd1b4d0-goog
Dmitry Vyukov
reviewerRe: [PATCH v6 04/18] khwasan, arm64: adjust shadow size for CONFIG_KASAN_HW
Quoted 29 lines · Andrey Konovalov
KWHASAN uses 1 shadow byte for 16 bytes of kernel memory, so it requires 1/16th of the kernel virtual address space for the shadow memory.
This commit sets
KASAN_SHADOW_SCALE_SHIFTto 4 when KHWASAN is enabled.Signed-off-by Andrey Konovalov <[email protected]>--- arch/arm64/Makefile | 2 +- arch/arm64/include/asm/memory.h | 13 +++++++++---- 2 files changed, 10 insertions(+), 5 deletions(-)arch/arm64/Makefile +1
-1@@ -94,7 +94,7 @@ endif # KASAN_SHADOW_OFFSET = VA_START + (1 << (VA_BITS - KASAN_SHADOW_SCALE_SHIFT)) # - (1 << (64 - KASAN_SHADOW_SCALE_SHIFT)) # in 32-bit arithmetic -KASAN_SHADOW_SCALE_SHIFT := 3 +KASAN_SHADOW_SCALE_SHIFT := $(if $(CONFIG_KASAN_HW), 4, 3) KASAN_SHADOW_OFFSET := $(shell printf "0x%08x00000000\n" $$(( \ (0xffffffff & (-1 << ($(CONFIG_ARM64_VA_BITS) - 32))) \ + (1 << ($(CONFIG_ARM64_VA_BITS) - 32 - $(KASAN_SHADOW_SCALE_SHIFT))) \arch/arm64/include/asm/memory.h +0
-0
arch/arm64/include/asm/memory.h line 77
Quoted 7 lines
arch/arm64/include/asm/memory.h +1
-3@@ -74,12 +74,17 @@#define KERNEL_END _end/* -* KASAN requires 1/8th of the kernel virtual address space for the shadow -* region. KASAN can bloat the stack significantly, so double the (minimum) -* stack size when KASAN is in use. +* KASAN and KHWASAN require 1/8th and 1/16th of the kernel virtual address
I am somewhat confused by the terminology.
"KASAN" is not actually "CONFIG_KASAN" below, it is actually "CONFIG_KASAN_GENERIC". While "KHWASAN" translates to "KASAN_HW" few lines later.
I think we need some consistent terminology for comments and config names until it's too late.
Quoted 17 lines
arch/arm64/include/asm/memory.h +8
-2+* space for the shadow region respectively. They can bloat the stack +* significantly, so double the (minimum) stack size when they are in use. */ -#ifdef CONFIG_KASAN +#ifdef CONFIG_KASAN_GENERIC #define KASAN_SHADOW_SCALE_SHIFT 3 +#endif +#ifdef CONFIG_KASAN_HW +#define KASAN_SHADOW_SCALE_SHIFT 4 +#endif +#ifdef CONFIG_KASAN #define KASAN_SHADOW_SIZE (UL(1) << (VA_BITS - KASAN_SHADOW_SCALE_SHIFT))#define KASAN_THREAD_SHIFT 1#else --2.19.0.rc0.228.g281dcd1b4d0-goog
Re: [PATCH v6 04/18] khwasan, arm64: adjust shadow size for CONFIG_KASAN_HW
Quoted 9 lines · Dmitry Vyukov
Quoted 5 lines · Andrey Konovalov
/*
- * KASAN requires 1/8th of the kernel virtual address space for the shadow
- * region. KASAN can bloat the stack significantly, so double the (minimum)
- * stack size when KASAN is in use.
+ * KASAN and KHWASAN require 1/8th and 1/16th of the kernel virtual address
Quoted 7 lines
I am somewhat confused by the terminology.
"KASAN" is not actually "CONFIG_KASAN" below, it is actually "CONFIG_KASAN_GENERIC". While "KHWASAN" translates to "KASAN_HW" few lines later.
I think we need some consistent terminology for comments and config names until it's too late.
As per offline discussion will rename in v7.
PATCH v6 5/18 khwasan: initialize shadow to 0xff
In reply to Andrey KonovalovA KHWASAN shadow memory cell contains a memory tag, that corresponds to the tag in the top byte of the pointer, that points to that memory. The native top byte value of kernel pointers is 0xff, so with KHWASAN we need to initialize shadow memory to 0xff. This commit does that.
--- arch/arm64/mm/kasan_init.c | 16 ++++++++++++++-- include/linux/kasan.h | 8 ++++++++ mm/kasan/common.c | 3 ++- 3 files changed, 24 insertions(+), 3 deletions(-)
arch/arm64/mm/kasan_init.c +14 -2
@@ -44,6 +44,15 @@ static phys_addr_t __init kasan_alloc_zeroed_page(int node)return __pa (p); } +static phys_addr_t __init kasan_alloc_raw_page (int node) +{ +void *p =memblock_virt_alloc_try_nid_raw (PAGE_SIZE ,PAGE_SIZE , +__pa (MAX_DMA_ADDRESS ), +MEMBLOCK_ALLOC_ACCESSIBLE , + node); +return __pa (p); +} +static pte_t *__init kasan_pte_offset (pmd_t *pmdp,unsigned long addr,int node,bool early) {
@@ -89,7 +98,9 @@ static void __init kasan_pte_populate(pmd_t *pmdp, unsigned long addr,do {phys_addr_t page_phys = early ?__pa_symbol (kasan_zero_page) - :kasan_alloc_zeroed_page (node); + :kasan_alloc_raw_page (node); +if (!early) +memset (__va (page_phys),KASAN_SHADOW_INIT ,PAGE_SIZE ); next = addr +PAGE_SIZE ;set_pte (ptep,pfn_pte (__phys_to_pfn (page_phys),PAGE_KERNEL )); }while (ptep++, addr = next, addr != end &&pte_none (READ_ONCE (*ptep)));
@@ -139,6 +150,7 @@ asmlinkage void __init kasan_early_init(void) KASAN_SHADOW_END - (1UL << (64 - KASAN_SHADOW_SCALE_SHIFT)));BUILD_BUG_ON (!IS_ALIGNED (KASAN_SHADOW_START ,PGDIR_SIZE ));BUILD_BUG_ON (!IS_ALIGNED (KASAN_SHADOW_END ,PGDIR_SIZE )); +kasan_pgd_populate (KASAN_SHADOW_START ,KASAN_SHADOW_END ,NUMA_NO_NODE , true); }
@@ -235,7 +247,7 @@ void __init kasan_init(void)set_pte (&kasan_zero_pte[i],pfn_pte (sym_to_pfn (kasan_zero_page),PAGE_KERNEL_RO )); -memset (kasan_zero_page,0 ,PAGE_SIZE ); +memset (kasan_zero_page,KASAN_SHADOW_INIT ,PAGE_SIZE );cpu_replace_ttbr1 (lm_alias (swapper_pg_dir));/* At this point kasan is fully initialized. Enable error messages */
include/linux/kasan.h +8 -0
@@ -150,6 +150,8 @@ static inline size_t kasan_metadata_size(struct kmem_cache *cache) { return 0; }#ifdef CONFIG_KASAN_GENERIC +#define KASAN_SHADOW_INIT 0 +void kasan_cache_shrink (struct kmem_cache *cache);void kasan_cache_shutdown (struct kmem_cache *cache);
@@ -160,4 +162,10 @@ static inline void kasan_cache_shutdown(struct kmem_cache *cache) {}#endif /* CONFIG_KASAN_GENERIC */ +#ifdef CONFIG_KASAN_HW + +#define KASAN_SHADOW_INIT 0xFF + +#endif /* CONFIG_KASAN_HW */ +#endif /* LINUX_KASAN_H */
mm/kasan/common.c +2 -1
@@ -474,11 +474,12 @@ int kasan_module_alloc(void *addr, size_t size) ret =__vmalloc_node_range (shadow_size,1 , shadow_start, shadow_start + shadow_size, -GFP_KERNEL | __GFP_ZERO, +GFP_KERNEL ,PAGE_KERNEL ,VM_NO_GUARD ,NUMA_NO_NODE ,__builtin_return_address (0 ));if (ret) { +__memset (ret,KASAN_SHADOW_INIT , shadow_size);find_vm_area (addr)->flags |=VM_KASAN ;kmemleak_ignore (ret);return 0 ;
Signature · 1 line
2.19.0.rc0.228.g281dcd1b4d0-goog
PATCH v6 6/18 khwasan, arm64: untag virt address in __kimg_to_phys and _virt_addr_is_linear
In reply to Andrey Konovalov__kimg_to_phys (which is used by virt_to_phys) and _virt_addr_is_linear (which is used by virt_addr_valid) assume that the top byte of the address is 0xff, which isn't always the case with KHWASAN enabled.
The solution is to reset the tag in those macros.
--- arch/arm64/include/asm/memory.h | 18 ++++++++++++++++++ 1 file changed, 18 insertions(+)
arch/arm64/include/asm/memory.h +18 -0
@@ -92,6 +92,13 @@#define KASAN_THREAD_SHIFT 0#endif +#ifdef CONFIG_KASAN_HW +#define KASAN_TAG_SHIFTED (tag) ((unsigned long)(tag) << 56) +#define KASAN_SET_TAG (addr, tag) (((addr) & ~KASAN_TAG_SHIFTED(0xff)) | \ + KASAN_TAG_SHIFTED(tag)) +#define KASAN_RESET_TAG (addr) KASAN_SET_TAG(addr, 0xff) +#endif +#define MIN_THREAD_SHIFT (14 + KASAN_THREAD_SHIFT)/*
@@ -232,7 +239,12 @@ static inline unsigned long kaslr_offset(void)#define __is_lm_address (addr) (!!((addr) & BIT(VA_BITS - 1)))#define __lm_to_phys (addr) (((addr) & ~PAGE_OFFSET) + PHYS_OFFSET) + +#ifdef CONFIG_KASAN_HW +#define __kimg_to_phys (addr) (KASAN_RESET_TAG(addr) - kimage_voffset) +#else #define __kimg_to_phys (addr) ((addr) - kimage_voffset) +#endif #define __virt_to_phys_nodebug (x) ({ \ phys_addr_t __x = (phys_addr_t)(x); \
@@ -308,7 +320,13 @@ static inline void *phys_to_virt(phys_addr_t x)#endif #endif +#ifdef CONFIG_KASAN_HW +#define _virt_addr_is_linear (kaddr) (KASAN_RESET_TAG((u64)(kaddr)) >= \ + PAGE_OFFSET) +#else #define _virt_addr_is_linear (kaddr) (((u64)(kaddr)) >= PAGE_OFFSET) +#endif +#define virt_addr_valid (kaddr) (_virt_addr_is_linear(kaddr) && \ _virt_addr_valid(kaddr))
Signature · 1 line
2.19.0.rc0.228.g281dcd1b4d0-goog
Dmitry Vyukov
reviewerRe: [PATCH v6 06/18] khwasan, arm64: untag virt address in __kimg_to_phys and _virt_addr_is_linear
Quoted 16 lines · Andrey Konovalov
__kimg_to_phys(which is used byvirt_to_phys) and_virt_addr_is_linear(which is used byvirt_addr_valid) assume that the top byte of the address is0xff, which isn't always the case with KHWASAN enabled.The solution is to reset the tag in those macros.
Signed-off-by Andrey Konovalov <[email protected]>--- arch/arm64/include/asm/memory.h | 18 ++++++++++++++++++ 1 file changed, 18 insertions(+)arch/arm64/include/asm/memory.h +0
-0
arch/arm64/include/asm/memory.h line 101
Quoted 10 lines
arch/arm64/include/asm/memory.h +7
-0@@ -92,6 +92,13 @@#define KASAN_THREAD_SHIFT 0#endif +#ifdef CONFIG_KASAN_HW +#define KASAN_TAG_SHIFTED (tag) ((unsigned long)(tag) << 56) +#define KASAN_SET_TAG (addr, tag) (((addr) & ~KASAN_TAG_SHIFTED(0xff)) | \ + KASAN_TAG_SHIFTED(tag)) +#define KASAN_RESET_TAG (addr) KASAN_SET_TAG(addr, 0xff) +#endif +
Wouldn't it be better to
#define KASAN_RESET_TAG(addr) addr
when CONFIG_KASAN_HW is not enabled, and then not duplicate the macros below? That's what we do in kasan.h for all hooks.
I see that a subsequent patch duplicates yet another macro in this file. While we could use:
#define __kimg_to_phys(addr) (KASAN_RESET_TAG(addr) - kimage_voffset)
with and without kasan. Duplicating them increases risk that somebody will change only the non-kasan version but forget kasan version.
Quoted 33 lines
arch/arm64/include/asm/memory.h +11
-1#define MIN_THREAD_SHIFT (14 + KASAN_THREAD_SHIFT)/* @@ -232,7 +239,12 @@ static inline unsigned long kaslr_offset(void)#define __is_lm_address (addr) (!!((addr) & BIT(VA_BITS - 1)))#define __lm_to_phys (addr) (((addr) & ~PAGE_OFFSET) + PHYS_OFFSET) + +#ifdef CONFIG_KASAN_HW +#define __kimg_to_phys (addr) (KASAN_RESET_TAG(addr) - kimage_voffset) +#else #define __kimg_to_phys (addr) ((addr) - kimage_voffset) +#endif #define __virt_to_phys_nodebug (x) ({ \ phys_addr_t __x = (phys_addr_t)(x); \@@ -308,7 +320,13 @@ static inline void *phys_to_virt(phys_addr_t x)#endif #endif +#ifdef CONFIG_KASAN_HW +#define _virt_addr_is_linear (kaddr) (KASAN_RESET_TAG((u64)(kaddr)) >= \ + PAGE_OFFSET) +#else #define _virt_addr_is_linear (kaddr) (((u64)(kaddr)) >= PAGE_OFFSET) +#endif +#define virt_addr_valid (kaddr) (_virt_addr_is_linear(kaddr) && \ _virt_addr_valid(kaddr)) --2.19.0.rc0.228.g281dcd1b4d0-goog
Re: [PATCH v6 06/18] khwasan, arm64: untag virt address in __kimg_to_phys and _virt_addr_is_linear
Quoted 18 lines · Dmitry Vyukov
arch/arm64/include/asm/memory.h line 101
Quoted 7 lines · Andrey Konovalov
arch/arm64/include/asm/memory.h +7
-0+#ifdef CONFIG_KASAN_HW +#define KASAN_TAG_SHIFTED (tag) ((unsigned long)(tag) << 56) +#define KASAN_SET_TAG (addr, tag) (((addr) & ~KASAN_TAG_SHIFTED(0xff)) | \ + KASAN_TAG_SHIFTED(tag)) +#define KASAN_RESET_TAG (addr) KASAN_SET_TAG(addr, 0xff) +#endif +Wouldn't it be better to
#defineKASAN_RESET_TAG(addr)addr
whenCONFIG_KASAN_HWis not enabled, and then not duplicate the macros below? That's what we do inkasan.hfor all hooks.
I see that a subsequent patch duplicates yet another macro in this file. While we could use:
Quoted 4 lines
#define
__kimg_to_phys(addr)(KASAN_RESET_TAG(addr)-kimage_voffset)with and without kasan. Duplicating them increases risk that somebody will change only the non-kasan version but forget kasan version.
Will do in v7.
PATCH v6 7/18 khwasan: add tag related helper functions
In reply to Andrey KonovalovThis commit adds a few helper functions, that are meant to be used to work with tags embedded in the top byte of kernel pointers: to set, to get or to reset (set to 0xff) the top byte.
--- arch/arm64/mm/kasan_init.c | 2 ++ include/linux/kasan.h | 29 +++++++++++++++++ mm/kasan/kasan.h | 55 ++++++++++++++++++++++++++++++++ mm/kasan/khwasan.c | 65 ++++++++++++++++++++++++++++++++++++++ 4 files changed, 151 insertions(+)
arch/arm64/mm/kasan_init.c +2 -0
@@ -250,6 +250,8 @@ void __init kasan_init(void)memset (kasan_zero_page,KASAN_SHADOW_INIT ,PAGE_SIZE );cpu_replace_ttbr1 (lm_alias (swapper_pg_dir)); +khwasan_init (); +/* At this point kasan is fully initialized. Enable error messages */ init_task.kasan_depth =0 ;pr_info ("KernelAddressSanitizer initialized\n" );
include/linux/kasan.h +29 -0
@@ -166,6 +166,35 @@ static inline void kasan_cache_shutdown(struct kmem_cache *cache) {}#define KASAN_SHADOW_INIT 0xFF +void khwasan_init (void ); + +void *khwasan_reset_tag (const void *addr); + +void *khwasan_preset_slub_tag (struct kmem_cache *cache,const void *addr); +void *khwasan_preset_slab_tag (struct kmem_cache *cache,unsigned int idx, +const void *addr); + +#else /* CONFIG_KASAN_HW */ + +static inline void khwasan_init (void ) { } + +static inline void *khwasan_reset_tag (const void *addr) +{ +return (void *)addr; +} + +static inline void *khwasan_preset_slub_tag (struct kmem_cache *cache, +const void *addr) +{ +return (void *)addr; +} + +static inline void *khwasan_preset_slab_tag (struct kmem_cache *cache, +unsigned int idx,const void *addr) +{ +return (void *)addr; +} +#endif /* CONFIG_KASAN_HW */ #endif /* LINUX_KASAN_H */
mm/kasan/kasan.h +55 -0
67 lines not on this page · raw message
mm/kasan/khwasan.c +65 -0
71 lines not on this page · raw message
Signature · 1 line
2.19.0.rc0.228.g281dcd1b4d0-goog
Dmitry Vyukov
reviewerRe: [PATCH v6 07/18] khwasan: add tag related helper functions
Quoted 174 lines · Andrey Konovalov
This commit adds a few helper functions, that are meant to be used to work with tags embedded in the top byte of kernel pointers: to set, to get or to reset (set to
0xff) the top byte.Signed-off-by Andrey Konovalov <[email protected]>--- arch/arm64/mm/kasan_init.c | 2 ++ include/linux/kasan.h | 29 +++++++++++++++++ mm/kasan/kasan.h | 55 ++++++++++++++++++++++++++++++++ mm/kasan/khwasan.c | 65 ++++++++++++++++++++++++++++++++++++++ 4 files changed, 151 insertions(+)arch/arm64/mm/kasan_init.c +2
-0@@ -250,6 +250,8 @@ void __init kasan_init(void)memset (kasan_zero_page,KASAN_SHADOW_INIT ,PAGE_SIZE );cpu_replace_ttbr1 (lm_alias (swapper_pg_dir)); +khwasan_init (); +/* At this point kasan is fully initialized. Enable error messages */ init_task.kasan_depth =0 ;pr_info ("KernelAddressSanitizer initialized\n" );include/linux/kasan.h +29
-0@@ -166,6 +166,35 @@ static inline void kasan_cache_shutdown(struct kmem_cache *cache) {}#define KASAN_SHADOW_INIT 0xFF +void khwasan_init (void ); + +void *khwasan_reset_tag (const void *addr); + +void *khwasan_preset_slub_tag (struct kmem_cache *cache,const void *addr); +void *khwasan_preset_slab_tag (struct kmem_cache *cache,unsigned int idx, +const void *addr); + +#else /* CONFIG_KASAN_HW */ + +static inline void khwasan_init (void ) { } + +static inline void *khwasan_reset_tag (const void *addr) +{ +return (void *)addr; +} + +static inline void *khwasan_preset_slub_tag (struct kmem_cache *cache, +const void *addr) +{ +return (void *)addr; +} + +static inline void *khwasan_preset_slab_tag (struct kmem_cache *cache, +unsigned int idx,const void *addr) +{ +return (void *)addr; +} +#endif /* CONFIG_KASAN_HW */ #endif /* LINUX_KASAN_H */ mm/kasan/kasan.h +55
-0@@ -8,6 +8,10 @@#define KASAN_SHADOW_SCALE_SIZE (1UL << KASAN_SHADOW_SCALE_SHIFT)#define KASAN_SHADOW_MASK (KASAN_SHADOW_SCALE_SIZE - 1) +#define KHWASAN_TAG_KERNEL 0xFF/* native kernel pointers tag */ +#define KHWASAN_TAG_INVALID 0xFE/* inaccessible memory tag */ +#define KHWASAN_TAG_MAX 0xFD/* maximum value for random tags */ +#define KASAN_FREE_PAGE 0xFF/* page was freed */ #define KASAN_PAGE_REDZONE 0xFE/* redzone for kmalloc_large allocations */ #define KASAN_KMALLOC_REDZONE 0xFC/* redzone inside slub object */ @@ -126,6 +130,57 @@ static inline void quarantine_reduce(void) { }static inline void quarantine_remove_cache (struct kmem_cache *cache) { }#endif +#ifdef CONFIG_KASAN_HW + +#define KHWASAN_TAG_SHIFT 56 +#define KHWASAN_TAG_MASK (0xFFUL << KHWASAN_TAG_SHIFT) + +u8 random_tag (void ); + +static inline void *set_tag (const void *addr,u8 tag) +{ +u64 a = (u64 )addr; + + a &= ~KHWASAN_TAG_MASK ; + a |= ((u64 )tag <<KHWASAN_TAG_SHIFT ); + +return (void *)a; +} + +static inline u8 get_tag (const void *addr) +{ +return (u8)((u64 )addr >>KHWASAN_TAG_SHIFT ); +} + +static inline void *reset_tag (const void *addr) +{ +return set_tag (addr,KHWASAN_TAG_KERNEL ); +} + +#else /* CONFIG_KASAN_HW */ + +static inline u8 random_tag (void ) +{ +return 0 ; +} + +static inline void *set_tag (const void *addr,u8 tag) +{ +return (void *)addr; +} + +static inline u8 get_tag (const void *addr) +{ +return 0 ; +} + +static inline void *reset_tag (const void *addr) +{ +return (void *)addr; +} + +#endif /* CONFIG_KASAN_HW */ +/* * Exported functions for interfaces called from assembly or from generated * code. Declarations here to avoid warning about missing declarations. mm/kasan/khwasan.c +28
-0@@ -38,6 +38,71 @@#include "kasan.h" #include "../slab.h" +static DEFINE_PER_CPU(u32, prng_state); + +void khwasan_init (void ) +{ +int cpu; + +for_each_possible_cpu (cpu) +per_cpu (prng_state, cpu) =get_random_u32 (); +} + +/* +* If a preemption happens between this_cpu_read and this_cpu_write, the only +* side effect is that we'll give a few allocated in different contexts objects +* the same tag. Since KHWASAN is meant to be used a probabilistic bug-detection +* debug feature, this doesn?t have significant negative impact. +* +* Ideally the tags use strong randomness to prevent any attempts to predict +* them during explicit exploit attempts. But strong randomness is expensive, +* and we did an intentional trade-off to use a PRNG. This non-atomic RMW +* sequence has in fact positive effect, since interrupts that randomly skew +* PRNG at unpredictable points do only good. +*/ +u8 random_tag (void ) +{ +u32 state =this_cpu_read (prng_state); + + state =1664525 * state +1013904223 ; +this_cpu_write (prng_state, state);
mm/kasan/khwasan.c line 78
Quoted 10 lines
mm/kasan/khwasan.c +10
-0+ +return (u8)(state % (KHWASAN_TAG_MAX +1 )); +} + +void *khwasan_reset_tag (const void *addr) +{ +return reset_tag (addr); +} + +void *khwasan_preset_slub_tag (struct kmem_cache *cache,const void *addr)
Can't we do this in the existing kasan_init_slab_obj() hook? It looks like it should do exactly this -- allow any one-time initialization for objects. We could extend it to accept index and return a new pointer.
If that does not work for some reason, I would try to at least unify the hook for slab/slub, e.g. pass idx=-1 from slub and then use random_tag().
It also seems that we do preset tag for slab multiple times (from slab_get_obj()). Using kasan_init_slab_obj() should resolve this too (hopefully we don't call it multiple times).
Quoted 33 lines
mm/kasan/khwasan.c +27
-1+{ +/* +* Since it's desirable to only call object contructors ones during +* slab allocation, we preassign tags to all such objects. +* Also preassign tags for SLAB_TYPESAFE_BY_RCU slabs to avoid +* use-after-free reports. +*/ +if (cache->ctor || cache->flags &SLAB_TYPESAFE_BY_RCU ) +return set_tag (addr,random_tag ()); +return (void *)addr; +} + +void *khwasan_preset_slab_tag (struct kmem_cache *cache,unsigned int idx, +const void *addr) +{ +/* +* See comment in khwasan_preset_slub_tag. +* For SLAB allocator we can't preassign tags randomly since the +* freelist is stored as an array of indexes instead of a linked +* list. Assign tags based on objects indexes, so that objects that +* are next to each other get different tags. +*/ +if (cache->ctor || cache->flags &SLAB_TYPESAFE_BY_RCU ) +return set_tag (addr, (u8 )idx); +return (void *)addr; +} +void check_memory_region (unsigned long addr,size_t size,bool write,unsigned long ret_ip) { --2.19.0.rc0.228.g281dcd1b4d0-goog
Re: [PATCH v6 07/18] khwasan: add tag related helper functions
Quoted 4 lines · Dmitry Vyukov
Quoted 1 line · Andrey Konovalov
+void *khwasan_preset_slub_tag(struct
kmem_cache*cache, const void *addr)
Quoted 10 lines
Can't we do this in the existing
kasan_init_slab_obj()hook? It looks like it should do exactly this -- allow any one-time initialization for objects. We could extend it to accept index and return a new pointer.
If that does not work for some reason, I would try to at least unify the hook for slab/slub, e.g. pass idx=-1 from slub and then userandom_tag().
It also seems that we do preset tag for slab multiple times (fromslab_get_obj()). Usingkasan_init_slab_obj()should resolve this too (hopefully we don't call it multiple times).
The issue is that SLAB stores freelist as an array of indexes instead of using an actual linked list like SLUB. So you can't store the tag in the pointer while the object is in the freelist, since there's no pointer. And, technically, we don't preset tags for SLAB, we just use the id as the tag every time a pointer is used, so perhaps we should rename the callback. As to unifying the callbacks, sure, we can do that.
Dmitry Vyukov
reviewerRe: [PATCH v6 07/18] khwasan: add tag related helper functions
Quoted 16 lines · Andrey Konovalov
Quoted 4 lines · Dmitry Vyukov
Quoted 1 line · Andrey Konovalov
+void *khwasan_preset_slub_tag(struct
kmem_cache*cache, const void *addr)Quoted 10 lines
Can't we do this in the existing
kasan_init_slab_obj()hook? It looks like it should do exactly this -- allow any one-time initialization for objects. We could extend it to accept index and return a new pointer.
If that does not work for some reason, I would try to at least unify the hook for slab/slub, e.g. pass idx=-1 from slub and then userandom_tag().
It also seems that we do preset tag for slab multiple times (fromslab_get_obj()). Usingkasan_init_slab_obj()should resolve this too (hopefully we don't call it multiple times).
Quoted 7 lines
The issue is that SLAB stores freelist as an array of indexes instead of using an actual linked list like SLUB. So you can't store the tag in the pointer while the object is in the freelist, since there's no pointer. And, technically, we don't preset tags for SLAB, we just use the id as the tag every time a pointer is used, so perhaps we should rename the callback. As to unifying the callbacks, sure, we can do that.
As per offline discussion: potentially we can use kasan_init_slab_obj() if we add tag in kmalloc hook by using obj_to_idx().
PATCH v6 8/18 khwasan: preassign tags to objects with ctors or SLAB_TYPESAFE_BY_RCU
In reply to Andrey KonovalovAn object constructor can initialize pointers within this objects based on the address of the object. Since the object address might be tagged, we need to assign a tag before calling constructor.
The implemented approach is to assign tags to objects with constructors when a slab is allocated and call constructors once as usual. The downside is that such object would always have the same tag when it is reallocated, so we won't catch use-after-frees on it.
Also pressign tags for objects from SLAB_TYPESAFE_BY_RCU caches, since they can be validy accessed after having been freed.
--- mm/slab.c | 6 +++++- mm/slub.c | 4 ++++ 2 files changed, 9 insertions(+), 1 deletion(-)
mm/slab.c +5 -1
@@ -403,7 +403,11 @@ static inline struct kmem_cache *virt_to_cache(const void *obj)static inline void *index_to_obj (struct kmem_cache *cache,struct page *page,unsigned int idx) { -return page->s_mem + cache->size * idx; +void *obj; + + obj = page->s_mem + cache->size * idx; + obj =khwasan_preset_slab_tag (cache, idx, obj); +return obj; }/*
mm/slub.c +4 -0
@@ -1531,12 +1531,14 @@ static bool shuffle_freelist(struct kmem_cache *s, struct page *page)/* First entry is used as the base of the freelist */ cur =next_freelist_entry (s, page, &pos, start, page_limit, freelist_count); + cur =khwasan_preset_slub_tag (s, cur); page->freelist = cur;for (idx =1 ; idx < page->objects; idx++) {setup_object (s, page, cur); next =next_freelist_entry (s, page, &pos, start, page_limit, freelist_count); + next =khwasan_preset_slub_tag (s, next);set_freepointer (s, cur, next); cur = next; }
@@ -1613,8 +1615,10 @@ static struct page *allocate_slab(struct kmem_cache *s, gfp_t flags, int node) shuffle =shuffle_freelist (s, page);if (!shuffle) { + start =khwasan_preset_slub_tag (s, start);for_each_object_idx (p, idx, s, start, page->objects) {setup_object (s, page, p); + p =khwasan_preset_slub_tag (s, p);if (likely (idx < page->objects))set_freepointer (s, p, p + s->size);else
Signature · 1 line
2.19.0.rc0.228.g281dcd1b4d0-goog
Re: [PATCH v6 08/18] khwasan: preassign tags to objects with ctors or SLAB_TYPESAFE_BY_RCU
For the slab pieces
Quoted 66 lines · Andrey Konovalov
An object constructor can initialize pointers within this objects based on the address of the object. Since the object address might be tagged, we need to assign a tag before calling constructor.
The implemented approach is to assign tags to objects with constructors when a slab is allocated and call constructors once as usual. The downside is that such object would always have the same tag when it is reallocated, so we won't catch use-after-frees on it.
Also pressign tags for objects from
SLAB_TYPESAFE_BY_RCUcaches, since they can be validy accessed after having been freed.Signed-off-by Andrey Konovalov <[email protected]>--- mm/slab.c | 6 +++++- mm/slub.c | 4 ++++ 2 files changed, 9 insertions(+), 1 deletion(-)mm/slab.c +5
-1@@ -403,7 +403,11 @@ static inline struct kmem_cache *virt_to_cache(const void *obj)static inline void *index_to_obj (struct kmem_cache *cache,struct page *page,unsigned int idx) { -return page->s_mem + cache->size * idx; +void *obj; + + obj = page->s_mem + cache->size * idx; + obj =khwasan_preset_slab_tag (cache, idx, obj); +return obj; }/* mm/slub.c +4
-0@@ -1531,12 +1531,14 @@ static bool shuffle_freelist(struct kmem_cache *s, struct page *page)/* First entry is used as the base of the freelist */ cur =next_freelist_entry (s, page, &pos, start, page_limit, freelist_count); + cur =khwasan_preset_slub_tag (s, cur); page->freelist = cur;for (idx =1 ; idx < page->objects; idx++) {setup_object (s, page, cur); next =next_freelist_entry (s, page, &pos, start, page_limit, freelist_count); + next =khwasan_preset_slub_tag (s, next);set_freepointer (s, cur, next); cur = next; }@@ -1613,8 +1615,10 @@ static struct page *allocate_slab(struct kmem_cache *s, gfp_t flags, int node) shuffle =shuffle_freelist (s, page);if (!shuffle) { + start =khwasan_preset_slub_tag (s, start);for_each_object_idx (p, idx, s, start, page->objects) {setup_object (s, page, p); + p =khwasan_preset_slub_tag (s, p);if (likely (idx < page->objects))set_freepointer (s, p, p + s->size);else
Dmitry Vyukov
reviewerRe: [PATCH v6 08/18] khwasan: preassign tags to objects with ctors or SLAB_TYPESAFE_BY_RCU
In reply to Andrey KonovalovQuoted 55 lines · Andrey Konovalov
An object constructor can initialize pointers within this objects based on the address of the object. Since the object address might be tagged, we need to assign a tag before calling constructor.
The implemented approach is to assign tags to objects with constructors when a slab is allocated and call constructors once as usual. The downside is that such object would always have the same tag when it is reallocated, so we won't catch use-after-frees on it.
Also pressign tags for objects from
SLAB_TYPESAFE_BY_RCUcaches, since they can be validy accessed after having been freed.Signed-off-by Andrey Konovalov <[email protected]>--- mm/slab.c | 6 +++++- mm/slub.c | 4 ++++ 2 files changed, 9 insertions(+), 1 deletion(-)mm/slab.c +5
-1@@ -403,7 +403,11 @@ static inline struct kmem_cache *virt_to_cache(const void *obj)static inline void *index_to_obj (struct kmem_cache *cache,struct page *page,unsigned int idx) { -return page->s_mem + cache->size * idx; +void *obj; + + obj = page->s_mem + cache->size * idx; + obj =khwasan_preset_slab_tag (cache, idx, obj); +return obj; }/* mm/slub.c +2
-0@@ -1531,12 +1531,14 @@ static bool shuffle_freelist(struct kmem_cache *s, struct page *page)/* First entry is used as the base of the freelist */ cur =next_freelist_entry (s, page, &pos, start, page_limit, freelist_count); + cur =khwasan_preset_slub_tag (s, cur); page->freelist = cur;for (idx =1 ; idx < page->objects; idx++) {setup_object (s, page, cur); next =next_freelist_entry (s, page, &pos, start, page_limit, freelist_count); + next =khwasan_preset_slub_tag (s, next);set_freepointer (s, cur, next); cur = next; }
mm/slub.c static struct page *allocate_slab(struct kmem_cache *s, gfp_t flags, int node) · line 1621
Quoted 7 lines
mm/slub.c +2
-0@@ -1613,8 +1615,10 @@ static struct page *allocate_slab(struct kmem_cache *s, gfp_t flags, int node) shuffle =shuffle_freelist (s, page);if (!shuffle) { + start =khwasan_preset_slub_tag (s, start);for_each_object_idx (p, idx, s, start, page->objects) {setup_object (s, page, p); + p =khwasan_preset_slub_tag (s, p);
As I commented in the previous patch, can't we do this in kasan_init_slab_obj(), which should be called in all the right places already?
Quoted 6 lines
mm/slub.c +0
-1if (likely (idx < page->objects))set_freepointer (s, p, p + s->size);else --2.19.0.rc0.228.g281dcd1b4d0-goog
Re: [PATCH v6 08/18] khwasan: preassign tags to objects with ctors or SLAB_TYPESAFE_BY_RCU
Quoted 9 lines · Dmitry Vyukov
mm/slub.c static struct page *allocate_slab(struct kmem_cache *s, gfp_t flags, int node) · line 1621
Quoted 5 lines · Andrey Konovalov
if (!shuffle) {mm/slub.c +2
-0+ start =khwasan_preset_slub_tag (s, start);for_each_object_idx (p, idx, s, start, page->objects) {setup_object (s, page, p); + p =khwasan_preset_slub_tag (s, p);
Quoted 4 lines
As I commented in the previous patch, can't we do this in
kasan_init_slab_obj(), which should be called in all the right places already?
As per offline discussion, will do in v7.
PATCH v6 9/18 khwasan, arm64: fix up fault handling logic
In reply to Andrey Konovalovshow_pte in arm64 fault handling relies on the fact that the top byte of a kernel pointer is 0xff, which isn't always the case with KHWASAN enabled. Reset the top byte.
--- arch/arm64/mm/fault.c | 3 +++ 1 file changed, 3 insertions(+)
arch/arm64/mm/fault.c +3 -0
@@ -32,6 +32,7 @@#include <linux/perf_event.h> #include <linux/preempt.h> #include <linux/hugetlb.h> +#include <linux/kasan.h> #include <asm/bug.h> #include <asm/cmpxchg.h>
@@ -134,6 +135,8 @@ void show_pte(unsigned long addr)pgd_t *pgdp;pgd_t pgd; + addr = (unsigned long )khwasan_reset_tag ((void *)addr); +if (addr <TASK_SIZE ) {/* TTBR0 */ mm = current->active_mm;
Signature · 1 line
2.19.0.rc0.228.g281dcd1b4d0-goog
PATCH v6 10/18 khwasan, arm64: enable top byte ignore for the kernel
In reply to Andrey KonovalovKHWASAN uses the Top Byte Ignore feature of arm64 CPUs to store a pointer tag in the top byte of each pointer. This commit enables the TCR_TBI1 bit, which enables Top Byte Ignore for the kernel, when KHWASAN is used.
--- arch/arm64/include/asm/pgtable-hwdef.h | 1 + arch/arm64/mm/proc.S | 8 +++++++- 2 files changed, 8 insertions(+), 1 deletion(-)
arch/arm64/include/asm/pgtable-hwdef.h +1 -0
@@ -289,6 +289,7 @@#define TCR_A1 (UL(1) << 22)#define TCR_ASID16 (UL(1) << 36)#define TCR_TBI0 (UL(1) << 37) +#define TCR_TBI1 (UL(1) << 38)#define TCR_HA (UL(1) << 39)#define TCR_HD (UL(1) << 40)#define TCR_NFD1 (UL(1) << 54)
arch/arm64/mm/proc.S +7 -1
@@ -47,6 +47,12 @@ /* PTWs cacheable, inner/outer WBWA */ #define TCR_CACHE_FLAGS TCR_IRGN_WBWA | TCR_ORGN_WBWA +#ifdef CONFIG_KASAN_HW +#define TCR_KASAN_FLAGS TCR_TBI1 +#else +#define TCR_KASAN_FLAGS 0 +#endif + #define MAIR(attr, mt) ((attr) << ((mt) * 8)) /*
@@ -440,7 +446,7 @@ ENTRY(__cpu_setup) */ ldr x10, =TCR_TxSZ(VA_BITS) | TCR_CACHE_FLAGS | TCR_SMP_FLAGS | \ TCR_TG_FLAGS | TCR_KASLR_FLAGS | TCR_ASID16 | \ - TCR_TBI0 | TCR_A1 + TCR_TBI0 | TCR_A1 | TCR_KASAN_FLAGS tcr_set_idmap_t0sz x10, x9 /*
Signature · 1 line
2.19.0.rc0.228.g281dcd1b4d0-goog
PATCH v6 11/18 khwasan, mm: perform untagged pointers comparison in krealloc
In reply to Andrey KonovalovThe krealloc function checks where the same buffer was reused or a new one allocated by comparing kernel pointers. KHWASAN changes memory tag on the krealloc'ed chunk of memory and therefore also changes the pointer tag of the returned pointer. Therefore we need to perform comparison on untagged (with tags reset) pointers to check whether it's the same memory region or not.
--- mm/slab_common.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
mm/slab_common.c +1 -1
@@ -1513,7 +1513,7 @@ void *krealloc(const void *p, size_t new_size, gfp_t flags) } ret =__do_krealloc (p, new_size, flags); -if (ret && p != ret) +if (ret &&khwasan_reset_tag (p) !=khwasan_reset_tag (ret))kfree (p);return ret;
Signature · 1 line
2.19.0.rc0.228.g281dcd1b4d0-goog
Re: [PATCH v6 11/18] khwasan, mm: perform untagged pointers comparison in krealloc
Quoted 26 lines · Andrey Konovalov
The krealloc function checks where the same buffer was reused or a new one allocated by comparing kernel pointers. KHWASAN changes memory tag on the krealloc'ed chunk of memory and therefore also changes the pointer tag of the returned pointer. Therefore we need to perform comparison on untagged (with tags reset) pointers to check whether it's the same memory region or not.
Signed-off-by Andrey Konovalov <[email protected]>--- mm/slab_common.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)mm/slab_common.c +1
-1@@ -1513,7 +1513,7 @@ void *krealloc(const void *p, size_t new_size, gfp_t flags) } ret =__do_krealloc (p, new_size, flags); -if (ret && p != ret) +if (ret &&khwasan_reset_tag (p) !=khwasan_reset_tag (ret))kfree (p);return ret;
PATCH v6 12/18 khwasan: split out kasan_report.c from report.c
In reply to Andrey KonovalovThis patch moves KASAN specific error reporting routines to kasan_report.c without any functional changes, leaving common error reporting code in report.c to be later reused by KHWASAN.
--- mm/kasan/Makefile | 4 +- mm/kasan/kasan.h | 7 ++ mm/kasan/kasan_report.c | 158 +++++++++++++++++++++++++ mm/kasan/khwasan_report.c | 39 +++++++ mm/kasan/report.c | 234 +++++++++----------------------------- 5 files changed, 257 insertions(+), 185 deletions(-) create mode 100644 mm/kasan/kasan_report.c create mode 100644 mm/kasan/khwasan_report.c
mm/kasan/Makefile +2 -2
@@ -14,5 +14,5 @@ CFLAGS_kasan.o := $(call cc-option, -fno-conserve-stack -fno-stack-protector) CFLAGS_khwasan.o := $(call cc-option, -fno-conserve-stack -fno-stack-protector) obj-$(CONFIG_KASAN) := common.o kasan_init.o report.o -obj-$(CONFIG_KASAN_GENERIC) += kasan.o quarantine.o -obj-$(CONFIG_KASAN_HW) += khwasan.o +obj-$(CONFIG_KASAN_GENERIC) += kasan.o kasan_report.o quarantine.o +obj-$(CONFIG_KASAN_HW) += khwasan.o khwasan_report.o
mm/kasan/kasan.h +7 -0
@@ -109,11 +109,18 @@ static inline const void *kasan_shadow_to_mem(const void *shadow_addr) << KASAN_SHADOW_SCALE_SHIFT); } +static inline bool addr_has_shadow (const void *addr) +{ +return (addr >=kasan_shadow_to_mem ((void *)KASAN_SHADOW_START )); +} +void kasan_poison_shadow (const void *address,size_t size,u8 value);void check_memory_region (unsigned long addr,size_t size,bool write,unsigned long ret_ip); +const char *get_bug_type (struct kasan_access_info *info); +void kasan_report (unsigned long addr,size_t size,bool is_write,unsigned long ip);void kasan_report_invalid_free (void *object,unsigned long ip);
mm/kasan/kasan_report.c new file +158 -0
158 lines not on this page · raw message
mm/kasan/khwasan_report.c new file +39 -0
@@ -0,0 +1,39 @@ +/* +* This file contains KHWASAN specific error reporting code. +* +* Copyright (c) 2014 Samsung Electronics Co., Ltd. +* Author: Andrey Ryabinin <[email protected]> +* +* Some code borrowed from https://github.com/xairy/kasan-prototype by +* Andrey Konovalov <[email protected]> +* +* This program is free software; you can redistribute it and/or modify +* it under the terms of the GNU General Public License version 2 as +* published by the Free Software Foundation. +* +*/ + +#include <linux/bitops.h> +#include <linux/ftrace.h> +#include <linux/init.h> +#include <linux/kernel.h> +#include <linux/mm.h> +#include <linux/printk.h> +#include <linux/sched.h> +#include <linux/slab.h> +#include <linux/stackdepot.h> +#include <linux/stacktrace.h> +#include <linux/string.h> +#include <linux/types.h> +#include <linux/kasan.h> +#include <linux/module.h> + +#include <asm/sections.h> + +#include "kasan.h" +#include "../slab.h" + +const char *get_bug_type (struct kasan_access_info *info) +{ +return "invalid-access" ; +}
mm/kasan/report.c +51 -183
315 lines not on this page · raw message
Signature · 1 line
2.19.0.rc0.228.g281dcd1b4d0-goog
PATCH v6 13/18 khwasan: add bug reporting routines
In reply to Andrey KonovalovThis commit adds rountines, that print KHWASAN error reports. Those are quite similar to KASAN, the difference is:
1. The way KHWASAN finds the first bad shadow cell (with a mismatching tag). KHWASAN compares memory tags from the shadow memory to the pointer tag.
2. KHWASAN reports all bugs with the "KASAN: invalid-access" header. This is done, so various external tools that already parse the kernel logs looking for KASAN reports wouldn't need to be changed.
--- include/linux/kasan.h | 3 +++ mm/kasan/kasan.h | 7 +++++ mm/kasan/kasan_report.c | 7 ++--- mm/kasan/khwasan_report.c | 21 +++++++++++++++ mm/kasan/report.c | 57 +++++++++++++++++++++------------------ 5 files changed, 64 insertions(+), 31 deletions(-)
include/linux/kasan.h +3 -0
@@ -174,6 +174,9 @@ void *khwasan_preset_slub_tag(struct kmem_cache *cache, const void *addr);void *khwasan_preset_slab_tag (struct kmem_cache *cache,unsigned int idx,const void *addr); +void kasan_report (unsigned long addr,size_t size, +bool write,unsigned long ip); +#else /* CONFIG_KASAN_HW */ static inline void khwasan_init (void ) { }
mm/kasan/kasan.h +7 -0
@@ -119,8 +119,15 @@ void kasan_poison_shadow(const void *address, size_t size, u8 value);void check_memory_region (unsigned long addr,size_t size,bool write,unsigned long ret_ip); +void *find_first_bad_addr (void *addr,size_t size);const char *get_bug_type (struct kasan_access_info *info); +#ifdef CONFIG_KASAN_HW +void print_tags (u8 addr_tag,const void *addr); +#else +static inline void print_tags (u8 addr_tag,const void *addr) { } +#endif +void kasan_report (unsigned long addr,size_t size,bool is_write,unsigned long ip);void kasan_report_invalid_free (void *object,unsigned long ip);
mm/kasan/kasan_report.c +2 -5
@@ -33,10 +33,10 @@#include "kasan.h" #include "../slab.h" -static const void *find_first_bad_addr (const void *addr,size_t size) +void *find_first_bad_addr (void *addr,size_t size) {u8 shadow_val = *(u8 *)kasan_mem_to_shadow (addr); -const void *first_bad_addr = addr; +void *first_bad_addr = addr;while (!shadow_val && first_bad_addr < addr + size) { first_bad_addr +=KASAN_SHADOW_SCALE_SIZE ;
@@ -50,9 +50,6 @@ static const char *get_shadow_bug_type(struct kasan_access_info *info)const char *bug_type ="unknown-crash" ;u8 *shadow_addr; - info->first_bad_addr =find_first_bad_addr (info->access_addr, - info->access_size); - shadow_addr = (u8 *)kasan_mem_to_shadow (info->first_bad_addr);/*
mm/kasan/khwasan_report.c +21 -0
@@ -37,3 +37,24 @@ const char *get_bug_type(struct kasan_access_info *info) {return "invalid-access" ; } + +void *find_first_bad_addr (void *addr,size_t size) +{ +u8 tag =get_tag (addr); +void *untagged_addr =reset_tag (addr); +u8 *shadow = (u8 *)kasan_mem_to_shadow (untagged_addr); +void *first_bad_addr = untagged_addr; + +while (*shadow == tag && first_bad_addr < untagged_addr + size) { + first_bad_addr +=KASAN_SHADOW_SCALE_SIZE ; + shadow = (u8 *)kasan_mem_to_shadow (first_bad_addr); + } +return first_bad_addr; +} + +void print_tags (u8 addr_tag,const void *addr) +{ +u8 *shadow = (u8 *)kasan_mem_to_shadow (addr); + +pr_err ("Pointer tag: [%02x], memory tag: [%02x]\n" , addr_tag, *shadow); +}
mm/kasan/report.c +31 -26
89 lines not on this page · raw message
Signature · 1 line
2.19.0.rc0.228.g281dcd1b4d0-goog
Dmitry Vyukov
reviewerRe: [PATCH v6 13/18] khwasan: add bug reporting routines
Quoted 39 lines · Andrey Konovalov
This commit adds rountines, that print KHWASAN error reports. Those are quite similar to KASAN, the difference is:
1. The way KHWASAN finds the first bad shadow cell (with a mismatching tag). KHWASAN compares memory tags from the shadow memory to the pointer tag.
2. KHWASAN reports all bugs with the "KASAN: invalid-access" header. This is done, so various external tools that already parse the kernel logs looking for KASAN reports wouldn't need to be changed.
Signed-off-by Andrey Konovalov <[email protected]>--- include/linux/kasan.h | 3 +++ mm/kasan/kasan.h | 7 +++++ mm/kasan/kasan_report.c | 7 ++--- mm/kasan/khwasan_report.c | 21 +++++++++++++++ mm/kasan/report.c | 57 +++++++++++++++++++++------------------ 5 files changed, 64 insertions(+), 31 deletions(-)include/linux/kasan.h +3
-0@@ -174,6 +174,9 @@ void *khwasan_preset_slub_tag(struct kmem_cache *cache, const void *addr);void *khwasan_preset_slab_tag (struct kmem_cache *cache,unsigned int idx,const void *addr); +void kasan_report (unsigned long addr,size_t size, +bool write,unsigned long ip); +#else /* CONFIG_KASAN_HW */ static inline void khwasan_init (void ) { }mm/kasan/kasan.h +0
-0
mm/kasan/kasan.h void kasan_poison_shadow(const void *address, size_t size, u8 value); · line 125
Quoted 7 lines
mm/kasan/kasan.h +2
-0@@ -119,8 +119,15 @@ void kasan_poison_shadow(const void *address, size_t size, u8 value);void check_memory_region (unsigned long addr,size_t size,bool write,unsigned long ret_ip); +void *find_first_bad_addr (void *addr,size_t size);const char *get_bug_type (struct kasan_access_info *info); +#ifdef CONFIG_KASAN_HW
We already have #ifdef CONFIG_KASAN_HW section below with additional functions for KASAN_HW and empty stubs otherwise. I would add this one there as well.
Quoted 42 lines
mm/kasan/kasan.h +5
-0+void print_tags (u8 addr_tag,const void *addr); +#else +static inline void print_tags (u8 addr_tag,const void *addr) { } +#endif +void kasan_report (unsigned long addr,size_t size,bool is_write,unsigned long ip);void kasan_report_invalid_free (void *object,unsigned long ip);mm/kasan/kasan_report.c +2
-5@@ -33,10 +33,10 @@#include "kasan.h" #include "../slab.h" -static const void *find_first_bad_addr (const void *addr,size_t size) +void *find_first_bad_addr (void *addr,size_t size) {u8 shadow_val = *(u8 *)kasan_mem_to_shadow (addr); -const void *first_bad_addr = addr; +void *first_bad_addr = addr;while (!shadow_val && first_bad_addr < addr + size) { first_bad_addr +=KASAN_SHADOW_SCALE_SIZE ;@@ -50,9 +50,6 @@ static const char *get_shadow_bug_type(struct kasan_access_info *info)const char *bug_type ="unknown-crash" ;u8 *shadow_addr; - info->first_bad_addr =find_first_bad_addr (info->access_addr, - info->access_size); - shadow_addr = (u8 *)kasan_mem_to_shadow (info->first_bad_addr);/* mm/kasan/khwasan_report.c +0
-0@@ -37,3 +37,24 @@ const char *get_bug_type(struct kasan_access_info *info) {return "invalid-access" ;
mm/kasan/khwasan_report.c const char *get_bug_type(struct kasan_access_info *info) · line 48
Quoted 10 lines
mm/kasan/khwasan_report.c +9
-0} + +void *find_first_bad_addr (void *addr,size_t size) +{ +u8 tag =get_tag (addr); +void *untagged_addr =reset_tag (addr); +u8 *shadow = (u8 *)kasan_mem_to_shadow (untagged_addr); +void *first_bad_addr = untagged_addr; + +while (*shadow == tag && first_bad_addr < untagged_addr + size) {
I think it's better to check that are within bounds before accessing shadow. Otherwise it's kinda potential out-of-bounds access ;) I know that we _should_ not do an oob here, but still. Also feels that this function can be shortened to something like:
u8 tag = get_tag(addr);
void *p = reset_tag(addr);
void *end = p + size;
while (p < end && tag == *(u8 *)kasan_mem_to_shadow(p))
p += KASAN_SHADOW_SCALE_SIZE;
return p;
Quoted 111 lines
mm/kasan/khwasan_report.c +12
-0+ first_bad_addr +=KASAN_SHADOW_SCALE_SIZE ; + shadow = (u8 *)kasan_mem_to_shadow (first_bad_addr); + } +return first_bad_addr; +} + +void print_tags (u8 addr_tag,const void *addr) +{ +u8 *shadow = (u8 *)kasan_mem_to_shadow (addr); + +pr_err ("Pointer tag: [%02x], memory tag: [%02x]\n" , addr_tag, *shadow); +}mm/kasan/report.c +31
-27@@ -64,11 +64,10 @@ static int __init kasan_set_multi_shot(char *str) }__setup ("kasan_multi_shot" , kasan_set_multi_shot); -static void print_error_description (struct kasan_access_info *info, -const char *bug_type) +static void print_error_description (struct kasan_access_info *info) {pr_err ("BUG: KASAN: %s in %pS\n" , - bug_type, (void *)info->ip); +get_bug_type (info), (void *)info->ip);pr_err ("%s of size %zu at addr %px by task %s/%d\n" , info->is_write ?"Write" :"Read" , info->access_size, info->access_addr, current->comm,task_pid_nr (current));@@ -272,6 +271,8 @@ void kasan_report_invalid_free(void *object, unsigned long ip)start_report (&flags);pr_err ("BUG: KASAN: double-free or invalid-free in %pS\n" , (void *)ip); +print_tags (get_tag (object),reset_tag (object)); + object =reset_tag (object);pr_err ("\n" );print_address_description (object);pr_err ("\n" );@@ -279,41 +280,45 @@ void kasan_report_invalid_free(void *object, unsigned long ip)end_report (&flags); } -static void kasan_report_error (struct kasan_access_info *info) -{ -unsigned long flags; - -start_report (&flags); - -print_error_description (info,get_bug_type (info)); -pr_err ("\n" ); - -if (!addr_has_shadow (info->access_addr)) { -dump_stack (); - }else { -print_address_description ((void *)info->access_addr); -pr_err ("\n" ); -print_shadow_for_address (info->first_bad_addr); - } - -end_report (&flags); -} -void kasan_report (unsigned long addr,size_t size,bool is_write,unsigned long ip) {struct kasan_access_info info; +void *tagged_addr; +void *untagged_addr; +unsigned long flags;if (likely (!report_enabled ()))return ;disable_trace_on_warning (); - info.access_addr = (void *)addr; - info.first_bad_addr = (void *)addr; + tagged_addr = (void *)addr; + untagged_addr =reset_tag (tagged_addr); + + info.access_addr = tagged_addr; +if (addr_has_shadow (untagged_addr)) + info.first_bad_addr =find_first_bad_addr (tagged_addr, size); +else + info.first_bad_addr = untagged_addr; info.access_size = size; info.is_write = is_write; info.ip = ip; -kasan_report_error (&info); +start_report (&flags); + +print_error_description (&info); +if (addr_has_shadow (untagged_addr)) +print_tags (get_tag (tagged_addr), info.first_bad_addr); +pr_err ("\n" ); + +if (addr_has_shadow (untagged_addr)) { +print_address_description (untagged_addr); +pr_err ("\n" ); +print_shadow_for_address (info.first_bad_addr); + }else { +dump_stack (); + } + +end_report (&flags); } --2.19.0.rc0.228.g281dcd1b4d0-goog
Re: [PATCH v6 13/18] khwasan: add bug reporting routines
Quoted 7 lines · Dmitry Vyukov
Quoted 1 line · Andrey Konovalov
+#ifdef
CONFIG_KASAN_HWWe already have #ifdef
CONFIG_KASAN_HWsection below with additional functions forKASAN_HWand empty stubs otherwise. I would add this one there as well.
Will do in v7.
Quoted 20 lines
Quoted 4 lines
(quoted diff) +3
-0+void print_tags(u8 addr_tag, const void *addr); +#else +static inline void print_tags(u8 addr_tag, const void *addr) { }Review
Quoted 10 lines
(quoted diff) +9
-0+#endif +void *find_first_bad_addr(void *addr, size_t size) +{ + u8 tag = get_tag(addr); + void *untagged_addr = reset_tag(addr); + u8 *shadow = (u8 *)kasan_mem_to_shadow(untagged_addr); + void *first_bad_addr = untagged_addr; + + while (*shadow == tag && first_bad_addr < untagged_addr + size) {I think it's better to check that are within bounds before accessing shadow. Otherwise it's kinda potential out-of-bounds access ;) I know that we _should_ not do an oob here, but still. Also feels that this function can be shortened to something like:
Quoted 7 lines
u8 tag =
get_tag(addr);
void *p =reset_tag(addr);
void *end = p + size;while (p < end && tag == *(u8 *)kasan_mem_to_shadow(p))
p += KASAN_SHADOW_SCALE_SIZE;return p;
Will do in v7.
PATCH v6 14/18 khwasan: add hooks implementation
In reply to Andrey KonovalovThis commit adds KHWASAN specific hooks implementation and adjusts common KASAN and KHWASAN ones.
1. When a new slab cache is created, KHWASAN rounds up the size of the objects in this cache to KASAN_SHADOW_SCALE_SIZE (== 16).
2. On each kmalloc KHWASAN generates a random tag, sets the shadow memory, that corresponds to this object to this tag, and embeds this tag value into the top byte of the returned pointer.
3. On each kfree KHWASAN poisons the shadow memory with a random tag to allow detection of use-after-free bugs.
The rest of the logic of the hook implementation is very much similar to the one provided by KASAN. KHWASAN saves allocation and free stack metadata to the slab object the same was KASAN does this.
--- mm/kasan/common.c | 82 +++++++++++++++++++++++++++++++++++----------- mm/kasan/kasan.h | 8 +++++ mm/kasan/khwasan.c | 40 ++++++++++++++++++++++ 3 files changed, 111 insertions(+), 19 deletions(-)
mm/kasan/common.c +63 -19
@@ -140,6 +140,9 @@ void kasan_poison_shadow(const void *address, size_t size, u8 value) {void *shadow_start, *shadow_end; +/* Perform shadow offset calculation based on untagged address */ + address =reset_tag (address); + shadow_start =kasan_mem_to_shadow (address); shadow_end =kasan_mem_to_shadow (address + size);
@@ -148,11 +151,20 @@ void kasan_poison_shadow(const void *address, size_t size, u8 value)void kasan_unpoison_shadow (const void *address,size_t size) { -kasan_poison_shadow (address, size,0 ); +u8 tag =get_tag (address); + +/* Perform shadow offset calculation based on untagged address */ + address =reset_tag (address); + +kasan_poison_shadow (address, size, tag);if (size &KASAN_SHADOW_MASK ) {u8 *shadow = (u8 *)kasan_mem_to_shadow (address + size); - *shadow = size &KASAN_SHADOW_MASK ; + +if (IS_ENABLED (CONFIG_KASAN_HW )) + *shadow = tag; +else + *shadow = size &KASAN_SHADOW_MASK ; } }
@@ -200,8 +212,9 @@ void kasan_unpoison_stack_above_sp_to(const void *watermark)void kasan_alloc_pages (struct page *page,unsigned int order) { -if (likely (!PageHighMem (page))) -kasan_unpoison_shadow (page_address (page),PAGE_SIZE << order); +if (unlikely (PageHighMem (page))) +return ; +kasan_unpoison_shadow (page_address (page),PAGE_SIZE << order); }void kasan_free_pages (struct page *page,unsigned int order)
@@ -235,6 +248,7 @@ void kasan_cache_create(struct kmem_cache *cache, unsigned int *size, slab_flags_t *flags) {unsigned int orig_size = *size; +unsigned int redzone_size =0 ;int redzone_adjust;/* Add alloc meta. */
@@ -242,20 +256,20 @@ void kasan_cache_create(struct kmem_cache *cache, unsigned int *size, *size +=sizeof (struct kasan_alloc_meta );/* Add free meta. */ -if (cache->flags &SLAB_TYPESAFE_BY_RCU || cache->ctor || - cache->object_size <sizeof (struct kasan_free_meta )) { +if (IS_ENABLED (CONFIG_KASAN_GENERIC ) && + (cache->flags &SLAB_TYPESAFE_BY_RCU || cache->ctor || + cache->object_size <sizeof (struct kasan_free_meta ))) { cache->kasan_info.free_meta_offset = *size; *size +=sizeof (struct kasan_free_meta ); } - redzone_adjust =optimal_redzone (cache->object_size) - - (*size - cache->object_size); + redzone_size =optimal_redzone (cache->object_size); + redzone_adjust = redzone_size - (*size - cache->object_size);if (redzone_adjust >0 ) *size += redzone_adjust; *size = min_t(unsigned int, KMALLOC_MAX_SIZE, -max (*size, cache->object_size + -optimal_redzone (cache->object_size))); +max (*size, cache->object_size + redzone_size));/* * If the metadata doesn't fit, don't enable KASAN at all.
@@ -268,6 +282,8 @@ void kasan_cache_create(struct kmem_cache *cache, unsigned int *size,return ; } + cache->align =round_up (cache->align,KASAN_SHADOW_SCALE_SIZE ); + *flags |=SLAB_KASAN ; }
@@ -328,15 +344,30 @@ void *kasan_slab_alloc(struct kmem_cache *cache, void *object, gfp_t flags)return kasan_kmalloc (cache, object, cache->object_size, flags); } +static inline bool shadow_invalid (u8 tag,s8 shadow_byte) +{ +if (IS_ENABLED (CONFIG_KASAN_GENERIC )) +return shadow_byte <0 || + shadow_byte >=KASAN_SHADOW_SCALE_SIZE ; +else +return tag != (u8 )shadow_byte; +} +static bool __kasan_slab_free (struct kmem_cache *cache,void *object,unsigned long ip,bool quarantine) {s8 shadow_byte; +u8 tag; +void *tagged_object;unsigned long rounded_up_size; + tag =get_tag (object); + tagged_object = object; + object =reset_tag (object); +if (unlikely (nearest_obj (cache,virt_to_head_page (object), object) != object)) { -kasan_report_invalid_free (object, ip); +kasan_report_invalid_free (tagged_object, ip);return true; }
@@ -345,20 +376,22 @@ static bool __kasan_slab_free(struct kmem_cache *cache, void *object,return false; shadow_byte =READ_ONCE (*(s8 *)kasan_mem_to_shadow (object)); -if (shadow_byte <0 || shadow_byte >=KASAN_SHADOW_SCALE_SIZE ) { -kasan_report_invalid_free (object, ip); +if (shadow_invalid (tag, shadow_byte)) { +kasan_report_invalid_free (tagged_object, ip);return true; } rounded_up_size =round_up (cache->object_size,KASAN_SHADOW_SCALE_SIZE );kasan_poison_shadow (object, rounded_up_size,KASAN_KMALLOC_FREE ); -if (!quarantine ||unlikely (!(cache->flags &SLAB_KASAN ))) +if ((IS_ENABLED (CONFIG_KASAN_GENERIC ) && !quarantine) || +unlikely (!(cache->flags &SLAB_KASAN )))return false;set_track (&get_alloc_info (cache, object)->free_track,GFP_NOWAIT );quarantine_put (get_free_info (cache, object), cache); -return true; + +return IS_ENABLED (CONFIG_KASAN_GENERIC ); }bool kasan_slab_free (struct kmem_cache *cache,void *object,unsigned long ip)
@@ -371,6 +404,7 @@ void *kasan_kmalloc(struct kmem_cache *cache, const void *object, size_t size, {unsigned long redzone_start;unsigned long redzone_end; +u8 tag;if (gfpflags_allow_blocking (flags))quarantine_reduce ();
@@ -383,14 +417,24 @@ void *kasan_kmalloc(struct kmem_cache *cache, const void *object, size_t size, redzone_end =round_up ((unsigned long )object + cache->object_size,KASAN_SHADOW_SCALE_SIZE ); -kasan_unpoison_shadow (object, size); +/* +* Objects with contructors and objects from SLAB_TYPESAFE_BY_RCU slabs +* have tags preassigned and are already tagged. +*/ +if (IS_ENABLED (CONFIG_KASAN_HW ) && + (cache->ctor || cache->flags &SLAB_TYPESAFE_BY_RCU )) + tag =get_tag (object); +else + tag =random_tag (); + +kasan_unpoison_shadow (set_tag (object, tag), size);kasan_poison_shadow ((void *)redzone_start, redzone_end - redzone_start,KASAN_KMALLOC_REDZONE );if (cache->flags &SLAB_KASAN )set_track (&get_alloc_info (cache, object)->alloc_track, flags); -return (void *)object; +return set_tag (object, tag); }EXPORT_SYMBOL (kasan_kmalloc);
@@ -440,7 +484,7 @@ void kasan_poison_kfree(void *ptr, unsigned long ip) page =virt_to_head_page (ptr);if (unlikely (!PageSlab (page))) { -if (ptr !=page_address (page)) { +if (reset_tag (ptr) !=page_address (page)) {kasan_report_invalid_free (ptr, ip);return ; }
@@ -453,7 +497,7 @@ void kasan_poison_kfree(void *ptr, unsigned long ip)void kasan_kfree_large (void *ptr,unsigned long ip) { -if (ptr !=page_address (virt_to_head_page (ptr))) +if (reset_tag (ptr) !=page_address (virt_to_head_page (ptr)))kasan_report_invalid_free (ptr, ip);/* The object will be poisoned by page_alloc. */ }
mm/kasan/kasan.h +8 -0
@@ -12,10 +12,18 @@#define KHWASAN_TAG_INVALID 0xFE/* inaccessible memory tag */ #define KHWASAN_TAG_MAX 0xFD/* maximum value for random tags */ +#ifdef CONFIG_KASAN_GENERIC #define KASAN_FREE_PAGE 0xFF/* page was freed */ #define KASAN_PAGE_REDZONE 0xFE/* redzone for kmalloc_large allocations */ #define KASAN_KMALLOC_REDZONE 0xFC/* redzone inside slub object */ #define KASAN_KMALLOC_FREE 0xFB/* object was freed (kmem_cache_free/kfree) */ +#else +#define KASAN_FREE_PAGE KHWASAN_TAG_INVALID +#define KASAN_PAGE_REDZONE KHWASAN_TAG_INVALID +#define KASAN_KMALLOC_REDZONE KHWASAN_TAG_INVALID +#define KASAN_KMALLOC_FREE KHWASAN_TAG_INVALID +#endif +#define KASAN_GLOBAL_REDZONE 0xFA/* redzone for global variable */ /*
mm/kasan/khwasan.c +40 -0
@@ -106,15 +106,52 @@ void *khwasan_preset_slab_tag(struct kmem_cache *cache, unsigned int idx,void check_memory_region (unsigned long addr,size_t size,bool write,unsigned long ret_ip) { +u8 tag; +u8 *shadow_first, *shadow_last, *shadow; +void *untagged_addr; + + tag =get_tag ((const void *)addr); + +/* Ignore accesses for pointers tagged with 0xff (native kernel +* pointer tag) to suppress false positives caused by kmap. +* +* Some kernel code was written to account for archs that don't keep +* high memory mapped all the time, but rather map and unmap particular +* pages when needed. Instead of storing a pointer to the kernel memory, +* this code saves the address of the page structure and offset within +* that page for later use. Those pages are then mapped and unmapped +* with kmap/kunmap when necessary and virt_to_page is used to get the +* virtual address of the page. For arm64 (that keeps the high memory +* mapped all the time), kmap is turned into a page_address call. + +* The issue is that with use of the page_address + virt_to_page +* sequence the top byte value of the original pointer gets lost (gets +* set to KHWASAN_TAG_KERNEL (0xFF). +*/ +if (tag ==KHWASAN_TAG_KERNEL ) +return ; + + untagged_addr =reset_tag ((const void *)addr); + shadow_first =kasan_mem_to_shadow (untagged_addr); + shadow_last =kasan_mem_to_shadow (untagged_addr + size -1 ); + +for (shadow = shadow_first; shadow <= shadow_last; shadow++) { +if (*shadow != tag) { +kasan_report (addr, size, write, ret_ip); +return ; + } + } }#define DEFINE_HWASAN_LOAD_STORE (size) \ void __hwasan_load##size##_noabort(unsigned long addr) \ { \ + check_memory_region(addr, size, false, _RET_IP_); \ } \ EXPORT_SYMBOL(__hwasan_load##size##_noabort); \ void __hwasan_store##size##_noabort(unsigned long addr) \ { \ + check_memory_region(addr, size, true, _RET_IP_); \ } \ EXPORT_SYMBOL(__hwasan_store##size##_noabort)
@@ -126,15 +163,18 @@ DEFINE_HWASAN_LOAD_STORE(16);void __hwasan_loadN_noabort (unsigned long addr,unsigned long size) { +check_memory_region (addr, size, false, _RET_IP_); }EXPORT_SYMBOL (__hwasan_loadN_noabort);void __hwasan_storeN_noabort (unsigned long addr,unsigned long size) { +check_memory_region (addr, size, true, _RET_IP_); }EXPORT_SYMBOL (__hwasan_storeN_noabort);void __hwasan_tag_memory (unsigned long addr,u8 tag,unsigned long size) { +kasan_poison_shadow ((void *)addr, size, tag); }EXPORT_SYMBOL (__hwasan_tag_memory);
Signature · 1 line
2.19.0.rc0.228.g281dcd1b4d0-goog
Dmitry Vyukov
reviewerRe: [PATCH v6 14/18] khwasan: add hooks implementation
Quoted 39 lines · Andrey Konovalov
This commit adds KHWASAN specific hooks implementation and adjusts common KASAN and KHWASAN ones.
1. When a new slab cache is created, KHWASAN rounds up the size of the objects in this cache to
KASAN_SHADOW_SCALE_SIZE(== 16).2. On each kmalloc KHWASAN generates a random tag, sets the shadow memory, that corresponds to this object to this tag, and embeds this tag value into the top byte of the returned pointer.
3. On each kfree KHWASAN poisons the shadow memory with a random tag to allow detection of use-after-free bugs.
The rest of the logic of the hook implementation is very much similar to the one provided by KASAN. KHWASAN saves allocation and free stack metadata to the slab object the same was KASAN does this.
Signed-off-by Andrey Konovalov <[email protected]>--- mm/kasan/common.c | 82 +++++++++++++++++++++++++++++++++++----------- mm/kasan/kasan.h | 8 +++++ mm/kasan/khwasan.c | 40 ++++++++++++++++++++++ 3 files changed, 111 insertions(+), 19 deletions(-)mm/kasan/common.c +3
-0@@ -140,6 +140,9 @@ void kasan_poison_shadow(const void *address, size_t size, u8 value) {void *shadow_start, *shadow_end; +/* Perform shadow offset calculation based on untagged address */ + address =reset_tag (address); + shadow_start =kasan_mem_to_shadow (address); shadow_end =kasan_mem_to_shadow (address + size);
mm/kasan/common.c void kasan_poison_shadow(const void *address, size_t size, u8 value) · line 156
Quoted 7 lines
mm/kasan/common.c +3
-1@@ -148,11 +151,20 @@ void kasan_poison_shadow(const void *address, size_t size, u8 value)void kasan_unpoison_shadow (const void *address,size_t size) { -kasan_poison_shadow (address, size,0 ); +u8 tag =get_tag (address); + +/* Perform shadow offset calculation based on untagged address */
The comment is not super-useful. It would be more useful to say why we need to do this.
Most callers explicitly untag pointer passed to this function, for some it's unclear if the pointer contains tag or not.
For example, __hwasan_tag_memory -- what does it accept? Tagged or untagged?
Quoted 4 lines
mm/kasan/common.c +3
-0+ address =reset_tag (address); + +kasan_poison_shadow (address, size, tag);
mm/kasan/common.c void kasan_poison_shadow(const void *address, size_t size, u8 value) · line 169
Quoted 10 lines
mm/kasan/common.c +5
-1if (size &KASAN_SHADOW_MASK ) {u8 *shadow = (u8 *)kasan_mem_to_shadow (address + size); - *shadow = size &KASAN_SHADOW_MASK ; + +if (IS_ENABLED (CONFIG_KASAN_HW )) + *shadow = tag; +else + *shadow = size &KASAN_SHADOW_MASK ; } }
It seems that this function is just different for kasan and khwasan. Currently for kasan we have:
kasan_poison_shadow(address, size, tag);
if (size & KASAN_SHADOW_MASK) {
u8 *shadow = (u8 *)kasan_mem_to_shadow(address + size);
*shadow = size & KASAN_SHADOW_MASK;
}But what we want to say for khwasan is:
kasan_poison_shadow(address, round_up(size, KASAN_SHADOW_SCALE_SIZE), get_tag(address));
Not sure if we want to keep a common implementation or just have separate implementations...
Quoted 14 lines
mm/kasan/common.c +3
-2@@ -200,8 +212,9 @@ void kasan_unpoison_stack_above_sp_to(const void *watermark)void kasan_alloc_pages (struct page *page,unsigned int order) { -if (likely (!PageHighMem (page))) -kasan_unpoison_shadow (page_address (page),PAGE_SIZE << order); +if (unlikely (PageHighMem (page))) +return ; +kasan_unpoison_shadow (page_address (page),PAGE_SIZE << order); }void kasan_free_pages (struct page *page,unsigned int order)
mm/kasan/common.c void kasan_cache_create(struct kmem_cache *cache, unsigned int *size, · line 251
Quoted 4 lines
mm/kasan/common.c +1
-0@@ -235,6 +248,7 @@ void kasan_cache_create(struct kmem_cache *cache, unsigned int *size, slab_flags_t *flags) {unsigned int orig_size = *size; +unsigned int redzone_size =0 ;
This variable seems to be always initialized below. We don't general initialize local variables in this case.
Quoted 179 lines
mm/kasan/common.c +45
-15int redzone_adjust;/* Add alloc meta. */ @@ -242,20 +256,20 @@ void kasan_cache_create(struct kmem_cache *cache, unsigned int *size, *size +=sizeof (struct kasan_alloc_meta );/* Add free meta. */ -if (cache->flags &SLAB_TYPESAFE_BY_RCU || cache->ctor || - cache->object_size <sizeof (struct kasan_free_meta )) { +if (IS_ENABLED (CONFIG_KASAN_GENERIC ) && + (cache->flags &SLAB_TYPESAFE_BY_RCU || cache->ctor || + cache->object_size <sizeof (struct kasan_free_meta ))) { cache->kasan_info.free_meta_offset = *size; *size +=sizeof (struct kasan_free_meta ); } - redzone_adjust =optimal_redzone (cache->object_size) - - (*size - cache->object_size); + redzone_size =optimal_redzone (cache->object_size); + redzone_adjust = redzone_size - (*size - cache->object_size);if (redzone_adjust >0 ) *size += redzone_adjust; *size = min_t(unsigned int, KMALLOC_MAX_SIZE, -max (*size, cache->object_size + -optimal_redzone (cache->object_size))); +max (*size, cache->object_size + redzone_size));/* * If the metadata doesn't fit, don't enable KASAN at all. @@ -268,6 +282,8 @@ void kasan_cache_create(struct kmem_cache *cache, unsigned int *size,return ; } + cache->align =round_up (cache->align,KASAN_SHADOW_SCALE_SIZE ); + *flags |=SLAB_KASAN ; }@@ -328,15 +344,30 @@ void *kasan_slab_alloc(struct kmem_cache *cache, void *object, gfp_t flags)return kasan_kmalloc (cache, object, cache->object_size, flags); } +static inline bool shadow_invalid (u8 tag,s8 shadow_byte) +{ +if (IS_ENABLED (CONFIG_KASAN_GENERIC )) +return shadow_byte <0 || + shadow_byte >=KASAN_SHADOW_SCALE_SIZE ; +else +return tag != (u8 )shadow_byte; +} +static bool __kasan_slab_free (struct kmem_cache *cache,void *object,unsigned long ip,bool quarantine) {s8 shadow_byte; +u8 tag; +void *tagged_object;unsigned long rounded_up_size; + tag =get_tag (object); + tagged_object = object; + object =reset_tag (object); +if (unlikely (nearest_obj (cache,virt_to_head_page (object), object) != object)) { -kasan_report_invalid_free (object, ip); +kasan_report_invalid_free (tagged_object, ip);return true; }@@ -345,20 +376,22 @@ static bool __kasan_slab_free(struct kmem_cache *cache, void *object,return false; shadow_byte =READ_ONCE (*(s8 *)kasan_mem_to_shadow (object)); -if (shadow_byte <0 || shadow_byte >=KASAN_SHADOW_SCALE_SIZE ) { -kasan_report_invalid_free (object, ip); +if (shadow_invalid (tag, shadow_byte)) { +kasan_report_invalid_free (tagged_object, ip);return true; } rounded_up_size =round_up (cache->object_size,KASAN_SHADOW_SCALE_SIZE );kasan_poison_shadow (object, rounded_up_size,KASAN_KMALLOC_FREE ); -if (!quarantine ||unlikely (!(cache->flags &SLAB_KASAN ))) +if ((IS_ENABLED (CONFIG_KASAN_GENERIC ) && !quarantine) || +unlikely (!(cache->flags &SLAB_KASAN )))return false;set_track (&get_alloc_info (cache, object)->free_track,GFP_NOWAIT );quarantine_put (get_free_info (cache, object), cache); -return true; + +return IS_ENABLED (CONFIG_KASAN_GENERIC ); }bool kasan_slab_free (struct kmem_cache *cache,void *object,unsigned long ip)@@ -371,6 +404,7 @@ void *kasan_kmalloc(struct kmem_cache *cache, const void *object, size_t size, {unsigned long redzone_start;unsigned long redzone_end; +u8 tag;if (gfpflags_allow_blocking (flags))quarantine_reduce ();@@ -383,14 +417,24 @@ void *kasan_kmalloc(struct kmem_cache *cache, const void *object, size_t size, redzone_end =round_up ((unsigned long )object + cache->object_size,KASAN_SHADOW_SCALE_SIZE ); -kasan_unpoison_shadow (object, size); +/* +* Objects with contructors and objects from SLAB_TYPESAFE_BY_RCU slabs +* have tags preassigned and are already tagged. +*/ +if (IS_ENABLED (CONFIG_KASAN_HW ) && + (cache->ctor || cache->flags &SLAB_TYPESAFE_BY_RCU )) + tag =get_tag (object); +else + tag =random_tag (); + +kasan_unpoison_shadow (set_tag (object, tag), size);kasan_poison_shadow ((void *)redzone_start, redzone_end - redzone_start,KASAN_KMALLOC_REDZONE );if (cache->flags &SLAB_KASAN )set_track (&get_alloc_info (cache, object)->alloc_track, flags); -return (void *)object; +return set_tag (object, tag); }EXPORT_SYMBOL (kasan_kmalloc);@@ -440,7 +484,7 @@ void kasan_poison_kfree(void *ptr, unsigned long ip) page =virt_to_head_page (ptr);if (unlikely (!PageSlab (page))) { -if (ptr !=page_address (page)) { +if (reset_tag (ptr) !=page_address (page)) {kasan_report_invalid_free (ptr, ip);return ; }@@ -453,7 +497,7 @@ void kasan_poison_kfree(void *ptr, unsigned long ip)void kasan_kfree_large (void *ptr,unsigned long ip) { -if (ptr !=page_address (virt_to_head_page (ptr))) +if (reset_tag (ptr) !=page_address (virt_to_head_page (ptr)))kasan_report_invalid_free (ptr, ip);/* The object will be poisoned by page_alloc. */ }mm/kasan/kasan.h +8
-0@@ -12,10 +12,18 @@#define KHWASAN_TAG_INVALID 0xFE/* inaccessible memory tag */ #define KHWASAN_TAG_MAX 0xFD/* maximum value for random tags */ +#ifdef CONFIG_KASAN_GENERIC #define KASAN_FREE_PAGE 0xFF/* page was freed */ #define KASAN_PAGE_REDZONE 0xFE/* redzone for kmalloc_large allocations */ #define KASAN_KMALLOC_REDZONE 0xFC/* redzone inside slub object */ #define KASAN_KMALLOC_FREE 0xFB/* object was freed (kmem_cache_free/kfree) */ +#else +#define KASAN_FREE_PAGE KHWASAN_TAG_INVALID +#define KASAN_PAGE_REDZONE KHWASAN_TAG_INVALID +#define KASAN_KMALLOC_REDZONE KHWASAN_TAG_INVALID +#define KASAN_KMALLOC_FREE KHWASAN_TAG_INVALID +#endif +#define KASAN_GLOBAL_REDZONE 0xFA/* redzone for global variable */ /* mm/kasan/khwasan.c +0
-0
mm/kasan/khwasan.c void *khwasan_preset_slab_tag(struct kmem_cache *cache, unsigned int idx, · line 115
Quoted 10 lines
mm/kasan/khwasan.c +7
-0@@ -106,15 +106,52 @@ void *khwasan_preset_slab_tag(struct kmem_cache *cache, unsigned int idx,void check_memory_region (unsigned long addr,size_t size,bool write,unsigned long ret_ip) { +u8 tag; +u8 *shadow_first, *shadow_last, *shadow; +void *untagged_addr; + + tag =get_tag ((const void *)addr); + +/* Ignore accesses for pointers tagged with 0xff (native kernel
/* on a separate line
Quoted 4 lines
mm/kasan/khwasan.c +4
-0+* pointer tag) to suppress false positives caused by kmap. +* +* Some kernel code was written to account for archs that don't keep +* high memory mapped all the time, but rather map and unmap particular
mm/kasan/khwasan.c void *khwasan_preset_slab_tag(struct kmem_cache *cache, unsigned int idx, · line 129
Quoted 10 lines
mm/kasan/khwasan.c +10
-0+* pages when needed. Instead of storing a pointer to the kernel memory, +* this code saves the address of the page structure and offset within +* that page for later use. Those pages are then mapped and unmapped +* with kmap/kunmap when necessary and virt_to_page is used to get the +* virtual address of the page. For arm64 (that keeps the high memory +* mapped all the time), kmap is turned into a page_address call. + +* The issue is that with use of the page_address + virt_to_page +* sequence the top byte value of the original pointer gets lost (gets +* set to KHWASAN_TAG_KERNEL (0xFF).
Missed closing bracket.
Quoted 50 lines
mm/kasan/khwasan.c +19
-1+*/ +if (tag ==KHWASAN_TAG_KERNEL ) +return ; + + untagged_addr =reset_tag ((const void *)addr); + shadow_first =kasan_mem_to_shadow (untagged_addr); + shadow_last =kasan_mem_to_shadow (untagged_addr + size -1 ); + +for (shadow = shadow_first; shadow <= shadow_last; shadow++) { +if (*shadow != tag) { +kasan_report (addr, size, write, ret_ip); +return ; + } + } }#define DEFINE_HWASAN_LOAD_STORE (size) \ void __hwasan_load##size##_noabort(unsigned long addr) \ { \ + check_memory_region(addr, size, false, _RET_IP_); \ } \ EXPORT_SYMBOL(__hwasan_load##size##_noabort); \ void __hwasan_store##size##_noabort(unsigned long addr) \ { \ + check_memory_region(addr, size, true, _RET_IP_); \ } \ EXPORT_SYMBOL(__hwasan_store##size##_noabort)@@ -126,15 +163,18 @@ DEFINE_HWASAN_LOAD_STORE(16);void __hwasan_loadN_noabort (unsigned long addr,unsigned long size) { +check_memory_region (addr, size, false, _RET_IP_); }EXPORT_SYMBOL (__hwasan_loadN_noabort);void __hwasan_storeN_noabort (unsigned long addr,unsigned long size) { +check_memory_region (addr, size, true, _RET_IP_); }EXPORT_SYMBOL (__hwasan_storeN_noabort);void __hwasan_tag_memory (unsigned long addr,u8 tag,unsigned long size) { +kasan_poison_shadow ((void *)addr, size, tag); }EXPORT_SYMBOL (__hwasan_tag_memory); --2.19.0.rc0.228.g281dcd1b4d0-goog
Re: [PATCH v6 14/18] khwasan: add hooks implementation
Quoted 9 lines · Dmitry Vyukov
mm/kasan/common.c void kasan_poison_shadow(const void *address, size_t size, u8 value) · line 156
Quoted 6 lines · Andrey Konovalov
void
kasan_unpoison_shadow(const void *address, size_t size){mm/kasan/common.c +3
-1-kasan_poison_shadow (address, size,0 ); +u8 tag =get_tag (address); + +/* Perform shadow offset calculation based on untagged address */
Quoted 5 lines
The comment is not super-useful. It would be more useful to say why we need to do this.
Most callers explicitly untag pointer passed to this function, for some it's unclear if the pointer contains tag or not.
For example,__hwasan_tag_memory-- what does it accept? Tagged or untagged?
There are some callers that pass tagged pointers to this functions, e.g. ksize or kasan_unpoison_object_data. I'll expand the comment.
Quoted 27 lines
Quoted 6 lines
mm/kasan/common.c +3
-0+ address =reset_tag (address); + +kasan_poison_shadow (address, size, tag);mm/kasan/common.c void kasan_poison_shadow(const void *address, size_t size, u8 value) · line 169
Quoted 10 lines
mm/kasan/common.c +5
-1if (size &KASAN_SHADOW_MASK ) {u8 *shadow = (u8 *)kasan_mem_to_shadow (address + size); - *shadow = size &KASAN_SHADOW_MASK ; + +if (IS_ENABLED (CONFIG_KASAN_HW )) + *shadow = tag; +else + *shadow = size &KASAN_SHADOW_MASK ; } }It seems that this function is just different for kasan and khwasan. Currently for kasan we have:
kasan_poison_shadow(address, size, tag);if (size & KASAN_SHADOW_MASK) { u8 *shadow = (u8 *)kasan_mem_to_shadow(address + size); *shadow = size & KASAN_SHADOW_MASK; }
Quoted 7 lines
But what we want to say for khwasan is:
kasan_poison_shadow(address, round_up(size, KASAN_SHADOW_SCALE_SIZE), get_tag(address));Not sure if we want to keep a common implementation or just have separate implementations...
As per offline discussion leaving as is.
Quoted 9 lines
mm/kasan/common.c void kasan_cache_create(struct kmem_cache *cache, unsigned int *size, · line 251
Quoted 6 lines
void
kasan_free_pages(struct page *page, unsigned int order)mm/kasan/common.c +1
-0@@ -235,6 +248,7 @@ void kasan_cache_create(struct kmem_cache *cache, unsigned int *size, slab_flags_t *flags) {unsigned int orig_size = *size; +unsigned int redzone_size =0 ;This variable seems to be always initialized below. We don't general initialize local variables in this case.
Will fix in v7.
Quoted 4 lines
Quoted 4 lines
mm/kasan/khwasan.c +1
-0void check_memory_region (unsigned long addr,size_t size,bool write,unsigned long ret_ip) { +u8 tag;
Quoted 8 lines
mm/kasan/khwasan.c void *khwasan_preset_slab_tag(struct kmem_cache *cache, unsigned int idx, · line 115
Quoted 6 lines
mm/kasan/khwasan.c +6
-0+u8 *shadow_first, *shadow_last, *shadow; +void *untagged_addr; + + tag =get_tag ((const void *)addr); + +/* Ignore accesses for pointers tagged with 0xff (native kernel /* on a separate line
Will fix in v7.
Quoted 16 lines
Quoted 5 lines
mm/kasan/khwasan.c +4
-0+* pointer tag) to suppress false positives caused by kmap. +* +* Some kernel code was written to account for archs that don't keep +* high memory mapped all the time, but rather map and unmap particular mm/kasan/khwasan.c void *khwasan_preset_slab_tag(struct kmem_cache *cache, unsigned int idx, · line 129
Quoted 10 lines
mm/kasan/khwasan.c +10
-0+* pages when needed. Instead of storing a pointer to the kernel memory, +* this code saves the address of the page structure and offset within +* that page for later use. Those pages are then mapped and unmapped +* with kmap/kunmap when necessary and virt_to_page is used to get the +* virtual address of the page. For arm64 (that keeps the high memory +* mapped all the time), kmap is turned into a page_address call. + +* The issue is that with use of the page_address + virt_to_page +* sequence the top byte value of the original pointer gets lost (gets +* set to KHWASAN_TAG_KERNEL (0xFF).
Quoted 1 line
Missed closing bracket.
Will fix in v7.
PATCH v6 15/18 khwasan, arm64: add brk handler for inline instrumentation
In reply to Andrey KonovalovKHWASAN inline instrumentation mode (which embeds checks of shadow memory into the generated code, instead of inserting a callback) generates a brk instruction when a tag mismatch is detected.
This commit add a KHWASAN brk handler, that decodes the immediate value passed to the brk instructions (to extract information about the memory access that triggered the mismatch), reads the register values (x0 contains the guilty address) and reports the bug.
--- arch/arm64/include/asm/brk-imm.h | 2 + arch/arm64/kernel/traps.c | 69 +++++++++++++++++++++++++++++++- 2 files changed, 69 insertions(+), 2 deletions(-)
arch/arm64/include/asm/brk-imm.h +2 -0
@@ -16,10 +16,12 @@* 0x400: for dynamic BRK instruction * 0x401: for compile time BRK instruction * 0x800: kernel-mode BUG() and WARN() traps +* 0x9xx: KHWASAN trap (allowed values 0x900 - 0x9ff) */ #define FAULT_BRK_IMM 0x100#define KGDB_DYN_DBG_BRK_IMM 0x400#define KGDB_COMPILED_DBG_BRK_IMM 0x401#define BUG_BRK_IMM 0x800 +#define KHWASAN_BRK_IMM 0x900#endif
arch/arm64/kernel/traps.c +67 -2
106 lines not on this page · raw message
Signature · 1 line
2.19.0.rc0.228.g281dcd1b4d0-goog
Dmitry Vyukov
reviewerRe: [PATCH v6 15/18] khwasan, arm64: add brk handler for inline instrumentation
Quoted 95 lines · Andrey Konovalov
KHWASAN inline instrumentation mode (which embeds checks of shadow memory into the generated code, instead of inserting a callback) generates a brk instruction when a tag mismatch is detected.
This commit add a KHWASAN brk handler, that decodes the immediate value passed to the brk instructions (to extract information about the memory access that triggered the mismatch), reads the register values (x0 contains the guilty address) and reports the bug.
Signed-off-by Andrey Konovalov <[email protected]>--- arch/arm64/include/asm/brk-imm.h | 2 + arch/arm64/kernel/traps.c | 69 +++++++++++++++++++++++++++++++- 2 files changed, 69 insertions(+), 2 deletions(-)arch/arm64/include/asm/brk-imm.h +2
-0@@ -16,10 +16,12 @@* 0x400: for dynamic BRK instruction * 0x401: for compile time BRK instruction * 0x800: kernel-mode BUG() and WARN() traps +* 0x9xx: KHWASAN trap (allowed values 0x900 - 0x9ff) */ #define FAULT_BRK_IMM 0x100#define KGDB_DYN_DBG_BRK_IMM 0x400#define KGDB_COMPILED_DBG_BRK_IMM 0x401#define BUG_BRK_IMM 0x800 +#define KHWASAN_BRK_IMM 0x900#endif arch/arm64/kernel/traps.c +29
-2@@ -35,6 +35,7 @@#include <linux/sizes.h> #include <linux/syscalls.h> #include <linux/mm_types.h> +#include <linux/kasan.h> #include <asm/atomic.h> #include <asm/bug.h> @@ -269,10 +270,14 @@ void arm64_notify_die(const char *str, struct pt_regs *regs, } } -void arm64_skip_faulting_instruction (struct pt_regs *regs,unsigned long size) +void __arm64_skip_faulting_instruction (struct pt_regs *regs,unsigned long size) { regs->pc += size; +} +void arm64_skip_faulting_instruction (struct pt_regs *regs,unsigned long size) +{ +__arm64_skip_faulting_instruction (regs, size);/* * If we were single stepping, we want to get the step exception after * we return from the trap. @@ -775,7 +780,7 @@ static int bug_handler(struct pt_regs *regs, unsigned int esr) }/* If thread survives, skip over the BUG instruction and continue: */ -arm64_skip_faulting_instruction (regs,AARCH64_INSN_SIZE ); +__arm64_skip_faulting_instruction (regs,AARCH64_INSN_SIZE );return DBG_HOOK_HANDLED ; }@@ -785,6 +790,59 @@ static struct break_hook bug_break_hook = { .fn = bug_handler, }; +#ifdef CONFIG_KASAN_HW + +#define KHWASAN_ESR_RECOVER 0x20 +#define KHWASAN_ESR_WRITE 0x10 +#define KHWASAN_ESR_SIZE_MASK 0x0f +#define KHWASAN_ESR_SIZE (esr) (1 << ((esr) & KHWASAN_ESR_SIZE_MASK)) + +static int khwasan_handler (struct pt_regs *regs,unsigned int esr) +{ +bool recover = esr &KHWASAN_ESR_RECOVER ; +bool write = esr &KHWASAN_ESR_WRITE ; +size_t size =KHWASAN_ESR_SIZE (esr); +u64 addr = regs->regs[0 ]; +u64 pc = regs->pc; + +if (user_mode (regs)) +return DBG_HOOK_ERROR ; + +kasan_report (addr, size, write, pc); + +/* +* The instrumentation allows to control whether we can proceed after
arch/arm64/kernel/traps.c static struct break_hook bug_break_hook = { · line 824
Quoted 10 lines
arch/arm64/kernel/traps.c +10
-0+* a crash was detected. This is done by passing the -recover flag to +* the compiler. Disabling recovery allows to generate more compact +* code. +* +* Unfortunately disabling recovery doesn't work for the kernel right +* now. KHWASAN reporting is disabled in some contexts (for example when +* the allocator accesses slab object metadata; same is true for KASAN; +* this is controlled by current->kasan_depth). All these accesses are +* detected by the tool, even though the reports for them are not +* printed.
I am not following this part. Slab accesses metadata. OK. This is detected as bad access. OK. Report is not printed. OK.
We skip BRK and resume execution. What is the problem?
Quoted 46 lines
arch/arm64/kernel/traps.c +28
-1+* +* This is something that might be fixed at some point in the future. +*/ +if (!recover) +die ("Oops - KHWASAN" , regs,0 ); + +/* If thread survives, skip over the brk instruction and continue: */ +__arm64_skip_faulting_instruction (regs,AARCH64_INSN_SIZE ); + return DBG_HOOK_HANDLED; +} + +#define KHWASAN_ESR_VAL (0xf2000000 | KHWASAN_BRK_IMM) +#define KHWASAN_ESR_MASK 0xffffff00 + +static struct break_hook khwasan_break_hook = { + .esr_val =KHWASAN_ESR_VAL , + .esr_mask =KHWASAN_ESR_MASK , + .fn = khwasan_handler, +}; +#endif +/* * Initial handler for AArch64 BRK exceptions * This handler only used until debug_traps_init(). @@ -792,6 +850,10 @@ static struct break_hook bug_break_hook = {int __init early_brk64 (unsigned long addr,unsigned int esr,struct pt_regs *regs) { +#ifdef CONFIG_KASAN_HW +if ((esr &KHWASAN_ESR_MASK ) ==KHWASAN_ESR_VAL ) +return khwasan_handler (regs, esr) !=DBG_HOOK_HANDLED ; +#endif return bug_handler (regs, esr) !=DBG_HOOK_HANDLED ; }@@ -799,4 +861,7 @@ int __init early_brk64(unsigned long addr, unsigned int esr,void __init trap_init (void ) {register_break_hook (&bug_break_hook); +#ifdef CONFIG_KASAN_HW +register_break_hook (&khwasan_break_hook); +#endif } --2.19.0.rc0.228.g281dcd1b4d0-goog
Re: [PATCH v6 15/18] khwasan, arm64: add brk handler for inline instrumentation
Quoted 29 lines · Dmitry Vyukov
Quoted 15 lines · Andrey Konovalov
arch/arm64/kernel/traps.c +15
-0+static int khwasan_handler (struct pt_regs *regs,unsigned int esr) +{ +bool recover = esr &KHWASAN_ESR_RECOVER ; +bool write = esr &KHWASAN_ESR_WRITE ; +size_t size =KHWASAN_ESR_SIZE (esr); +u64 addr = regs->regs[0 ]; +u64 pc = regs->pc; + +if (user_mode (regs)) +return DBG_HOOK_ERROR ; + +kasan_report (addr, size, write, pc); + +/* +* The instrumentation allows to control whether we can proceed after arch/arm64/kernel/traps.c static struct break_hook bug_break_hook = { · line 824
Quoted 10 lines
arch/arm64/kernel/traps.c +10
-0+* a crash was detected. This is done by passing the -recover flag to +* the compiler. Disabling recovery allows to generate more compact +* code. +* +* Unfortunately disabling recovery doesn't work for the kernel right +* now. KHWASAN reporting is disabled in some contexts (for example when +* the allocator accesses slab object metadata; same is true for KASAN; +* this is controlled by current->kasan_depth). All these accesses are +* detected by the tool, even though the reports for them are not +* printed.
Quoted 6 lines
I am not following this part. Slab accesses metadata. OK. This is detected as bad access. OK. Report is not printed. OK.
We skip BRK and resume execution. What is the problem?
When the kernel is compiled with -fsanitize=kernel-hwaddress without any additional flags (like it's done now with KASAN_HW) everything works as you described and there's no problem. However if one were to recompile the kernel with hwasan recovery disabled, KHWASAN wouldn't work due to the reasons described in the comment. Should I make it more clear?
Quoted 8 lines
Quoted 8 lines
arch/arm64/kernel/traps.c +5
-0+* +* This is something that might be fixed at some point in the future. +*/ +if (!recover) +die ("Oops - KHWASAN" , regs,0 );
Dmitry Vyukov
reviewerRe: [PATCH v6 15/18] khwasan, arm64: add brk handler for inline instrumentation
In reply to Andrey KonovalovQuoted 100 lines · Andrey Konovalov
KHWASAN inline instrumentation mode (which embeds checks of shadow memory into the generated code, instead of inserting a callback) generates a brk instruction when a tag mismatch is detected.
This commit add a KHWASAN brk handler, that decodes the immediate value passed to the brk instructions (to extract information about the memory access that triggered the mismatch), reads the register values (x0 contains the guilty address) and reports the bug.
Signed-off-by Andrey Konovalov <[email protected]>--- arch/arm64/include/asm/brk-imm.h | 2 + arch/arm64/kernel/traps.c | 69 +++++++++++++++++++++++++++++++- 2 files changed, 69 insertions(+), 2 deletions(-)arch/arm64/include/asm/brk-imm.h +2
-0@@ -16,10 +16,12 @@* 0x400: for dynamic BRK instruction * 0x401: for compile time BRK instruction * 0x800: kernel-mode BUG() and WARN() traps +* 0x9xx: KHWASAN trap (allowed values 0x900 - 0x9ff) */ #define FAULT_BRK_IMM 0x100#define KGDB_DYN_DBG_BRK_IMM 0x400#define KGDB_COMPILED_DBG_BRK_IMM 0x401#define BUG_BRK_IMM 0x800 +#define KHWASAN_BRK_IMM 0x900#endif arch/arm64/kernel/traps.c +34
-2@@ -35,6 +35,7 @@#include <linux/sizes.h> #include <linux/syscalls.h> #include <linux/mm_types.h> +#include <linux/kasan.h> #include <asm/atomic.h> #include <asm/bug.h> @@ -269,10 +270,14 @@ void arm64_notify_die(const char *str, struct pt_regs *regs, } } -void arm64_skip_faulting_instruction (struct pt_regs *regs,unsigned long size) +void __arm64_skip_faulting_instruction (struct pt_regs *regs,unsigned long size) { regs->pc += size; +} +void arm64_skip_faulting_instruction (struct pt_regs *regs,unsigned long size) +{ +__arm64_skip_faulting_instruction (regs, size);/* * If we were single stepping, we want to get the step exception after * we return from the trap. @@ -775,7 +780,7 @@ static int bug_handler(struct pt_regs *regs, unsigned int esr) }/* If thread survives, skip over the BUG instruction and continue: */ -arm64_skip_faulting_instruction (regs,AARCH64_INSN_SIZE ); +__arm64_skip_faulting_instruction (regs,AARCH64_INSN_SIZE );return DBG_HOOK_HANDLED ; }@@ -785,6 +790,59 @@ static struct break_hook bug_break_hook = { .fn = bug_handler, }; +#ifdef CONFIG_KASAN_HW + +#define KHWASAN_ESR_RECOVER 0x20 +#define KHWASAN_ESR_WRITE 0x10 +#define KHWASAN_ESR_SIZE_MASK 0x0f +#define KHWASAN_ESR_SIZE (esr) (1 << ((esr) & KHWASAN_ESR_SIZE_MASK)) + +static int khwasan_handler (struct pt_regs *regs,unsigned int esr) +{ +bool recover = esr &KHWASAN_ESR_RECOVER ; +bool write = esr &KHWASAN_ESR_WRITE ; +size_t size =KHWASAN_ESR_SIZE (esr); +u64 addr = regs->regs[0 ]; +u64 pc = regs->pc; + +if (user_mode (regs)) +return DBG_HOOK_ERROR ; + +kasan_report (addr, size, write, pc); + +/* +* The instrumentation allows to control whether we can proceed after +* a crash was detected. This is done by passing the -recover flag to +* the compiler. Disabling recovery allows to generate more compact +* code. +* +* Unfortunately disabling recovery doesn't work for the kernel right
arch/arm64/kernel/traps.c static struct break_hook bug_break_hook = { · line 829
Quoted 10 lines
arch/arm64/kernel/traps.c +10
-0+* now. KHWASAN reporting is disabled in some contexts (for example when +* the allocator accesses slab object metadata; same is true for KASAN; +* this is controlled by current->kasan_depth). All these accesses are +* detected by the tool, even though the reports for them are not +* printed. +* +* This is something that might be fixed at some point in the future. +*/ +if (!recover) +die ("Oops - KHWASAN" , regs,0 );
Why die and not panic? Die seems to be much less used function, and it calls panic anyway, and we call panic in kasan_report if panic_on_warn is set.
Quoted 40 lines
arch/arm64/kernel/traps.c +22
-1+/* If thread survives, skip over the brk instruction and continue: */ +__arm64_skip_faulting_instruction (regs,AARCH64_INSN_SIZE ); + return DBG_HOOK_HANDLED; +} + +#define KHWASAN_ESR_VAL (0xf2000000 | KHWASAN_BRK_IMM) +#define KHWASAN_ESR_MASK 0xffffff00 + +static struct break_hook khwasan_break_hook = { + .esr_val =KHWASAN_ESR_VAL , + .esr_mask =KHWASAN_ESR_MASK , + .fn = khwasan_handler, +}; +#endif +/* * Initial handler for AArch64 BRK exceptions * This handler only used until debug_traps_init(). @@ -792,6 +850,10 @@ static struct break_hook bug_break_hook = {int __init early_brk64 (unsigned long addr,unsigned int esr,struct pt_regs *regs) { +#ifdef CONFIG_KASAN_HW +if ((esr &KHWASAN_ESR_MASK ) ==KHWASAN_ESR_VAL ) +return khwasan_handler (regs, esr) !=DBG_HOOK_HANDLED ; +#endif return bug_handler (regs, esr) !=DBG_HOOK_HANDLED ; }@@ -799,4 +861,7 @@ int __init early_brk64(unsigned long addr, unsigned int esr,void __init trap_init (void ) {register_break_hook (&bug_break_hook); +#ifdef CONFIG_KASAN_HW +register_break_hook (&khwasan_break_hook); +#endif } --2.19.0.rc0.228.g281dcd1b4d0-goog
Re: [PATCH v6 15/18] khwasan, arm64: add brk handler for inline instrumentation
Quoted 1 line · Dmitry Vyukov
On Wed, Aug 29, 2018 at 1:35 PM, Andrey Konovalov <[email protected]> wrote:
[...]
Quoted 26 lines
Quoted 26 lines
arch/arm64/kernel/traps.c +26
-0+static int khwasan_handler (struct pt_regs *regs,unsigned int esr) +{ +bool recover = esr &KHWASAN_ESR_RECOVER ; +bool write = esr &KHWASAN_ESR_WRITE ; +size_t size =KHWASAN_ESR_SIZE (esr); +u64 addr = regs->regs[0 ]; +u64 pc = regs->pc; + +if (user_mode (regs)) +return DBG_HOOK_ERROR ; + +kasan_report (addr, size, write, pc); + +/* +* The instrumentation allows to control whether we can proceed after +* a crash was detected. This is done by passing the -recover flag to +* the compiler. Disabling recovery allows to generate more compact +* code. +* +* Unfortunately disabling recovery doesn't work for the kernel right +* now. KHWASAN reporting is disabled in some contexts (for example when +* the allocator accesses slab object metadata; same is true for KASAN; +* this is controlled by current->kasan_depth). All these accesses are +* detected by the tool, even though the reports for them are not +* printed. +*
Quoted 8 lines
arch/arm64/kernel/traps.c static struct break_hook bug_break_hook = { · line 829
Quoted 4 lines
arch/arm64/kernel/traps.c +4
-0+* This is something that might be fixed at some point in the future. +*/ +if (!recover) +die ("Oops - KHWASAN" , regs,0 );Why die and not panic? Die seems to be much less used function, and it calls panic anyway, and we call panic in
kasan_reportifpanic_on_warnis set.
die() is vaguely equivalent to BUG(); die() and BUG() normally only terminate the current process, which may or may not leave the system somewhat usable, while panic() always brings down the whole system. AFAIK panic() shouldn't be used unless you're in some very low-level code where you know that trying to just kill the current process can't work and the entire system is broken beyond repair.
If KASAN traps on some random memory access, there's a good chance that just killing the current process will allow at least parts of the system to continue. I'm not sure whether BUG() or die() is more appropriate here, but I think it definitely should not be a panic().
Dmitry Vyukov
reviewerRe: [PATCH v6 15/18] khwasan, arm64: add brk handler for inline instrumentation
Quoted 45 lines · Jann Horn
Quoted 1 line · Dmitry Vyukov
On Wed, Aug 29, 2018 at 1:35 PM, Andrey Konovalov <[email protected]> wrote:
[...]
Quoted 26 lines
Quoted 26 lines
arch/arm64/kernel/traps.c +26
-0+static int khwasan_handler (struct pt_regs *regs,unsigned int esr) +{ +bool recover = esr &KHWASAN_ESR_RECOVER ; +bool write = esr &KHWASAN_ESR_WRITE ; +size_t size =KHWASAN_ESR_SIZE (esr); +u64 addr = regs->regs[0 ]; +u64 pc = regs->pc; + +if (user_mode (regs)) +return DBG_HOOK_ERROR ; + +kasan_report (addr, size, write, pc); + +/* +* The instrumentation allows to control whether we can proceed after +* a crash was detected. This is done by passing the -recover flag to +* the compiler. Disabling recovery allows to generate more compact +* code. +* +* Unfortunately disabling recovery doesn't work for the kernel right +* now. KHWASAN reporting is disabled in some contexts (for example when +* the allocator accesses slab object metadata; same is true for KASAN; +* this is controlled by current->kasan_depth). All these accesses are +* detected by the tool, even though the reports for them are not +* printed. +* Quoted 8 lines
arch/arm64/kernel/traps.c static struct break_hook bug_break_hook = { · line 829
Quoted 4 lines
arch/arm64/kernel/traps.c +4
-0+* This is something that might be fixed at some point in the future. +*/ +if (!recover) +die ("Oops - KHWASAN" , regs,0 );Why die and not panic? Die seems to be much less used function, and it calls panic anyway, and we call panic in
kasan_reportifpanic_on_warnis set.
die()is vaguely equivalent toBUG();die()andBUG()normally only terminate the current process, which may or may not leave the system somewhat usable, whilepanic()always brings down the whole system. AFAIKpanic()shouldn't be used unless you're in some very low-level code where you know that trying to just kill the current process can't work and the entire system is broken beyond repair.
Quoted 4 lines
If KASAN traps on some random memory access, there's a good chance that just killing the current process will allow at least parts of the system to continue. I'm not sure whether
BUG()ordie()is more appropriate here, but I think it definitely should not be apanic().
Nick, do you know if die() will be enough to catch problems on Android phones? panic_on_warn would turn this into panic, but I guess one does not want panic_on_warn on a canary phone.
Nick Desaulniers
reviewerRe: [PATCH v6 15/18] khwasan, arm64: add brk handler for inline instrumentation
Quoted 53 lines · Dmitry Vyukov
Quoted 45 lines · Jann Horn
Quoted 1 line · Dmitry Vyukov
On Wed, Aug 29, 2018 at 1:35 PM, Andrey Konovalov <[email protected]> wrote:
[...]
Quoted 26 lines
Quoted 26 lines
arch/arm64/kernel/traps.c +26
-0+static int khwasan_handler (struct pt_regs *regs,unsigned int esr) +{ +bool recover = esr &KHWASAN_ESR_RECOVER ; +bool write = esr &KHWASAN_ESR_WRITE ; +size_t size =KHWASAN_ESR_SIZE (esr); +u64 addr = regs->regs[0 ]; +u64 pc = regs->pc; + +if (user_mode (regs)) +return DBG_HOOK_ERROR ; + +kasan_report (addr, size, write, pc); + +/* +* The instrumentation allows to control whether we can proceed after +* a crash was detected. This is done by passing the -recover flag to +* the compiler. Disabling recovery allows to generate more compact +* code. +* +* Unfortunately disabling recovery doesn't work for the kernel right +* now. KHWASAN reporting is disabled in some contexts (for example when +* the allocator accesses slab object metadata; same is true for KASAN; +* this is controlled by current->kasan_depth). All these accesses are +* detected by the tool, even though the reports for them are not +* printed. +* Quoted 8 lines
arch/arm64/kernel/traps.c static struct break_hook bug_break_hook = { · line 829
Quoted 4 lines
arch/arm64/kernel/traps.c +4
-0+* This is something that might be fixed at some point in the future. +*/ +if (!recover) +die ("Oops - KHWASAN" , regs,0 );Why die and not panic? Die seems to be much less used function, and it calls panic anyway, and we call panic in
kasan_reportifpanic_on_warnis set.
die()is vaguely equivalent toBUG();die()andBUG()normally only terminate the current process, which may or may not leave the system somewhat usable, whilepanic()always brings down the whole system. AFAIKpanic()shouldn't be used unless you're in some very low-level code where you know that trying to just kill the current process can't work and the entire system is broken beyond repair.Quoted 4 lines
If KASAN traps on some random memory access, there's a good chance that just killing the current process will allow at least parts of the system to continue. I'm not sure whether
BUG()ordie()is more appropriate here, but I think it definitely should not be apanic().
Quoted 3 lines
Nick, do you know if
die()will be enough to catch problems on Android phones?panic_on_warnwould turn this into panic, but I guess one does not wantpanic_on_warnon a canary phone.
die() has arch specific implementations, so looking at:
arch/arm64/kernel/traps.c:196#die
it looks like panic is invoked if in_interrupt() or panic_on_oops(), which is a configure option. So maybe the config for KHWASAN should also enable that? Otherwise seems easy to forget. But maybe that should remain configurable separately?
Looking at the kernel configs for the Pixel 2, it does seem like CONFIG_PANIC_ON_OOPS=y is already enabled.
https://android.googlesource.com/kernel/msm/+/android-msm-wahoo-4.4-pie/arch/arm64/configs/wahoo_defconfig#746
Specifically to catch problems on Android, our internal debug builds can report on panics, but not oops, IIUC.
Signature · 2 lines
Thanks, ~Nick Desaulniers
Re: [PATCH v6 15/18] khwasan, arm64: add brk handler for inline instrumentation
Quoted 59 lines · Nick Desaulniers
Quoted 53 lines · Dmitry Vyukov
Quoted 45 lines · Jann Horn
Quoted 1 line · Dmitry Vyukov
On Wed, Aug 29, 2018 at 1:35 PM, Andrey Konovalov <[email protected]> wrote:
[...]
Quoted 26 lines
Quoted 26 lines
arch/arm64/kernel/traps.c +26
-0+static int khwasan_handler (struct pt_regs *regs,unsigned int esr) +{ +bool recover = esr &KHWASAN_ESR_RECOVER ; +bool write = esr &KHWASAN_ESR_WRITE ; +size_t size =KHWASAN_ESR_SIZE (esr); +u64 addr = regs->regs[0 ]; +u64 pc = regs->pc; + +if (user_mode (regs)) +return DBG_HOOK_ERROR ; + +kasan_report (addr, size, write, pc); + +/* +* The instrumentation allows to control whether we can proceed after +* a crash was detected. This is done by passing the -recover flag to +* the compiler. Disabling recovery allows to generate more compact +* code. +* +* Unfortunately disabling recovery doesn't work for the kernel right +* now. KHWASAN reporting is disabled in some contexts (for example when +* the allocator accesses slab object metadata; same is true for KASAN; +* this is controlled by current->kasan_depth). All these accesses are +* detected by the tool, even though the reports for them are not +* printed. +* Quoted 8 lines
arch/arm64/kernel/traps.c static struct break_hook bug_break_hook = { · line 829
Quoted 4 lines
arch/arm64/kernel/traps.c +4
-0+* This is something that might be fixed at some point in the future. +*/ +if (!recover) +die ("Oops - KHWASAN" , regs,0 );Why die and not panic? Die seems to be much less used function, and it calls panic anyway, and we call panic in
kasan_reportifpanic_on_warnis set.
die()is vaguely equivalent toBUG();die()andBUG()normally only terminate the current process, which may or may not leave the system somewhat usable, whilepanic()always brings down the whole system. AFAIKpanic()shouldn't be used unless you're in some very low-level code where you know that trying to just kill the current process can't work and the entire system is broken beyond repair.Quoted 4 lines
If KASAN traps on some random memory access, there's a good chance that just killing the current process will allow at least parts of the system to continue. I'm not sure whether
BUG()ordie()is more appropriate here, but I think it definitely should not be apanic().Quoted 3 lines
Nick, do you know if
die()will be enough to catch problems on Android phones?panic_on_warnwould turn this into panic, but I guess one does not wantpanic_on_warnon a canary phone.
Quoted 8 lines
die()has arch specific implementations, so looking at:arch/arm64/kernel/traps.c:196#die
it looks like panic is invoked if
in_interrupt()orpanic_on_oops(), which is a configure option. So maybe the config for KHWASAN should also enable that? Otherwise seems easy to forget. But maybe that should remain configurable separately?
In the upstream kernel, it is desirable to be able to discover bugs and debug them from inside the running system. When you detect a bug that makes it impossible for the current task to continue safely, you're supposed to use something like BUG() to terminate the task; if you can continue safely, you're supposed to use WARN(). Either way, you should *not* just shoot down the whole kernel unless the system is corrupted so badly that trying to keep running would be pointless.
You can configure the kernel such that it crashes the device when such an event occurs, and doing so can sometimes be beneficial; but please don't hardcode such policies into the upstream kernel.
Dmitry Vyukov
reviewerRe: [PATCH v6 15/18] khwasan, arm64: add brk handler for inline instrumentation
In reply to Nick DesaulniersQuoted 62 lines · Nick Desaulniers' via kasan-dev
Quoted 53 lines · Dmitry Vyukov
Quoted 45 lines · Jann Horn
Quoted 1 line · Dmitry Vyukov
On Wed, Aug 29, 2018 at 1:35 PM, Andrey Konovalov <[email protected]> wrote:
[...]
Quoted 26 lines
Quoted 26 lines
arch/arm64/kernel/traps.c +26
-0+static int khwasan_handler (struct pt_regs *regs,unsigned int esr) +{ +bool recover = esr &KHWASAN_ESR_RECOVER ; +bool write = esr &KHWASAN_ESR_WRITE ; +size_t size =KHWASAN_ESR_SIZE (esr); +u64 addr = regs->regs[0 ]; +u64 pc = regs->pc; + +if (user_mode (regs)) +return DBG_HOOK_ERROR ; + +kasan_report (addr, size, write, pc); + +/* +* The instrumentation allows to control whether we can proceed after +* a crash was detected. This is done by passing the -recover flag to +* the compiler. Disabling recovery allows to generate more compact +* code. +* +* Unfortunately disabling recovery doesn't work for the kernel right +* now. KHWASAN reporting is disabled in some contexts (for example when +* the allocator accesses slab object metadata; same is true for KASAN; +* this is controlled by current->kasan_depth). All these accesses are +* detected by the tool, even though the reports for them are not +* printed. +* Quoted 8 lines
arch/arm64/kernel/traps.c static struct break_hook bug_break_hook = { · line 829
Quoted 4 lines
arch/arm64/kernel/traps.c +4
-0+* This is something that might be fixed at some point in the future. +*/ +if (!recover) +die ("Oops - KHWASAN" , regs,0 );Why die and not panic? Die seems to be much less used function, and it calls panic anyway, and we call panic in
kasan_reportifpanic_on_warnis set.
die()is vaguely equivalent toBUG();die()andBUG()normally only terminate the current process, which may or may not leave the system somewhat usable, whilepanic()always brings down the whole system. AFAIKpanic()shouldn't be used unless you're in some very low-level code where you know that trying to just kill the current process can't work and the entire system is broken beyond repair.Quoted 4 lines
If KASAN traps on some random memory access, there's a good chance that just killing the current process will allow at least parts of the system to continue. I'm not sure whether
BUG()ordie()is more appropriate here, but I think it definitely should not be apanic().Quoted 3 lines
Nick, do you know if
die()will be enough to catch problems on Android phones?panic_on_warnwould turn this into panic, but I guess one does not wantpanic_on_warnon a canary phone.
die()has arch specific implementations, so looking at:arch/arm64/kernel/traps.c:196#die
Quoted 8 lines
it looks like panic is invoked if
in_interrupt()orpanic_on_oops(), which is a configure option. So maybe the config for KHWASAN should also enable that? Otherwise seems easy to forget. But maybe that should remain configurable separately?Looking at the kernel configs for the Pixel 2, it does seem like
CONFIG_PANIC_ON_OOPS=yis already enabled.
https://android.googlesource.com/kernel/msm/+/android-msm-wahoo-4.4-pie/arch/arm64/configs/wahoo_defconfig#746
Then I think we are good here.
Quoted 2 lines
Specifically to catch problems on Android, our internal debug builds can report on panics, but not oops, IIUC.
PATCH v6 16/18 khwasan, mm, arm64: tag non slab memory allocated via pagealloc
In reply to Andrey KonovalovKWHASAN doesn't check memory accesses through pointers tagged with 0xff. When page_address is used to get pointer to memory that corresponds to some page, the tag of the resulting pointer gets set to 0xff, even though the allocated memory might have been tagged differently.
For slab pages it's impossible to recover the correct tag to return from page_address, since the page might contain multiple slab objects tagged with different values, and we can't know in advance which one of them is going to get accessed. For non slab pages however, we can recover the tag in page_address, since the whole page was marked with the same tag.
This patch adds tagging to non slab memory allocated with pagealloc. To set the tag of the pointer returned from page_address, the tag gets stored to page->flags when the memory gets allocated.
--- arch/arm64/include/asm/memory.h | 10 ++++++++++ include/linux/mm.h | 29 +++++++++++++++++++++++++++++ include/linux/page-flags-layout.h | 10 ++++++++++ mm/cma.c | 11 +++++++++++ mm/kasan/common.c | 17 +++++++++++++++-- mm/page_alloc.c | 1 + 6 files changed, 76 insertions(+), 2 deletions(-)
arch/arm64/include/asm/memory.h +10 -0
@@ -312,7 +312,17 @@ static inline void *phys_to_virt(phys_addr_t x)#define __virt_to_pgoff (kaddr) (((u64)(kaddr) & ~PAGE_OFFSET) / PAGE_SIZE * sizeof(struct page))#define __page_to_voff (kaddr) (((u64)(kaddr) & ~VMEMMAP_START) * PAGE_SIZE / sizeof(struct page)) +#ifndef CONFIG_KASAN_HW #define page_to_virt (page) ((void *)((__page_to_voff(page)) | PAGE_OFFSET)) +#else +#define page_to_virt (page) ({ \ + unsigned long __addr = \ + ((__page_to_voff(page)) | PAGE_OFFSET); \ + __addr = KASAN_SET_TAG(__addr, page_kasan_tag(page)); \ + ((void *)__addr); \ +}) +#endif +#define virt_to_page (vaddr) ((struct page *)((__virt_to_pgoff(vaddr)) | VMEMMAP_START))#define _virt_addr_valid (kaddr) pfn_valid((((u64)(kaddr) & ~PAGE_OFFSET) \
include/linux/mm.h +29 -0
@@ -804,6 +804,7 @@ vm_fault_t finish_mkwrite_fault(struct vm_fault *vmf);#define NODES_PGOFF (SECTIONS_PGOFF - NODES_WIDTH)#define ZONES_PGOFF (NODES_PGOFF - ZONES_WIDTH)#define LAST_CPUPID_PGOFF (ZONES_PGOFF - LAST_CPUPID_WIDTH) +#define KASAN_TAG_PGOFF (LAST_CPUPID_PGOFF - KASAN_TAG_WIDTH)/* * Define the bit shifts to access each section. For non-existent
@@ -814,6 +815,7 @@ vm_fault_t finish_mkwrite_fault(struct vm_fault *vmf);#define NODES_PGSHIFT (NODES_PGOFF * (NODES_WIDTH != 0))#define ZONES_PGSHIFT (ZONES_PGOFF * (ZONES_WIDTH != 0))#define LAST_CPUPID_PGSHIFT (LAST_CPUPID_PGOFF * (LAST_CPUPID_WIDTH != 0)) +#define KASAN_TAG_PGSHIFT (KASAN_TAG_PGOFF * (KASAN_TAG_WIDTH != 0))/* NODE:ZONE or SECTION:ZONE is used to ID a zone for the buddy allocator */ #ifdef NODE_NOT_IN_PAGE_FLAGS
@@ -836,6 +838,7 @@ vm_fault_t finish_mkwrite_fault(struct vm_fault *vmf);#define NODES_MASK ((1UL << NODES_WIDTH) - 1)#define SECTIONS_MASK ((1UL << SECTIONS_WIDTH) - 1)#define LAST_CPUPID_MASK ((1UL << LAST_CPUPID_SHIFT) - 1) +#define KASAN_TAG_MASK ((1UL << KASAN_TAG_WIDTH) - 1)#define ZONEID_MASK ((1UL << ZONEID_SHIFT) - 1)static inline enum zone_type page_zonenum (const struct page *page)
@@ -1081,6 +1084,32 @@ static inline bool cpupid_match_pid(struct task_struct *task, int cpupid) }#endif /* CONFIG_NUMA_BALANCING */ +#ifdef CONFIG_KASAN_HW +static inline u8 page_kasan_tag (const struct page *page) +{ +return (page->flags >>KASAN_TAG_PGSHIFT ) &KASAN_TAG_MASK ; +} + +static inline void page_kasan_tag_set (struct page *page,u8 tag) +{ + page->flags &= ~(KASAN_TAG_MASK <<KASAN_TAG_PGSHIFT ); + page->flags |= (tag &KASAN_TAG_MASK ) <<KASAN_TAG_PGSHIFT ; +} + +static inline void page_kasan_tag_reset (struct page *page) +{ +page_kasan_tag_set (page,0xff ); +} +#else +static inline u8 page_kasan_tag (const struct page *page) +{ +return 0xff ; +} + +static inline void page_kasan_tag_set (struct page *page,u8 tag) { } +static inline void page_kasan_tag_reset (struct page *page) { } +#endif +static inline struct zone *page_zone (const struct page *page) {return &NODE_DATA (page_to_nid (page))->node_zones[page_zonenum (page)];
include/linux/page-flags-layout.h +10 -0
@@ -82,6 +82,16 @@#define LAST_CPUPID_WIDTH 0#endif +#ifdef CONFIG_KASAN_HW +#define KASAN_TAG_WIDTH 8 +#if SECTIONS_WIDTH +NODES_WIDTH +ZONES_WIDTH +LAST_CPUPID_WIDTH +KASAN_TAG_WIDTH \ + >BITS_PER_LONG -NR_PAGEFLAGS +#error "KASAN: not enough bits in page flags for tag" +#endif +#else +#define KASAN_TAG_WIDTH 0 +#endif +/* * We are going to use the flags for the page to node mapping if its in * there. This includes the case where there is no node, so it is implicit.
mm/cma.c +11 -0
@@ -407,6 +407,7 @@ struct page *cma_alloc(struct cma *cma, size_t count, unsigned int align,unsigned long pfn =-1 ;unsigned long start =0 ;unsigned long bitmap_maxno, bitmap_no, bitmap_count; +size_t i;struct page *page =NULL ;int ret = -ENOMEM ;
@@ -466,6 +467,16 @@ struct page *cma_alloc(struct cma *cma, size_t count, unsigned int align,trace_cma_alloc (pfn, page, count, align); +/* +* CMA can allocate multiple page blocks, which results in different +* blocks being marked with different tags. Reset the tags to ignore +* those page blocks. +*/ +if (page) { +for (i =0 ; i < count; i++) +page_kasan_tag_reset (page + i); + } +if (ret && !no_warn) {pr_err ("%s: alloc failed, req-size: %zu pages, ret: %d\n" , __func__, count, ret);
mm/kasan/common.c +15 -2
@@ -212,8 +212,15 @@ void kasan_unpoison_stack_above_sp_to(const void *watermark)void kasan_alloc_pages (struct page *page,unsigned int order) { +u8 tag; +unsigned long i; +if (unlikely (PageHighMem (page)))return ; + + tag =random_tag (); +for (i =0 ; i < (1 << order); i++) +page_kasan_tag_set (page + i, tag);kasan_unpoison_shadow (page_address (page),PAGE_SIZE << order); }
@@ -311,6 +318,12 @@ struct kasan_free_meta *get_free_info(struct kmem_cache *cache,void kasan_poison_slab (struct page *page) { +unsigned long i; + +if (IS_ENABLED (CONFIG_SLAB )) + page->s_mem =reset_tag (page->s_mem); +for (i =0 ; i < (1 <<compound_order (page)); i++) +page_kasan_tag_reset (page + i);kasan_poison_shadow (page_address (page),PAGE_SIZE <<compound_order (page),KASAN_KMALLOC_REDZONE );
@@ -484,7 +497,7 @@ void kasan_poison_kfree(void *ptr, unsigned long ip) page =virt_to_head_page (ptr);if (unlikely (!PageSlab (page))) { -if (reset_tag (ptr) !=page_address (page)) { +if (ptr !=page_address (page)) {kasan_report_invalid_free (ptr, ip);return ; }
@@ -497,7 +510,7 @@ void kasan_poison_kfree(void *ptr, unsigned long ip)void kasan_kfree_large (void *ptr,unsigned long ip) { -if (reset_tag (ptr) !=page_address (virt_to_head_page (ptr))) +if (ptr !=page_address (virt_to_head_page (ptr)))kasan_report_invalid_free (ptr, ip);/* The object will be poisoned by page_alloc. */ }
mm/page_alloc.c +1 -0
@@ -1177,6 +1177,7 @@ static void __meminit __init_single_page(struct page *page, unsigned long pfn,init_page_count (page);page_mapcount_reset (page);page_cpupid_reset_last (page); +page_kasan_tag_reset (page);INIT_LIST_HEAD (&page->lru);#ifdef WANT_PAGE_VIRTUAL
Signature · 1 line
2.19.0.rc0.228.g281dcd1b4d0-goog
Re: [PATCH v6 16/18] khwasan, mm, arm64: tag non slab memory allocated via pagealloc
mm/kasan/common.c struct kasan_free_meta *get_free_info(struct kmem_cache *cache, · line 324
Quoted 6 lines · Andrey Konovalov
void
kasan_poison_slab(struct page *page)
{mm/kasan/common.c +4
-0+unsigned long i; + +if (IS_ENABLED (CONFIG_SLAB )) + page->s_mem =reset_tag (page->s_mem);
Why reinitialize here, instead of single initialization in alloc_slabmgmt()?
Quoted 5 lines
mm/kasan/common.c +2
-0+for (i =0 ; i < (1 <<compound_order (page)); i++) +page_kasan_tag_reset (page + i);kasan_poison_shadow (page_address (page),PAGE_SIZE <<compound_order (page),KASAN_KMALLOC_REDZONE );
Re: [PATCH v6 16/18] khwasan, mm, arm64: tag non slab memory allocated via pagealloc
Quoted 4 lines · Andrey Ryabinin
On 08/29/2018 02:35 PM, Andrey Konovalov wrote:
Quoted 8 lines
mm/kasan/common.c struct kasan_free_meta *get_free_info(struct kmem_cache *cache, · line 324
Quoted 6 lines
void
kasan_poison_slab(struct page *page)
{mm/kasan/common.c +4
-0+unsigned long i; + +if (IS_ENABLED (CONFIG_SLAB )) + page->s_mem =reset_tag (page->s_mem);Why reinitialize here, instead of single initialization in
alloc_slabmgmt()?
Hm, don't see why I did it this way, looks odd to me as well. Will fix in v7, thanks!
PATCH v6 17/18 khwasan: update kasan documentation
In reply to Andrey KonovalovThis patch updates KASAN documentation to reflect the addition of KHWASAN.
--- Documentation/dev-tools/kasan.rst | 213 +++++++++++++++++------------- 1 file changed, 123 insertions(+), 90 deletions(-)
Documentation/dev-tools/kasan.rst +123 -90
@@ -8,11 +8,19 @@ KernelAddressSANitizer (KASAN) is a dynamic memory error detector. It provides a fast and comprehensive solution for finding use-after-free and out-of-bounds bugs. -KASAN uses compile-time instrumentation for checking every memory access, -therefore you will need a GCC version 4.9.2 or later. GCC 5.0 or later is -required for detection of out-of-bounds accesses to stack or global variables. +KASAN has two modes: classic KASAN (a classic version, similar to user space +ASan) and KHWASAN (a version based on memory tagging, similar to user space +HWASan). -Currently KASAN is supported only for the x86_64 and arm64 architectures. +KASAN uses compile-time instrumentation to insert validity checks before every +memory access, and therefore requires a compiler version that supports that. +For classic KASAN you need GCC version 4.9.2 or later. GCC 5.0 or later is +required for detection of out-of-bounds accesses on stack and global variables. +KHWASAN in turns is only supported in clang and requires revision 330044 or +later. + +Currently classic KASAN is supported for the x86_64, arm64 and xtensa +architectures, and KHWASAN is supported only for arm64. Usage -----
@@ -21,12 +29,14 @@ To enable KASAN configure kernel with:: CONFIG_KASAN = y -and choose between CONFIG_KASAN_OUTLINE and CONFIG_KASAN_INLINE. Outline and -inline are compiler instrumentation types. The former produces smaller binary -the latter is 1.1 - 2 times faster. Inline instrumentation requires a GCC +and choose between CONFIG_KASAN_GENERIC (to enable classic KASAN) and +CONFIG_KASAN_HW (to enabled KHWASAN). You also need to choose choose between +CONFIG_KASAN_OUTLINE and CONFIG_KASAN_INLINE. Outline and inline are compiler +instrumentation types. The former produces smaller binary while the latter is +1.1 - 2 times faster. For classic KASAN inline instrumentation requires GCC version 5.0 or later. -KASAN works with both SLUB and SLAB memory allocators. +Both KASAN modes work with both SLUB and SLAB memory allocators. For better bug detection and nicer reporting, enable CONFIG_STACKTRACE. To disable instrumentation for specific files or directories, add a line
@@ -43,85 +53,80 @@ similar to the following to the respective kernel Makefile: Error reports ~~~~~~~~~~~~~ -A typical out of bounds access report looks like this:: +A typical out-of-bounds access classic KASAN report looks like this:: ================================================================== - BUG: AddressSanitizer: out of bounds access in kmalloc_oob_right+0x65/0x75 [test_kasan] at addr ffff8800693bc5d3 - Write of size 1 by task modprobe/1689 - ============================================================================= - BUG kmalloc-128 (Not tainted): kasan error - ----------------------------------------------------------------------------- - - Disabling lock debugging due to kernel taint - INFO: Allocated in kmalloc_oob_right+0x3d/0x75 [test_kasan] age=0 cpu=0 pid=1689 - __slab_alloc+0x4b4/0x4f0 - kmem_cache_alloc_trace+0x10b/0x190 - kmalloc_oob_right+0x3d/0x75 [test_kasan] - init_module+0x9/0x47 [test_kasan] - do_one_initcall+0x99/0x200 - load_module+0x2cb3/0x3b20 - SyS_finit_module+0x76/0x80 - system_call_fastpath+0x12/0x17 - INFO: Slab 0xffffea0001a4ef00 objects=17 used=7 fp=0xffff8800693bd728 flags=0x100000000004080 - INFO: Object 0xffff8800693bc558 @offset=1368 fp=0xffff8800693bc720 - - Bytes b4 ffff8800693bc548: 00 00 00 00 00 00 00 00 5a 5a 5a 5a 5a 5a 5a 5a ........ZZZZZZZZ - Object ffff8800693bc558: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b kkkkkkkkkkkkkkkk - Object ffff8800693bc568: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b kkkkkkkkkkkkkkkk - Object ffff8800693bc578: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b kkkkkkkkkkkkkkkk - Object ffff8800693bc588: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b kkkkkkkkkkkkkkkk - Object ffff8800693bc598: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b kkkkkkkkkkkkkkkk - Object ffff8800693bc5a8: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b kkkkkkkkkkkkkkkk - Object ffff8800693bc5b8: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b kkkkkkkkkkkkkkkk - Object ffff8800693bc5c8: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b a5 kkkkkkkkkkkkkkk. - Redzone ffff8800693bc5d8: cc cc cc cc cc cc cc cc ........ - Padding ffff8800693bc718: 5a 5a 5a 5a 5a 5a 5a 5a ZZZZZZZZ - CPU: 0 PID: 1689 Comm: modprobe Tainted: G B 3.18.0-rc1-mm1+ #98 - Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS rel-1.7.5-0-ge51488c-20140602_164612-nilsson.home.kraxel.org 04/01/2014 - ffff8800693bc000 0000000000000000 ffff8800693bc558 ffff88006923bb78 - ffffffff81cc68ae 00000000000000f3 ffff88006d407600 ffff88006923bba8 - ffffffff811fd848 ffff88006d407600 ffffea0001a4ef00 ffff8800693bc558 + BUG: KASAN: slab-out-of-bounds in kmalloc_oob_right+0xa8/0xbc [test_kasan] + Write of size 1 at addr ffff8800696f3d3b by task insmod/2734 + + CPU: 0 PID: 2734 Comm: insmod Not tainted 4.15.0+ #98 + Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.10.2-1 04/01/2014 Call Trace: - [<ffffffff81cc68ae>] dump_stack+0x46/0x58 - [<ffffffff811fd848>] print_trailer+0xf8/0x160 - [<ffffffffa00026a7>] ? kmem_cache_oob+0xc3/0xc3 [test_kasan] - [<ffffffff811ff0f5>] object_err+0x35/0x40 - [<ffffffffa0002065>] ? kmalloc_oob_right+0x65/0x75 [test_kasan] - [<ffffffff8120b9fa>] kasan_report_error+0x38a/0x3f0 - [<ffffffff8120a79f>] ? kasan_poison_shadow+0x2f/0x40 - [<ffffffff8120b344>] ? kasan_unpoison_shadow+0x14/0x40 - [<ffffffff8120a79f>] ? kasan_poison_shadow+0x2f/0x40 - [<ffffffffa00026a7>] ? kmem_cache_oob+0xc3/0xc3 [test_kasan] - [<ffffffff8120a995>] __asan_store1+0x75/0xb0 - [<ffffffffa0002601>] ? kmem_cache_oob+0x1d/0xc3 [test_kasan] - [<ffffffffa0002065>] ? kmalloc_oob_right+0x65/0x75 [test_kasan] - [<ffffffffa0002065>] kmalloc_oob_right+0x65/0x75 [test_kasan] - [<ffffffffa00026b0>] init_module+0x9/0x47 [test_kasan] - [<ffffffff810002d9>] do_one_initcall+0x99/0x200 - [<ffffffff811e4e5c>] ? __vunmap+0xec/0x160 - [<ffffffff81114f63>] load_module+0x2cb3/0x3b20 - [<ffffffff8110fd70>] ? m_show+0x240/0x240 - [<ffffffff81115f06>] SyS_finit_module+0x76/0x80 - [<ffffffff81cd3129>] system_call_fastpath+0x12/0x17 + __dump_stack lib/dump_stack.c:17 + dump_stack+0x83/0xbc lib/dump_stack.c:53 + print_address_description+0x73/0x280 mm/kasan/report.c:254 + kasan_report_error mm/kasan/report.c:352 + kasan_report+0x10e/0x220 mm/kasan/report.c:410 + __asan_report_store1_noabort+0x17/0x20 mm/kasan/report.c:505 + kmalloc_oob_right+0xa8/0xbc [test_kasan] lib/test_kasan.c:42 + kmalloc_tests_init+0x16/0x769 [test_kasan] + do_one_initcall+0x9e/0x240 init/main.c:832 + do_init_module+0x1b6/0x542 kernel/module.c:3462 + load_module+0x6042/0x9030 kernel/module.c:3786 + SYSC_init_module+0x18f/0x1c0 kernel/module.c:3858 + SyS_init_module+0x9/0x10 kernel/module.c:3841 + do_syscall_64+0x198/0x480 arch/x86/entry/common.c:287 + entry_SYSCALL_64_after_hwframe+0x21/0x86 arch/x86/entry/entry_64.S:251 + RIP: 0033:0x7fdd79df99da + RSP: 002b:00007fff2229bdf8 EFLAGS: 00000202 ORIG_RAX: 00000000000000af + RAX: ffffffffffffffda RBX: 000055c408121190 RCX: 00007fdd79df99da + RDX: 00007fdd7a0b8f88 RSI: 0000000000055670 RDI: 00007fdd7a47e000 + RBP: 000055c4081200b0 R08: 0000000000000003 R09: 0000000000000000 + R10: 00007fdd79df5d0a R11: 0000000000000202 R12: 00007fdd7a0b8f88 + R13: 000055c408120090 R14: 0000000000000000 R15: 0000000000000000 + + Allocated by task 2734: + save_stack+0x43/0xd0 mm/kasan/common.c:176 + set_track+0x20/0x30 mm/kasan/common.c:188 + kasan_kmalloc+0x9a/0xc0 mm/kasan/kasan.c:372 + kmem_cache_alloc_trace+0xcd/0x1a0 mm/slub.c:2761 + kmalloc ./include/linux/slab.h:512 + kmalloc_oob_right+0x56/0xbc [test_kasan] lib/test_kasan.c:36 + kmalloc_tests_init+0x16/0x769 [test_kasan] + do_one_initcall+0x9e/0x240 init/main.c:832 + do_init_module+0x1b6/0x542 kernel/module.c:3462 + load_module+0x6042/0x9030 kernel/module.c:3786 + SYSC_init_module+0x18f/0x1c0 kernel/module.c:3858 + SyS_init_module+0x9/0x10 kernel/module.c:3841 + do_syscall_64+0x198/0x480 arch/x86/entry/common.c:287 + entry_SYSCALL_64_after_hwframe+0x21/0x86 arch/x86/entry/entry_64.S:251 + + The buggy address belongs to the object at ffff8800696f3cc0 + which belongs to the cache kmalloc-128 of size 128 + The buggy address is located 123 bytes inside of + 128-byte region [ffff8800696f3cc0, ffff8800696f3d40) + The buggy address belongs to the page: + page:ffffea0001a5bcc0 count:1 mapcount:0 mapping: (null) index:0x0 + flags: 0x100000000000100(slab) + raw: 0100000000000100 0000000000000000 0000000000000000 0000000180150015 + raw: ffffea0001a8ce40 0000000300000003 ffff88006d001640 0000000000000000 + page dumped because: kasan: bad access detected + Memory state around the buggy address: - ffff8800693bc300: fc fc fc fc fc fc fc fc fc fc fc fc fc fc fc fc - ffff8800693bc380: fc fc 00 00 00 00 00 00 00 00 00 00 00 00 00 fc - ffff8800693bc400: fc fc fc fc fc fc fc fc fc fc fc fc fc fc fc fc - ffff8800693bc480: fc fc fc fc fc fc fc fc fc fc fc fc fc fc fc fc - ffff8800693bc500: fc fc fc fc fc fc fc fc fc fc fc 00 00 00 00 00 - >ffff8800693bc580: 00 00 00 00 00 00 00 00 00 00 03 fc fc fc fc fc - ^ - ffff8800693bc600: fc fc fc fc fc fc fc fc fc fc fc fc fc fc fc fc - ffff8800693bc680: fc fc fc fc fc fc fc fc fc fc fc fc fc fc fc fc - ffff8800693bc700: fc fc fc fc fb fb fb fb fb fb fb fb fb fb fb fb - ffff8800693bc780: fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb - ffff8800693bc800: fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb + ffff8800696f3c00: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 fc + ffff8800696f3c80: fc fc fc fc fc fc fc fc 00 00 00 00 00 00 00 00 + >ffff8800696f3d00: 00 00 00 00 00 00 00 03 fc fc fc fc fc fc fc fc + ^ + ffff8800696f3d80: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 fc fc + ffff8800696f3e00: fc fc fc fc fc fc fc fc fb fb fb fb fb fb fb fb ================================================================== -The header of the report discribe what kind of bug happened and what kind of -access caused it. It's followed by the description of the accessed slub object -(see 'SLUB Debug output' section in Documentation/vm/slub.rst for details) and -the description of the accessed memory page. +The header of the report provides a short summary of what kind of bug happened +and what kind of access caused it. It's followed by a stack trace of the bad +access, a stack trace of where the accessed memory was allocated (in case bad +access happens on a slab object), and a stack trace of where the object was +freed (in case of a use-after-free bug report). Next comes a description of +the accessed slab object and information about the accessed memory page. In the last section the report shows memory state around the accessed address. Reading this part requires some understanding of how KASAN works.
@@ -138,18 +143,24 @@ inaccessible memory like redzones or freed memory (see mm/kasan/kasan.h). In the report above the arrows point to the shadow byte 03, which means that the accessed address is partially accessible. +For KHWASAN this last report section shows the memory tags around the accessed +address (see Implementation details section). + Implementation details ---------------------- +Classic KASAN +~~~~~~~~~~~~~ + From a high level, our approach to memory error detection is similar to that of kmemcheck: use shadow memory to record whether each byte of memory is safe -to access, and use compile-time instrumentation to check shadow memory on each -memory access. +to access, and use compile-time instrumentation to insert checks of shadow +memory on each memory access. -AddressSanitizer dedicates 1/8 of kernel memory to its shadow memory -(e.g. 16TB to cover 128TB on x86_64) and uses direct mapping with a scale and -offset to translate a memory address to its corresponding shadow address. +Classic KASAN dedicates 1/8th of kernel memory to its shadow memory (e.g. 16TB +to cover 128TB on x86_64) and uses direct mapping with a scale and offset to +translate a memory address to its corresponding shadow address. Here is the function which translates an address to its corresponding shadow address::
@@ -162,12 +173,34 @@ address:: where ``KASAN_SHADOW_SCALE_SHIFT = 3``. -Compile-time instrumentation used for checking memory accesses. Compiler inserts -function calls (__asan_load*(addr), __asan_store*(addr)) before each memory -access of size 1, 2, 4, 8 or 16. These functions check whether memory access is -valid or not by checking corresponding shadow memory. +Compile-time instrumentation is used to insert memory access checks. Compiler +inserts function calls (__asan_load*(addr), __asan_store*(addr)) before each +memory access of size 1, 2, 4, 8 or 16. These functions check whether memory +access is valid or not by checking corresponding shadow memory. GCC 5.0 has possibility to perform inline instrumentation. Instead of making function calls GCC directly inserts the code to check the shadow memory. This option significantly enlarges kernel but it gives x1.1-x2 performance boost over outline instrumented kernel. + +KHWASAN +~~~~~~~ + +KHWASAN uses the Top Byte Ignore (TBI) feature of modern arm64 CPUs to store +a pointer tag in the top byte of kernel pointers. KHWASAN also uses shadow +memory to store memory tags associated with each 16-byte memory cell (therefore +it dedicates 1/16th of the kernel memory for shadow memory). + +On each memory allocation KHWASAN generates a random tag, tags allocated memory +with this tag, and embeds this tag into the returned pointer. KHWASAN uses +compile-time instrumentation to insert checks before each memory access. These +checks make sure that tag of the memory that is being accessed is equal to tag +of the pointer that is used to access this memory. In case of a tag mismatch +KHWASAN prints a bug report. + +KHWASAN also has two instrumentation modes (outline, that emits callbacks to +check memory accesses; and inline, that performs the shadow memory checks +inline). With outline instrumentation mode, a bug report is simply printed +from the function that performs the access check. With inline instrumentation +a brk instruction is emitted by the compiler, and a dedicated brk handler is +used to print KHWASAN reports.
Signature · 1 line
2.19.0.rc0.228.g281dcd1b4d0-goog
Dmitry Vyukov
reviewerRe: [PATCH v6 17/18] khwasan: update kasan documentation
Quoted 18 lines · Andrey Konovalov
This patch updates KASAN documentation to reflect the addition of KHWASAN.
Signed-off-by Andrey Konovalov <[email protected]>--- Documentation/dev-tools/kasan.rst | 213 +++++++++++++++++------------- 1 file changed, 123 insertions(+), 90 deletions(-)Documentation/dev-tools/kasan.rst +0
-3@@ -8,11 +8,19 @@ KernelAddressSANitizer (KASAN) is a dynamic memory error detector. It provides a fast and comprehensive solution for finding use-after-free and out-of-bounds bugs. -KASAN uses compile-time instrumentation for checking every memory access, -therefore you will need a GCC version 4.9.2 or later. GCC 5.0 or later is -required for detection of out-of-bounds accesses to stack or global variables.
Documentation/dev-tools/kasan.rst KernelAddressSANitizer (KASAN) is a dynamic memory error detector. It provides · line 19
Quoted 10 lines
Documentation/dev-tools/kasan.rst +8
-1+KASAN has two modes: classic KASAN (a classic version, similar to user space +ASan) and KHWASAN (a version based on memory tagging, similar to user space +HWASan). -Currently KASAN is supported only for the x86_64 and arm64 architectures. +KASAN uses compile-time instrumentation to insert validity checks before every +memory access, and therefore requires a compiler version that supports that. +For classic KASAN you need GCC version 4.9.2 or later. GCC 5.0 or later is +required for detection of out-of-bounds accesses on stack and global variables. +KHWASAN in turns is only supported in clang and requires revision 330044 or
in turn?
Quoted 8 lines
Documentation/dev-tools/kasan.rst +4
-0+later. + +Currently classic KASAN is supported for the x86_64, arm64 and xtensa +architectures, and KHWASAN is supported only for arm64. Usage -----
Documentation/dev-tools/kasan.rst To enable KASAN configure kernel with:: · line 33
Quoted 8 lines
Documentation/dev-tools/kasan.rst +2
-3@@ -21,12 +29,14 @@ To enable KASAN configure kernel with:: CONFIG_KASAN = y -and choose between CONFIG_KASAN_OUTLINE and CONFIG_KASAN_INLINE. Outline and -inline are compiler instrumentation types. The former produces smaller binary -the latter is 1.1 - 2 times faster. Inline instrumentation requires a GCC +and choose between CONFIG_KASAN_GENERIC (to enable classic KASAN) and +CONFIG_KASAN_HW (to enabled KHWASAN). You also need to choose choose between
to enable
Quoted 73 lines
Documentation/dev-tools/kasan.rst +10
-51+CONFIG_KASAN_OUTLINE and CONFIG_KASAN_INLINE. Outline and inline are compiler +instrumentation types. The former produces smaller binary while the latter is +1.1 - 2 times faster. For classic KASAN inline instrumentation requires GCC version 5.0 or later. -KASAN works with both SLUB and SLAB memory allocators. +Both KASAN modes work with both SLUB and SLAB memory allocators. For better bug detection and nicer reporting, enable CONFIG_STACKTRACE. To disable instrumentation for specific files or directories, add a line@@ -43,85 +53,80 @@ similar to the following to the respective kernel Makefile: Error reports ~~~~~~~~~~~~~ -A typical out of bounds access report looks like this:: +A typical out-of-bounds access classic KASAN report looks like this:: ================================================================== - BUG: AddressSanitizer: out of bounds access in kmalloc_oob_right+0x65/0x75 [test_kasan] at addr ffff8800693bc5d3 - Write of size 1 by task modprobe/1689 - ============================================================================= - BUG kmalloc-128 (Not tainted): kasan error - ----------------------------------------------------------------------------- - - Disabling lock debugging due to kernel taint - INFO: Allocated in kmalloc_oob_right+0x3d/0x75 [test_kasan] age=0 cpu=0 pid=1689 - __slab_alloc+0x4b4/0x4f0 - kmem_cache_alloc_trace+0x10b/0x190 - kmalloc_oob_right+0x3d/0x75 [test_kasan] - init_module+0x9/0x47 [test_kasan] - do_one_initcall+0x99/0x200 - load_module+0x2cb3/0x3b20 - SyS_finit_module+0x76/0x80 - system_call_fastpath+0x12/0x17 - INFO: Slab 0xffffea0001a4ef00 objects=17 used=7 fp=0xffff8800693bd728 flags=0x100000000004080 - INFO: Object 0xffff8800693bc558 @offset=1368 fp=0xffff8800693bc720 - - Bytes b4 ffff8800693bc548: 00 00 00 00 00 00 00 00 5a 5a 5a 5a 5a 5a 5a 5a ........ZZZZZZZZ - Object ffff8800693bc558: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b kkkkkkkkkkkkkkkk - Object ffff8800693bc568: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b kkkkkkkkkkkkkkkk - Object ffff8800693bc578: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b kkkkkkkkkkkkkkkk - Object ffff8800693bc588: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b kkkkkkkkkkkkkkkk - Object ffff8800693bc598: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b kkkkkkkkkkkkkkkk - Object ffff8800693bc5a8: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b kkkkkkkkkkkkkkkk - Object ffff8800693bc5b8: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b kkkkkkkkkkkkkkkk - Object ffff8800693bc5c8: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b a5 kkkkkkkkkkkkkkk. - Redzone ffff8800693bc5d8: cc cc cc cc cc cc cc cc ........ - Padding ffff8800693bc718: 5a 5a 5a 5a 5a 5a 5a 5a ZZZZZZZZ - CPU: 0 PID: 1689 Comm: modprobe Tainted: G B 3.18.0-rc1-mm1+ #98 - Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS rel-1.7.5-0-ge51488c-20140602_164612-nilsson.home.kraxel.org 04/01/2014 - ffff8800693bc000 0000000000000000 ffff8800693bc558 ffff88006923bb78 - ffffffff81cc68ae 00000000000000f3 ffff88006d407600 ffff88006923bba8 - ffffffff811fd848 ffff88006d407600 ffffea0001a4ef00 ffff8800693bc558 + BUG: KASAN: slab-out-of-bounds in kmalloc_oob_right+0xa8/0xbc [test_kasan] + Write of size 1 at addr ffff8800696f3d3b by task insmod/2734 + + CPU: 0 PID: 2734 Comm: insmod Not tainted 4.15.0+ #98 + Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.10.2-1 04/01/2014 Call Trace: - [<ffffffff81cc68ae>] dump_stack+0x46/0x58 - [<ffffffff811fd848>] print_trailer+0xf8/0x160 - [<ffffffffa00026a7>] ? kmem_cache_oob+0xc3/0xc3 [test_kasan] - [<ffffffff811ff0f5>] object_err+0x35/0x40 - [<ffffffffa0002065>] ? kmalloc_oob_right+0x65/0x75 [test_kasan] - [<ffffffff8120b9fa>] kasan_report_error+0x38a/0x3f0 - [<ffffffff8120a79f>] ? kasan_poison_shadow+0x2f/0x40 - [<ffffffff8120b344>] ? kasan_unpoison_shadow+0x14/0x40 - [<ffffffff8120a79f>] ? kasan_poison_shadow+0x2f/0x40 - [<ffffffffa00026a7>] ? kmem_cache_oob+0xc3/0xc3 [test_kasan] - [<ffffffff8120a995>] __asan_store1+0x75/0xb0 - [<ffffffffa0002601>] ? kmem_cache_oob+0x1d/0xc3 [test_kasan] - [<ffffffffa0002065>] ? kmalloc_oob_right+0x65/0x75 [test_kasan] - [<ffffffffa0002065>] kmalloc_oob_right+0x65/0x75 [test_kasan]
Documentation/dev-tools/kasan.rst similar to the following to the respective kernel Makefile: · line 67
Quoted 10 lines
Documentation/dev-tools/kasan.rst +3
-7- [<ffffffffa00026b0>] init_module+0x9/0x47 [test_kasan] - [<ffffffff810002d9>] do_one_initcall+0x99/0x200 - [<ffffffff811e4e5c>] ? __vunmap+0xec/0x160 - [<ffffffff81114f63>] load_module+0x2cb3/0x3b20 - [<ffffffff8110fd70>] ? m_show+0x240/0x240 - [<ffffffff81115f06>] SyS_finit_module+0x76/0x80 - [<ffffffff81cd3129>] system_call_fastpath+0x12/0x17 + __dump_stack lib/dump_stack.c:17 + dump_stack+0x83/0xbc lib/dump_stack.c:53 + print_address_description+0x73/0x280 mm/kasan/report.c:254
KASAN does not print line numbers per se.
I think we need to show unmodified output to not confuse readers (probably remove the useless ? lines).
Quoted 151 lines
Documentation/dev-tools/kasan.rst +95
-26+ kasan_report_error mm/kasan/report.c:352 + kasan_report+0x10e/0x220 mm/kasan/report.c:410 + __asan_report_store1_noabort+0x17/0x20 mm/kasan/report.c:505 + kmalloc_oob_right+0xa8/0xbc [test_kasan] lib/test_kasan.c:42 + kmalloc_tests_init+0x16/0x769 [test_kasan] + do_one_initcall+0x9e/0x240 init/main.c:832 + do_init_module+0x1b6/0x542 kernel/module.c:3462 + load_module+0x6042/0x9030 kernel/module.c:3786 + SYSC_init_module+0x18f/0x1c0 kernel/module.c:3858 + SyS_init_module+0x9/0x10 kernel/module.c:3841 + do_syscall_64+0x198/0x480 arch/x86/entry/common.c:287 + entry_SYSCALL_64_after_hwframe+0x21/0x86 arch/x86/entry/entry_64.S:251 + RIP: 0033:0x7fdd79df99da + RSP: 002b:00007fff2229bdf8 EFLAGS: 00000202 ORIG_RAX: 00000000000000af + RAX: ffffffffffffffda RBX: 000055c408121190 RCX: 00007fdd79df99da + RDX: 00007fdd7a0b8f88 RSI: 0000000000055670 RDI: 00007fdd7a47e000 + RBP: 000055c4081200b0 R08: 0000000000000003 R09: 0000000000000000 + R10: 00007fdd79df5d0a R11: 0000000000000202 R12: 00007fdd7a0b8f88 + R13: 000055c408120090 R14: 0000000000000000 R15: 0000000000000000 + + Allocated by task 2734: + save_stack+0x43/0xd0 mm/kasan/common.c:176 + set_track+0x20/0x30 mm/kasan/common.c:188 + kasan_kmalloc+0x9a/0xc0 mm/kasan/kasan.c:372 + kmem_cache_alloc_trace+0xcd/0x1a0 mm/slub.c:2761 + kmalloc ./include/linux/slab.h:512 + kmalloc_oob_right+0x56/0xbc [test_kasan] lib/test_kasan.c:36 + kmalloc_tests_init+0x16/0x769 [test_kasan] + do_one_initcall+0x9e/0x240 init/main.c:832 + do_init_module+0x1b6/0x542 kernel/module.c:3462 + load_module+0x6042/0x9030 kernel/module.c:3786 + SYSC_init_module+0x18f/0x1c0 kernel/module.c:3858 + SyS_init_module+0x9/0x10 kernel/module.c:3841 + do_syscall_64+0x198/0x480 arch/x86/entry/common.c:287 + entry_SYSCALL_64_after_hwframe+0x21/0x86 arch/x86/entry/entry_64.S:251 + + The buggy address belongs to the object at ffff8800696f3cc0 + which belongs to the cache kmalloc-128 of size 128 + The buggy address is located 123 bytes inside of + 128-byte region [ffff8800696f3cc0, ffff8800696f3d40) + The buggy address belongs to the page: + page:ffffea0001a5bcc0 count:1 mapcount:0 mapping: (null) index:0x0 + flags: 0x100000000000100(slab) + raw: 0100000000000100 0000000000000000 0000000000000000 0000000180150015 + raw: ffffea0001a8ce40 0000000300000003 ffff88006d001640 0000000000000000 + page dumped because: kasan: bad access detected + Memory state around the buggy address: - ffff8800693bc300: fc fc fc fc fc fc fc fc fc fc fc fc fc fc fc fc - ffff8800693bc380: fc fc 00 00 00 00 00 00 00 00 00 00 00 00 00 fc - ffff8800693bc400: fc fc fc fc fc fc fc fc fc fc fc fc fc fc fc fc - ffff8800693bc480: fc fc fc fc fc fc fc fc fc fc fc fc fc fc fc fc - ffff8800693bc500: fc fc fc fc fc fc fc fc fc fc fc 00 00 00 00 00 - >ffff8800693bc580: 00 00 00 00 00 00 00 00 00 00 03 fc fc fc fc fc - ^ - ffff8800693bc600: fc fc fc fc fc fc fc fc fc fc fc fc fc fc fc fc - ffff8800693bc680: fc fc fc fc fc fc fc fc fc fc fc fc fc fc fc fc - ffff8800693bc700: fc fc fc fc fb fb fb fb fb fb fb fb fb fb fb fb - ffff8800693bc780: fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb - ffff8800693bc800: fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb + ffff8800696f3c00: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 fc + ffff8800696f3c80: fc fc fc fc fc fc fc fc 00 00 00 00 00 00 00 00 + >ffff8800696f3d00: 00 00 00 00 00 00 00 03 fc fc fc fc fc fc fc fc + ^ + ffff8800696f3d80: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 fc fc + ffff8800696f3e00: fc fc fc fc fc fc fc fc fb fb fb fb fb fb fb fb ================================================================== -The header of the report discribe what kind of bug happened and what kind of -access caused it. It's followed by the description of the accessed slub object -(see 'SLUB Debug output' section in Documentation/vm/slub.rst for details) and -the description of the accessed memory page. +The header of the report provides a short summary of what kind of bug happened +and what kind of access caused it. It's followed by a stack trace of the bad +access, a stack trace of where the accessed memory was allocated (in case bad +access happens on a slab object), and a stack trace of where the object was +freed (in case of a use-after-free bug report). Next comes a description of +the accessed slab object and information about the accessed memory page. In the last section the report shows memory state around the accessed address. Reading this part requires some understanding of how KASAN works.@@ -138,18 +143,24 @@ inaccessible memory like redzones or freed memory (see mm/kasan/kasan.h). In the report above the arrows point to the shadow byte 03, which means that the accessed address is partially accessible. +For KHWASAN this last report section shows the memory tags around the accessed +address (see Implementation details section). Implementation details ---------------------- +Classic KASAN +~~~~~~~~~~~~~ + From a high level, our approach to memory error detection is similar to that of kmemcheck: use shadow memory to record whether each byte of memory is safe -to access, and use compile-time instrumentation to check shadow memory on each -memory access. +to access, and use compile-time instrumentation to insert checks of shadow +memory on each memory access. -AddressSanitizer dedicates 1/8 of kernel memory to its shadow memory -(e.g. 16TB to cover 128TB on x86_64) and uses direct mapping with a scale and -offset to translate a memory address to its corresponding shadow address. +Classic KASAN dedicates 1/8th of kernel memory to its shadow memory (e.g. 16TB +to cover 128TB on x86_64) and uses direct mapping with a scale and offset to +translate a memory address to its corresponding shadow address. Here is the function which translates an address to its corresponding shadow address::@@ -162,12 +173,34 @@ address:: where ``KASAN_SHADOW_SCALE_SHIFT = 3``. -Compile-time instrumentation used for checking memory accesses. Compiler inserts -function calls (__asan_load*(addr), __asan_store*(addr)) before each memory -access of size 1, 2, 4, 8 or 16. These functions check whether memory access is -valid or not by checking corresponding shadow memory. +Compile-time instrumentation is used to insert memory access checks. Compiler +inserts function calls (__asan_load*(addr), __asan_store*(addr)) before each +memory access of size 1, 2, 4, 8 or 16. These functions check whether memory +access is valid or not by checking corresponding shadow memory. GCC 5.0 has possibility to perform inline instrumentation. Instead of making function calls GCC directly inserts the code to check the shadow memory. This option significantly enlarges kernel but it gives x1.1-x2 performance boost over outline instrumented kernel. + +KHWASAN +~~~~~~~ + +KHWASAN uses the Top Byte Ignore (TBI) feature of modern arm64 CPUs to store +a pointer tag in the top byte of kernel pointers. KHWASAN also uses shadow +memory to store memory tags associated with each 16-byte memory cell (therefore +it dedicates 1/16th of the kernel memory for shadow memory). + +On each memory allocation KHWASAN generates a random tag, tags allocated memory +with this tag, and embeds this tag into the returned pointer. KHWASAN uses +compile-time instrumentation to insert checks before each memory access. These +checks make sure that tag of the memory that is being accessed is equal to tag +of the pointer that is used to access this memory. In case of a tag mismatch +KHWASAN prints a bug report. + +KHWASAN also has two instrumentation modes (outline, that emits callbacks to +check memory accesses; and inline, that performs the shadow memory checks +inline). With outline instrumentation mode, a bug report is simply printed +from the function that performs the access check. With inline instrumentation +a brk instruction is emitted by the compiler, and a dedicated brk handler is +used to print KHWASAN reports. --2.19.0.rc0.228.g281dcd1b4d0-goog
Re: [PATCH v6 17/18] khwasan: update kasan documentation
Quoted 22 lines · Dmitry Vyukov
Documentation/dev-tools/kasan.rst KernelAddressSANitizer (KASAN) is a dynamic memory error detector. It provides · line 19
Quoted 8 lines · Andrey Konovalov
This patch updates KASAN documentation to reflect the addition of KHWASAN.
Documentation/dev-tools/kasan.rst +5
-1-Currently KASAN is supported only for the x86_64 and arm64 architectures. +KASAN uses compile-time instrumentation to insert validity checks before every +memory access, and therefore requires a compiler version that supports that. +For classic KASAN you need GCC version 4.9.2 or later. GCC 5.0 or later is +required for detection of out-of-bounds accesses on stack and global variables. +KHWASAN in turns is only supported in clang and requires revision 330044 orin turn?
Documentation/dev-tools/kasan.rst To enable KASAN configure kernel with:: · line 33
Quoted 5 lines
Documentation/dev-tools/kasan.rst +2
-3-and choose between CONFIG_KASAN_OUTLINE and CONFIG_KASAN_INLINE. Outline and -inline are compiler instrumentation types. The former produces smaller binary -the latter is 1.1 - 2 times faster. Inline instrumentation requires a GCC +and choose between CONFIG_KASAN_GENERIC (to enable classic KASAN) and +CONFIG_KASAN_HW (to enabled KHWASAN). You also need to choose choose betweento enable
Quoted 6 lines
Quoted 1 line
+
print_address_description+0x73/0x280mm/kasan/report.c:254KASAN does not print line numbers per se.
I think we need to show unmodified output to not confuse readers (probably remove the useless ? lines).
Will fix all in v7.
PATCH v6 18/18 kasan: add SPDX-License-Identifier mark to source files
In reply to Andrey KonovalovThis patch adds a "SPDX-License-Identifier: GPL-2.0" mark to all source files under mm/kasan.
--- mm/kasan/common.c | 1 + mm/kasan/kasan.c | 1 + mm/kasan/kasan_init.c | 1 + mm/kasan/kasan_report.c | 1 + mm/kasan/khwasan.c | 1 + mm/kasan/khwasan_report.c | 1 + mm/kasan/quarantine.c | 1 + mm/kasan/report.c | 1 + 8 files changed, 8 insertions(+)
mm/kasan/common.c +1 -0
@@ -1,3 +1,4 @@ +// SPDX-License-Identifier: GPL-2.0 /* * This file contains common KASAN and KHWASAN code. *
mm/kasan/kasan.c +1 -0
@@ -1,3 +1,4 @@ +// SPDX-License-Identifier: GPL-2.0 /* * This file contains core KASAN code. *
mm/kasan/kasan_init.c +1 -0
@@ -1,3 +1,4 @@ +// SPDX-License-Identifier: GPL-2.0 /* * This file contains some kasan initialization code. *
mm/kasan/kasan_report.c +1 -0
@@ -1,3 +1,4 @@ +// SPDX-License-Identifier: GPL-2.0 /* * This file contains KASAN specific error reporting code. *
mm/kasan/khwasan.c +1 -0
@@ -1,3 +1,4 @@ +// SPDX-License-Identifier: GPL-2.0 /* * This file contains core KHWASAN code. *
mm/kasan/khwasan_report.c +1 -0
@@ -1,3 +1,4 @@ +// SPDX-License-Identifier: GPL-2.0 /* * This file contains KHWASAN specific error reporting code. *
mm/kasan/quarantine.c +1 -0
@@ -1,3 +1,4 @@ +// SPDX-License-Identifier: GPL-2.0 /* * KASAN quarantine. *
mm/kasan/report.c +1 -0
@@ -1,3 +1,4 @@ +// SPDX-License-Identifier: GPL-2.0 /* * This file contains common KASAN and KHWASAN error reporting code. *
Signature · 1 line
2.19.0.rc0.228.g281dcd1b4d0-goog
Andrew Morton
maintainerQuoted 2 lines · On Wed, 29 Aug 2018 13:35:04 +0200 Andrey Konovalov <[email protected]>…
This patchset adds a new mode to KASAN [1], which is called KHWASAN (Kernel HardWare assisted Address SANitizer).
We're at v6 and there are no reviewed-by's or acked-by's to be seen. Is that a fair commentary on what has been happening, or have people been remiss in sending and gathering such things?
Nick Desaulniers
reviewerQuoted 10 lines · Andrew Morton
Quoted 2 lines · On Wed, 29 Aug 2018 13:35:04 +0200 Andrey Konovalov <[email protected]>…
This patchset adds a new mode to KASAN [1], which is called KHWASAN (Kernel HardWare assisted Address SANitizer).
We're at v6 and there are no reviewed-by's or acked-by's to be seen. Is that a fair commentary on what has been happening, or have people been remiss in sending and gathering such things?
I'm anxious to use these for Pixel Android devices. Looks like the series has been aggregating changes from valuable feedback. Maybe if the ARM maintainers and KASAN maintainers could Ack or Nack these, we could decide to merge these or what needs more work?
Signature · 2 lines
Thanks, ~Nick Desaulniers
Quoted 8 lines · Andrew Morton
Quoted 2 lines · On Wed, 29 Aug 2018 13:35:04 +0200 Andrey Konovalov <[email protected]>…
This patchset adds a new mode to KASAN [1], which is called KHWASAN (Kernel HardWare assisted Address SANitizer).
We're at v6 and there are no reviewed-by's or acked-by's to be seen. Is that a fair commentary on what has been happening, or have people been remiss in sending and gathering such things?
I still have concerns about the consequences of merging this as anything other than a debug option [1]. Unfortunately, merging it as a debug option defeats the whole point, so I think we need to spend more effort on developing tools that can help us to find and fix the subtle bugs which will arise from enabling tagged pointers in the kernel.
Will
[1] http://lists.infradead.org/pipermail/linux-arm-kernel/2018-August/596077.html
Quoted 10 lines · Will Deacon
Quoted 8 lines · Andrew Morton
Quoted 2 lines · On Wed, 29 Aug 2018 13:35:04 +0200 Andrey Konovalov <[email protected]>…
This patchset adds a new mode to KASAN [1], which is called KHWASAN (Kernel HardWare assisted Address SANitizer).
We're at v6 and there are no reviewed-by's or acked-by's to be seen. Is that a fair commentary on what has been happening, or have people been remiss in sending and gathering such things?
Quoted 5 lines
I still have concerns about the consequences of merging this as anything other than a debug option [1]. Unfortunately, merging it as a debug option defeats the whole point, so I think we need to spend more effort on developing tools that can help us to find and fix the subtle bugs which will arise from enabling tagged pointers in the kernel.
I totally don't mind calling it a debug option. Do I need to somehow specify it somewhere?
Why does it defeat the point? The point is to ease KASAN-like testing on devices with limited memory.
Nick Desaulniers
reviewerQuoted 18 lines · Andrey Konovalov
Quoted 10 lines · Will Deacon
Quoted 8 lines · Andrew Morton
Quoted 2 lines · On Wed, 29 Aug 2018 13:35:04 +0200 Andrey Konovalov <[email protected]>…
This patchset adds a new mode to KASAN [1], which is called KHWASAN (Kernel HardWare assisted Address SANitizer).
We're at v6 and there are no reviewed-by's or acked-by's to be seen. Is that a fair commentary on what has been happening, or have people been remiss in sending and gathering such things?
Quoted 5 lines
I still have concerns about the consequences of merging this as anything other than a debug option [1]. Unfortunately, merging it as a debug option defeats the whole point, so I think we need to spend more effort on developing tools that can help us to find and fix the subtle bugs which will arise from enabling tagged pointers in the kernel.
Quoted 5 lines
I totally don't mind calling it a debug option. Do I need to somehow specify it somewhere?
Why does it defeat the point? The point is to ease KASAN-like testing on devices with limited memory.
I don't disagree with using it strictly for debug. When I say I want the series for Pixel phones, I should have been clearer that my intent is for a limited pool of internal testers to walk around with KHWASAN enabled devices; not general end users. It's hard enough today to get anyone to test KASAN/ASAN on their "daily driver" due to the memory usage and resulting performance.
We don't ship KASAN or KUBSAN on by default to end users (nor plan to); it's used strictly for fuzzing through syzkaller (or by brave "dogfooders" on the internal kernel teams). KHWASAN would let these dogfooders go from being brave to fearless.
Signature · 2 lines
Thanks, ~Nick Desaulniers
Quoted 11 lines · Andrey Konovalov
Quoted 10 lines · Will Deacon
Quoted 8 lines · Andrew Morton
Quoted 2 lines · On Wed, 29 Aug 2018 13:35:04 +0200 Andrey Konovalov <[email protected]>…
This patchset adds a new mode to KASAN [1], which is called KHWASAN (Kernel HardWare assisted Address SANitizer).
We're at v6 and there are no reviewed-by's or acked-by's to be seen. Is that a fair commentary on what has been happening, or have people been remiss in sending and gathering such things?
Quoted 8 lines
Quoted 5 lines
I still have concerns about the consequences of merging this as anything other than a debug option [1]. Unfortunately, merging it as a debug option defeats the whole point, so I think we need to spend more effort on developing tools that can help us to find and fix the subtle bugs which will arise from enabling tagged pointers in the kernel.
I totally don't mind calling it a debug option. Do I need to somehow specify it somewhere?
Ok, sorry, I completely misunderstood you earlier on then! For some reason I thought you wanted this on by default.
In which case, I'm ok with the overall idea as long as we make the caveats clear in the Kconfig text. In particular, that enabling this option may introduce problems relating to pointer casting and comparison, but can offer better coverage and lower memory consumption than a fully software-based KASAN solution.
Will
Quoted 21 lines · Will Deacon
Quoted 11 lines · Andrey Konovalov
Quoted 10 lines · Will Deacon
Quoted 8 lines · Andrew Morton
Quoted 2 lines · On Wed, 29 Aug 2018 13:35:04 +0200 Andrey Konovalov <[email protected]>…
This patchset adds a new mode to KASAN [1], which is called KHWASAN (Kernel HardWare assisted Address SANitizer).
We're at v6 and there are no reviewed-by's or acked-by's to be seen. Is that a fair commentary on what has been happening, or have people been remiss in sending and gathering such things?
Quoted 8 lines
Quoted 5 lines
I still have concerns about the consequences of merging this as anything other than a debug option [1]. Unfortunately, merging it as a debug option defeats the whole point, so I think we need to spend more effort on developing tools that can help us to find and fix the subtle bugs which will arise from enabling tagged pointers in the kernel.
I totally don't mind calling it a debug option. Do I need to somehow specify it somewhere?
Quoted 8 lines
Ok, sorry, I completely misunderstood you earlier on then! For some reason I thought you wanted this on by default.
In which case, I'm ok with the overall idea as long as we make the caveats clear in the
Kconfigtext. In particular, that enabling this option may introduce problems relating to pointer casting and comparison, but can offer better coverage and lower memory consumption than a fully software-based KASAN solution.
Great! I'll explicitly call it debug feature and mention the caveats in v7. Thanks!
Quoted 12 lines · Andrey Konovalov
Andrey Konovalov (18):
khwasan, mm: change kasan hooks signatures
khwasan: move common kasan and khwasan code tocommon.c
khwasan: addCONFIG_KASAN_GENERICandCONFIG_KASAN_HWkhwasan, arm64: adjust shadow size forCONFIG_KASAN_HW
khwasan: initialize shadow to0xff
khwasan, arm64: untag virt address in__kimg_to_physand
_virt_addr_is_linear
khwasan: add tag related helper functions
khwasan: preassign tags to objects with ctors orSLAB_TYPESAFE_BY_RCUkhwasan, arm64: fix up fault handling logic
khwasan, arm64: enable top byte ignore for the kernel
Quoted 9 lines
khwasan, mm: perform untagged pointers comparison in krealloc
khwasan: split outkasan_report.cfromreport.c
khwasan: add bug reporting routines
khwasan: add hooks implementation
khwasan, arm64: add brk handler for inline instrumentation khwasan, mm, arm64: tag non slab memory allocated via pagealloc
khwasan: update kasan documentation
kasan: add SPDX-License-Identifier mark to source files
Aside from nit in 16/18 patch looks fine for me.