string: Add load_unaligned_zeropad() code path to sized_strscpy()
Subsystems (6)
v4 cover string: Add load_unaligned_zeropad() code path to sized_strscpy()
This series fixes an issue where strscpy() would sometimes trigger a false positive KASAN report with MTE.
v4:
- clarify commit message
- improve comment
v3:
- simplify test case
Peter Collingbourne (1):
string: Add load_unaligned_zeropad() code path to sized_strscpy()
Vincenzo Frascino (1):
kasan: Add strscpy() test to trigger tag fault on arm64
lib/string.c | 13 ++++++++++--- mm/kasan/kasan_test_c.c | 16 ++++++++++++++++ 2 files changed, 26 insertions(+), 3 deletions(-)
Signature · 1 line
2.49.0.472.ge94155a9ec-goog
PATCH v4 1/2 string: Add load_unaligned_zeropad() code path to sized_strscpy()
The call to read_word_at_a_time() in sized_strscpy() is problematic with MTE because it may trigger a tag check fault when reading across a tag granule (16 bytes) boundary. To make this code MTE compatible, let's start using load_unaligned_zeropad() on architectures where it is available (i.e. architectures that define CONFIG_DCACHE_WORD_ACCESS). Because load_unaligned_zeropad() takes care of page boundaries as well as tag granule boundaries, also disable the code preventing crossing page boundaries when using load_unaligned_zeropad().
---
v2:
- new approach
lib/string.c | 13 ++++++++++--- 1 file changed, 10 insertions(+), 3 deletions(-)
lib/string.c +10 -3
@@ -119,6 +119,7 @@ ssize_t sized_strscpy(char *dest, const char *src, size_t count)if (count ==0 ||WARN_ON_ONCE (count >INT_MAX ))return -E2BIG ; +#ifndef CONFIG_DCACHE_WORD_ACCESS #ifdef CONFIG_HAVE_EFFICIENT_UNALIGNED_ACCESS /* * If src is unaligned, don't cross a page boundary,
@@ -133,12 +134,14 @@ ssize_t sized_strscpy(char *dest, const char *src, size_t count)/* If src or dest is unaligned, don't do word-at-a-time. */ if (((long ) dest | (long ) src) & (sizeof (long ) -1 )) max =0 ; +#endif #endif /* -* read_word_at_a_time() below may read uninitialized bytes after the -* trailing zero and use them in comparisons. Disable this optimization -* under KMSAN to prevent false positive reports. +* load_unaligned_zeropad() or read_word_at_a_time() below may read +* uninitialized bytes after the trailing zero and use them in +* comparisons. Disable this optimization under KMSAN to prevent +* false positive reports. */ if (IS_ENABLED (CONFIG_KMSAN )) max =0 ;
@@ -146,7 +149,11 @@ ssize_t sized_strscpy(char *dest, const char *src, size_t count)while (max >=sizeof (unsigned long )) {unsigned long c, data; +#ifdef CONFIG_DCACHE_WORD_ACCESS + c =load_unaligned_zeropad (src+res); +#else c =read_word_at_a_time (src+res); +#endif if (has_zero (c, &data, &constants)) { data =prep_zero_mask (c, data, &constants); data =create_zero_mask (data);
Signature · 1 line
2.49.0.472.ge94155a9ec-goog
Quoted 32 lines · Peter Collingbourne
lib/string.c +6
-3@@ -119,6 +119,7 @@ ssize_t sized_strscpy(char *dest, const char *src, size_t count)if (count ==0 ||WARN_ON_ONCE (count >INT_MAX ))return -E2BIG ; +#ifndef CONFIG_DCACHE_WORD_ACCESS #ifdef CONFIG_HAVE_EFFICIENT_UNALIGNED_ACCESS /* * If src is unaligned, don't cross a page boundary, @@ -133,12 +134,14 @@ ssize_t sized_strscpy(char *dest, const char *src, size_t count)/* If src or dest is unaligned, don't do word-at-a-time. */ if (((long ) dest | (long ) src) & (sizeof (long ) -1 )) max =0 ; +#endif #endif /* -* read_word_at_a_time() below may read uninitialized bytes after the -* trailing zero and use them in comparisons. Disable this optimization -* under KMSAN to prevent false positive reports. +* load_unaligned_zeropad() or read_word_at_a_time() below may read +* uninitialized bytes after the trailing zero and use them in +* comparisons. Disable this optimization under KMSAN to prevent +* false positive reports. */ if (IS_ENABLED (CONFIG_KMSAN )) max =0 ;@@ -146,7 +149,11 @@ ssize_t sized_strscpy(char *dest, const char *src, size_t count)while (max >=sizeof (unsigned long )) {
lib/string.c ssize_t sized_strscpy(char *dest, const char *src, size_t count) · line 159
Quoted 10 lines
lib/string.c +4
-0unsigned long c, data; +#ifdef CONFIG_DCACHE_WORD_ACCESS + c =load_unaligned_zeropad (src+res); +#else c =read_word_at_a_time (src+res); +#endif if (has_zero (c, &data, &constants)) { data =prep_zero_mask (c, data, &constants); data =create_zero_mask (data);
Kees mentioned the scenario where this crosses the page boundary and we pad the source with zeros. It's probably fine but there are 70+ cases where the strscpy() return value is checked, I only looked at a couple.
Could we at least preserve the behaviour with regards to page boundaries and keep the existing 'max' limiting logic? If I read the code correctly, a fall back to reading one byte at a time from an unmapped page would panic. We also get this behaviour if src[0] is reading from an invalid address, though for arm64 the panic would be in ex_handler_load_unaligned_zeropad() when count >= 8.
Reading across tag granule (but not across page boundary) and causing a tag check fault would result in padding but we can live with this and only architectures that do MTE-style tag checking would get the new behaviour.
What I haven't checked is whether a tag check fault in ex_handler_load_unaligned_zeropad() would confuse the KASAN logic for MTE (it would be a second tag check fault while processing the first). At a quick look, it seems ok but it might be worth checking.
Signature · 1 line
Catalin
Quoted 36 lines · Catalin Marinas
Quoted 34 lines · Peter Collingbourne
lib/string.c +6
-3@@ -119,6 +119,7 @@ ssize_t sized_strscpy(char *dest, const char *src, size_t count)if (count ==0 ||WARN_ON_ONCE (count >INT_MAX ))return -E2BIG ; +#ifndef CONFIG_DCACHE_WORD_ACCESS #ifdef CONFIG_HAVE_EFFICIENT_UNALIGNED_ACCESS /* * If src is unaligned, don't cross a page boundary, @@ -133,12 +134,14 @@ ssize_t sized_strscpy(char *dest, const char *src, size_t count)/* If src or dest is unaligned, don't do word-at-a-time. */ if (((long ) dest | (long ) src) & (sizeof (long ) -1 )) max =0 ; +#endif #endif /* -* read_word_at_a_time() below may read uninitialized bytes after the -* trailing zero and use them in comparisons. Disable this optimization -* under KMSAN to prevent false positive reports. +* load_unaligned_zeropad() or read_word_at_a_time() below may read +* uninitialized bytes after the trailing zero and use them in +* comparisons. Disable this optimization under KMSAN to prevent +* false positive reports. */ if (IS_ENABLED (CONFIG_KMSAN )) max =0 ;@@ -146,7 +149,11 @@ ssize_t sized_strscpy(char *dest, const char *src, size_t count)while (max >=sizeof (unsigned long )) {unsigned long c, data;
Quoted 12 lines
lib/string.c ssize_t sized_strscpy(char *dest, const char *src, size_t count) · line 159
Quoted 8 lines
lib/string.c +4
-0+#ifdef CONFIG_DCACHE_WORD_ACCESS + c =load_unaligned_zeropad (src+res); +#else c =read_word_at_a_time (src+res); +#endif if (has_zero (c, &data, &constants)) { data =prep_zero_mask (c, data, &constants); data =create_zero_mask (data);Kees mentioned the scenario where this crosses the page boundary and we pad the source with zeros. It's probably fine but there are 70+ cases where the
strscpy()return value is checked, I only looked at a couple.
The return value is the same with/without the patch, it's the number of bytes copied before the null terminator (i.e. not including the extra nulls now written).
Quoted 6 lines
Could we at least preserve the behaviour with regards to page boundaries and keep the existing 'max' limiting logic? If I read the code correctly, a fall back to reading one byte at a time from an unmapped page would panic. We also get this behaviour if src[0] is reading from an invalid address, though for arm64 the panic would be in
ex_handler_load_unaligned_zeropad()when count >= 8.
So do you think that the code should continue to panic if the source string is unterminated because of a page boundary? I don't have a strong opinion but maybe that's something that we should only do if some error checking option is turned on?
Quoted 4 lines
Reading across tag granule (but not across page boundary) and causing a tag check fault would result in padding but we can live with this and only architectures that do MTE-style tag checking would get the new behaviour.
By "padding" do you mean the extra (up to sizeof(unsigned long)) nulls now written to the destination? It seems unlikely that code would deliberately depend on the nulls not being written, the number of nulls written is not part of the documented interface contract and will vary right now depending on how close the source string is to a page boundary. If code is accidentally depending on nulls not being written, that's almost certainly a bug anyway (because of the page boundary thing) and we should fix it if discovered by this change.
Quoted 4 lines
What I haven't checked is whether a tag check fault in
ex_handler_load_unaligned_zeropad()would confuse the KASAN logic for MTE (it would be a second tag check fault while processing the first). At a quick look, it seems ok but it might be worth checking.
Yes, that works, and I added a test case for that in v5. The stack trace looks like this:
[ 21.969736] Call trace: [ 21.969739] show_stack+0x18/0x24 (C) [ 21.969756] __dump_stack+0x28/0x38 [ 21.969764] dump_stack_lvl+0x54/0x6c [ 21.969770] print_address_description+0x7c/0x274 [ 21.969780] print_report+0x90/0xe8 [ 21.969789] kasan_report+0xf0/0x150 [ 21.969799] __do_kernel_fault+0x5c/0x1cc [ 21.969808] do_bad_area+0x30/0xec [ 21.969816] do_tag_check_fault+0x20/0x30 [ 21.969824] do_mem_abort+0x3c/0x8c [ 21.969832] el1_abort+0x3c/0x5c [ 21.969840] el1h_64_sync_handler+0x50/0xcc [ 21.969847] el1h_64_sync+0x6c/0x70 [ 21.969854] fixup_exception+0xb0/0xe4 (P) [ 21.969865] __do_kernel_fault+0x80/0x1cc [ 21.969873] do_bad_area+0x30/0xec [ 21.969881] do_tag_check_fault+0x20/0x30 [ 21.969889] do_mem_abort+0x3c/0x8c [ 21.969896] el1_abort+0x3c/0x5c [ 21.969905] el1h_64_sync_handler+0x50/0xcc [ 21.969912] el1h_64_sync+0x6c/0x70 [ 21.969917] sized_strscpy+0x30/0x114 (P) [ 21.969929] kunit_try_run_case+0x64/0x160 [ 21.969939] kunit_generic_run_threadfn_adapter+0x28/0x4c [ 21.969950] kthread+0x1c4/0x208 [ 21.969956] ret_from_fork+0x10/0x20
Peter
Quoted 49 lines · Peter Collingbourne
Quoted 35 lines · Catalin Marinas
Quoted 34 lines · Peter Collingbourne
lib/string.c +6
-3@@ -119,6 +119,7 @@ ssize_t sized_strscpy(char *dest, const char *src, size_t count)if (count ==0 ||WARN_ON_ONCE (count >INT_MAX ))return -E2BIG ; +#ifndef CONFIG_DCACHE_WORD_ACCESS #ifdef CONFIG_HAVE_EFFICIENT_UNALIGNED_ACCESS /* * If src is unaligned, don't cross a page boundary, @@ -133,12 +134,14 @@ ssize_t sized_strscpy(char *dest, const char *src, size_t count)/* If src or dest is unaligned, don't do word-at-a-time. */ if (((long ) dest | (long ) src) & (sizeof (long ) -1 )) max =0 ; +#endif #endif /* -* read_word_at_a_time() below may read uninitialized bytes after the -* trailing zero and use them in comparisons. Disable this optimization -* under KMSAN to prevent false positive reports. +* load_unaligned_zeropad() or read_word_at_a_time() below may read +* uninitialized bytes after the trailing zero and use them in +* comparisons. Disable this optimization under KMSAN to prevent +* false positive reports. */ if (IS_ENABLED (CONFIG_KMSAN )) max =0 ;@@ -146,7 +149,11 @@ ssize_t sized_strscpy(char *dest, const char *src, size_t count)while (max >=sizeof (unsigned long )) {unsigned long c, data;Quoted 12 lines
lib/string.c ssize_t sized_strscpy(char *dest, const char *src, size_t count) · line 159
Quoted 8 lines
lib/string.c +4
-0+#ifdef CONFIG_DCACHE_WORD_ACCESS + c =load_unaligned_zeropad (src+res); +#else c =read_word_at_a_time (src+res); +#endif if (has_zero (c, &data, &constants)) { data =prep_zero_mask (c, data, &constants); data =create_zero_mask (data);Kees mentioned the scenario where this crosses the page boundary and we pad the source with zeros. It's probably fine but there are 70+ cases where the
strscpy()return value is checked, I only looked at a couple.
Quoted 3 lines
The return value is the same with/without the patch, it's the number of bytes copied before the null terminator (i.e. not including the extra nulls now written).
I was thinking of the -E2BIG return but you are right, the patch wouldn't change this. If, for example, you read 8 bytes across a page boundary and it faults, load_unaligned_zeropad() returns fewer characters copied, implying the source was null-terminated. read_word_at_a_time(), OTOH, panics in the next byte-at-a-time loop. But it wouldn't return -E2BIG either, so it doesn't matter for the caller.
Quoted 11 lines
Quoted 6 lines
Could we at least preserve the behaviour with regards to page boundaries and keep the existing 'max' limiting logic? If I read the code correctly, a fall back to reading one byte at a time from an unmapped page would panic. We also get this behaviour if src[0] is reading from an invalid address, though for arm64 the panic would be in
ex_handler_load_unaligned_zeropad()when count >= 8.So do you think that the code should continue to panic if the source string is unterminated because of a page boundary? I don't have a strong opinion but maybe that's something that we should only do if some error checking option is turned on?
It's mostly about keeping the current behaviour w.r.t. page boundaries. Not a strong opinion either. The change would be to not read across page boundaries.
Quoted 7 lines
Quoted 4 lines
Reading across tag granule (but not across page boundary) and causing a tag check fault would result in padding but we can live with this and only architectures that do MTE-style tag checking would get the new behaviour.
By "padding" do you mean the extra (up to sizeof(unsigned long)) nulls now written to the destination?
No, I meant the padding of the source when a fault occurs. The write to the destination would only be a single '\0' byte. It's the destination safe termination vs. panic above.
Quoted 7 lines
Quoted 4 lines
What I haven't checked is whether a tag check fault in
ex_handler_load_unaligned_zeropad()would confuse the KASAN logic for MTE (it would be a second tag check fault while processing the first). At a quick look, it seems ok but it might be worth checking.Yes, that works, and I added a test case for that in v5. The stack trace looks like this:
Thanks for checking.
Signature · 1 line
Catalin
Quoted 1 line · On Wed, 2 Apr 2025 17:08:51 -0700 Peter Collingbourne <[email protected]> wrote:
On Wed, Apr 2, 2025 at 1:10 PM Catalin Marinas <[email protected]> wrote:
..
Quoted 5 lines
Quoted 4 lines
Reading across tag granule (but not across page boundary) and causing a tag check fault would result in padding but we can live with this and only architectures that do MTE-style tag checking would get the new behaviour.
Quoted 8 lines
By "padding" do you mean the extra (up to sizeof(unsigned long)) nulls now written to the destination? It seems unlikely that code would deliberately depend on the nulls not being written, the number of nulls written is not part of the documented interface contract and will vary right now depending on how close the source string is to a page boundary. If code is accidentally depending on nulls not being written, that's almost certainly a bug anyway (because of the page boundary thing) and we should fix it if discovered by this change.
There was an issue with one of the copy routines writing beyond the expected point in a destination buffer.
I can't remember the full details, but it would match strscpy().
David
PATCH v4 2/2 kasan: Add strscpy() test to trigger tag fault on arm64
In reply to Peter CollingbourneFrom: Vincenzo Frascino <[email protected]>
When we invoke strscpy() with a maximum size of N bytes, it assumes that:
- It can always read N bytes from the source.
- It always write N bytes (zero-padded) to the destination.
On aarch64 with Memory Tagging Extension enabled if we pass an N that is bigger then the source buffer, it would previously trigger an MTE fault.
Implement a KASAN KUnit test that triggers the issue with the previous implementation of read_word_at_a_time() on aarch64 with MTE enabled.
---
v4:
- clarify commit message
- improve comment
v3:
- simplify test case
v2:
- rebased
- fixed test failure
mm/kasan/kasan_test_c.c | 16 ++++++++++++++++ 1 file changed, 16 insertions(+)
mm/kasan/kasan_test_c.c +16 -0
@@ -1570,6 +1570,7 @@ static void kasan_memcmp(struct kunit *test)static void kasan_strings (struct kunit *test) {char *ptr; +char *src;size_t size =24 ;/*
@@ -1581,6 +1582,21 @@ static void kasan_strings(struct kunit *test) ptr =kmalloc (size,GFP_KERNEL | __GFP_ZERO);KUNIT_ASSERT_NOT_ERR_OR_NULL (test, ptr); + src =kmalloc (KASAN_GRANULE_SIZE ,GFP_KERNEL | __GFP_ZERO); +strscpy (src,"f0cacc1a0000000" ,KASAN_GRANULE_SIZE ); + +/* +* Make sure that strscpy() does not trigger KASAN if it overreads into +* poisoned memory. +* +* The expected size does not include the terminator '\0' +* so it is (KASAN_GRANULE_SIZE - 2) == +* KASAN_GRANULE_SIZE - ("initial removed character" + "\0"). +*/ +KUNIT_EXPECT_EQ (test,KASAN_GRANULE_SIZE -2 , +strscpy (ptr, src +1 ,KASAN_GRANULE_SIZE )); + +kfree (src);kfree (ptr);/*
Signature · 1 line
2.49.0.472.ge94155a9ec-goog