d99e3140a4: BUG:KCSAN:data-race_in_folio_remove_rmap_ptes/print_report
Subsystems (3)
kernel test robot
build botHello,
kernel test robot noticed "BUG:KCSAN:data-race_in_folio_remove_rmap_ptes/print_report" on:
commit: d99e3140a4d33e26066183ff727d8f02f56bec64 ("mm: turn folio_test_hugetlb into a PageType")
https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git master
[test failed on linus/master c760b3725e52403dc1b28644fb09c47a83cacea6]
[test failed on linux-next/master 3689b0ef08b70e4e03b82ebd37730a03a672853a]
in testcase: trinity
version: trinity-i386-abe9de86-1_20230429
with following parameters:
runtime: 300s group: group-04 nr_groups: 5
compiler: gcc-13
test machine: qemu-system-x86_64 -enable-kvm -cpu SandyBridge -smp 2 -m 16G
(please refer to attached dmesg/kmsg for entire log/backtrace)
we noticed this issue does not always happen. we also noticed there are different random KCSAN issues for both this commit and its parent. but below 4 only happen on this commit with not small rate and keep clean on parent.
fd1a745ce03e3794 d99e3140a4d33e26066183ff727
---------------- ---------------------------
fail:runs %reproduction fail:runs
| | |
:106 35% 37:192 dmesg.BUG:KCSAN:data-race_in_folio_add_file_rmap_ptes/print_report
:106 29% 31:192 dmesg.BUG:KCSAN:data-race_in_folio_dup_file_rmap_ptes/print_report
:106 103% 109:192 dmesg.BUG:KCSAN:data-race_in_folio_remove_rmap_ptes/print_report
:106 21% 22:192 dmesg.BUG:KCSAN:data-race_in_folio_try_dup_anon_rmap_ptes/print_report
If you fix the issue in a separate patch/commit (i.e. not just a new version of the same patch/commit), kindly add following tags
| Reported-by: kernel test robot <[email protected]>
| Closes: https://lore.kernel.org/oe-lkp/[email protected]
[ 66.557105][ T404] BUG: KCSAN: data-race in folio_remove_rmap_ptes / print_report [ 66.557726][ T404] [ 66.557923][ T404] read-write (marked) to 0xffffea000443ec30 of 4 bytes by task 405 on cpu 0: [ 66.558647][ T404] folio_remove_rmap_ptes (arch/x86/include/asm/atomic.h:79 (discriminator 1) include/linux/atomic/atomic-arch-fallback.h:2319 (discriminator 1) include/linux/atomic/atomic-instrumented.h:1421 (discriminator 1) mm/rmap.c:1514 (discriminator 1) mm/rmap.c:1590 (discriminator 1)) [ 66.559106][ T404] zap_present_ptes (mm/memory.c:1506 mm/memory.c:1563) [ 66.559525][ T404] zap_pte_range (mm/memory.c:1608 (discriminator 1)) [ 66.559919][ T404] zap_pmd_range+0x11c/0x18a [ 66.560359][ T404] unmap_page_range (mm/memory.c:1751 mm/memory.c:1772 mm/memory.c:1793) [ 66.560773][ T404] unmap_single_vma (mm/memory.c:1841) [ 66.561184][ T404] unmap_vmas (mm/memory.c:1885) [ 66.561552][ T404] exit_mmap (mm/mmap.c:3268) [ 66.561910][ T404] __mmput (kernel/fork.c:1347) [ 66.562235][ T404] mmput (kernel/fork.c:1369) [ 66.562555][ T404] exec_mmap (fs/exec.c:1053) [ 66.562901][ T404] begin_new_exec (fs/exec.c:1310) [ 66.563280][ T404] load_elf_binary (fs/binfmt_elf.c:1006 (discriminator 1)) [ 66.563678][ T404] search_binary_handler (fs/exec.c:1780) [ 66.564106][ T404] exec_binprm (fs/exec.c:1821) [ 66.564461][ T404] bprm_execve (fs/exec.c:1872) [ 66.564829][ T404] do_execveat_common+0x286/0x2af [ 66.565284][ T404] compat_do_execve (fs/exec.c:2081) [ 66.565663][ T404] __ia32_compat_sys_execve (fs/exec.c:2144) [ 66.566100][ T404] ia32_sys_call (kbuild/obj/consumer/x86_64-randconfig-016-20230920/./arch/x86/include/generated/asm/syscalls_32.h:12) [ 66.566529][ T404] __do_fast_syscall_32 (arch/x86/entry/common.c:165 arch/x86/entry/common.c:321) [ 66.566946][ T404] do_fast_syscall_32 (arch/x86/entry/common.c:346 (discriminator 1)) [ 66.567369][ T404] do_SYSENTER_32 (arch/x86/entry/common.c:385) [ 66.567762][ T404] entry_SYSENTER_compat_after_hwframe (arch/x86/entry/entry_64_compat.S:122) [ 66.568307][ T404] [ 66.568510][ T404] read to 0xffffea000443ec30 of 4 bytes by task 404 on cpu 1: [ 66.569119][ T404] print_report (kernel/kcsan/report.c:396) [ 66.569495][ T404] kcsan_report_known_origin (kernel/kcsan/report.c:692) [ 66.569959][ T404] kcsan_setup_watchpoint (kernel/kcsan/core.c:678) [ 66.570402][ T404] __tsan_read4 (kernel/kcsan/core.c:1024) [ 66.570787][ T404] __folio_rmap_sanity_checks (include/linux/page-flags.h:1045 include/linux/rmap.h:201) [ 66.571241][ T404] folio_remove_rmap_ptes (include/linux/instrumented.h:97 include/linux/atomic/atomic-instrumented.h:1420 mm/rmap.c:1514 mm/rmap.c:1590) [ 66.571679][ T404] zap_present_ptes (mm/memory.c:1506 mm/memory.c:1563) [ 66.572076][ T404] zap_pte_range (mm/memory.c:1608 (discriminator 1)) [ 66.572456][ T404] zap_pmd_range+0x11c/0x18a [ 66.572877][ T404] unmap_page_range (mm/memory.c:1751 mm/memory.c:1772 mm/memory.c:1793) [ 66.573274][ T404] unmap_single_vma (mm/memory.c:1841) [ 66.573670][ T404] unmap_vmas (mm/memory.c:1885) [ 66.574020][ T404] exit_mmap (mm/mmap.c:3268) [ 66.574364][ T404] __mmput (kernel/fork.c:1347) [ 66.574717][ T404] mmput (kernel/fork.c:1369) [ 66.575033][ T404] exec_mmap (fs/exec.c:1053) [ 66.575401][ T404] begin_new_exec (fs/exec.c:1310) [ 66.575811][ T404] load_elf_binary (fs/binfmt_elf.c:1006 (discriminator 1)) [ 66.576234][ T404] search_binary_handler (fs/exec.c:1780) [ 66.576683][ T404] exec_binprm (fs/exec.c:1821) [ 66.577064][ T404] bprm_execve (fs/exec.c:1872) [ 66.577455][ T404] do_execveat_common+0x286/0x2af [ 66.577939][ T404] compat_do_execve (fs/exec.c:2081) [ 66.578316][ T404] __ia32_compat_sys_execve (fs/exec.c:2144) [ 66.582840][ T404] ia32_sys_call (kbuild/obj/consumer/x86_64-randconfig-016-20230920/./arch/x86/include/generated/asm/syscalls_32.h:12) [ 66.583256][ T404] __do_fast_syscall_32 (arch/x86/entry/common.c:165 arch/x86/entry/common.c:321) [ 66.583687][ T404] do_fast_syscall_32 (arch/x86/entry/common.c:346 (discriminator 1)) [ 66.584104][ T404] do_SYSENTER_32 (arch/x86/entry/common.c:385) [ 66.584491][ T404] entry_SYSENTER_compat_after_hwframe (arch/x86/entry/entry_64_compat.S:122) [ 66.585022][ T404] [ 66.585226][ T404] value changed: 0x00000016 -> 0x00000015 [ 66.585694][ T404] [ 66.585891][ T404] Reported by Kernel Concurrency Sanitizer on: [ 66.586387][ T404] CPU: 1 PID: 404 Comm: run Not tainted 6.9.0-rc4-00021-gd99e3140a4d3 #1 [ 66.587057][ T404] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.16.2-debian-1.16.2-1 04/01/2014 [ 66.587876][ T404] ==================================================================
The kernel config and materials to reproduce are available at:
https://download.01.org/0day-ci/archive/20240528/[email protected]
Signature · 2 lines
0-DAY CI Kernel Test Service https://github.com/intel/lkp-tests/wiki
Am 28.05.24 um 09:11 schrieb kernel test robot:
Quoted 25 lines
Hello,
kernel test robot noticed "BUG:KCSAN:data-race_in_folio_remove_rmap_ptes/print_report" on:
commit:
d99e3140a4d33e26066183ff727d8f02f56bec64("mm: turnfolio_test_hugetlbinto a PageType")
https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git master[test failed on linus/master
c760b3725e52403dc1b28644fb09c47a83cacea6]
[test failed on linux-next/master3689b0ef08b70e4e03b82ebd37730a03a672853a]in testcase: trinity
version: trinity-i386-abe9de86-1_20230429
with following parameters:runtime: 300s group: group-04 nr_groups: 5compiler: gcc-13
test machine: qemu-system-x86_64 -enable-kvm -cpu SandyBridge -smp 2 -m 16G
Quoted 7 lines
(please refer to attached dmesg/kmsg for entire log/backtrace)
we noticed this issue does not always happen. we also noticed there are different random KCSAN issues for both this commit and its parent. but below 4 only happen on this commit with not small rate and keep clean on parent.
Likely that's just a page_type check racing against concurrent mapcount changes.
In __folio_rmap_sanity_checks() we check
VM_WARN_ON_FOLIO(folio_test_hugetlb(folio), folio);
To make sure we don't get hugetlb folios in the wrong rmap code path. That can easily race with concurrent mapcount changes, just like any other page_type checks that end up in folio_test_type/page_has_type e.g., from PFN walkers.
Load tearing in these functions shouldn't really result in false positives (what we care about), but READ_ONCE shouldn't hurt or make a difference.
From b03dc9bf27571442d886d8da624a4e4f737433f2 Mon Sep 17 00:00:00 2001
From: David Hildenbrand <[email protected]>
Date: Tue, 28 May 2024 09:37:20 +0200
Subject: [PATCH] mm: read page_type using READ_ONCE
KCSAN complains about possible data races: while we check for a page_type -- for example for sanity checks -- we might concurrently modify the mapcount that overlays page_type.
Let's use READ_ONCE to avoid laod tearing (shouldn't make a difference) and to make KCSAN happy.
Note: nothing should really be broken besides wrong KCSAN complaints.
--- include/linux/page-flags.h | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-)
include/linux/page-flags.h +3 -3
@@ -955,9 +955,9 @@ PAGEFLAG_FALSE(HasHWPoisoned, has_hwpoisoned)#define PG_slab 0x00001000#define PageType (page, flag) \ - ((page->page_type & (PAGE_TYPE_BASE | flag)) == PAGE_TYPE_BASE) + ((READ_ONCE(page->page_type) & (PAGE_TYPE_BASE | flag)) == PAGE_TYPE_BASE)#define folio_test_type (folio, flag) \ - ((folio->page.page_type & (PAGE_TYPE_BASE | flag)) == PAGE_TYPE_BASE) + ((READ_ONCE(folio->page.page_type) & (PAGE_TYPE_BASE | flag)) == PAGE_TYPE_BASE)static inline int page_type_has_type (unsigned int page_type) {
@@ -966,7 +966,7 @@ static inline int page_type_has_type(unsigned int page_type)static inline int page_has_type (const struct page *page) { -return page_type_has_type (page->page_type); +return page_type_has_type (READ_ONCE (page->page_type)); }#define FOLIO_TYPE_OPS (lname, fname) \
Signature · 7 lines
2.45.1 -- Thanks, David / dhildenb
Oscar Salvador
Reviewed-byQuoted 5 lines · David Hildenbrand
Likely that's just a
page_typecheck racing against concurrent mapcount changes.In
__folio_rmap_sanity_checks()we checkVM_WARN_ON_FOLIO(folio_test_hugetlb(folio), folio);
Yeah, and that "collides" with
last = atomic_add_negative(-1, &page->_mapcount)
from __folio_remove_rmap.
Quoted 18 lines
To make sure we don't get hugetlb folios in the wrong rmap code path. That can easily race with concurrent mapcount changes, just like any other
page_typechecks that end up in folio_test_type/page_has_type e.g., from PFN walkers.Load tearing in these functions shouldn't really result in false positives (what we care about), but
READ_ONCEshouldn't hurt or make a difference.From
b03dc9bf27571442d886d8da624a4e4f737433f2Mon Sep 17 00:00:00 2001
From: David Hildenbrand <[email protected]>
Date: Tue, 28 May 2024 09:37:20 +0200
Subject: [PATCH] mm: readpage_typeusingREAD_ONCEKCSAN complains about possible data races: while we check for a
page_type-- for example for sanity checks -- we might concurrently modify the mapcount that overlayspage_type.
Quoted 8 lines
Let's use
READ_ONCEto avoid laod tearing (shouldn't make a difference) and to make KCSAN happy.Note: nothing should really be broken besides wrong KCSAN complaints.
Reported-by kernel test robot <[email protected]>Signed-off-by David Hildenbrand <[email protected]>
Thanks!
Signature · 2 lines
Oscar Salvador SUSE Labs
Miaohe Lin
Reviewed-byQuoted 58 lines · David Hildenbrand
Am 28.05.24 um 09:11 schrieb kernel test robot:
Quoted 25 lines
Hello,
kernel test robot noticed "BUG:KCSAN:data-race_in_folio_remove_rmap_ptes/print_report" on:
commit:
d99e3140a4d33e26066183ff727d8f02f56bec64("mm: turnfolio_test_hugetlbinto a PageType")
https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git master[test failed on linus/master
c760b3725e52403dc1b28644fb09c47a83cacea6] [test failed on linux-next/master3689b0ef08b70e4e03b82ebd37730a03a672853a]in testcase: trinity
version: trinity-i386-abe9de86-1_20230429
with following parameters:runtime: 300s
group: group-04
nr_groups: 5compiler: gcc-13
test machine: qemu-system-x86_64 -enable-kvm -cpu SandyBridge -smp 2 -m 16GQuoted 7 lines
(please refer to attached dmesg/kmsg for entire log/backtrace)
we noticed this issue does not always happen. we also noticed there are different random KCSAN issues for both this commit and its parent. but below 4 only happen on this commit with not small rate and keep clean on parent.
Likely that's just a
page_typecheck racing against concurrent mapcount changes.In
__folio_rmap_sanity_checks()we checkVM_WARN_ON_FOLIO(folio_test_hugetlb(folio), folio);To make sure we don't get hugetlb folios in the wrong rmap code path. That can easily race with concurrent mapcount changes, just like any other
page_typechecks that end up in folio_test_type/page_has_type e.g., from PFN walkers.Load tearing in these functions shouldn't really result in false positives (what we care about), but
READ_ONCEshouldn't hurt or make a difference.From
b03dc9bf27571442d886d8da624a4e4f737433f2Mon Sep 17 00:00:00 2001
From: David Hildenbrand <[email protected]>
Date: Tue, 28 May 2024 09:37:20 +0200
Subject: [PATCH] mm: readpage_typeusingREAD_ONCEKCSAN complains about possible data races: while we check for a
page_type-- for example for sanity checks -- we might concurrently modify the mapcount that overlayspage_type.
Quoted 8 lines
Let's use
READ_ONCEto avoid laod tearing (shouldn't make a difference) and to make KCSAN happy.Note: nothing should really be broken besides wrong KCSAN complaints.
Reported-by kernel test robot <[email protected]>Signed-off-by David Hildenbrand <[email protected]>
LGTM. Thanks for fixing.
Thanks.
.