Skip to contentSkip to conversation

d99e3140a4: BUG:KCSAN:data-race_in_folio_remove_rmap_ptes/print_report

linus:master kernel test robot 4 participants 4 messages May 28, 2024

Subsystems (3)
lore
Share & tools
lore

kernel test robot

author build bot
#0 lore raw

Hello,

kernel test robot noticed "BUG:KCSAN:data-race_in_folio_remove_rmap_ptes/print_report" on:

commit: d99e3140a4d33e26066183ff727d8f02f56bec64 ("mm: turn folio_test_hugetlb into a PageType")
https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git master

[test failed on linus/master c760b3725e52403dc1b28644fb09c47a83cacea6]
[test failed on linux-next/master 3689b0ef08b70e4e03b82ebd37730a03a672853a]

in testcase: trinity
version: trinity-i386-abe9de86-1_20230429
with following parameters:

	runtime: 300s
	group: group-04
	nr_groups: 5

compiler: gcc-13
test machine: qemu-system-x86_64 -enable-kvm -cpu SandyBridge -smp 2 -m 16G

(please refer to attached dmesg/kmsg for entire log/backtrace)

we noticed this issue does not always happen. we also noticed there are different random KCSAN issues for both this commit and its parent. but below 4 only happen on this commit with not small rate and keep clean on parent.

fd1a745ce03e3794 d99e3140a4d33e26066183ff727
---------------- ---------------------------
fail:runs %reproduction fail:runs
| | |
:106 35% 37:192 dmesg.BUG:KCSAN:data-race_in_folio_add_file_rmap_ptes/print_report
:106 29% 31:192 dmesg.BUG:KCSAN:data-race_in_folio_dup_file_rmap_ptes/print_report
:106 103% 109:192 dmesg.BUG:KCSAN:data-race_in_folio_remove_rmap_ptes/print_report
:106 21% 22:192 dmesg.BUG:KCSAN:data-race_in_folio_try_dup_anon_rmap_ptes/print_report

If you fix the issue in a separate patch/commit (i.e. not just a new version of the same patch/commit), kindly add following tags
| Reported-by: kernel test robot <[email protected]>
| Closes: https://lore.kernel.org/oe-lkp/[email protected]

[   66.557105][  T404] BUG: KCSAN: data-race in folio_remove_rmap_ptes / print_report
[   66.557726][  T404]
[   66.557923][  T404] read-write (marked) to 0xffffea000443ec30 of 4 bytes by task 405 on cpu 0:
[ 66.558647][ T404] folio_remove_rmap_ptes (arch/x86/include/asm/atomic.h:79 (discriminator 1) include/linux/atomic/atomic-arch-fallback.h:2319 (discriminator 1) include/linux/atomic/atomic-instrumented.h:1421 (discriminator 1) mm/rmap.c:1514 (discriminator 1) mm/rmap.c:1590 (discriminator 1)) 
[ 66.559106][ T404] zap_present_ptes (mm/memory.c:1506 mm/memory.c:1563) 
[ 66.559525][ T404] zap_pte_range (mm/memory.c:1608 (discriminator 1)) 
[ 66.559919][ T404] zap_pmd_range+0x11c/0x18a 
[ 66.560359][ T404] unmap_page_range (mm/memory.c:1751 mm/memory.c:1772 mm/memory.c:1793) 
[ 66.560773][ T404] unmap_single_vma (mm/memory.c:1841) 
[ 66.561184][ T404] unmap_vmas (mm/memory.c:1885) 
[ 66.561552][ T404] exit_mmap (mm/mmap.c:3268) 
[ 66.561910][ T404] __mmput (kernel/fork.c:1347) 
[ 66.562235][ T404] mmput (kernel/fork.c:1369) 
[ 66.562555][ T404] exec_mmap (fs/exec.c:1053) 
[ 66.562901][ T404] begin_new_exec (fs/exec.c:1310) 
[ 66.563280][ T404] load_elf_binary (fs/binfmt_elf.c:1006 (discriminator 1)) 
[ 66.563678][ T404] search_binary_handler (fs/exec.c:1780) 
[ 66.564106][ T404] exec_binprm (fs/exec.c:1821) 
[ 66.564461][ T404] bprm_execve (fs/exec.c:1872) 
[ 66.564829][ T404] do_execveat_common+0x286/0x2af 
[ 66.565284][ T404] compat_do_execve (fs/exec.c:2081) 
[ 66.565663][ T404] __ia32_compat_sys_execve (fs/exec.c:2144) 
[ 66.566100][ T404] ia32_sys_call (kbuild/obj/consumer/x86_64-randconfig-016-20230920/./arch/x86/include/generated/asm/syscalls_32.h:12) 
[ 66.566529][ T404] __do_fast_syscall_32 (arch/x86/entry/common.c:165 arch/x86/entry/common.c:321) 
[ 66.566946][ T404] do_fast_syscall_32 (arch/x86/entry/common.c:346 (discriminator 1)) 
[ 66.567369][ T404] do_SYSENTER_32 (arch/x86/entry/common.c:385) 
[ 66.567762][ T404] entry_SYSENTER_compat_after_hwframe (arch/x86/entry/entry_64_compat.S:122) 
[   66.568307][  T404]
[   66.568510][  T404] read to 0xffffea000443ec30 of 4 bytes by task 404 on cpu 1:
[ 66.569119][ T404] print_report (kernel/kcsan/report.c:396) 
[ 66.569495][ T404] kcsan_report_known_origin (kernel/kcsan/report.c:692) 
[ 66.569959][ T404] kcsan_setup_watchpoint (kernel/kcsan/core.c:678) 
[ 66.570402][ T404] __tsan_read4 (kernel/kcsan/core.c:1024) 
[ 66.570787][ T404] __folio_rmap_sanity_checks (include/linux/page-flags.h:1045 include/linux/rmap.h:201) 
[ 66.571241][ T404] folio_remove_rmap_ptes (include/linux/instrumented.h:97 include/linux/atomic/atomic-instrumented.h:1420 mm/rmap.c:1514 mm/rmap.c:1590) 
[ 66.571679][ T404] zap_present_ptes (mm/memory.c:1506 mm/memory.c:1563) 
[ 66.572076][ T404] zap_pte_range (mm/memory.c:1608 (discriminator 1)) 
[ 66.572456][ T404] zap_pmd_range+0x11c/0x18a 
[ 66.572877][ T404] unmap_page_range (mm/memory.c:1751 mm/memory.c:1772 mm/memory.c:1793) 
[ 66.573274][ T404] unmap_single_vma (mm/memory.c:1841) 
[ 66.573670][ T404] unmap_vmas (mm/memory.c:1885) 
[ 66.574020][ T404] exit_mmap (mm/mmap.c:3268) 
[ 66.574364][ T404] __mmput (kernel/fork.c:1347) 
[ 66.574717][ T404] mmput (kernel/fork.c:1369) 
[ 66.575033][ T404] exec_mmap (fs/exec.c:1053) 
[ 66.575401][ T404] begin_new_exec (fs/exec.c:1310) 
[ 66.575811][ T404] load_elf_binary (fs/binfmt_elf.c:1006 (discriminator 1)) 
[ 66.576234][ T404] search_binary_handler (fs/exec.c:1780) 
[ 66.576683][ T404] exec_binprm (fs/exec.c:1821) 
[ 66.577064][ T404] bprm_execve (fs/exec.c:1872) 
[ 66.577455][ T404] do_execveat_common+0x286/0x2af 
[ 66.577939][ T404] compat_do_execve (fs/exec.c:2081) 
[ 66.578316][ T404] __ia32_compat_sys_execve (fs/exec.c:2144) 
[ 66.582840][ T404] ia32_sys_call (kbuild/obj/consumer/x86_64-randconfig-016-20230920/./arch/x86/include/generated/asm/syscalls_32.h:12) 
[ 66.583256][ T404] __do_fast_syscall_32 (arch/x86/entry/common.c:165 arch/x86/entry/common.c:321) 
[ 66.583687][ T404] do_fast_syscall_32 (arch/x86/entry/common.c:346 (discriminator 1)) 
[ 66.584104][ T404] do_SYSENTER_32 (arch/x86/entry/common.c:385) 
[ 66.584491][ T404] entry_SYSENTER_compat_after_hwframe (arch/x86/entry/entry_64_compat.S:122) 
[   66.585022][  T404]
[   66.585226][  T404] value changed: 0x00000016 -> 0x00000015
[   66.585694][  T404]
[   66.585891][  T404] Reported by Kernel Concurrency Sanitizer on:
[   66.586387][  T404] CPU: 1 PID: 404 Comm: run Not tainted 6.9.0-rc4-00021-gd99e3140a4d3 #1
[   66.587057][  T404] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.16.2-debian-1.16.2-1 04/01/2014
[   66.587876][  T404] ==================================================================

The kernel config and materials to reproduce are available at:
https://download.01.org/0day-ci/archive/20240528/[email protected]

Signature · 2 lines
0-DAY CI Kernel Test Service
https://github.com/intel/lkp-tests/wiki
#1 lore raw

Am 28.05.24 um 09:11 schrieb kernel test robot:

Quoted 25 lines

Hello,

kernel test robot noticed "BUG:KCSAN:data-race_in_folio_remove_rmap_ptes/print_report" on:

commit: d99e3140a4d33e26066183ff727d8f02f56bec64 ("mm: turn folio_test_hugetlb into a PageType")
https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git master

[test failed on linus/master c760b3725e52403dc1b28644fb09c47a83cacea6]
[test failed on linux-next/master 3689b0ef08b70e4e03b82ebd37730a03a672853a]

in testcase: trinity
version: trinity-i386-abe9de86-1_20230429
with following parameters:

	runtime: 300s
	group: group-04
	nr_groups: 5

compiler: gcc-13
test machine: qemu-system-x86_64 -enable-kvm -cpu SandyBridge -smp 2 -m 16G

Quoted 7 lines

(please refer to attached dmesg/kmsg for entire log/backtrace)

we noticed this issue does not always happen. we also noticed there are different random KCSAN issues for both this commit and its parent. but below 4 only happen on this commit with not small rate and keep clean on parent.

Likely that's just a page_type check racing against concurrent mapcount changes.

In __folio_rmap_sanity_checks() we check

	VM_WARN_ON_FOLIO(folio_test_hugetlb(folio), folio);

To make sure we don't get hugetlb folios in the wrong rmap code path. That can easily race with concurrent mapcount changes, just like any other page_type checks that end up in folio_test_type/page_has_type e.g., from PFN walkers.

Load tearing in these functions shouldn't really result in false positives (what we care about), but READ_ONCE shouldn't hurt or make a difference.

From b03dc9bf27571442d886d8da624a4e4f737433f2 Mon Sep 17 00:00:00 2001
From: David Hildenbrand <[email protected]>
Date: Tue, 28 May 2024 09:37:20 +0200
Subject: [PATCH] mm: read page_type using READ_ONCE

KCSAN complains about possible data races: while we check for a page_type -- for example for sanity checks -- we might concurrently modify the mapcount that overlays page_type.

Let's use READ_ONCE to avoid laod tearing (shouldn't make a difference) and to make KCSAN happy.

Note: nothing should really be broken besides wrong KCSAN complaints.

---
 include/linux/page-flags.h | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)
include/linux/page-flags.h +3 -3
index 104078afe0b1..e46ccbb9aa58 100644
@@ -955,9 +955,9 @@ PAGEFLAG_FALSE(HasHWPoisoned, has_hwpoisoned)
 #define PG_slab		0x00001000
 
 #define PageType(page, flag)						\
-	((page->page_type & (PAGE_TYPE_BASE | flag)) == PAGE_TYPE_BASE)
+	((READ_ONCE(page->page_type) & (PAGE_TYPE_BASE | flag)) == PAGE_TYPE_BASE)
 #define folio_test_type(folio, flag)					\
-	((folio->page.page_type & (PAGE_TYPE_BASE | flag)) == PAGE_TYPE_BASE)
+	((READ_ONCE(folio->page.page_type) & (PAGE_TYPE_BASE | flag))  == PAGE_TYPE_BASE)
 
 static inline int page_type_has_type(unsigned int page_type)
 {
@@ -966,7 +966,7 @@ static inline int page_type_has_type(unsigned int page_type)
 
 static inline int page_has_type(const struct page *page)
 {
-	return page_type_has_type(page->page_type);
+	return page_type_has_type(READ_ONCE(page->page_type));
 }
 
 #define FOLIO_TYPE_OPS(lname, fname)					\
Signature · 7 lines
2.45.1


-- 
Thanks,

David / dhildenb
Reviewed-by
#2 lore raw
Quoted 5 lines · David Hildenbrand

Likely that's just a page_type check racing against concurrent mapcount changes.

In __folio_rmap_sanity_checks() we check

	VM_WARN_ON_FOLIO(folio_test_hugetlb(folio), folio);

Yeah, and that "collides" with

last = atomic_add_negative(-1, &page->_mapcount)

from __folio_remove_rmap.

Quoted 18 lines

To make sure we don't get hugetlb folios in the wrong rmap code path. That can easily race with concurrent mapcount changes, just like any other page_type checks that end up in folio_test_type/page_has_type e.g., from PFN walkers.

Load tearing in these functions shouldn't really result in false positives (what we care about), but READ_ONCE shouldn't hurt or make a difference.

From b03dc9bf27571442d886d8da624a4e4f737433f2 Mon Sep 17 00:00:00 2001
From: David Hildenbrand <[email protected]>
Date: Tue, 28 May 2024 09:37:20 +0200
Subject: [PATCH] mm: read page_type using READ_ONCE

KCSAN complains about possible data races: while we check for a page_type -- for example for sanity checks -- we might concurrently modify the mapcount that overlays page_type.

Quoted 8 lines

Let's use READ_ONCE to avoid laod tearing (shouldn't make a difference) and to make KCSAN happy.

Note: nothing should really be broken besides wrong KCSAN complaints.

Thanks!

Signature · 2 lines
Oscar Salvador
SUSE Labs
Reviewed-by
#3 lore raw
In reply to David Hildenbrand
Quoted 58 lines · David Hildenbrand

Am 28.05.24 um 09:11 schrieb kernel test robot:

Quoted 25 lines

Hello,

kernel test robot noticed "BUG:KCSAN:data-race_in_folio_remove_rmap_ptes/print_report" on:

commit: d99e3140a4d33e26066183ff727d8f02f56bec64 ("mm: turn folio_test_hugetlb into a PageType")
https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git master

[test failed on linus/master      c760b3725e52403dc1b28644fb09c47a83cacea6] [test failed on linux-next/master 3689b0ef08b70e4e03b82ebd37730a03a672853a]

in testcase: trinity
version: trinity-i386-abe9de86-1_20230429
with following parameters:

    runtime: 300s
    group: group-04
    nr_groups: 5

compiler: gcc-13
test machine: qemu-system-x86_64 -enable-kvm -cpu SandyBridge -smp 2 -m 16G

Quoted 7 lines

(please refer to attached dmesg/kmsg for entire log/backtrace)

we noticed this issue does not always happen. we also noticed there are different random KCSAN issues for both this commit and its parent. but below 4 only happen on this commit with not small rate and keep clean on parent.

Likely that's just a page_type check racing against concurrent mapcount changes.

In __folio_rmap_sanity_checks() we check VM_WARN_ON_FOLIO(folio_test_hugetlb(folio), folio);

To make sure we don't get hugetlb folios in the wrong rmap code path. That can easily race with concurrent mapcount changes, just like any other page_type checks that end up in folio_test_type/page_has_type e.g., from PFN walkers.

Load tearing in these functions shouldn't really result in false positives (what we care about), but READ_ONCE shouldn't hurt or make a difference.

From b03dc9bf27571442d886d8da624a4e4f737433f2 Mon Sep 17 00:00:00 2001
From: David Hildenbrand <[email protected]>
Date: Tue, 28 May 2024 09:37:20 +0200
Subject: [PATCH] mm: read page_type using READ_ONCE

KCSAN complains about possible data races: while we check for a page_type -- for example for sanity checks -- we might concurrently modify the mapcount that overlays page_type.

Quoted 8 lines

Let's use READ_ONCE to avoid laod tearing (shouldn't make a difference) and to make KCSAN happy.

Note: nothing should really be broken besides wrong KCSAN complaints.

LGTM. Thanks for fixing.

Thanks.
.