mm/vmstat: annotate data race in zone_page_state_snapshot()
Subsystems (2)
PATCH 1/1 mm/vmstat: annotate data race in zone_page_state_snapshot()
From: Hui Zhu <[email protected]>
KCSAN reports a data-race on v6.6 which also exists on mainline:
BUG: KCSAN: data-race in zone_page_state_snapshot root/klinux/./include/linux/vmstat.h:232 [inline] BUG: KCSAN: data-race in allow_direct_reclaim+0x150/0x290 root/klinux/mm/vmscan.c:6328
race at unknown origin, with read to 0xffff9e0575410740 of 1 bytes by task 209280 on cpu 15:
zone_page_state_snapshot root/klinux/./include/linux/vmstat.h:232 [inline] allow_direct_reclaim+0x150/0x290 root/klinux/mm/vmscan.c:6328 throttle_direct_reclaim+0x1c4/0x480 root/klinux/mm/vmscan.c:6402 try_to_free_pages+0x177/0x590 root/klinux/mm/vmscan.c:6466 __perform_reclaim root/klinux/mm/page_alloc.c:3966 [inline] __alloc_pages_direct_reclaim root/klinux/mm/page_alloc.c:3988 [inline] __alloc_pages_slowpath.constprop.0+0x3b3/0xdf0 root/klinux/mm/page_alloc.c:4395 __alloc_pages_noprof+0x28a/0x2d0 root/klinux/mm/page_alloc.c:4794 alloc_pages_mpol_noprof+0x181/0x470 root/klinux/mm/mempolicy.c:2270 alloc_pages_noprof+0x7a/0xe0 root/klinux/mm/mempolicy.c:2347 io_mem_alloc_compound root/klinux/io_uring/io_uring.c:2848 [inline] io_pages_map+0xb9/0x330 root/klinux/io_uring/io_uring.c:2892 io_allocate_scq_urings+0x231/0x300 root/klinux/io_uring/io_uring.c:3933 io_uring_create+0x29c/0x9c0 root/klinux/io_uring/io_uring.c:4091 io_uring_setup+0xb6/0x100 root/klinux/io_uring/io_uring.c:4203 __do_sys_io_uring_setup root/klinux/io_uring/io_uring.c:4230 [inline] __se_sys_io_uring_setup root/klinux/io_uring/io_uring.c:4224 [inline] __x64_sys_io_uring_setup+0x66/0xe0 root/klinux/io_uring/io_uring.c:4224 x64_sys_call+0x1816/0x1d90 root/klinux/./arch/x86/include/generated/asm/syscalls_64.h:426 do_syscall_x64 root/klinux/arch/x86/entry/common.c:51 [inline] do_syscall_64+0x70/0x2a0 root/klinux/arch/x86/entry/common.c:81 entry_SYSCALL_64_after_hwframe+0x76/0xe0
value changed: 0xdd -> 0xb6
Reported by Kernel Concurrency Sanitizer on:
CPU: 15 PID: 209280 Comm: syz-executor.13 Tainted: G S ------- ---- 6.6.89+ #1250 Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS rel-1.16.3-0-ga6ed6b701f0a-prebuilt.qemu.org 04/01/2014
Found by KCSAN testing on an older kernel; the same race still exists on mainline.
zone_page_state_snapshot() sums the per-cpu vm_stat_diff[] deltas of all online CPUs with a plain load, while the allocation and free fast paths update the local delta through this_cpu_cmpxchg() and friends. Those are inline asm, and page_alloc.o is excluded from KCSAN, so the writer is not instrumented and the report is of unknown origin.
The race is benign: the function already documents that the result cannot be exactly accurate, and every consumer is a reclaim or compaction heuristic that tolerates an estimate. Annotate the read with data_race(), following commit af1c31acc853 ("mm/vmstat: annotate data race for zone->free_area[order].nr_free").
No functional change intended.
--- include/linux/vmstat.h | 8 +++++++- 1 file changed, 7 insertions(+), 1 deletion(-)
include/linux/vmstat.h +7 -1
@@ -236,9 +236,15 @@ static inline unsigned long zone_page_state_snapshot(struct zone *zone,long x =atomic_long_read (&zone->vm_stat[item]);#ifdef CONFIG_SMP +struct per_cpu_zonestat __percpu *pzstats = zone->per_cpu_zonestats;int cpu; +for_each_online_cpu (cpu) - x +=per_cpu_ptr (zone->per_cpu_zonestats, cpu)->vm_stat_diff[item]; +/* +* Access to vm_stat_diff is lockless as the result is only +* an estimate. Use data_race to avoid KCSAN warning. +*/ + x +=data_race (per_cpu_ptr (pzstats, cpu)->vm_stat_diff[item]);if (x <0 ) x =0 ;
Signature · 1 line
2.43.0
Quoted 52 lines · Hui Zhu
From: Hui Zhu <[email protected]>
KCSAN reports a data-race on v6.6 which also exists on mainline:
BUG: KCSAN: data-race in zone_page_state_snapshot root/klinux/./include/linux/vmstat.h:232 [inline] BUG: KCSAN: data-race in allow_direct_reclaim+0x150/0x290 root/klinux/mm/vmscan.c:6328race at unknown origin, with read to
0xffff9e0575410740of 1 bytes by task 209280 on cpu 15:zone_page_state_snapshot root/klinux/./include/linux/vmstat.h:232 [inline] allow_direct_reclaim+0x150/0x290 root/klinux/mm/vmscan.c:6328 throttle_direct_reclaim+0x1c4/0x480 root/klinux/mm/vmscan.c:6402 try_to_free_pages+0x177/0x590 root/klinux/mm/vmscan.c:6466 __perform_reclaim root/klinux/mm/page_alloc.c:3966 [inline] __alloc_pages_direct_reclaim root/klinux/mm/page_alloc.c:3988 [inline] __alloc_pages_slowpath.constprop.0+0x3b3/0xdf0 root/klinux/mm/page_alloc.c:4395 __alloc_pages_noprof+0x28a/0x2d0 root/klinux/mm/page_alloc.c:4794 alloc_pages_mpol_noprof+0x181/0x470 root/klinux/mm/mempolicy.c:2270 alloc_pages_noprof+0x7a/0xe0 root/klinux/mm/mempolicy.c:2347 io_mem_alloc_compound root/klinux/io_uring/io_uring.c:2848 [inline] io_pages_map+0xb9/0x330 root/klinux/io_uring/io_uring.c:2892 io_allocate_scq_urings+0x231/0x300 root/klinux/io_uring/io_uring.c:3933 io_uring_create+0x29c/0x9c0 root/klinux/io_uring/io_uring.c:4091 io_uring_setup+0xb6/0x100 root/klinux/io_uring/io_uring.c:4203 __do_sys_io_uring_setup root/klinux/io_uring/io_uring.c:4230 [inline] __se_sys_io_uring_setup root/klinux/io_uring/io_uring.c:4224 [inline] __x64_sys_io_uring_setup+0x66/0xe0 root/klinux/io_uring/io_uring.c:4224 x64_sys_call+0x1816/0x1d90 root/klinux/./arch/x86/include/generated/asm/syscalls_64.h:426 do_syscall_x64 root/klinux/arch/x86/entry/common.c:51 [inline] do_syscall_64+0x70/0x2a0 root/klinux/arch/x86/entry/common.c:81 entry_SYSCALL_64_after_hwframe+0x76/0xe0value changed:
0xdd->0xb6Reported by Kernel Concurrency Sanitizer on:
CPU: 15 PID: 209280 Comm: syz-executor.13 Tainted: G S ------- ---- 6.6.89+ #1250 Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS rel-1.16.3-0-ga6ed6b701f0a-prebuilt.qemu.org 04/01/2014Found by KCSAN testing on an older kernel; the same race still exists on mainline.
zone_page_state_snapshot()sums the per-cpu vm_stat_diff[] deltas of all online CPUs with a plain load, while the allocation and free fast paths update the local delta throughthis_cpu_cmpxchg()and friends. Those are inline asm, and page_alloc.o is excluded from KCSAN, so the writer is not instrumented and the report is of unknown origin.The race is benign: the function already documents that the result cannot be exactly accurate, and every consumer is a reclaim or compaction heuristic that tolerates an estimate. Annotate the read with
data_race(), following commitaf1c31acc853("mm/vmstat: annotate data race for zone->free_area[order].nr_free").
Quoted 3 lines
No functional change intended.
Signed-off-by Hui Zhu <[email protected]>
Thanks!
Quoted 27 lines
--- include/linux/vmstat.h | 8 +++++++- 1 file changed, 7 insertions(+), 1 deletion(-)include/linux/vmstat.h +7
-1@@ -236,9 +236,15 @@ static inline unsigned long zone_page_state_snapshot(struct zone *zone,long x =atomic_long_read (&zone->vm_stat[item]);#ifdef CONFIG_SMP +struct per_cpu_zonestat __percpu *pzstats = zone->per_cpu_zonestats;int cpu; +for_each_online_cpu (cpu) - x +=per_cpu_ptr (zone->per_cpu_zonestats, cpu)->vm_stat_diff[item]; +/* +* Access to vm_stat_diff is lockless as the result is only +* an estimate. Use data_race to avoid KCSAN warning. +*/ + x +=data_race (per_cpu_ptr (pzstats, cpu)->vm_stat_diff[item]);if (x <0 ) x =0 ;Signature · 1 line
2.43.0
Signature · 2 lines
Michal Hocko SUSE Labs