Skip to contentSkip to conversation

mm/vmstat: annotate data race in zone_page_state_snapshot()

PATCH In review Hui Zhu 2 participants 2 messages Oct 28 – Oct 29

Subsystems (2)
lore
Share & tools
lore

Hui Zhu

author
#0 lore raw

PATCH 1/1 mm/vmstat: annotate data race in zone_page_state_snapshot()

From: Hui Zhu <[email protected]>

KCSAN reports a data-race on v6.6 which also exists on mainline:

BUG: KCSAN: data-race in zone_page_state_snapshot root/klinux/./include/linux/vmstat.h:232 [inline]
BUG: KCSAN: data-race in allow_direct_reclaim+0x150/0x290 root/klinux/mm/vmscan.c:6328

race at unknown origin, with read to 0xffff9e0575410740 of 1 bytes by task 209280 on cpu 15:

zone_page_state_snapshot root/klinux/./include/linux/vmstat.h:232 [inline]
allow_direct_reclaim+0x150/0x290 root/klinux/mm/vmscan.c:6328
throttle_direct_reclaim+0x1c4/0x480 root/klinux/mm/vmscan.c:6402
try_to_free_pages+0x177/0x590 root/klinux/mm/vmscan.c:6466
__perform_reclaim root/klinux/mm/page_alloc.c:3966 [inline]
__alloc_pages_direct_reclaim root/klinux/mm/page_alloc.c:3988 [inline]
__alloc_pages_slowpath.constprop.0+0x3b3/0xdf0 root/klinux/mm/page_alloc.c:4395
__alloc_pages_noprof+0x28a/0x2d0 root/klinux/mm/page_alloc.c:4794
alloc_pages_mpol_noprof+0x181/0x470 root/klinux/mm/mempolicy.c:2270
alloc_pages_noprof+0x7a/0xe0 root/klinux/mm/mempolicy.c:2347
io_mem_alloc_compound root/klinux/io_uring/io_uring.c:2848 [inline]
io_pages_map+0xb9/0x330 root/klinux/io_uring/io_uring.c:2892
io_allocate_scq_urings+0x231/0x300 root/klinux/io_uring/io_uring.c:3933
io_uring_create+0x29c/0x9c0 root/klinux/io_uring/io_uring.c:4091
io_uring_setup+0xb6/0x100 root/klinux/io_uring/io_uring.c:4203
__do_sys_io_uring_setup root/klinux/io_uring/io_uring.c:4230 [inline]
__se_sys_io_uring_setup root/klinux/io_uring/io_uring.c:4224 [inline]
__x64_sys_io_uring_setup+0x66/0xe0 root/klinux/io_uring/io_uring.c:4224
x64_sys_call+0x1816/0x1d90 root/klinux/./arch/x86/include/generated/asm/syscalls_64.h:426
do_syscall_x64 root/klinux/arch/x86/entry/common.c:51 [inline]
do_syscall_64+0x70/0x2a0 root/klinux/arch/x86/entry/common.c:81
entry_SYSCALL_64_after_hwframe+0x76/0xe0

value changed: 0xdd -> 0xb6

Reported by Kernel Concurrency Sanitizer on:

CPU: 15 PID: 209280 Comm: syz-executor.13 Tainted: G S ------- ---- 6.6.89+ #1250
Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS rel-1.16.3-0-ga6ed6b701f0a-prebuilt.qemu.org 04/01/2014

Found by KCSAN testing on an older kernel; the same race still exists on mainline.

zone_page_state_snapshot() sums the per-cpu vm_stat_diff[] deltas of all online CPUs with a plain load, while the allocation and free fast paths update the local delta through this_cpu_cmpxchg() and friends. Those are inline asm, and page_alloc.o is excluded from KCSAN, so the writer is not instrumented and the report is of unknown origin.

The race is benign: the function already documents that the result cannot be exactly accurate, and every consumer is a reclaim or compaction heuristic that tolerates an estimate. Annotate the read with data_race(), following commit af1c31acc853 ("mm/vmstat: annotate data race for zone->free_area[order].nr_free").

No functional change intended.

---
 include/linux/vmstat.h | 8 +++++++-
 1 file changed, 7 insertions(+), 1 deletion(-)
include/linux/vmstat.h +7 -1
index 5b31d8e7ae405..9df3587a29685 100644
@@ -236,9 +236,15 @@ static inline unsigned long zone_page_state_snapshot(struct zone *zone,
 	long x = atomic_long_read(&zone->vm_stat[item]);
 
 #ifdef CONFIG_SMP
+	struct per_cpu_zonestat __percpu *pzstats = zone->per_cpu_zonestats;
 	int cpu;
+
 	for_each_online_cpu(cpu)
-		x += per_cpu_ptr(zone->per_cpu_zonestats, cpu)->vm_stat_diff[item];
+		/*
+		 * Access to vm_stat_diff is lockless as the result is only
+		 * an estimate. Use data_race to avoid KCSAN warning.
+		 */
+		x += data_race(per_cpu_ptr(pzstats, cpu)->vm_stat_diff[item]);
 
 	if (x < 0)
 		x = 0;
Signature · 1 line
2.43.0
reviewer Acked-by on v1 1/1
#1 lore raw
Quoted 52 lines · Hui Zhu

From: Hui Zhu <[email protected]>

KCSAN reports a data-race on v6.6 which also exists on mainline:

BUG: KCSAN: data-race in zone_page_state_snapshot root/klinux/./include/linux/vmstat.h:232 [inline]
BUG: KCSAN: data-race in allow_direct_reclaim+0x150/0x290 root/klinux/mm/vmscan.c:6328

race at unknown origin, with read to 0xffff9e0575410740 of 1 bytes by task 209280 on cpu 15:

zone_page_state_snapshot root/klinux/./include/linux/vmstat.h:232 [inline]
allow_direct_reclaim+0x150/0x290 root/klinux/mm/vmscan.c:6328
throttle_direct_reclaim+0x1c4/0x480 root/klinux/mm/vmscan.c:6402
try_to_free_pages+0x177/0x590 root/klinux/mm/vmscan.c:6466
__perform_reclaim root/klinux/mm/page_alloc.c:3966 [inline]
__alloc_pages_direct_reclaim root/klinux/mm/page_alloc.c:3988 [inline]
__alloc_pages_slowpath.constprop.0+0x3b3/0xdf0 root/klinux/mm/page_alloc.c:4395
__alloc_pages_noprof+0x28a/0x2d0 root/klinux/mm/page_alloc.c:4794
alloc_pages_mpol_noprof+0x181/0x470 root/klinux/mm/mempolicy.c:2270
alloc_pages_noprof+0x7a/0xe0 root/klinux/mm/mempolicy.c:2347
io_mem_alloc_compound root/klinux/io_uring/io_uring.c:2848 [inline]
io_pages_map+0xb9/0x330 root/klinux/io_uring/io_uring.c:2892
io_allocate_scq_urings+0x231/0x300 root/klinux/io_uring/io_uring.c:3933
io_uring_create+0x29c/0x9c0 root/klinux/io_uring/io_uring.c:4091
io_uring_setup+0xb6/0x100 root/klinux/io_uring/io_uring.c:4203
__do_sys_io_uring_setup root/klinux/io_uring/io_uring.c:4230 [inline]
__se_sys_io_uring_setup root/klinux/io_uring/io_uring.c:4224 [inline]
__x64_sys_io_uring_setup+0x66/0xe0 root/klinux/io_uring/io_uring.c:4224
x64_sys_call+0x1816/0x1d90 root/klinux/./arch/x86/include/generated/asm/syscalls_64.h:426
do_syscall_x64 root/klinux/arch/x86/entry/common.c:51 [inline]
do_syscall_64+0x70/0x2a0 root/klinux/arch/x86/entry/common.c:81
entry_SYSCALL_64_after_hwframe+0x76/0xe0

value changed: 0xdd -> 0xb6

Reported by Kernel Concurrency Sanitizer on:

CPU: 15 PID: 209280 Comm: syz-executor.13 Tainted: G S ------- ---- 6.6.89+ #1250
Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS rel-1.16.3-0-ga6ed6b701f0a-prebuilt.qemu.org 04/01/2014

Found by KCSAN testing on an older kernel; the same race still exists on mainline.

zone_page_state_snapshot() sums the per-cpu vm_stat_diff[] deltas of all online CPUs with a plain load, while the allocation and free fast paths update the local delta through this_cpu_cmpxchg() and friends. Those are inline asm, and page_alloc.o is excluded from KCSAN, so the writer is not instrumented and the report is of unknown origin.

The race is benign: the function already documents that the result cannot be exactly accurate, and every consumer is a reclaim or compaction heuristic that tolerates an estimate. Annotate the read with data_race(), following commit af1c31acc853 ("mm/vmstat: annotate data race for zone->free_area[order].nr_free").

Quoted 3 lines

No functional change intended.

Thanks!

Quoted 27 lines
---
 include/linux/vmstat.h | 8 +++++++-
 1 file changed, 7 insertions(+), 1 deletion(-)
include/linux/vmstat.h +7 -1
index 5b31d8e7ae405..9df3587a29685 100644
@@ -236,9 +236,15 @@ static inline unsigned long zone_page_state_snapshot(struct zone *zone,
 	long x = atomic_long_read(&zone->vm_stat[item]);
 
 #ifdef CONFIG_SMP
+	struct per_cpu_zonestat __percpu *pzstats = zone->per_cpu_zonestats;
 	int cpu;
+
 	for_each_online_cpu(cpu)
-		x += per_cpu_ptr(zone->per_cpu_zonestats, cpu)->vm_stat_diff[item];
+		/*
+		 * Access to vm_stat_diff is lockless as the result is only
+		 * an estimate. Use data_race to avoid KCSAN warning.
+		 */
+		x += data_race(per_cpu_ptr(pzstats, cpu)->vm_stat_diff[item]);
 
 	if (x < 0)
 		x = 0;
Signature · 1 line
2.43.0
Signature · 2 lines
Michal Hocko
SUSE Labs