mm: assert exclusive nid/zonenum bits at the page/folio access sites
Subsystems (3)
PATCH v3 1/1 mm: assert exclusive nid/zonenum bits at the page/folio access sites
From: Hui Zhu <[email protected]>
KCSAN reports a data race between page_to_nid()/folio_pgdat() reading page->flags and folio_trylock()/folio_lock() concurrently doing test_and_set_bit_lock(PG_locked, ...) on the same word, e.g.:
BUG: KCSAN: data-race in __lruvec_stat_mod_folio / shmem_get_folio_gfp
The node id and zone id occupy fixed bit-ranges of page->flags that are set once at page init and never modified afterwards, so they can never overlap with the low PG_locked/PG_waiters bits touched by the folio lock path.
ASSERT_EXCLUSIVE_BITS(mdf.f, ...) inside memdesc_nid()/memdesc_zonenum() checks a by-value copy of the flags word, not the actual shared page->flags/folio->flags being modified concurrently, so it doesn't reliably assert anything about the real race. Move the assertion to page_to_nid(), folio_nid(), page_zonenum() and folio_zonenum(), where flags is dereferenced directly from the page/folio.
---
Changelog:
v3:
According to the comments of Andrew and Sashiko, move ASSERT_EXCLUSIVE_BITS out of memdesc_nid()/memdesc_zonenum() into the page/folio call sites.
v2:
According to the comments of David, remove useless comments and use ASSERT_EXCLUSIVE_BITS() in memdesc_nid() instead of data_race() in page_to_nid().
include/linux/mm.h | 4 ++++ include/linux/mmzone.h | 3 ++- 2 files changed, 6 insertions(+), 1 deletion(-)
include/linux/mm.h +4 -0
@@ -2296,11 +2296,15 @@ static inline int memdesc_nid(memdesc_flags_t mdf)static inline int page_to_nid (const struct page *page) { +ASSERT_EXCLUSIVE_BITS (PF_POISONED_CHECK (page)->flags, +NODES_MASK <<NODES_PGSHIFT );return memdesc_nid (PF_POISONED_CHECK (page)->flags); }static inline int folio_nid (const struct folio *folio) { +ASSERT_EXCLUSIVE_BITS (folio->flags, +NODES_MASK <<NODES_PGSHIFT );return memdesc_nid (folio->flags); }
include/linux/mmzone.h +2 -1
@@ -1274,17 +1274,18 @@ static inline bool zone_is_empty(const struct zone *zone)static inline enum zone_type memdesc_zonenum (memdesc_flags_t flags) { -ASSERT_EXCLUSIVE_BITS (flags.f,ZONES_MASK <<ZONES_PGSHIFT );return (flags.f >>ZONES_PGSHIFT ) &ZONES_MASK ; }static inline enum zone_type page_zonenum (const struct page *page) { +ASSERT_EXCLUSIVE_BITS (page->flags,ZONES_MASK <<ZONES_PGSHIFT );return memdesc_zonenum (page->flags); }static inline enum zone_type folio_zonenum (const struct folio *folio) { +ASSERT_EXCLUSIVE_BITS (folio->flags,ZONES_MASK <<ZONES_PGSHIFT );return memdesc_zonenum (folio->flags); }
Signature · 1 line
2.43.0
Andrew Morton
maintainerQuoted 11 lines · On Thu, 25 Jun 2026 10:40:19 +0800 Hui Zhu <[email protected]> wrote:
KCSAN reports a data race between page_to_nid()/folio_pgdat() reading
page->flagsand folio_trylock()/folio_lock() concurrently doingtest_and_set_bit_lock(PG_locked, ...)on the same word, e.g.:BUG: KCSAN: data-race in __lruvec_stat_mod_folio / shmem_get_folio_gfpThe node id and zone id occupy fixed bit-ranges of
page->flagsthat are set once at page init and never modified afterwards, so they can never overlap with the low PG_locked/PG_waiters bits touched by the folio lock path.
Quoted 6 lines
ASSERT_EXCLUSIVE_BITS(mdf.f, ...)inside memdesc_nid()/memdesc_zonenum() checks a by-value copy of the flags word, not the actual shared page->flags/folio->flags being modified concurrently, so it doesn't reliably assert anything about the real race. Move the assertion topage_to_nid(),folio_nid(),page_zonenum()andfolio_zonenum(), where flags is dereferenced directly from the page/folio.
Thanks. Sashiko is worried about CONFIG_NUMA=n:
https://sashiko.dev/#/patchset/[email protected]
I'm a bit surprised that these functions even exist on NUMA=n. That we don't have simply
#else
static inline int folio_nid(const struct folio *folio)
{
return 0;
}#endif