Skip to contentSkip to conversation

mm/page_counter: fix various data races

PATCH In review Qian Cai 5 participants 10 messages Jan 29, 2020 – Feb 11, 2020

Subsystems (2)
lore
Share & tools
lore

Qian Cai

author
#0 lore raw

PATCH 1/1 mm/page_counter: fix various data races

The commit 3e32cb2e0a12 ("mm: memcontrol: lockless page counters") could had memcg->memsw->watermark been accessed concurrently as reported by KCSAN,

Reported by Kernel Concurrency Sanitizer on:

 BUG: KCSAN: data-race in page_counter_try_charge / page_counter_try_charge

read to 0xffff8fb18c4cd190 of 8 bytes by task 1081 on cpu 59:

  page_counter_try_charge+0x4d/0x150 mm/page_counter.c:138
  try_charge+0x131/0xd50 mm/memcontrol.c:2405
  __memcg_kmem_charge_memcg+0x58/0x140
  __memcg_kmem_charge+0xcc/0x280
  __alloc_pages_nodemask+0x1e1/0x450
  alloc_pages_current+0xa6/0x120
  pte_alloc_one+0x17/0xd0
  __pte_alloc+0x3a/0x1f0
  copy_p4d_range+0xc36/0x1990
  copy_page_range+0x21d/0x360
  dup_mmap+0x5f5/0x7a0
  dup_mm+0xa2/0x240
  copy_process+0x1b3f/0x3460
  _do_fork+0xaa/0xa20
  __x64_sys_clone+0x13b/0x170
  do_syscall_64+0x91/0xb47
  entry_SYSCALL_64_after_hwframe+0x49/0xbe

write to 0xffff8fb18c4cd190 of 8 bytes by task 1153 on cpu 120:

  page_counter_try_charge+0x5b/0x150 mm/page_counter.c:139
  try_charge+0x131/0xd50 mm/memcontrol.c:2405
  mem_cgroup_try_charge+0x159/0x460
  mem_cgroup_try_charge_delay+0x3d/0xa0
  wp_page_copy+0x14d/0x930
  do_wp_page+0x107/0x7b0
  __handle_mm_fault+0xce6/0xd40
  handle_mm_fault+0xfc/0x2f0
  do_page_fault+0x263/0x6f9
  page_fault+0x34/0x40

Since watermark could be compared or set to garbage due to load or store tearing which would change the code logic, fix it by adding a pair of READ_ONCE() and WRITE_ONCE() in those places.

Fixes 3e32cb2e0a12 ("mm: memcontrol: lockless page counters")
---
 mm/page_counter.c | 8 ++++----
 1 file changed, 4 insertions(+), 4 deletions(-)
mm/page_counter.c +4 -4
index de31470655f6..a17841150906 100644
@@ -82,8 +82,8 @@ void page_counter_charge(struct page_counter *counter, unsigned long nr_pages)
 		 * This is indeed racy, but we can live with some
 		 * inaccuracy in the watermark.
 		 */
-		if (new > c->watermark)
-			c->watermark = new;
+		if (new > READ_ONCE(c->watermark))
+			WRITE_ONCE(c->watermark, new);
 	}
 }
 
@@ -135,8 +135,8 @@ bool page_counter_try_charge(struct page_counter *counter,
 		 * Just like with failcnt, we can live with some
 		 * inaccuracy in the watermark.
 		 */
-		if (new > c->watermark)
-			c->watermark = new;
+		if (new > READ_ONCE(c->watermark))
+			WRITE_ONCE(c->watermark, new);
 	}
 	return true;
 
Signature · 1 line
2.21.0 (Apple Git-122.2)
#1 lore raw
Quoted 64 lines · Qian Cai

The commit 3e32cb2e0a12 ("mm: memcontrol: lockless page counters") could had memcg->memsw->watermark been accessed concurrently as reported by KCSAN,

Reported by Kernel Concurrency Sanitizer on:

 BUG: KCSAN: data-race in page_counter_try_charge / page_counter_try_charge

read to 0xffff8fb18c4cd190 of 8 bytes by task 1081 on cpu 59:

  page_counter_try_charge+0x4d/0x150 mm/page_counter.c:138
  try_charge+0x131/0xd50 mm/memcontrol.c:2405
  __memcg_kmem_charge_memcg+0x58/0x140
  __memcg_kmem_charge+0xcc/0x280
  __alloc_pages_nodemask+0x1e1/0x450
  alloc_pages_current+0xa6/0x120
  pte_alloc_one+0x17/0xd0
  __pte_alloc+0x3a/0x1f0
  copy_p4d_range+0xc36/0x1990
  copy_page_range+0x21d/0x360
  dup_mmap+0x5f5/0x7a0
  dup_mm+0xa2/0x240
  copy_process+0x1b3f/0x3460
  _do_fork+0xaa/0xa20
  __x64_sys_clone+0x13b/0x170
  do_syscall_64+0x91/0xb47
  entry_SYSCALL_64_after_hwframe+0x49/0xbe

write to 0xffff8fb18c4cd190 of 8 bytes by task 1153 on cpu 120:

  page_counter_try_charge+0x5b/0x150 mm/page_counter.c:139
  try_charge+0x131/0xd50 mm/memcontrol.c:2405
  mem_cgroup_try_charge+0x159/0x460
  mem_cgroup_try_charge_delay+0x3d/0xa0
  wp_page_copy+0x14d/0x930
  do_wp_page+0x107/0x7b0
  __handle_mm_fault+0xce6/0xd40
  handle_mm_fault+0xfc/0x2f0
  do_page_fault+0x263/0x6f9
  page_fault+0x34/0x40

Since watermark could be compared or set to garbage due to load or store tearing which would change the code logic, fix it by adding a pair of READ_ONCE() and WRITE_ONCE() in those places.

Fixes 3e32cb2e0a12 ("mm: memcontrol: lockless page counters")
---
 mm/page_counter.c | 8 ++++----
 1 file changed, 4 insertions(+), 4 deletions(-)
mm/page_counter.c +2 -2
index de31470655f6..a17841150906 100644
@@ -82,8 +82,8 @@ void page_counter_charge(struct page_counter *counter, unsigned long nr_pages)
 		 * This is indeed racy, but we can live with some
 		 * inaccuracy in the watermark.
 		 */
-		if (new > c->watermark)
-			c->watermark = new;
+		if (new > READ_ONCE(c->watermark))
+			WRITE_ONCE(c->watermark, new);
 	}
 }
 

mm/page_counter.c bool page_counter_try_charge(struct page_counter *counter, · line 139

Quoted 7 lines
mm/page_counter.c +2 -2
@@ -135,8 +135,8 @@ bool page_counter_try_charge(struct page_counter *counter,
 		 * Just like with failcnt, we can live with some
 		 * inaccuracy in the watermark.
 		 */
-		if (new > c->watermark)
-			c->watermark = new;
+		if (new > READ_ONCE(c->watermark))
+			WRITE_ONCE(c->watermark, new);

So, if this is racy, isn't it a problem that that "new" could suddenly be < c->watermark (concurrent writer). So you would use the "higher" watermark.

Signature · 3 lines
Thanks,

David / dhildenb
#2 lore raw
Quoted 64 lines · David Hildenbrand
Quoted 63 lines · Qian Cai

The commit 3e32cb2e0a12 ("mm: memcontrol: lockless page counters") could had memcg->memsw->watermark been accessed concurrently as reported by KCSAN,

Reported by Kernel Concurrency Sanitizer on:

 BUG: KCSAN: data-race in page_counter_try_charge / page_counter_try_charge

read to 0xffff8fb18c4cd190 of 8 bytes by task 1081 on cpu 59:

  page_counter_try_charge+0x4d/0x150 mm/page_counter.c:138
  try_charge+0x131/0xd50 mm/memcontrol.c:2405
  __memcg_kmem_charge_memcg+0x58/0x140
  __memcg_kmem_charge+0xcc/0x280
  __alloc_pages_nodemask+0x1e1/0x450
  alloc_pages_current+0xa6/0x120
  pte_alloc_one+0x17/0xd0
  __pte_alloc+0x3a/0x1f0
  copy_p4d_range+0xc36/0x1990
  copy_page_range+0x21d/0x360
  dup_mmap+0x5f5/0x7a0
  dup_mm+0xa2/0x240
  copy_process+0x1b3f/0x3460
  _do_fork+0xaa/0xa20
  __x64_sys_clone+0x13b/0x170
  do_syscall_64+0x91/0xb47
  entry_SYSCALL_64_after_hwframe+0x49/0xbe

write to 0xffff8fb18c4cd190 of 8 bytes by task 1153 on cpu 120:

  page_counter_try_charge+0x5b/0x150 mm/page_counter.c:139
  try_charge+0x131/0xd50 mm/memcontrol.c:2405
  mem_cgroup_try_charge+0x159/0x460
  mem_cgroup_try_charge_delay+0x3d/0xa0
  wp_page_copy+0x14d/0x930
  do_wp_page+0x107/0x7b0
  __handle_mm_fault+0xce6/0xd40
  handle_mm_fault+0xfc/0x2f0
  do_page_fault+0x263/0x6f9
  page_fault+0x34/0x40

Since watermark could be compared or set to garbage due to load or store tearing which would change the code logic, fix it by adding a pair of READ_ONCE() and WRITE_ONCE() in those places.

Fixes 3e32cb2e0a12 ("mm: memcontrol: lockless page counters")
---
 mm/page_counter.c | 8 ++++----
 1 file changed, 4 insertions(+), 4 deletions(-)
mm/page_counter.c +2 -2
index de31470655f6..a17841150906 100644
@@ -82,8 +82,8 @@ void page_counter_charge(struct page_counter *counter, unsigned long nr_pages)
 		 * This is indeed racy, but we can live with some
 		 * inaccuracy in the watermark.
 		 */
-		if (new > c->watermark)
-			c->watermark = new;
+		if (new > READ_ONCE(c->watermark))
+			WRITE_ONCE(c->watermark, new);
 	}
 }
 
Quoted 12 lines

mm/page_counter.c bool page_counter_try_charge(struct page_counter *counter, · line 139

Quoted 8 lines
mm/page_counter.c +2 -2
@@ -135,8 +135,8 @@ bool page_counter_try_charge(struct page_counter *counter,
 		 * Just like with failcnt, we can live with some
 		 * inaccuracy in the watermark.
 		 */
-		if (new > c->watermark)
-			c->watermark = new;
+		if (new > READ_ONCE(c->watermark))
+			WRITE_ONCE(c->watermark, new);

So, if this is racy, isn't it a problem that that "new" could suddenly be < c->watermark (concurrent writer). So you would use the "higher" watermark.

s/use/lose/ :)

Signature · 3 lines
Thanks,

David / dhildenb
maintainer Acked-by on v1 1/1
#3 lore raw
In reply to Qian Cai
Quoted 38 lines · Qian Cai

The commit 3e32cb2e0a12 ("mm: memcontrol: lockless page counters") could had memcg->memsw->watermark been accessed concurrently as reported by KCSAN,

Reported by Kernel Concurrency Sanitizer on:

 BUG: KCSAN: data-race in page_counter_try_charge / page_counter_try_charge

read to 0xffff8fb18c4cd190 of 8 bytes by task 1081 on cpu 59:

  page_counter_try_charge+0x4d/0x150 mm/page_counter.c:138
  try_charge+0x131/0xd50 mm/memcontrol.c:2405
  __memcg_kmem_charge_memcg+0x58/0x140
  __memcg_kmem_charge+0xcc/0x280
  __alloc_pages_nodemask+0x1e1/0x450
  alloc_pages_current+0xa6/0x120
  pte_alloc_one+0x17/0xd0
  __pte_alloc+0x3a/0x1f0
  copy_p4d_range+0xc36/0x1990
  copy_page_range+0x21d/0x360
  dup_mmap+0x5f5/0x7a0
  dup_mm+0xa2/0x240
  copy_process+0x1b3f/0x3460
  _do_fork+0xaa/0xa20
  __x64_sys_clone+0x13b/0x170
  do_syscall_64+0x91/0xb47
  entry_SYSCALL_64_after_hwframe+0x49/0xbe

write to 0xffff8fb18c4cd190 of 8 bytes by task 1153 on cpu 120:

  page_counter_try_charge+0x5b/0x150 mm/page_counter.c:139
  try_charge+0x131/0xd50 mm/memcontrol.c:2405
  mem_cgroup_try_charge+0x159/0x460
  mem_cgroup_try_charge_delay+0x3d/0xa0
  wp_page_copy+0x14d/0x930
  do_wp_page+0x107/0x7b0
  __handle_mm_fault+0xce6/0xd40
  handle_mm_fault+0xfc/0x2f0
  do_page_fault+0x263/0x6f9
  page_fault+0x34/0x40
Quoted 3 lines

Since watermark could be compared or set to garbage due to load or store tearing which would change the code logic, fix it by adding a pair of READ_ONCE() and WRITE_ONCE() in those places.

There is no actual problem and the report is false positive, right? While compilers are free to do all sorts of stuff do we have any actual proof that this particular path would ever be problematic.

I do not oppose to the patch. {READ,WRITE}_ONCE actually makes some sense to me, but the changelog should be more clear on how serious this is.

Quoted 2 lines
Fixes 3e32cb2e0a12 ("mm: memcontrol: lockless page counters")
Quoted 32 lines
---
 mm/page_counter.c | 8 ++++----
 1 file changed, 4 insertions(+), 4 deletions(-)
mm/page_counter.c +4 -4
index de31470655f6..a17841150906 100644
@@ -82,8 +82,8 @@ void page_counter_charge(struct page_counter *counter, unsigned long nr_pages)
 		 * This is indeed racy, but we can live with some
 		 * inaccuracy in the watermark.
 		 */
-		if (new > c->watermark)
-			c->watermark = new;
+		if (new > READ_ONCE(c->watermark))
+			WRITE_ONCE(c->watermark, new);
 	}
 }
 
@@ -135,8 +135,8 @@ bool page_counter_try_charge(struct page_counter *counter,
 		 * Just like with failcnt, we can live with some
 		 * inaccuracy in the watermark.
 		 */
-		if (new > c->watermark)
-			c->watermark = new;
+		if (new > READ_ONCE(c->watermark))
+			WRITE_ONCE(c->watermark, new);
 	}
 	return true;
 
Signature · 1 line
2.21.0 (Apple Git-122.2)
Signature · 2 lines
Michal Hocko
SUSE Labs
#4 lore raw
Quoted 4 lines · Michal Hocko
Quoted 2 lines
Fixes 3e32cb2e0a12 ("mm: memcontrol: lockless page counters")

Please include

Reported-by [email protected]

for https://syzkaller.appspot.com/bug?id=744097b8b91cecd8b035a6f746bb12e4efc7669f .

By the way, can READ_ONCE()/WRITE_ONCE() really solve this warning? The link above says read/write on the same location ( mm/page_counter.c:129 ). I don't know how READ_ONCE()/WRITE_ONCE() can solve the race.

Quoted 34 lines
Quoted 33 lines
---
 mm/page_counter.c | 8 ++++----
 1 file changed, 4 insertions(+), 4 deletions(-)
mm/page_counter.c +4 -4
index de31470655f6..a17841150906 100644
@@ -82,8 +82,8 @@ void page_counter_charge(struct page_counter *counter, unsigned long nr_pages)
 		 * This is indeed racy, but we can live with some
 		 * inaccuracy in the watermark.
 		 */
-		if (new > c->watermark)
-			c->watermark = new;
+		if (new > READ_ONCE(c->watermark))
+			WRITE_ONCE(c->watermark, new);
 	}
 }
 
@@ -135,8 +135,8 @@ bool page_counter_try_charge(struct page_counter *counter,
 		 * Just like with failcnt, we can live with some
 		 * inaccuracy in the watermark.
 		 */
-		if (new > c->watermark)
-			c->watermark = new;
+		if (new > READ_ONCE(c->watermark))
+			WRITE_ONCE(c->watermark, new);
 	}
 	return true;
 
Signature · 1 line
2.21.0 (Apple Git-122.2)
#5 lore raw
Quoted 9 lines · Tetsuo Handa
Quoted 4 lines · Michal Hocko
Quoted 2 lines
Fixes 3e32cb2e0a12 ("mm: memcontrol: lockless page counters")

Please include

Quoted 7 lines
Reported-by [email protected]

for https://syzkaller.appspot.com/bug?id=744097b8b91cecd8b035a6f746bb12e4efc7669f .

By the way, can READ_ONCE()/WRITE_ONCE() really solve this warning? The link above says read/write on the same location ( mm/page_counter.c:129 ). I don't know how READ_ONCE()/WRITE_ONCE() can solve the race.

It avoids the *data* race, with *_ONCE telling the compiler to not optimize the accesses in concurrency-unfriendly ways. Since *_ONCE is used, it conveys clear intent that the code here is meant to be concurrent, and KCSAN stops complaining (and assumes that the *logic* is correct).

The race itself is still there, but as per comment in the file, apparently fine and not a logic bug.

Quoted 35 lines
Quoted 34 lines
Quoted 33 lines
---
 mm/page_counter.c | 8 ++++----
 1 file changed, 4 insertions(+), 4 deletions(-)
mm/page_counter.c +4 -5
index de31470655f6..a17841150906 100644
@@ -82,8 +82,8 @@ void page_counter_charge(struct page_counter *counter, unsigned long nr_pages)
              * This is indeed racy, but we can live with some
              * inaccuracy in the watermark.
              */
-            if (new > c->watermark)
-                    c->watermark = new;
+            if (new > READ_ONCE(c->watermark))
+                    WRITE_ONCE(c->watermark, new);
     }
 }
 
@@ -135,8 +135,8 @@ bool page_counter_try_charge(struct page_counter *counter,
              * Just like with failcnt, we can live with some
              * inaccuracy in the watermark.
              */
-            if (new > c->watermark)
-                    c->watermark = new;
+            if (new > READ_ONCE(c->watermark))
+                    WRITE_ONCE(c->watermark, new);
     }
     return true;
 
--

2.21.0 (Apple Git-122.2)

#6 lore raw
Quoted 9 lines · Marco Elver
Quoted 3 lines

By the way, can READ_ONCE()/WRITE_ONCE() really solve this warning? The link above says read/write on the same location ( mm/page_counter.c:129 ). I don't know how READ_ONCE()/WRITE_ONCE() can solve the race.

It avoids the *data* race, with *_ONCE telling the compiler to not optimize the accesses in concurrency-unfriendly ways. Since *_ONCE is used, it conveys clear intent that the code here is meant to be concurrent, and KCSAN stops complaining (and assumes that the *logic* is correct).

I see. Unlike c->failcnt++ which involves read-modify-write, *_ONCE() can be used for simple read (like c->watermark) or simple write (like c->watermark = new) case.

author
#7 lore raw
In reply to Tetsuo Handa
Quoted 7 lines · Tetsuo Handa

By the way, can READ_ONCE()/WRITE_ONCE() really solve this warning? The link above says read/write on the same location ( mm/page_counter.c:129 ). I don't know how READ_ONCE()/WRITE_ONCE() can solve the race.

That looks like a different one where it complains about c->failcnt++. I’ll send a separate patch for that.

#8 lore raw
Quoted 10 lines · Qian Cai
Quoted 7 lines · Tetsuo Handa

By the way, can READ_ONCE()/WRITE_ONCE() really solve this warning? The link above says read/write on the same location ( mm/page_counter.c:129 ). I don't know how READ_ONCE()/WRITE_ONCE() can solve the race.

That looks like a different one where it complains about c->failcnt++. I’ll send a separate patch for that.

Ah, then this patch is meant for mm/page_counter.c:138 versus page_counter.c:139 race which was closed as invalid at https://syzkaller.appspot.com/bug?id=871391ec080746185a2dd437c54d75fcd1ef0ef8 .