Skip to contentSkip to conversation

md/raid5: fix lockless max_nr_stripes reads

PATCH Applied Chen Cheng 2 participants 3 messages Jun 24 – Jul 30 MD RAID

lore
Share & tools
lore

Chen Cheng

author
#0 lore raw

PATCH 1/1 md/raid5: fix lockless max_nr_stripes reads

From: Chen Cheng <[email protected]>

max_nr_stripes is updated under cache_size_mutex in the stripe cache grow/shrink paths, while is_inactive_blocked() and raid5_end_read_request() read it without that lock.

Use READ_ONCE() for those reads in lockless path to match the WRITE_ONCE() updates and avoid KCSAN data race reports.

A similar issue was previously fixed in commit-id:

	dfd2bf436709b2bccb78c2dda550dde93700efa7.
Fixes 0009fad033370 ("raid5 improve too many read errors msg by adding limits")
Fixes 3514da58be9c4 ("md/raid5: Make is_inactive_blocked() helper")

KCSAN report:
=================

BUG: KCSAN: data-race in grow_one_stripe / is_inactive_blocked

write (marked) to 0xffff8f01f0b5a268 of 4 bytes by task 12616 on cpu 9:

 grow_one_stripe+0x2d8/0x320
 raid5d+0xb57/0xba0
 md_thread+0x15a/0x2d0
 [..........]

read to 0xffff8f01f0b5a268 of 4 bytes by task 12670 on cpu 11:

 is_inactive_blocked+0x97/0xc0
 raid5_get_active_stripe+0x2fd/0xa70
 raid5_make_request+0x4aa/0x2940
 [..........]

value changed: 0x000003b9 -> 0x000003ba

---
 drivers/md/raid5.c | 8 ++++----
 1 file changed, 4 insertions(+), 4 deletions(-)
drivers/md/raid5.c +4 -4
index cb58b4353995..cacdf4211628 100644
@@ -799,11 +799,11 @@ static bool is_inactive_blocked(struct r5conf *conf, int hash)
 
 	if (!test_bit(R5_INACTIVE_BLOCKED, &conf->cache_state))
 		return true;
 
 	return (atomic_read(&conf->active_stripes) <
-		(conf->max_nr_stripes * 3 / 4));
+		(READ_ONCE(conf->max_nr_stripes) * 3 / 4));
 }
 
 struct stripe_head *raid5_get_active_stripe(struct r5conf *conf,
 		struct stripe_request_ctx *ctx, sector_t sector,
 		unsigned int flags)
@@ -2767,10 +2767,11 @@ static void raid5_end_read_request(struct bio * bi)
 		if (atomic_read(&rdev->read_errors))
 			atomic_set(&rdev->read_errors, 0);
 	} else {
 		int retry = 0;
 		int set_bad = 0;
+		int max_nr_stripes = READ_ONCE(conf->max_nr_stripes);
 
 		clear_bit(R5_UPTODATE, &sh->dev[i].flags);
 		if (!(bi->bi_status == BLK_STS_PROTECTION))
 			atomic_inc(&rdev->read_errors);
 		if (test_bit(R5_ReadRepl, &sh->dev[i].flags))
@@ -2792,17 +2793,16 @@ static void raid5_end_read_request(struct bio * bi)
 			pr_warn_ratelimited(
 				"md/raid:%s: read error NOT corrected!! (sector %llu on %pg).\n",
 				mdname(conf->mddev),
 				(unsigned long long)s,
 				rdev->bdev);
-		} else if (atomic_read(&rdev->read_errors)
-			 > conf->max_nr_stripes) {
+		} else if (atomic_read(&rdev->read_errors) > max_nr_stripes) {
 			if (!test_bit(Faulty, &rdev->flags)) {
 				pr_warn("md/raid:%s: %d read_errors > %d stripes\n",
 				    mdname(conf->mddev),
 				    atomic_read(&rdev->read_errors),
-				    conf->max_nr_stripes);
+				    max_nr_stripes);
 				pr_warn("md/raid:%s: Too many read errors, failing device %pg.\n",
 				    mdname(conf->mddev), rdev->bdev);
 			}
 		} else
 			retry = 1;
Signature · 1 line
2.54.0
maintainer Reviewed-by on v1 1/1
#1 lore raw
Quoted 32 lines · Chen Cheng

From: Chen Cheng<[email protected]>

max_nr_stripes is updated under cache_size_mutex in the stripe cache grow/shrink paths, while is_inactive_blocked() and raid5_end_read_request() read it without that lock.

Use READ_ONCE() for those reads in lockless path to match the WRITE_ONCE() updates and avoid KCSAN data race reports.

A similar issue was previously fixed in commit-id:

	dfd2bf436709b2bccb78c2dda550dde93700efa7.
Fixes 0009fad033370 ("raid5 improve too many read errors msg by adding limits")
Fixes 3514da58be9c4 ("md/raid5: Make is_inactive_blocked() helper")

KCSAN report:
=================

BUG: KCSAN: data-race in grow_one_stripe / is_inactive_blocked

write (marked) to 0xffff8f01f0b5a268 of 4 bytes by task 12616 on cpu 9:

  grow_one_stripe+0x2d8/0x320
  raid5d+0xb57/0xba0
  md_thread+0x15a/0x2d0
  [..........]

read to 0xffff8f01f0b5a268 of 4 bytes by task 12670 on cpu 11:

  is_inactive_blocked+0x97/0xc0
  raid5_get_active_stripe+0x2fd/0xa70
  raid5_make_request+0x4aa/0x2940
  [..........]
Quoted 6 lines

value changed: 0x000003b9 -> 0x000003ba

---
  drivers/md/raid5.c | 8 ++++----
  1 file changed, 4 insertions(+), 4 deletions(-)
Signature · 2 lines
Thanks,
Kuai
maintainer Applied
#2 lore raw
In reply to Chen Cheng
Quoted 32 lines · Chen Cheng

From: Chen Cheng<[email protected]>

max_nr_stripes is updated under cache_size_mutex in the stripe cache grow/shrink paths, while is_inactive_blocked() and raid5_end_read_request() read it without that lock.

Use READ_ONCE() for those reads in lockless path to match the WRITE_ONCE() updates and avoid KCSAN data race reports.

A similar issue was previously fixed in commit-id:

	dfd2bf436709b2bccb78c2dda550dde93700efa7.
Fixes 0009fad033370 ("raid5 improve too many read errors msg by adding limits")
Fixes 3514da58be9c4 ("md/raid5: Make is_inactive_blocked() helper")

KCSAN report:
=================

BUG: KCSAN: data-race in grow_one_stripe / is_inactive_blocked

write (marked) to 0xffff8f01f0b5a268 of 4 bytes by task 12616 on cpu 9:

  grow_one_stripe+0x2d8/0x320
  raid5d+0xb57/0xba0
  md_thread+0x15a/0x2d0
  [..........]

read to 0xffff8f01f0b5a268 of 4 bytes by task 12670 on cpu 11:

  is_inactive_blocked+0x97/0xc0
  raid5_get_active_stripe+0x2fd/0xa70
  raid5_make_request+0x4aa/0x2940
  [..........]
Quoted 6 lines

value changed: 0x000003b9 -> 0x000003ba

---
  drivers/md/raid5.c | 8 ++++----
  1 file changed, 4 insertions(+), 4 deletions(-)

Applied to md-7.3

Signature · 2 lines
Thanks,
Kuai