md/raid10: fixes, atomic write handling, and error-path cleanup
v4 cover md/raid10: fixes, atomic write handling, and error-path cleanup
Hi,
This v4 of series contains a mix of bug fixes and cleanups for RAID10, along with a related atomic write fix for RAID1.
Changes in v4:
- Improve the commit message of patch 2.
- Add Reviewed-by tag from John Garry to patch 2.
- Link to v3: https://lore.kernel.org/linux-raid/[email protected]/
Changes in v3:
- Set chunk_sectors to BARRIER_UNIT_SECTOR_SIZE instead of setting atomic_write_hw_unit_max.
- Avoid enabling write-behind when the atomic write exceeds the
write-behind limit.
- Link to v2: https://lore.kernel.org/linux-raid/[email protected]/
Changes in v2:
- Expand the commit message to explain why the
allow_barrier()/wait_barrier() pair is no longer needed.
- Drop the early atomic write split check from raid1_write_request().
- Advertise the atomic write size limit via queue limits.
- Disable write-behind instead of failing atomic writes when the
BIO_MAX_VECS limit is encountered.
- Drop the early atomic write split check from raid10_write_request()
and rely on queue limits instead.
- Fix a compilation error (bi -> bio).
- Link to v1: https://lore.kernel.org/linux-raid/[email protected]/
Thanks,
Abd-alrhman,
Abd-Alrhman Masalkhi (7):
md/raid10: fix r10bio leak in raid10_write_request() error paths md/raid1: restrict atomic write limits and handle runtime constraints md/raid10: consistently fail atomic writes that require splitting md/raid10: remove unnecessary barrier around bio_submit_split_bioset() md/raid10: replace wait loop with wait_event_idle() md/raid10: simplify write request error handling md/raid10: simplify read request error handling
drivers/md/raid1.c | 25 ++++------ drivers/md/raid10.c | 118 +++++++++++++++++++++----------------------- 2 files changed, 65 insertions(+), 78 deletions(-)
Signature · 1 line
2.43.0
PATCH v4 1/7 md/raid10: fix r10bio leak in raid10_write_request() error paths
When raid10_write_request() fails because REQ_NOWAIT is set, the allocated r10_bio is not freed before returning, resulting in a memory leak. Free r10_bio before returning from the REQ_NOWAIT error paths.
---
Changes in v4:
- No changes.
- Link to v3: https://lore.kernel.org/linux-raid/[email protected]/
Changes in v3:
- No changes.
- Link to v2: https://lore.kernel.org/linux-raid/[email protected]/
Changes in v2:
- No changes.
- Link to v1: https://lore.kernel.org/linux-raid/[email protected]/
--- drivers/md/raid10.c | 2 ++ 1 file changed, 2 insertions(+)
drivers/md/raid10.c +2 -0
@@ -1365,6 +1365,7 @@ static bool raid10_write_request(struct mddev *mddev, struct bio *bio,/* Bail out if REQ_NOWAIT is set for the bio */ if (bio->bi_opf &REQ_NOWAIT ) {bio_wouldblock_error (bio); +free_r10bio (r10_bio);return false; }for (;;) {
@@ -1398,6 +1399,7 @@ static bool raid10_write_request(struct mddev *mddev, struct bio *bio,if (bio->bi_opf &REQ_NOWAIT ) {allow_barrier (conf);bio_wouldblock_error (bio); +free_r10bio (r10_bio);return false; } mddev_add_trace_msg(conf->mddev,
Signature · 1 line
2.43.0
[email protected]
AI reviewRe: [PATCH v4 1/7] md/raid10: fix r10bio leak in raid10_write_request() error paths
Thank you for your contribution! Sashiko AI review found 2 potential issue(s) to consider:
Thank you for your contribution! Sashiko AI review found 2 potential issue(s) to consider:
Pre-existing issues:
- [High] Allocation of r10_bio with GFP_NOIO can sleep, violating REQ_NOWAIT semantics.
- [High] Functions in the IO path (wait_blocked_dev() and wait_barrier()) sleep unconditionally, violating REQ_NOWAIT semantics. --
--- Patch [1]: [PATCH v4 1/7] md/raid10: fix r10bio leak in raid10_write_request() error paths --- commit d87114f7731cb53cc142eafbf2338e5b721f84ad
Author: Abd-Alrhman Masalkhi <[email protected]>
md/raid10: fix r10bio leak in raid10_write_request() error paths
When raid10_write_request() fails because REQ_NOWAIT is set, the allocated r10_bio is not freed before returning, resulting in a memory leak. Free r10_bio before returning from the REQ_NOWAIT error paths.
drivers/md/raid10.c static bool raid10_write_request(struct mddev *mddev, struct bio *bio, · line 1370
Quoted 11 lines
drivers/md/raid10.c +1
-0@@ -1365,6 +1365,7 @@ static bool raid10_write_request(struct mddev *mddev, struct bio *bio,/* Bail out if REQ_NOWAIT is set for the bio */ if (bio->bi_opf &REQ_NOWAIT ) {bio_wouldblock_error (bio); +free_r10bio (r10_bio);return false; }
[Severity: High]
This isn't a bug introduced by this patch, but while looking at REQ_NOWAIT handling, it appears the initial allocation of r10_bio in __make_request() and raid10_handle_discard() ignores the REQ_NOWAIT flag.
Both functions call:
r10_bio = mempool_alloc(&conf->r10bio_pool, GFP_NOIO);
Since GFP_NOIO includes __GFP_DIRECT_RECLAIM, this can sleep if the mempool is depleted.
Could this violate the strict non-blocking guarantees of REQ_NOWAIT IO and potentially starve async workers?
Quoted 1 line
drivers/md/raid10.c +0
-0for (;;) {
[ ... ]
drivers/md/raid10.c static bool raid10_write_request(struct mddev *mddev, struct bio *bio, · line 1404
Quoted 7 lines
drivers/md/raid10.c +1
-0@@ -1398,6 +1399,7 @@ static bool raid10_write_request(struct mddev *mddev, struct bio *bio,if (bio->bi_opf &REQ_NOWAIT ) {allow_barrier (conf);bio_wouldblock_error (bio); +free_r10bio (r10_bio);return false; }
[Severity: High]
This is a pre-existing issue, but further down in the IO path, several functions seem to sleep unconditionally even for REQ_NOWAIT bios.
For example, wait_blocked_dev() does:
md_wait_for_blocked_rdev(blocked_rdev, mddev);
wait_barrier(conf, false);
goto retry_wait;Also, in raid10_write_request() and raid10_read_request(), when a bio spans a chunk boundary and is split, the code unconditionally blocks:
allow_barrier(conf);
bio = bio_submit_split_bioset(bio, max_sectors, &conf->bio_split);
wait_barrier(conf, false);Could these calls to wait_barrier(conf, false) and md_wait_for_blocked_rdev() deadlock async submitters that rely on the non-blocking guarantee?
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=1
Re: [PATCH v4 1/7] md/raid10: fix r10bio leak in raid10_write_request() error paths
In reply to Abd-Alrhman MasalkhiQuoted 8 lines · Abd-Alrhman Masalkhi
When
raid10_write_request()fails becauseREQ_NOWAITis set, the allocatedr10_biois not freed before returning, resulting in a memory leak. Freer10_biobefore returning from theREQ_NOWAITerror paths.Fixes c9aa889b035f ("md: raid10 add nowait support")Reported-by sashiko-bot<[email protected]>Signed-off-by Abd-Alrhman Masalkhi<[email protected]>
Signature · 2 lines
Thanks, Kuai
PATCH v4 2/7 md/raid1: restrict atomic write limits and handle runtime constraints
In reply to Abd-Alrhman MasalkhiRestrict the RAID1 atomic write limits by setting chunk_sectors to BARRIER_UNIT_SECTOR_SIZE so that atomic writes never straddle a barrier unit.
A bio that passes block-layer validation may still become unserviceable within RAID1 due to bad blocks or write-behind constraints. In the former case, complete the bio with EIO. In the latter case, disable write-behind rather than failing the bio with EIO.
---
Changes in v4:
- Improve the commit message.
- Add Reviewed-by tag from John Garry.
- Link to v3: https://lore.kernel.org/linux-raid/[email protected]/
Changes in v3:
- Set chunk_sectors to BARRIER_UNIT_SECTOR_SIZE instead of setting atomic_write_hw_unit_max.
- Avoid enabling write-behind when the atomic write exceeds the
write-behind limit.
- Link to v2: https://lore.kernel.org/linux-raid/[email protected]/
Changes in v2:
- Drop the early atomic write split check from raid1_write_request().
- Advertise the atomic write size limit via queue limits.
- Disable write-behind instead of failing atomic writes when the
BIO_MAX_VECS limit is encountered.
- Link to v1: https://lore.kernel.org/linux-raid/[email protected]/
--- drivers/md/raid1.c | 25 ++++++++++--------------- 1 file changed, 10 insertions(+), 15 deletions(-)
drivers/md/raid1.c +10 -15
@@ -1522,6 +1522,7 @@ static bool raid1_write_request(struct mddev *mddev, struct bio *bio,int first_clone;bool write_behind = false;bool nowait = bio->bi_opf &REQ_NOWAIT ; +bool atomic = bio->bi_opf &REQ_ATOMIC ;bool is_discard =op_is_discard (bio->bi_opf);sector_t sector = bio->bi_iter.bi_sector;
@@ -1577,7 +1578,8 @@ static bool raid1_write_request(struct mddev *mddev, struct bio *bio,* write-mostly, which means we could allocate write behind * bio later. */ -if (!is_discard && rdev &&test_bit (WriteMostly, &rdev->flags)) +if (!is_discard && rdev &&test_bit (WriteMostly, &rdev->flags) && + (!atomic || max_sectors <=BIO_MAX_VECS * (PAGE_SIZE >>9 ))) write_behind = true; r1_bio->bios[i] =NULL ;
@@ -1603,20 +1605,6 @@ static bool raid1_write_request(struct mddev *mddev, struct bio *bio, }if (is_bad) {int good_sectors; - -/* -* We cannot atomically write this, so just -* error in that case. It could be possible to -* atomically write other mirrors, but the -* complexity of supporting that is not worth -* the benefit. -*/ -if (bio->bi_opf &REQ_ATOMIC ) { - bio->bi_status =BLK_STS_NOTSUPP ; -bio_endio (bio); - goto err_dec_pending; - } - good_sectors = first_bad - sector;if (good_sectors < max_sectors) max_sectors = good_sectors;
@@ -1636,7 +1624,13 @@ static bool raid1_write_request(struct mddev *mddev, struct bio *bio,if (write_behind && mddev->bitmap) max_sectors =min_t (int, max_sectors,BIO_MAX_VECS * (PAGE_SIZE >>9 )); +if (max_sectors <bio_sectors (bio)) { +if (atomic) { +bio_io_error (bio); + goto err_dec_pending; + } + bio =bio_submit_split_bioset (bio, max_sectors, &conf->bio_split);if (!bio)
@@ -3228,6 +3222,7 @@ static int raid1_set_limits(struct mddev *mddev)md_init_stacking_limits (&lim); lim.max_write_zeroes_sectors =0 ; lim.max_hw_wzeroes_unmap_sectors =0 ; + lim.chunk_sectors =BARRIER_UNIT_SECTOR_SIZE ; lim.logical_block_size = mddev->logical_block_size; lim.features |=BLK_FEAT_ATOMIC_WRITES ; lim.features |=BLK_FEAT_PCI_P2PDMA ;
Signature · 1 line
2.43.0
Re: [PATCH v4 2/7] md/raid1: restrict atomic write limits and handle runtime constraints
Quoted 32 lines · Abd-Alrhman Masalkhi
Restrict the RAID1 atomic write limits by setting
chunk_sectorstoBARRIER_UNIT_SECTOR_SIZEso that atomic writes never straddle a barrier unit.A bio that passes block-layer validation may still become unserviceable within RAID1 due to bad blocks or write-behind constraints. In the former case, complete the bio with EIO. In the latter case, disable write-behind rather than failing the bio with EIO.
Fixes f2a38abf5f1c ("md/raid1: Atomic write support")Fixes a4c55c902670 ("md/raid1: simplify raid1_write_request() error handling")Reviewed-by John Garry<[email protected]>Signed-off-by Abd-Alrhman Masalkhi<[email protected]>---
Changes in v4:
- Improve the commit message.
- Add Reviewed-by tag from John Garry.
- Link to v3:https://lore.kernel.org/linux-raid/[email protected]/Changes in v3:
- Setchunk_sectorstoBARRIER_UNIT_SECTOR_SIZEinstead of settingatomic_write_hw_unit_max.
- Avoid enabling write-behind when the atomic write exceeds the
write-behind limit.
- Link to v2:https://lore.kernel.org/linux-raid/[email protected]/Changes in v2:
- Drop the early atomic write split check fromraid1_write_request().
- Advertise the atomic write size limit via queue limits.
- Disable write-behind instead of failing atomic writes when the
BIO_MAX_VECSlimit is encountered.
- Link to v1:https://lore.kernel.org/linux-raid/[email protected]/
Quoted 3 lines
--- drivers/md/raid1.c | 25 ++++++++++--------------- 1 file changed, 10 insertions(+), 15 deletions(-)
Signature · 2 lines
Thanks, Kuai
PATCH v4 3/7 md/raid10: consistently fail atomic writes that require splitting
In reply to Abd-Alrhman MasalkhiRAID10 currently handles one badblock path explicitly by failing atomic writes with EIO. However, another badblock path can also reduce the writable range and force the bio through bio_submit_split_bioset(), which implicitly completes the bio with EINVAL.
Fix this by handling atomic writes in the common split check. If RAID10 determines that an atomic write would require splitting, complete the bio with EIO.
---
Changes in v4:
- No changes.
- Link to v3: https://lore.kernel.org/linux-raid/[email protected]/
Changes in v3:
- No changes.
- Link to v2: https://lore.kernel.org/linux-raid/[email protected]/
Changes in v2:
- Drop the early atomic write split check from raid10_write_request()
and rely on queue limits instead.
- Link to v1: https://lore.kernel.org/linux-raid/[email protected]/
--- drivers/md/raid10.c | 14 ++++---------- 1 file changed, 4 insertions(+), 10 deletions(-)
drivers/md/raid10.c +4 -10
@@ -1356,6 +1356,7 @@ static bool raid10_write_request(struct mddev *mddev, struct bio *bio,int i, k;sector_t sectors;int max_sectors; +bool atomic = bio->bi_opf &REQ_ATOMIC ; if ((mddev_is_clustered(mddev) && mddev->cluster_ops->area_resyncing(mddev, WRITE,
@@ -1464,16 +1465,6 @@ static bool raid10_write_request(struct mddev *mddev, struct bio *bio,if (is_bad) {int good_sectors; -/* -* We cannot atomically write this, so just -* error in that case. It could be possible to -* atomically write other mirrors, but the -* complexity of supporting that is not worth -* the benefit. -*/ -if (bio->bi_opf &REQ_ATOMIC ) - goto err_handle; - good_sectors = first_bad - dev_sector;if (good_sectors < max_sectors) max_sectors = good_sectors;
@@ -1493,6 +1484,9 @@ static bool raid10_write_request(struct mddev *mddev, struct bio *bio, r10_bio->sectors = max_sectors;if (r10_bio->sectors <bio_sectors (bio)) { +if (atomic) + goto err_handle; +allow_barrier (conf); bio =bio_submit_split_bioset (bio, r10_bio->sectors, &conf->bio_split);
Signature · 1 line
2.43.0
[email protected]
AI reviewRe: [PATCH v4 3/7] md/raid10: consistently fail atomic writes that require splitting
Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
Pre-existing issues:
- [High] Concurrent disk removal during a failed or split write causes a NULL pointer dereference in the error handling path. --
--- Patch [3]: [PATCH v4 3/7] md/raid10: consistently fail atomic writes that require splitting --- commit a682adb077534fe2522eacd5e431dab764a60d3f
Author: Abd-Alrhman Masalkhi <[email protected]>
md/raid10: consistently fail atomic writes that require splitting
This patch addresses atomic writes in RAID10 by checking if they require splitting. If an atomic write would be split due to badblocks or other limits, it now correctly fails the bio with EIO rather than allowing the implicit EINVAL from bio_submit_split_bioset().
Quoted 4 lines
drivers/md/raid10.c +0
-0
[ ... ]
drivers/md/raid10.c static bool raid10_write_request(struct mddev *mddev, struct bio *bio, · line 1488
Quoted 6 lines
drivers/md/raid10.c +2
-0@@ -1493,6 +1484,9 @@ static bool raid10_write_request(struct mddev *mddev, struct bio *bio, r10_bio->sectors = max_sectors;if (r10_bio->sectors <bio_sectors (bio)) { +if (atomic) + goto err_handle;
[Severity: High]
This is a pre-existing issue, but can this jump to the error handling path trigger a NULL pointer dereference if a disk removal happens concurrently?
If raid10_write_request() initiates an IO and skips a Faulty primary rdev, it uses the replacement (setting repl_bio and incrementing rrdev->nr_pending). If md_check_recovery() concurrently calls raid10_remove_disk() for the Faulty primary, it can promote the replacement to primary and set p->replacement to NULL.
When this atomic check fails, it jumps to the error unwinding loop which does:
struct md_rdev *rrdev = conf->mirrors[d].replacement;
...
if (r10_bio->devs[k].repl_bio) {
rdev_dec_pending(rrdev, mddev);
r10_bio->devs[k].repl_bio = NULL;
}Since replacement is now NULL, rrdev is NULL, and rdev_dec_pending() will dereference it.
Looking at raid10_end_write_request(), it implements a fallback to handle concurrent promotion by falling back to conf->mirrors[d].rdev when replacement is NULL.
Does the error unwinding loop in raid10_write_request() need a similar fallback?
Quoted 4 lines
drivers/md/raid10.c +1
-0+allow_barrier (conf); bio =bio_submit_split_bioset (bio, r10_bio->sectors, &conf->bio_split);
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=3
Re: [PATCH v4 3/7] md/raid10: consistently fail atomic writes that require splitting
In reply to Abd-Alrhman MasalkhiQuoted 20 lines · Abd-Alrhman Masalkhi
RAID10 currently handles one badblock path explicitly by failing atomic writes with EIO. However, another badblock path can also reduce the writable range and force the bio through
bio_submit_split_bioset(), which implicitly completes the bio with EINVAL.Fix this by handling atomic writes in the common split check. If RAID10 determines that an atomic write would require splitting, complete the bio with EIO.
Fixes a1d9b4fd42d9 ("md/raid10: Atomic write support")Signed-off-by Abd-Alrhman Masalkhi<[email protected]>---
Changes in v4:
- No changes.
- Link to v3:https://lore.kernel.org/linux-raid/[email protected]/Changes in v3:
- No changes.
- Link to v2:https://lore.kernel.org/linux-raid/[email protected]/
Quoted 7 lines
Changes in v2:
- Drop the early atomic write split check fromraid10_write_request()
and rely on queue limits instead.
- Link to v1:https://lore.kernel.org/linux-raid/[email protected]/--- drivers/md/raid10.c | 14 ++++---------- 1 file changed, 4 insertions(+), 10 deletions(-)
Signature · 2 lines
Thanks, Kuai
Re: [PATCH v4 3/7] md/raid10: consistently fail atomic writes that require splitting
In reply to Abd-Alrhman MasalkhiQuoted 3 lines · Abd-Alrhman Masalkhi
RAID10 currently handles one badblock path explicitly by failing atomic writes with EIO. However, another badblock path can also reduce the writable range and force the bio through
bio_submit_split_bioset(),
What is this other badblock path which you are referring to?
Quoted 9 lines
which implicitly completes the bio with EINVAL.
Fix this by handling atomic writes in the common split check. If RAID10 determines that an atomic write would require splitting, complete the bio with EIO.
Fixes a1d9b4fd42d9 ("md/raid10: Atomic write support")Signed-off-by Abd-Alrhman Masalkhi<[email protected]>---
Re: [PATCH v4 3/7] md/raid10: consistently fail atomic writes that require splitting
Quoted 7 lines · John Garry
Quoted 3 lines · Abd-Alrhman Masalkhi
RAID10 currently handles one badblock path explicitly by failing atomic writes with EIO. However, another badblock path can also reduce the writable range and force the bio through
bio_submit_split_bioset(),What is this other badblock path which you are referring to?
The other path is when the requested write overlaps a bad-block region from the beginning, but the intended write extends beyond the bad-block region. In that case, the current code silently reduces max_sectors. As a result, we end up calling bio_submit_split_bioset(), which completes the bio with -EINVAL rather than -EIO. This is the same issue that existed in raid1 and was fixed by patch 2.
Quoted 10 lines
Quoted 9 lines
which implicitly completes the bio with EINVAL.
Fix this by handling atomic writes in the common split check. If RAID10 determines that an atomic write would require splitting, complete the bio with EIO.
Fixes a1d9b4fd42d9 ("md/raid10: Atomic write support")Signed-off-by Abd-Alrhman Masalkhi<[email protected]>---
Signature · 2 lines
Best Regards, Abd-Alrhman
John Garry
Reviewed-byRe: [PATCH v4 3/7] md/raid10: consistently fail atomic writes that require splitting
Quoted 6 lines · Abd-Alrhman Masalkhi
Quoted 5 lines · John Garry
Quoted 3 lines · Abd-Alrhman Masalkhi
RAID10 currently handles one badblock path explicitly by failing atomic writes with EIO. However, another badblock path can also reduce the writable range and force the bio through
bio_submit_split_bioset(),
Quoted 8 lines
Quoted 2 lines
What is this other badblock path which you are referring to?
The other path is when the requested write overlaps a bad-block region from the beginning, but the intended write extends beyond the bad-block region. In that case, the current code silently reduces
max_sectors. As a result, we end up callingbio_submit_split_bioset(), which completes the bio with-EINVALrather than-EIO. This is the same issue that existed in raid1 and was fixed by patch 2.
OK, it would be nice to add a concise version of that info to the commit log.
Re: [PATCH v4 3/7] md/raid10: consistently fail atomic writes that require splitting
Hi John,
Quoted 16 lines · John Garry
Quoted 6 lines · Abd-Alrhman Masalkhi
Quoted 5 lines · John Garry
Quoted 3 lines · Abd-Alrhman Masalkhi
RAID10 currently handles one badblock path explicitly by failing atomic writes with EIO. However, another badblock path can also reduce the writable range and force the bio through
bio_submit_split_bioset(),Quoted 8 lines
Quoted 2 lines
What is this other badblock path which you are referring to?
The other path is when the requested write overlaps a bad-block region from the beginning, but the intended write extends beyond the bad-block region. In that case, the current code silently reduces
max_sectors. As a result, we end up callingbio_submit_split_bioset(), which completes the bio with-EINVALrather than-EIO. This is the same issue that existed in raid1 and was fixed by patch 2.
Quoted 3 lines
OK, it would be nice to add a concise version of that info to the commit log.
Thanks. I'll update the commit log accordingly and send v5.
Quoted 1 line
Reviewed-by John Garry <[email protected]>
Signature · 2 lines
Best Regards, Abd-Alrhman
PATCH v4 4/7 md/raid10: remove unnecessary barrier around bio_submit_split_bioset()
In reply to Abd-Alrhman Masalkhiraid10_write_request() drops the barrier before calling bio_submit_split_bioset() and reacquires it afterwards. This is no longer necessary because the split bio cannot re-enter raid10_write_request() while the barrier is held.
The allow_barrier()/wait_barrier() pair was introduced by commit e820d55cb99d ("md: fix raid10 hang issue caused by barrier") when submit_flushes() called md_handle_request() directly, allowing re-entry into raid10_write_request(). Since v5.2, submit_flushes() has instead gone through submit_bio(), eliminating that recursion. submit_flushes() was later removed entirely by commit b75197e86e6d ("md: Remove flush handling").
Currently, raid10_write_request() is only entered from the bio submission path, so the split bio submitted by bio_submit_split_bioset() cannot recurse back into wait_barrier().
Remove the redundant allow_barrier()/wait_barrier() pair around bio_submit_split_bioset().
---
Changes in v4:
- No changes.
- Link to v3: https://lore.kernel.org/linux-raid/[email protected]/
Changes in v3:
- No changes.
- Link to v2: https://lore.kernel.org/linux-raid/[email protected]/
Changes in v2:
- Expand the commit message to explain why the
allow_barrier()/wait_barrier() pair is no longer needed.
- Link to v1: https://lore.kernel.org/linux-raid/[email protected]/
--- drivers/md/raid10.c | 2 -- 1 file changed, 2 deletions(-)
drivers/md/raid10.c +0 -2
@@ -1487,10 +1487,8 @@ static bool raid10_write_request(struct mddev *mddev, struct bio *bio,if (atomic) goto err_handle; -allow_barrier (conf); bio =bio_submit_split_bioset (bio, r10_bio->sectors, &conf->bio_split); -wait_barrier (conf, false);if (!bio) {set_bit (R10BIO_Returned, &r10_bio->state); goto err_handle;
Signature · 1 line
2.43.0
Re: [PATCH v4 4/7] md/raid10: remove unnecessary barrier around bio_submit_split_bioset()
Quoted 30 lines · Abd-Alrhman Masalkhi
raid10_write_request()drops the barrier before callingbio_submit_split_bioset()and reacquires it afterwards. This is no longer necessary because the split bio cannot re-enterraid10_write_request()while the barrier is held.The allow_barrier()/wait_barrier() pair was introduced by commit
e820d55cb99d("md: fix raid10 hang issue caused by barrier") whensubmit_flushes()calledmd_handle_request()directly, allowing re-entry intoraid10_write_request(). Since v5.2,submit_flushes()has instead gone throughsubmit_bio(), eliminating that recursion.submit_flushes()was later removed entirely by commitb75197e86e6d("md: Remove flush handling").Currently,
raid10_write_request()is only entered from the bio submission path, so the split bio submitted bybio_submit_split_bioset()cannot recurse back intowait_barrier().Remove the redundant allow_barrier()/wait_barrier() pair around
bio_submit_split_bioset().Signed-off-by Abd-Alrhman Masalkhi<[email protected]>---
Changes in v4:
- No changes.
- Link to v3:https://lore.kernel.org/linux-raid/[email protected]/Changes in v3:
- No changes.
- Link to v2:https://lore.kernel.org/linux-raid/[email protected]/
Quoted 7 lines
Changes in v2:
- Expand the commit message to explain why the
allow_barrier()/wait_barrier() pair is no longer needed.
- Link to v1:https://lore.kernel.org/linux-raid/[email protected]/--- drivers/md/raid10.c | 2 -- 1 file changed, 2 deletions(-)
Signature · 2 lines
Thanks, Kuai
PATCH v4 5/7 md/raid10: replace wait loop with wait_event_idle()
In reply to Abd-Alrhman MasalkhiThe wait loop is equivalent to wait_event_idle() and can be simplified by usaing it for improving readability.
---
Changes in v4:
- No changes.
- Link to v3: https://lore.kernel.org/linux-raid/[email protected]/
Changes in v3:
- No changes.
- Link to v2: https://lore.kernel.org/linux-raid/[email protected]/
Changes in v2:
- No changes.
- Link to v1: https://lore.kernel.org/linux-raid/[email protected]/
--- drivers/md/raid10.c | 15 +++++---------- 1 file changed, 5 insertions(+), 10 deletions(-)
drivers/md/raid10.c +5 -10
@@ -1362,22 +1362,17 @@ static bool raid10_write_request(struct mddev *mddev, struct bio *bio, mddev->cluster_ops->area_resyncing(mddev, WRITE, bio->bi_iter.bi_sector, bio_end_sector(bio)))) { -DEFINE_WAIT (w);/* Bail out if REQ_NOWAIT is set for the bio */ if (bio->bi_opf &REQ_NOWAIT ) {bio_wouldblock_error (bio);free_r10bio (r10_bio);return false; } -for (;;) { -prepare_to_wait (&conf->wait_barrier, - &w,TASK_IDLE ); -if (!mddev->cluster_ops->area_resyncing (mddev,WRITE , - bio->bi_iter.bi_sector,bio_end_sector (bio))) -break ; -schedule (); - } -finish_wait (&conf->wait_barrier, &w); + +wait_event_idle (conf->wait_barrier, + !mddev->cluster_ops->area_resyncing (mddev,WRITE , + bio->bi_iter.bi_sector, +bio_end_sector (bio))); } sectors = r10_bio->sectors;
Signature · 1 line
2.43.0
Re: [PATCH v4 5/7] md/raid10: replace wait loop with wait_event_idle()
Quoted 13 lines · Abd-Alrhman Masalkhi
The wait loop is equivalent to
wait_event_idle()and can be simplified by usaing it for improving readability.Signed-off-by Abd-Alrhman Masalkhi<[email protected]>---
Changes in v4:
- No changes.
- Link to v3:https://lore.kernel.org/linux-raid/[email protected]/Changes in v3:
- No changes.
- Link to v2:https://lore.kernel.org/linux-raid/[email protected]/
Quoted 6 lines
Changes in v2:
- No changes.
- Link to v1:https://lore.kernel.org/linux-raid/[email protected]/--- drivers/md/raid10.c | 15 +++++---------- 1 file changed, 5 insertions(+), 10 deletions(-)
Signature · 2 lines
Thanks, Kuai
PATCH v4 6/7 md/raid10: simplify write request error handling
In reply to Abd-Alrhman Masalkhiraid10_write_request() currently handles bio completion, barrier handling, and r10_bio lifetime management in several different error paths. This results in duplicated cleanup logic and increases the risk of introducing bugs in future modifications.
Move bio_wouldblock_error() handling to the callers of regular_request_wait(), consolidate the write error paths, and free r10_bio from a single location in __make_request() when raid10_write_request() fails.
It remove redundant local copies of r10_bio->sectors and use a single max_sectors variable throughout the function.
---
Changes in v4:
- No changes.
- Link to v3: https://lore.kernel.org/linux-raid/[email protected]/
Changes in v3:
- No changes.
- Link to v2: https://lore.kernel.org/linux-raid/[email protected]/
Changes in v2:
- No changes.
- Link to v1: https://lore.kernel.org/linux-raid/[email protected]/
--- drivers/md/raid10.c | 58 ++++++++++++++++++++++----------------------- 1 file changed, 28 insertions(+), 30 deletions(-)
drivers/md/raid10.c +28 -30
@@ -1123,18 +1123,16 @@ static bool regular_request_wait(struct mddev *mddev, struct r10conf *conf, struct bio *bio, sector_t sectors) {/* Bail out if REQ_NOWAIT is set for the bio */ -if (!wait_barrier (conf, bio->bi_opf &REQ_NOWAIT )) { -bio_wouldblock_error (bio); +if (!wait_barrier (conf, bio->bi_opf &REQ_NOWAIT ))return false; - } +while (test_bit (MD_RECOVERY_RESHAPE , &mddev->recovery) && bio->bi_iter.bi_sector < conf->reshape_progress && bio->bi_iter.bi_sector + sectors > conf->reshape_progress) {allow_barrier (conf); -if (bio->bi_opf &REQ_NOWAIT ) { -bio_wouldblock_error (bio); +if (bio->bi_opf &REQ_NOWAIT )return false; - } +mddev_add_trace_msg (conf->mddev,"raid10 wait reshape" ); wait_event(conf->wait_barrier, conf->reshape_progress <= bio->bi_iter.bi_sector ||
@@ -1192,6 +1190,7 @@ static void raid10_read_request(struct mddev *mddev, struct bio *bio, }if (!regular_request_wait (mddev, conf, bio, r10_bio->sectors)) { +bio_wouldblock_error (bio);free_r10bio (r10_bio);return ; }
@@ -1354,8 +1353,8 @@ static bool raid10_write_request(struct mddev *mddev, struct bio *bio, {struct r10conf *conf = mddev->private;int i, k; -sector_t sectors; -int max_sectors; +int max_sectors = r10_bio->sectors; +bool nowait = bio->bi_opf &REQ_NOWAIT ;bool atomic = bio->bi_opf &REQ_ATOMIC ; if ((mddev_is_clustered(mddev) &&
@@ -1363,9 +1362,8 @@ static bool raid10_write_request(struct mddev *mddev, struct bio *bio, bio->bi_iter.bi_sector, bio_end_sector(bio)))) {/* Bail out if REQ_NOWAIT is set for the bio */ -if (bio->bi_opf &REQ_NOWAIT ) { +if (nowait) {bio_wouldblock_error (bio); -free_r10bio (r10_bio);return false; }
@@ -1375,28 +1373,25 @@ static bool raid10_write_request(struct mddev *mddev, struct bio *bio, bio_end_sector(bio))); } - sectors = r10_bio->sectors; -if (!regular_request_wait (mddev, conf, bio, sectors)) { -free_r10bio (r10_bio); +if (!regular_request_wait (mddev, conf, bio, max_sectors)) { +bio_wouldblock_error (bio);return false; }if (test_bit (MD_RECOVERY_RESHAPE , &mddev->recovery) && (mddev->reshape_backwards ? (bio->bi_iter.bi_sector < conf->reshape_safe && - bio->bi_iter.bi_sector + sectors > conf->reshape_progress) - : (bio->bi_iter.bi_sector + sectors > conf->reshape_safe && + bio->bi_iter.bi_sector + max_sectors > conf->reshape_progress) + : (bio->bi_iter.bi_sector + max_sectors > conf->reshape_safe && bio->bi_iter.bi_sector < conf->reshape_progress))) {/* Need to update reshape_position in metadata */ mddev->reshape_position = conf->reshape_progress;set_mask_bits (&mddev->sb_flags,0 ,BIT (MD_SB_CHANGE_DEVS ) |BIT (MD_SB_CHANGE_PENDING ));md_wakeup_thread (mddev->thread); -if (bio->bi_opf &REQ_NOWAIT ) { -allow_barrier (conf); +if (nowait) {bio_wouldblock_error (bio); -free_r10bio (r10_bio); -return false; + goto err_allow_barrier; }mddev_add_trace_msg (conf->mddev,"raid10 wait reshape metadata" );
@@ -1421,8 +1416,6 @@ static bool raid10_write_request(struct mddev *mddev, struct bio *bio,wait_blocked_dev (mddev, r10_bio); - max_sectors = r10_bio->sectors; -for (i =0 ; i < conf->copies; i++) {int d = r10_bio->devs[i].devnum;struct md_rdev *rdev, *rrdev;
@@ -1479,15 +1472,15 @@ static bool raid10_write_request(struct mddev *mddev, struct bio *bio, r10_bio->sectors = max_sectors;if (r10_bio->sectors <bio_sectors (bio)) { -if (atomic) - goto err_handle; +if (atomic) { +bio_io_error (bio); + goto err_dec_pending; + } bio =bio_submit_split_bioset (bio, r10_bio->sectors, &conf->bio_split); -if (!bio) { -set_bit (R10BIO_Returned, &r10_bio->state); - goto err_handle; - } +if (!bio) + goto err_dec_pending; r10_bio->master_bio = bio; }
@@ -1505,7 +1498,7 @@ static bool raid10_write_request(struct mddev *mddev, struct bio *bio,one_write_done (r10_bio);return true; -err_handle: +err_dec_pending:for (k =0 ; k < i; k++) {int d = r10_bio->devs[k].devnum;struct md_rdev *rdev = conf->mirrors[d].rdev;
@@ -1521,7 +1514,9 @@ static bool raid10_write_request(struct mddev *mddev, struct bio *bio, } } -raid_end_bio_io (r10_bio); +err_allow_barrier: +allow_barrier (conf); +return false; }
@@ -1546,8 +1541,11 @@ static bool __make_request(struct mddev *mddev, struct bio *bio, int sectors) ret = true;if (bio_data_dir (bio) ==READ )raid10_read_request (mddev, bio, r10_bio); -else +else { ret =raid10_write_request (mddev, bio, r10_bio); +if (!ret) +free_r10bio (r10_bio); + }return ret; }
Signature · 1 line
2.43.0
PATCH v4 7/7 md/raid10: simplify read request error handling
In reply to Abd-Alrhman Masalkhiraid10_read_request() currently handles bio completion, barrier handling, and r10_bio lifetime management in several different error paths. This results in duplicated cleanup logic and increases the risk of introducing bugs in future modifications.
Make raid10_read_request() return a status to its callers, consolidate the read error paths, and free r10_bio from a single location in the callers. Since the callers allocate r10_bio, they should also be responsible for freeing it when the request fails.
This makes the read path follow the same ownership model as the write path and simplifies the error handling flow.
---
Changes in v4:
- No changes.
- Link to v3: https://lore.kernel.org/linux-raid/[email protected]/
Changes in v3:
- No changes.
- Link to v2: https://lore.kernel.org/linux-raid/[email protected]/
Changes in v2:
- Fix a compilation error (bi -> bio).
- Link to v1: https://lore.kernel.org/linux-raid/[email protected]/
--- drivers/md/raid10.c | 45 +++++++++++++++++++++++++-------------------- 1 file changed, 25 insertions(+), 20 deletions(-)
drivers/md/raid10.c +25 -20
@@ -1143,7 +1143,7 @@ static bool regular_request_wait(struct mddev *mddev, struct r10conf *conf,return true; } -static void raid10_read_request (struct mddev *mddev,struct bio *bio, +static bool raid10_read_request (struct mddev *mddev,struct bio *bio,struct r10bio *r10_bio) {struct r10conf *conf = mddev->private;
@@ -1191,8 +1191,7 @@ static void raid10_read_request(struct mddev *mddev, struct bio *bio,if (!regular_request_wait (mddev, conf, bio, r10_bio->sectors)) {bio_wouldblock_error (bio); -free_r10bio (r10_bio); -return ; +return false; } rdev =read_balance (conf, r10_bio, &max_sectors);
@@ -1202,8 +1201,8 @@ static void raid10_read_request(struct mddev *mddev, struct bio *bio,mdname (mddev), b, (unsigned long long)r10_bio->sector); } -raid_end_bio_io (r10_bio); -return ; +bio_io_error (bio); + goto err_allow_barrier; }if (err_rdev) pr_err_ratelimited("md/raid10:%s: %pg: redirecting sector %llu to another mirror\n",
@@ -1215,10 +1214,8 @@ static void raid10_read_request(struct mddev *mddev, struct bio *bio, bio =bio_submit_split_bioset (bio, max_sectors, &conf->bio_split);wait_barrier (conf, false); -if (!bio) { -set_bit (R10BIO_Returned, &r10_bio->state); - goto err_handle; - } +if (!bio) + goto err_dec_pending; r10_bio->master_bio = bio; r10_bio->sectors = max_sectors;
@@ -1244,10 +1241,16 @@ static void raid10_read_request(struct mddev *mddev, struct bio *bio, read_bio->bi_private = r10_bio;mddev_trace_remap (mddev, read_bio, r10_bio->sector);submit_bio_noacct (read_bio); -return ; -err_handle: + +return true; + +err_dec_pending:atomic_dec (&rdev->nr_pending); -raid_end_bio_io (r10_bio); + +err_allow_barrier: +allow_barrier (conf); + +return false; } static void raid10_write_one_disk(struct mddev *mddev, struct r10bio *r10_bio,
@@ -1538,14 +1541,13 @@ static bool __make_request(struct mddev *mddev, struct bio *bio, int sectors)memset (r10_bio->devs,0 ,sizeof (r10_bio->devs[0 ]) * conf->geo.raid_disks); - ret = true;if (bio_data_dir (bio) ==READ ) -raid10_read_request (mddev, bio, r10_bio); -else { + ret =raid10_read_request (mddev, bio, r10_bio); +else ret =raid10_write_request (mddev, bio, r10_bio); -if (!ret) -free_r10bio (r10_bio); - } + +if (!ret) +free_r10bio (r10_bio);return ret; }
@@ -1875,6 +1877,7 @@ static bool raid10_make_request(struct mddev *mddev, struct bio *bio)sector_t chunk_mask = (conf->geo.chunk_mask & conf->prev.chunk_mask);int chunk_sects = chunk_mask +1 ;int sectors =bio_sectors (bio); +bool write =bio_data_dir (bio) ==WRITE ;if (unlikely (bio->bi_opf &REQ_PREFLUSH ) &&md_flush_request (mddev, bio))
@@ -1898,7 +1901,7 @@ static bool raid10_make_request(struct mddev *mddev, struct bio *bio) sectors = chunk_sects - (bio->bi_iter.bi_sector & (chunk_sects -1 )); -if (!__make_request (mddev, bio, sectors)) +if (!__make_request (mddev, bio, sectors) && write)md_write_end (mddev);/* In case raid10d snuck in to freeze_array */
@@ -2866,7 +2869,9 @@ static void handle_read_error(struct mddev *mddev, struct r10bio *r10_bio)rdev_dec_pending (rdev, mddev); r10_bio->state =0 ; -raid10_read_request (mddev, r10_bio->master_bio, r10_bio); +if (!raid10_read_request (mddev, r10_bio->master_bio, r10_bio)) +free_r10bio (r10_bio); +/* * allow_barrier after re-submit to ensure no sync io * can be issued while regular io pending.
Signature · 1 line
2.43.0
yu kuai
maintainerRe: [PATCH v4 7/7] md/raid10: simplify read request error handling
Hi,
Quoted 23 lines · Abd-Alrhman Masalkhi
raid10_read_request()currently handles bio completion, barrier handling, andr10_biolifetime management in several different error paths. This results in duplicated cleanup logic and increases the risk of introducing bugs in future modifications.Make
raid10_read_request()return a status to its callers, consolidate the read error paths, and freer10_biofrom a single location in the callers. Since the callers allocater10_bio, they should also be responsible for freeing it when the request fails.This makes the read path follow the same ownership model as the write path and simplifies the error handling flow.
Signed-off-by Abd-Alrhman Masalkhi <[email protected]>---
Changes in v4:
- No changes.
- Link to v3: https://lore.kernel.org/linux-raid/[email protected]/Changes in v3:
- No changes.
- Link to v2: https://lore.kernel.org/linux-raid/[email protected]/
Quoted 6 lines
Changes in v2:
- Fix a compilation error (bi -> bio).
- Link to v1: https://lore.kernel.org/linux-raid/[email protected]/--- drivers/md/raid10.c | 45 +++++++++++++++++++++++++-------------------- 1 file changed, 25 insertions(+), 20 deletions(-)
I don't think patch 6 and 7 are actually simplifications, and 7 in particular introduces a fragile dependency I'd rather not take as-is. The diff lines also said this and I'd rather not take as-is.
Quoted 116 lines
drivers/md/raid10.c +25
-20@@ -1143,7 +1143,7 @@ static bool regular_request_wait(struct mddev *mddev, struct r10conf *conf,return true; } -static void raid10_read_request (struct mddev *mddev,struct bio *bio, +static bool raid10_read_request (struct mddev *mddev,struct bio *bio,struct r10bio *r10_bio) {struct r10conf *conf = mddev->private;@@ -1191,8 +1191,7 @@ static void raid10_read_request(struct mddev *mddev, struct bio *bio,if (!regular_request_wait (mddev, conf, bio, r10_bio->sectors)) {bio_wouldblock_error (bio); -free_r10bio (r10_bio); -return ; +return false; } rdev =read_balance (conf, r10_bio, &max_sectors);@@ -1202,8 +1201,8 @@ static void raid10_read_request(struct mddev *mddev, struct bio *bio,mdname (mddev), b, (unsigned long long)r10_bio->sector); } -raid_end_bio_io (r10_bio); -return ; +bio_io_error (bio); + goto err_allow_barrier; }if (err_rdev) pr_err_ratelimited("md/raid10:%s: %pg: redirecting sector %llu to another mirror\n",@@ -1215,10 +1214,8 @@ static void raid10_read_request(struct mddev *mddev, struct bio *bio, bio =bio_submit_split_bioset (bio, max_sectors, &conf->bio_split);wait_barrier (conf, false); -if (!bio) { -set_bit (R10BIO_Returned, &r10_bio->state); - goto err_handle; - } +if (!bio) + goto err_dec_pending; r10_bio->master_bio = bio; r10_bio->sectors = max_sectors;@@ -1244,10 +1241,16 @@ static void raid10_read_request(struct mddev *mddev, struct bio *bio, read_bio->bi_private = r10_bio;mddev_trace_remap (mddev, read_bio, r10_bio->sector);submit_bio_noacct (read_bio); -return ; -err_handle: + +return true; + +err_dec_pending:atomic_dec (&rdev->nr_pending); -raid_end_bio_io (r10_bio); + +err_allow_barrier: +allow_barrier (conf); + +return false; } static void raid10_write_one_disk(struct mddev *mddev, struct r10bio *r10_bio,@@ -1538,14 +1541,13 @@ static bool __make_request(struct mddev *mddev, struct bio *bio, int sectors)memset (r10_bio->devs,0 ,sizeof (r10_bio->devs[0 ]) * conf->geo.raid_disks); - ret = true;if (bio_data_dir (bio) ==READ ) -raid10_read_request (mddev, bio, r10_bio); -else { + ret =raid10_read_request (mddev, bio, r10_bio); +else ret =raid10_write_request (mddev, bio, r10_bio); -if (!ret) -free_r10bio (r10_bio); - } + +if (!ret) +free_r10bio (r10_bio);return ret; }@@ -1875,6 +1877,7 @@ static bool raid10_make_request(struct mddev *mddev, struct bio *bio)sector_t chunk_mask = (conf->geo.chunk_mask & conf->prev.chunk_mask);int chunk_sects = chunk_mask +1 ;int sectors =bio_sectors (bio); +bool write =bio_data_dir (bio) ==WRITE ;if (unlikely (bio->bi_opf &REQ_PREFLUSH ) &&md_flush_request (mddev, bio))@@ -1898,7 +1901,7 @@ static bool raid10_make_request(struct mddev *mddev, struct bio *bio) sectors = chunk_sects - (bio->bi_iter.bi_sector & (chunk_sects -1 )); -if (!__make_request (mddev, bio, sectors)) +if (!__make_request (mddev, bio, sectors) && write)md_write_end (mddev);/* In case raid10d snuck in to freeze_array */ @@ -2866,7 +2869,9 @@ static void handle_read_error(struct mddev *mddev, struct r10bio *r10_bio)rdev_dec_pending (rdev, mddev); r10_bio->state =0 ; -raid10_read_request (mddev, r10_bio->master_bio, r10_bio); +if (!raid10_read_request (mddev, r10_bio->master_bio, r10_bio)) +free_r10bio (r10_bio); +/* * allow_barrier after re-submit to ensure no sync io * can be issued while regular io pending.
Signature · 2 lines
Thanks, Kuai
Re: [PATCH v4 7/7] md/raid10: simplify read request error handling
Hi Kuai,
Thanks for the feedback.
Quoted 33 lines · yu kuai
Hi,
Quoted 23 lines · Abd-Alrhman Masalkhi
raid10_read_request()currently handles bio completion, barrier handling, andr10_biolifetime management in several different error paths. This results in duplicated cleanup logic and increases the risk of introducing bugs in future modifications.Make
raid10_read_request()return a status to its callers, consolidate the read error paths, and freer10_biofrom a single location in the callers. Since the callers allocater10_bio, they should also be responsible for freeing it when the request fails.This makes the read path follow the same ownership model as the write path and simplifies the error handling flow.
Signed-off-by Abd-Alrhman Masalkhi <[email protected]>---
Changes in v4:
- No changes.
- Link to v3: https://lore.kernel.org/linux-raid/[email protected]/Changes in v3:
- No changes.
- Link to v2: https://lore.kernel.org/linux-raid/[email protected]/Quoted 6 lines
Changes in v2:
- Fix a compilation error (bi -> bio).
- Link to v1: https://lore.kernel.org/linux-raid/[email protected]/--- drivers/md/raid10.c | 45 +++++++++++++++++++++++++-------------------- 1 file changed, 25 insertions(+), 20 deletions(-)
Quoted 4 lines
I don't think patch 6 and 7 are actually simplifications, and 7 in particular introduces a fragile dependency I'd rather not take as-is. The diff lines also said this and I'd rather not take as-is.
I see your point about the fragile dependency, especially in handle_read_error(). I'll drop those patchs from this series and revisit the cleanup in a future submission if I can find a better approach that doesn't introduce that dependency.
Quoted 120 lines
Quoted 116 lines
drivers/md/raid10.c +25
-20@@ -1143,7 +1143,7 @@ static bool regular_request_wait(struct mddev *mddev, struct r10conf *conf,return true; } -static void raid10_read_request (struct mddev *mddev,struct bio *bio, +static bool raid10_read_request (struct mddev *mddev,struct bio *bio,struct r10bio *r10_bio) {struct r10conf *conf = mddev->private;@@ -1191,8 +1191,7 @@ static void raid10_read_request(struct mddev *mddev, struct bio *bio,if (!regular_request_wait (mddev, conf, bio, r10_bio->sectors)) {bio_wouldblock_error (bio); -free_r10bio (r10_bio); -return ; +return false; } rdev =read_balance (conf, r10_bio, &max_sectors);@@ -1202,8 +1201,8 @@ static void raid10_read_request(struct mddev *mddev, struct bio *bio,mdname (mddev), b, (unsigned long long)r10_bio->sector); } -raid_end_bio_io (r10_bio); -return ; +bio_io_error (bio); + goto err_allow_barrier; }if (err_rdev) pr_err_ratelimited("md/raid10:%s: %pg: redirecting sector %llu to another mirror\n",@@ -1215,10 +1214,8 @@ static void raid10_read_request(struct mddev *mddev, struct bio *bio, bio =bio_submit_split_bioset (bio, max_sectors, &conf->bio_split);wait_barrier (conf, false); -if (!bio) { -set_bit (R10BIO_Returned, &r10_bio->state); - goto err_handle; - } +if (!bio) + goto err_dec_pending; r10_bio->master_bio = bio; r10_bio->sectors = max_sectors;@@ -1244,10 +1241,16 @@ static void raid10_read_request(struct mddev *mddev, struct bio *bio, read_bio->bi_private = r10_bio;mddev_trace_remap (mddev, read_bio, r10_bio->sector);submit_bio_noacct (read_bio); -return ; -err_handle: + +return true; + +err_dec_pending:atomic_dec (&rdev->nr_pending); -raid_end_bio_io (r10_bio); + +err_allow_barrier: +allow_barrier (conf); + +return false; } static void raid10_write_one_disk(struct mddev *mddev, struct r10bio *r10_bio,@@ -1538,14 +1541,13 @@ static bool __make_request(struct mddev *mddev, struct bio *bio, int sectors)memset (r10_bio->devs,0 ,sizeof (r10_bio->devs[0 ]) * conf->geo.raid_disks); - ret = true;if (bio_data_dir (bio) ==READ ) -raid10_read_request (mddev, bio, r10_bio); -else { + ret =raid10_read_request (mddev, bio, r10_bio); +else ret =raid10_write_request (mddev, bio, r10_bio); -if (!ret) -free_r10bio (r10_bio); - } + +if (!ret) +free_r10bio (r10_bio);return ret; }@@ -1875,6 +1877,7 @@ static bool raid10_make_request(struct mddev *mddev, struct bio *bio)sector_t chunk_mask = (conf->geo.chunk_mask & conf->prev.chunk_mask);int chunk_sects = chunk_mask +1 ;int sectors =bio_sectors (bio); +bool write =bio_data_dir (bio) ==WRITE ;if (unlikely (bio->bi_opf &REQ_PREFLUSH ) &&md_flush_request (mddev, bio))@@ -1898,7 +1901,7 @@ static bool raid10_make_request(struct mddev *mddev, struct bio *bio) sectors = chunk_sects - (bio->bi_iter.bi_sector & (chunk_sects -1 )); -if (!__make_request (mddev, bio, sectors)) +if (!__make_request (mddev, bio, sectors) && write)md_write_end (mddev);/* In case raid10d snuck in to freeze_array */ @@ -2866,7 +2869,9 @@ static void handle_read_error(struct mddev *mddev, struct r10bio *r10_bio)rdev_dec_pending (rdev, mddev); r10_bio->state =0 ; -raid10_read_request (mddev, r10_bio->master_bio, r10_bio); +if (!raid10_read_request (mddev, r10_bio->master_bio, r10_bio)) +free_r10bio (r10_bio); +/* * allow_barrier after re-submit to ensure no sync io * can be issued while regular io pending. Signature · 2 lines
Thanks, Kuai
Signature · 2 lines
Best Regards, Abd-Alrhman
Hi,
Quoted 32 lines · Abd-Alrhman Masalkhi
Hi,
This v4 of series contains a mix of bug fixes and cleanups for RAID10, along with a related atomic write fix for RAID1.
Changes in v4:
- Improve the commit message of patch 2.
- Add Reviewed-by tag from John Garry to patch 2.
- Link to v3: https://lore.kernel.org/linux-raid/[email protected]/Changes in v3:
- Setchunk_sectorstoBARRIER_UNIT_SECTOR_SIZEinstead of settingatomic_write_hw_unit_max.
- Avoid enabling write-behind when the atomic write exceeds the
write-behind limit.
- Link to v2: https://lore.kernel.org/linux-raid/[email protected]/Changes in v2:
- Expand the commit message to explain why the
allow_barrier()/wait_barrier() pair is no longer needed.
- Drop the early atomic write split check fromraid1_write_request().
- Advertise the atomic write size limit via queue limits.
- Disable write-behind instead of failing atomic writes when the
BIO_MAX_VECSlimit is encountered.
- Drop the early atomic write split check fromraid10_write_request()
and rely on queue limits instead.
- Fix a compilation error (bi -> bio).
- Link to v1: https://lore.kernel.org/linux-raid/[email protected]/Thanks,
Abd-alrhman,
Quoted 8 lines
Abd-Alrhman Masalkhi (7):
md/raid10: fix r10bio leak inraid10_write_request()error paths md/raid1: restrict atomic write limits and handle runtime constraints md/raid10: consistently fail atomic writes that require splitting md/raid10: remove unnecessary barrier aroundbio_submit_split_bioset()md/raid10: replace wait loop withwait_event_idle()md/raid10: simplify write request error handling md/raid10: simplify read request error handling
Applied patch 2,3,4 to md-7.3
Both patch 1 and patch 5 is not needed after your nowait patch.
Quoted 5 lines
drivers/md/raid1.c | 25 ++++------ drivers/md/raid10.c | 118 +++++++++++++++++++++----------------------- 2 files changed, 65 insertions(+), 78 deletions(-)
Signature · 2 lines
Thanks, Kuai