btrfs: raid56: minor fixes to rmw_rbio()
cover btrfs: raid56: minor fixes to rmw_rbio()
I was checking the bio_list_lock usage as some call sites doesn't look sane, e.g. spin_lock(); set_bit(); spin_unlock();.
But during the cleanup and reviewing, LLM exposed two new bugs:
- rbio_orig_end_io() can race with lock_stripe_add()
And rbio_orig_end_io() didn't hold the bio_list_lock, under certain race condition, a bio can even be missing.
Fixed by the first patch.
- Failed rbio can still be cached
Fixed by the second patch.
Qu Wenruo (2):
btrfs: fix racy bio_list changes for failed RMW rbio
btrfs: don't cache rbio if an error is hit
fs/btrfs/raid56.c | 35 +++++++++++++++++++++++------------ 1 file changed, 23 insertions(+), 12 deletions(-)
Signature · 1 line
2.56.0
PATCH 1/2 btrfs: fix racy bio_list changes for failed RMW rbio
If an RMW work failed early, e.g. alloc_rbio_parity_pages() failed, it won't get RBIO_RMW_LOCKED_BIT set, and get rbio_orig_end_io() called.
In that case, there can be a race between bio_list_get() on the rbio, and new incoming bio merging into the rbio:
A is the rbio that rbio_orig_end_io() is running on. a is the higher layer bio that generated the rbio A.
B is the new rbio that will be merged into A. b is the higher layer bio that generated the rbio B.
rbio_orig_end_io(A) worker | lock_stripe_add(B)
--------------------------------------+---------------------------------
rbio_orig_end_io(A) | lock_stripe_add(B) |- cur = bio_list_get(A) | |- rbio_can_merge(A, B) | |- bio = A->bio_list.head; | | which returned true | | which bio a is returned. | | | | | | | |- A->bio_list.tail = NULL; | | | | | | | | | |- merge_rbio(A, B) | | | | |- bio_list_merge_init() | | | | |- if (A->bio_list.tail) | | | | | A->bio_list.tail is | | | | | still NULL | | | | | | | | | |- A->bio_list.head = b | | | | |- A->bio_list.tail = b | |- A->bio_list.head = NULL; | |- return 1; | |- return bio; | Caller expect rbio A to | which is bio a. | complete bio b. | |- extra = bio_list_get() | Which is NULL | |- rbio_endio_bio_list(cur);
Which only ends bio a.
In above case, the bio_list race can get bio b lost, and never ended. Thankfully this race window is very tiny, so not practical in the real world.
Fix it by setting RBIO_RMW_LOCKED_BIT for error path before calling rbio_orig_end_io().
And since we're ensuring all RMW rbios have RBIO_RMW_LOCKED_BIT bit set before calling rbio_orig_end_io(), add two ASSERT()s for that and remove the old comment.
--- fs/btrfs/raid56.c | 32 ++++++++++++++++++++------------ 1 file changed, 20 insertions(+), 12 deletions(-)
fs/btrfs/raid56.c +20 -12
@@ -874,7 +874,10 @@ static void rbio_endio_bio_list(struct bio *cur, blk_status_t status)static void rbio_orig_end_io (struct btrfs_raid_bio *rbio,blk_status_t status) {struct bio *cur =bio_list_get (&rbio->bio_list); -struct bio *extra; + +/* An RMW rbio must have RBIO_RMW_LOCKED_BIT set. */ +if (rbio->operation ==BTRFS_RBIO_WRITE ) +ASSERT (test_bit (RBIO_RMW_LOCKED_BIT , &rbio->flags));kfree (rbio->csum_buf);bitmap_free (rbio->csum_bitmap);
@@ -888,21 +891,11 @@ static void rbio_orig_end_io(struct btrfs_raid_bio *rbio, blk_status_t status)*/ bitmap_clear (&rbio->dbitmap,0 , rbio->stripe_nsectors); -/* -* At this moment, rbio->bio_list is empty, however since rbio does not -* always have RBIO_RMW_LOCKED_BIT set and rbio is still linked on the -* hash list, rbio may be merged with others so that rbio->bio_list -* becomes non-empty. -* Once unlock_stripe() is done, rbio->bio_list will not be updated any -* more and we can call bio_endio() on all queued bios. -*/ unlock_stripe (rbio); - extra =bio_list_get (&rbio->bio_list); +ASSERT (bio_list_empty (&rbio->bio_list));free_raid_bio (rbio);rbio_endio_bio_list (cur, status); -if (extra) -rbio_endio_bio_list (extra, status); }/*
@@ -2464,6 +2457,21 @@ static void rmw_rbio(struct btrfs_raid_bio *rbio) } } out: +/* +* If we hit an error early, the rbio won't get +* RBIO_RMW_LOCKED_BIT bit set, and can cause unexpected +* merge during rbio_orig_end_io(). +* +* So set that flag before calling rbio_orig_end_io(), and +* we have to hold the bio_list_lock, to make sure +* rbio_can_merge() and merge_rbio() are seeing a stable +* flag. +*/ +if (ret <0 ) { +spin_lock (&rbio->bio_list_lock); +set_bit (RBIO_RMW_LOCKED_BIT , &rbio->flags); +spin_unlock (&rbio->bio_list_lock); + }rbio_orig_end_io (rbio,errno_to_blk_status (ret)); }
Signature · 1 line
2.56.0
PATCH 2/2 btrfs: don't cache rbio if an error is hit
In reply to Qu WenruoIf rmw_rbio() hit an error after cache_rbio_pages() call, the rbio still have RBIO_CACHE_READY_BIT set, thus can be utilized for future page stealing.
But that rbio didn't really reach the disk, thus using the pages for future steal can cause data inconsistency between in-memory cache and on-disk data.
Fix the problem by clearing the RBIO_CACHE_READY_BIT on the error path of rmw_rbio().
--- fs/btrfs/raid56.c | 3 +++ 1 file changed, 3 insertions(+)
fs/btrfs/raid56.c +3 -0
@@ -2471,6 +2471,9 @@ static void rmw_rbio(struct btrfs_raid_bio *rbio)spin_lock (&rbio->bio_list_lock);set_bit (RBIO_RMW_LOCKED_BIT , &rbio->flags);spin_unlock (&rbio->bio_list_lock); + +/* Failed write, don't cache for page stealing. */ +clear_bit (RBIO_CACHE_READY_BIT , &rbio->flags); }rbio_orig_end_io (rbio,errno_to_blk_status (ret)); }
Signature · 1 line
2.56.0
Quoted 10 lines · Qu Wenruo
I was checking the
bio_list_lockusage as some call sites doesn't look sane, e.g.spin_lock();set_bit();spin_unlock();.But during the cleanup and reviewing, LLM exposed two new bugs:
-
rbio_orig_end_io()can race withlock_stripe_add()
Andrbio_orig_end_io()didn't hold thebio_list_lock, under certain race condition, a bio can even be missing.
Quoted 8 lines
Fixed by the first patch.
- Failed rbio can still be cached
Fixed by the second patch.Qu Wenruo (2):
btrfs: fix racybio_listchanges for failed RMW rbio
btrfs: don't cache rbio if an error is hit
Thanks.
Quoted 8 lines
fs/btrfs/raid56.c | 35 +++++++++++++++++++++++------------ 1 file changed, 23 insertions(+), 12 deletions(-)--
2.56.0