btrfs: extra debug output for sector size < page size cases
cover btrfs: extra debug output for sector size < page size cases
The first patch is the long existing bug that full subpage bitmap dump is not working for checked bitmap.
Thankfully even myself is not affected by the bug.
The second one is for a crash I hit where ASSERT() got triggered in btrfs_folio_set_locked() after a btrfs_run_delalloc_range() failure.
The last one is for the btrfs_run_delalloc_range() failure, which is not that rare in my environment, I guess the unsafe cache mode for my aarch64 VM makes it too easy to hit ENOSPC.
But ENOSPC from btrfs_run_delalloc_range() itself is already a problem for our data/metadata space reservation code, thus it should be outputted even for non-debug build.
Qu Wenruo (3):
btrfs: subpage: fix the bitmap dump for the locked flags
btrfs: subpage: dump the involved bitmap when ASSERT() failed
btrfs: add extra error messages for extent_writepage() failure
fs/btrfs/extent_io.c | 16 +++++++++++++++ fs/btrfs/subpage.c | 47 ++++++++++++++++++++++++++++++++------------ 2 files changed, 50 insertions(+), 13 deletions(-)
Signature · 1 line
2.47.0
PATCH 1/3 btrfs: subpage: fix the bitmap dump for the locked flags
We're dumping the locked bitmap into the @checked_bitmap variable, causing incorrect values during debug.
Thankfuklly even during my development I haven't hit a case where I need to dump the locked bitmap.
But for the sake of consistency, fix it by dumpping the locked bitmap into @locked_bitmap variable for output.
--- fs/btrfs/subpage.c | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-)
fs/btrfs/subpage.c +4 -2
@@ -716,6 +716,7 @@ void __cold btrfs_subpage_dump_bitmap(const struct btrfs_fs_info *fs_info,unsigned long writeback_bitmap;unsigned long ordered_bitmap;unsigned long checked_bitmap; +unsigned long locked_bitmap;unsigned long flags;ASSERT (folio_test_private (folio) &&folio_get_private (folio));
@@ -728,15 +729,16 @@ void __cold btrfs_subpage_dump_bitmap(const struct btrfs_fs_info *fs_info,GET_SUBPAGE_BITMAP (subpage, fs_info, writeback, &writeback_bitmap);GET_SUBPAGE_BITMAP (subpage, fs_info, ordered, &ordered_bitmap);GET_SUBPAGE_BITMAP (subpage, fs_info, checked, &checked_bitmap); -GET_SUBPAGE_BITMAP (subpage, fs_info, locked, &checked_bitmap); +GET_SUBPAGE_BITMAP (subpage, fs_info, locked, &locked_bitmap);spin_unlock_irqrestore (&subpage->lock, flags);dump_page (folio_page (folio,0 ),"btrfs subpage dump" );btrfs_warn (fs_info, -"start=%llu len=%u page=%llu, bitmaps uptodate=%*pbl dirty=%*pbl writeback=%*pbl ordered=%*pbl checked=%*pbl" , +"start=%llu len=%u page=%llu, bitmaps uptodate=%*pbl dirty=%*pbl locked=%*pbl writeback=%*pbl ordered=%*pbl checked=%*pbl" , start, len,folio_pos (folio), sectors_per_page, &uptodate_bitmap, sectors_per_page, &dirty_bitmap, + sectors_per_page, &locked_bitmap, sectors_per_page, &writeback_bitmap, sectors_per_page, &ordered_bitmap, sectors_per_page, &checked_bitmap);
Signature · 1 line
2.47.0
PATCH 2/3 btrfs: subpage: dump the involved bitmap when ASSERT() failed
In reply to Qu WenruoFor btrfs_folio_assert_not_dirty() and btrfs_folio_set_lock(), we call bitmap_test_range_all_zero() to ensure the involved range has not bit set.
However with my recent enhanced delalloc range error handling, I'm hitting the ASSERT() inside btrfs_folio_set_lock(), and is wondering if it's some error handling not properly cleanup the locked bitmap but directly unlock the page.
So add some extra dumpping for the ASSERTs to dump the involved bitmap to help debug.
--- fs/btrfs/subpage.c | 41 ++++++++++++++++++++++++++++++----------- 1 file changed, 30 insertions(+), 11 deletions(-)
fs/btrfs/subpage.c +30 -11
@@ -635,6 +635,28 @@ IMPLEMENT_BTRFS_PAGE_OPS(ordered, folio_set_ordered, folio_clear_ordered,IMPLEMENT_BTRFS_PAGE_OPS (checked, folio_set_checked, folio_clear_checked, folio_test_checked); +#define GET_SUBPAGE_BITMAP (subpage, fs_info, name, dst) \ +{ \ + const int sectors_per_page = fs_info->sectors_per_page; \ + \ + ASSERT(sectors_per_page < BITS_PER_LONG); \ + *dst = bitmap_read(subpage->bitmaps, \ + sectors_per_page * btrfs_bitmap_nr_##name, \ + sectors_per_page); \ +} + +#define subpage_dump_bitmap (fs_info, folio, name, start, len) \ +{ \ + struct btrfs_subpage *subpage = folio_get_private(folio); \ + unsigned long bitmap; \ + \ + GET_SUBPAGE_BITMAP(subpage, fs_info, name, &bitmap); \ + btrfs_warn(fs_info, \ + "dumpping bitmap start=%llu len=%u folio=%llu" #name "_bitmap=%*pbl", \ + start, len, folio_pos(folio), \ + fs_info->sectors_per_page, &bitmap); \ +} +/* * Make sure not only the page dirty bit is cleared, but also subpage dirty bit * is cleared.
@@ -660,6 +682,10 @@ void btrfs_folio_assert_not_dirty(const struct btrfs_fs_info *fs_info, subpage =folio_get_private (folio);ASSERT (subpage);spin_lock_irqsave (&subpage->lock, flags); +if (unlikely (!bitmap_test_range_all_zero (subpage->bitmaps, start_bit, nbits))) { +subpage_dump_bitmap (fs_info, folio, dirty, start, len); +ASSERT (bitmap_test_range_all_zero (subpage->bitmaps, start_bit, nbits)); + }ASSERT (bitmap_test_range_all_zero (subpage->bitmaps, start_bit, nbits));spin_unlock_irqrestore (&subpage->lock, flags); }
@@ -689,23 +715,16 @@ void btrfs_folio_set_lock(const struct btrfs_fs_info *fs_info, nbits = len >> fs_info->sectorsize_bits;spin_lock_irqsave (&subpage->lock, flags);/* Target range should not yet be locked. */ -ASSERT (bitmap_test_range_all_zero (subpage->bitmaps, start_bit, nbits)); +if (unlikely (!bitmap_test_range_all_zero (subpage->bitmaps, start_bit, nbits))) { +subpage_dump_bitmap (fs_info, folio, locked, start, len); +ASSERT (bitmap_test_range_all_zero (subpage->bitmaps, start_bit, nbits)); + }bitmap_set (subpage->bitmaps, start_bit, nbits); ret =atomic_add_return (nbits, &subpage->nr_locked);ASSERT (ret <= fs_info->sectors_per_page);spin_unlock_irqrestore (&subpage->lock, flags); } -#define GET_SUBPAGE_BITMAP (subpage, fs_info, name, dst) \ -{ \ - const int sectors_per_page = fs_info->sectors_per_page; \ - \ - ASSERT(sectors_per_page < BITS_PER_LONG); \ - *dst = bitmap_read(subpage->bitmaps, \ - sectors_per_page * btrfs_bitmap_nr_##name, \ - sectors_per_page); \ -} -void __cold btrfs_subpage_dump_bitmap (const struct btrfs_fs_info *fs_info,struct folio *folio,u64 start,u32 len) {
Signature · 1 line
2.47.0
PATCH 3/3 btrfs: add extra error messages for extent_writepage() failure
In reply to Qu WenruoDuring my development on aarch64 with 64K page size and 4K sector size, I'm hitting several problems related to error handling of extent_writepage(), most of them are caused by btrfs_run_delalloc_range() failure with -ENOSPC error.
This error itself is already a problem for our data/metadata space reservation code, but we also need extra info like the submit_bitmap to calculate if the error handling is doing its job correctly.
So add two btrfs_err_rl()s to indicate the errors that is affecting the error handling of extent_writepage(), which has already saved me several times during debugging.
--- fs/btrfs/extent_io.c | 16 ++++++++++++++++ 1 file changed, 16 insertions(+)
fs/btrfs/extent_io.c +16 -0
@@ -1240,6 +1240,15 @@ static noinline_for_stack int writepage_delalloc(struct btrfs_inode *inode, found_start, found_start + found_len -1 , wbc); +if (unlikely (ret <0 )) +btrfs_err_rl (fs_info, +"failed to run delalloc range, root %lld ino %llu folio %llu submit_bitmap %*pbl start %llu len %u: %d" , + inode->root->root_key.objectid, +btrfs_ino (inode), +folio_pos (folio), + fs_info->sectors_per_page, + &bio_ctrl->submit_bitmap, + found_start, found_len, ret); }else {/* * We've hit an error during previous delalloc range,
@@ -1506,6 +1515,13 @@ static int extent_writepage(struct folio *folio, struct btrfs_bio_ctrl *bio_ctrl PAGE_SIZE, bio_ctrl, i_size);if (ret ==1 )return 0 ; +if (ret <0 ) +btrfs_err_rl (fs_info, +"failed to submit blocks, root %lld ino %llu folio %llu submit_bitmap %*pbl: %d" , +BTRFS_I (inode)->root->root_key.objectid, +btrfs_ino (BTRFS_I (inode)), +folio_pos (folio), fs_info->sectors_per_page, + &bio_ctrl->submit_bitmap, ret); bio_ctrl->wbc->nr_to_write--;
Signature · 1 line
2.47.0
Quoted 11 lines · Qu Wenruo
The first patch is the long existing bug that full subpage bitmap dump is not working for checked bitmap.
Thankfully even myself is not affected by the bug.The second one is for a crash I hit where
ASSERT()got triggered inbtrfs_folio_set_locked()after abtrfs_run_delalloc_range()failure.The last one is for the
btrfs_run_delalloc_range()failure, which is not that rare in my environment, I guess the unsafe cache mode for my aarch64 VM makes it too easy to hit ENOSPC.
Quoted 8 lines
But ENOSPC from
btrfs_run_delalloc_range()itself is already a problem for our data/metadata space reservation code, thus it should be outputted even for non-debug build.Qu Wenruo (3):
btrfs: subpage: fix the bitmap dump for the locked flags
btrfs: subpage: dump the involved bitmap whenASSERT()failed
btrfs: add extra error messages forextent_writepage()failure
Re: [PATCH 2/3] btrfs: subpage: dump the involved bitmap when ASSERT() failed
In reply to Qu Wenruofs/btrfs/subpage.c IMPLEMENT_BTRFS_PAGE_OPS(ordered, folio_set_ordered, folio_clear_ordered, · line 655
Quoted 8 lines · Qu Wenruo
fs/btrfs/subpage.c +8
-0+#define subpage_dump_bitmap (fs_info, folio, name, start, len) \ +{ \ + struct btrfs_subpage *subpage = folio_get_private(folio); \ + unsigned long bitmap; \ + \ + GET_SUBPAGE_BITMAP(subpage, fs_info, name, &bitmap); \ + btrfs_warn(fs_info, \ + "dumpping bitmap start=%llu len=%u folio=%llu" #name "_bitmap=%*pbl", \
The stringified #name would be next to the folio=number, please add one " ".
Quoted 3 lines
fs/btrfs/subpage.c +3
-0+ start, len,folio_pos (folio), \ + fs_info->sectors_per_page, &bitmap); \ +}
Quoted 16 lines · David Sterba
Quoted 15 lines · Qu Wenruo
The first patch is the long existing bug that full subpage bitmap dump is not working for checked bitmap.
Thankfully even myself is not affected by the bug.The second one is for a crash I hit where
ASSERT()got triggered inbtrfs_folio_set_locked()after abtrfs_run_delalloc_range()failure.The last one is for the
btrfs_run_delalloc_range()failure, which is not that rare in my environment, I guess the unsafe cache mode for my aarch64 VM makes it too easy to hit ENOSPC.But ENOSPC from
btrfs_run_delalloc_range()itself is already a problem for our data/metadata space reservation code, thus it should be outputted even for non-debug build.
Quoted 6 lines
Quoted 4 lines
Qu Wenruo (3):
btrfs: subpage: fix the bitmap dump for the locked flags
btrfs: subpage: dump the involved bitmap whenASSERT()failed
btrfs: add extra error messages forextent_writepage()failureReviewed-by David Sterba <[email protected]>
Thanks for the review.
Although I'd prefer this series to be merged after the double accounting fix (which also affects non-subpage cases)
The problem is in the 3rd patch, which is touching the code just after btrfs_run_delalloc_range() returned.
That area is also where the double accounting fix is touching.
To make backport a little easier, I'd prefer make the fix first, then the extra debug patches.
Thanks,
Qu
Quoted 31 lines · Qu Wenruo
Quoted 16 lines · David Sterba
Quoted 15 lines · Qu Wenruo
The first patch is the long existing bug that full subpage bitmap dump is not working for checked bitmap.
Thankfully even myself is not affected by the bug.The second one is for a crash I hit where
ASSERT()got triggered inbtrfs_folio_set_locked()after abtrfs_run_delalloc_range()failure.The last one is for the
btrfs_run_delalloc_range()failure, which is not that rare in my environment, I guess the unsafe cache mode for my aarch64 VM makes it too easy to hit ENOSPC.But ENOSPC from
btrfs_run_delalloc_range()itself is already a problem for our data/metadata space reservation code, thus it should be outputted even for non-debug build.Quoted 6 lines
Quoted 4 lines
Qu Wenruo (3):
btrfs: subpage: fix the bitmap dump for the locked flags
btrfs: subpage: dump the involved bitmap whenASSERT()failed
btrfs: add extra error messages forextent_writepage()failureReviewed-by David Sterba <[email protected]>Thanks for the review.
Although I'd prefer this series to be merged after the double accounting fix (which also affects non-subpage cases)
Quoted 7 lines
The problem is in the 3rd patch, which is touching the code just after
btrfs_run_delalloc_range()returned.That area is also where the double accounting fix is touching.
To make backport a little easier, I'd prefer make the fix first, then the extra debug patches.
Right, no objections here. As you've been finding the problems any debugging and reporting improvement is good.