btrfs: the missing 4 patches to implement metadata write path
cover btrfs: the missing 4 patches to implement metadata write path
When adding the comments for subpage metadata code, I inserted the comment patch into the wrong position, and then use that patch as a separator between data and metadata write path.
Thus the submitted metadata write path patchset lacks the real functions to submit subpage metadata write bio.
Qu Wenruo (4):
btrfs: introduce end_bio_subpage_eb_writepage() function
btrfs: introduce write_one_subpage_eb() function
btrfs: make lock_extent_buffer_for_io() to be subpage compatible
btrfs: introduce submit_eb_subpage() to submit a subpage metadata page
fs/btrfs/extent_io.c | 293 ++++++++++++++++++++++++++++++++++++++----- 1 file changed, 263 insertions(+), 30 deletions(-)
Signature · 1 line
2.31.1
PATCH 1/4 btrfs: introduce end_bio_subpage_eb_writepage() function
The new function, end_bio_subpage_eb_writepage(), will handle the metadata writeback endio.
The major differences involved are:
- How to grab extent buffer
Now page::private is a pointer to btrfs_subpage, we can no longer grab extent buffer directly.
Thus we need to use the bv_offset to locate the extent buffer manually and iterate through the whole range.
- Use btrfs_subpage_end_writeback() caller This helper will handle the subpage writeback for us.
Since this function is executed under endio context, when grabbing extent buffers it can't grab eb->refs_lock as that lock is not designed to be grabbed under hardirq context.
So here introduce a helper, find_extent_buffer_nospinlock(), for such situation, and convert find_extent_buffer() to use that helper.
--- fs/btrfs/extent_io.c | 135 +++++++++++++++++++++++++++++++++---------- 1 file changed, 106 insertions(+), 29 deletions(-)
fs/btrfs/extent_io.c +106 -29
@@ -4080,13 +4080,97 @@ static void set_btree_ioerr(struct page *page, struct extent_buffer *eb) } } +/* +* This is the endio specific version which won't touch any unsafe spinlock +* in endio context. +*/ +static struct extent_buffer *find_extent_buffer_nospinlock ( +struct btrfs_fs_info *fs_info,u64 start) +{ +struct extent_buffer *eb; + +rcu_read_lock (); + eb =radix_tree_lookup (&fs_info->buffer_radix, + start >> fs_info->sectorsize_bits); +if (eb &&atomic_inc_not_zero (&eb->refs)) { +rcu_read_unlock (); +return eb; + } +rcu_read_unlock (); +return NULL ; +} +/* +* The endio function for subpage extent buffer write. +* +* Unlike end_bio_extent_buffer_writepage(), we only call end_page_writeback() +* after all extent buffers in the page has finished their writeback. +*/ +static void end_bio_subpage_eb_writepage (struct btrfs_fs_info *fs_info, +struct bio *bio) +{ +struct bio_vec *bvec; +struct bvec_iter_all iter_all; + +ASSERT (!bio_flagged (bio,BIO_CLONED )); +bio_for_each_segment_all (bvec, bio, iter_all) { +struct page *page = bvec->bv_page; +u64 bvec_start =page_offset (page) + bvec->bv_offset; +u64 bvec_end = bvec_start + bvec->bv_len -1 ; +u64 cur_bytenr = bvec_start; + +ASSERT (IS_ALIGNED (bvec->bv_len, fs_info->nodesize)); + +/* Iterate through all extent buffers in the range */ +while (cur_bytenr <= bvec_end) { +struct extent_buffer *eb; +int done; + +/* +* Here we can't use find_extent_buffer(), as it may +* try to lock eb->refs_lock, which is not safe in endio +* context. +*/ + eb =find_extent_buffer_nospinlock (fs_info, cur_bytenr); +ASSERT (eb); + + cur_bytenr = eb->start + eb->len; + +ASSERT (test_bit (EXTENT_BUFFER_WRITEBACK , &eb->bflags)); + done =atomic_dec_and_test (&eb->io_pages); +ASSERT (done); + +if (bio->bi_status || +test_bit (EXTENT_BUFFER_WRITE_ERR , &eb->bflags)) { +ClearPageUptodate (page); +set_btree_ioerr (page, eb); + } + +btrfs_subpage_clear_writeback (fs_info, page, eb->start, + eb->len); +end_extent_buffer_writeback (eb); +/* +* free_extent_buffer() will grab spinlock which is not +* safe in endio context. Thus here we manually dec +* the ref. +*/ +atomic_dec (&eb->refs); + } + } +bio_put (bio); +} +static void end_bio_extent_buffer_writepage (struct bio *bio) { +struct btrfs_fs_info *fs_info;struct bio_vec *bvec;struct extent_buffer *eb;int done;struct bvec_iter_all iter_all; + fs_info =btrfs_sb (bio_first_page_all (bio)->mapping->host->i_sb); +if (fs_info->sectorsize <PAGE_SIZE ) +return end_bio_subpage_eb_writepage (fs_info, bio); +ASSERT (!bio_flagged (bio,BIO_CLONED ));bio_for_each_segment_all (bvec, bio, iter_all) {struct page *page = bvec->bv_page;
@@ -5467,36 +5551,29 @@ struct extent_buffer *find_extent_buffer(struct btrfs_fs_info *fs_info, {struct extent_buffer *eb; -rcu_read_lock (); - eb =radix_tree_lookup (&fs_info->buffer_radix, - start >> fs_info->sectorsize_bits); -if (eb &&atomic_inc_not_zero (&eb->refs)) { -rcu_read_unlock (); -/* -* Lock our eb's refs_lock to avoid races with -* free_extent_buffer. When we get our eb it might be flagged -* with EXTENT_BUFFER_STALE and another task running -* free_extent_buffer might have seen that flag set, -* eb->refs == 2, that the buffer isn't under IO (dirty and -* writeback flags not set) and it's still in the tree (flag -* EXTENT_BUFFER_TREE_REF set), therefore being in the process -* of decrementing the extent buffer's reference count twice. -* So here we could race and increment the eb's reference count, -* clear its stale flag, mark it as dirty and drop our reference -* before the other task finishes executing free_extent_buffer, -* which would later result in an attempt to free an extent -* buffer that is dirty. -*/ -if (test_bit (EXTENT_BUFFER_STALE , &eb->bflags)) { -spin_lock (&eb->refs_lock); -spin_unlock (&eb->refs_lock); - } -mark_extent_buffer_accessed (eb,NULL ); -return eb; + eb =find_extent_buffer_nospinlock (fs_info, start); +if (!eb) +return NULL ; +/* +* Lock our eb's refs_lock to avoid races with free_extent_buffer(). +* When we get our eb it might be flagged with EXTENT_BUFFER_STALE and +* another task running free_extent_buffer() might have seen that flag +* set, eb->refs == 2, that the buffer isn't under IO (dirty and +* writeback flags not set) and it's still in the tree (flag +* EXTENT_BUFFER_TREE_REF set), therefore being in the process +* of decrementing the extent buffer's reference count twice. +* So here we could race and increment the eb's reference count, +* clear its stale flag, mark it as dirty and drop our reference +* before the other task finishes executing free_extent_buffer, +* which would later result in an attempt to free an extent +* buffer that is dirty. +*/ +if (test_bit (EXTENT_BUFFER_STALE , &eb->bflags)) { +spin_lock (&eb->refs_lock); +spin_unlock (&eb->refs_lock); } -rcu_read_unlock (); - -return NULL ; +mark_extent_buffer_accessed (eb,NULL ); +return eb; }#ifdef CONFIG_BTRFS_FS_RUN_SANITY_TESTS
Signature · 1 line
2.31.1
PATCH 2/4 btrfs: introduce write_one_subpage_eb() function
In reply to Qu WenruoThe new function, write_one_subpage_eb(), as a subroutine for subpage metadata write, will handle the extent buffer bio submission.
The major differences between the new write_one_subpage_eb() and write_one_eb() is:
- No page locking
When entering write_one_subpage_eb() the page is no longer locked. We only lock the page for its status update, and unlock immediately. Now we completely rely on extent io tree locking.
- Extra bitmap update along with page status update
Now page dirty and writeback is controlled by btrfs_subpage::dirty_bitmap and btrfs_subpage::writeback_bitmap. They both follow the schema that any sector is dirty/writeback, then the full page get dirty/writeback.
- When to update the nr_written number
Now we take a short cut, if we have cleared the last dirty bit of the page, we update nr_written.
This is not completely perfect, but should emulate the old behavior good enough.
--- fs/btrfs/extent_io.c | 55 ++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 55 insertions(+)
fs/btrfs/extent_io.c +55 -0
@@ -4196,6 +4196,58 @@ static void end_bio_extent_buffer_writepage(struct bio *bio)bio_put (bio); } +/* +* Unlike the work in write_one_eb(), we rely completely on extent locking. +* Page locking is only utizlied at minimal to keep the VM code happy. +* +* Caller should still call write_one_eb() other than this function directly. +* As write_one_eb() has extra prepration before submitting the extent buffer. +*/ +static int write_one_subpage_eb (struct extent_buffer *eb, +struct writeback_control *wbc, +struct extent_page_data *epd) +{ +struct btrfs_fs_info *fs_info = eb->fs_info; +struct page *page = eb->pages[0 ]; +unsigned int write_flags =wbc_to_write_flags (wbc) |REQ_META ; +bool no_dirty_ebs = false; +int ret; + +/* clear_page_dirty_for_io() in subpage helper need page locked. */ +lock_page (page); +btrfs_subpage_set_writeback (fs_info, page, eb->start, eb->len); + +/* If we're the last dirty bit to update nr_written */ + no_dirty_ebs =btrfs_subpage_clear_and_test_dirty (fs_info, page, + eb->start, eb->len); +if (no_dirty_ebs) +clear_page_dirty_for_io (page); + + ret =submit_extent_page (REQ_OP_WRITE | write_flags, wbc, page, + eb->start, eb->len, eb->start -page_offset (page), + &epd->bio, end_bio_extent_buffer_writepage,0 ,0 ,0 , + false); +if (ret) { +btrfs_subpage_clear_writeback (fs_info, page, eb->start, + eb->len); +set_btree_ioerr (page, eb); +unlock_page (page); + +if (atomic_dec_and_test (&eb->io_pages)) +end_extent_buffer_writeback (eb); +return -EIO ; + } +unlock_page (page); +/* +* Submission finishes without problem, if no range of the page is +* dirty anymore, we have submitted a page. +* Update the nr_written in wbc. +*/ +if (no_dirty_ebs) +update_nr_written (wbc,1 ); +return ret; +} + static noinline_for_stack int write_one_eb(struct extent_buffer *eb,struct writeback_control *wbc,struct extent_page_data *epd)
@@ -4227,6 +4279,9 @@ static noinline_for_stack int write_one_eb(struct extent_buffer *eb,memzero_extent_buffer (eb, start, end - start); } +if (eb->fs_info->sectorsize <PAGE_SIZE ) +return write_one_subpage_eb (eb, wbc, epd); +for (i =0 ; i < num_pages; i++) {struct page *p = eb->pages[i];
Signature · 1 line
2.31.1
PATCH 3/4 btrfs: make lock_extent_buffer_for_io() to be subpage compatible
In reply to Qu WenruoFor subpage metadata, we don't use page locking at all. So just skip the page locking part for subpage.
All the remaining routine can be reused.
--- fs/btrfs/extent_io.c | 8 +++++++- 1 file changed, 7 insertions(+), 1 deletion(-)
fs/btrfs/extent_io.c +7 -1
@@ -3967,7 +3967,13 @@ static noinline_for_stack int lock_extent_buffer_for_io(struct extent_buffer *ebbtrfs_tree_unlock (eb); -if (!ret) +/* +* Either we don't need to submit any tree block, or we're submitting +* subpage. +* Subpage metadata doesn't use page locking at all, so we can skip +* the page locking. +*/ +if (!ret || fs_info->sectorsize <PAGE_SIZE )return ret; num_pages =num_extent_pages (eb);
Signature · 1 line
2.31.1
PATCH 4/4 btrfs: introduce submit_eb_subpage() to submit a subpage metadata page
In reply to Qu WenruoThe new function, submit_eb_subpage(), will submit all the dirty extent buffers in the page.
The major difference between submit_eb_page() and submit_eb_subpage() is:
- How to grab extent buffer
Now we use find_extent_buffer_nospinlock() other than using page::private.
All other different handling is already done in functions like lock_extent_buffer_for_io() and write_one_eb().
--- fs/btrfs/extent_io.c | 95 ++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 95 insertions(+)
fs/btrfs/extent_io.c +95 -0
@@ -4323,6 +4323,98 @@ static noinline_for_stack int write_one_eb(struct extent_buffer *eb,return ret; } +/* +* Submit one subpage btree page. +* +* The main difference between submit_eb_page() is: +* - Page locking +* For subpage, we don't rely on page locking at all. +* +* - Flush write bio +* We only flush bio if we may be unable to fit current extent buffers into +* current bio. +* +* Return >=0 for the number of submitted extent buffers. +* Return <0 for fatal error. +*/ +static int submit_eb_subpage (struct page *page, +struct writeback_control *wbc, +struct extent_page_data *epd) +{ +struct btrfs_fs_info *fs_info =btrfs_sb (page->mapping->host->i_sb); +int submitted =0 ; +u64 page_start =page_offset (page); +int bit_start =0 ; +int nbits =BTRFS_SUBPAGE_BITMAP_SIZE ; +int sectors_per_node = fs_info->nodesize >> fs_info->sectorsize_bits; +int ret; + +/* Lock and write each dirty extent buffers in the range */ +while (bit_start < nbits) { +struct btrfs_subpage *subpage = (struct btrfs_subpage *)page->private; +struct extent_buffer *eb; +unsigned long flags; +u64 start; + +/* +* Take private lock to ensure the subpage won't be detached +* halfway. +*/ +spin_lock (&page->mapping->private_lock); +if (!PagePrivate (page)) { +spin_unlock (&page->mapping->private_lock); +break ; + } +spin_lock_irqsave (&subpage->lock, flags); +if (!((1 << bit_start) & subpage->dirty_bitmap)) { +spin_unlock_irqrestore (&subpage->lock, flags); +spin_unlock (&page->mapping->private_lock); + bit_start++; +continue ; + } + + start = page_start + bit_start * fs_info->sectorsize; + bit_start += sectors_per_node; + +/* +* Here we just want to grab the eb without touching extra +* spin locks. So here we call find_extent_buffer_nospinlock(). +*/ + eb =find_extent_buffer_nospinlock (fs_info, start); +spin_unlock_irqrestore (&subpage->lock, flags); +spin_unlock (&page->mapping->private_lock); + +/* +* The eb has already reached 0 refs thus find_extent_buffer() +* doesn't return it. We don't need to write back such eb +* anyway. +*/ +if (!eb) +continue ; + + ret =lock_extent_buffer_for_io (eb, epd); +if (ret ==0 ) { +free_extent_buffer (eb); +continue ; + } +if (ret <0 ) { +free_extent_buffer (eb); + goto cleanup; + } + ret =write_one_eb (eb, wbc, epd); +free_extent_buffer (eb); +if (ret <0 ) + goto cleanup; + submitted++; + } +return submitted; + +cleanup: +/* We hit error, end bio for the submitted extent buffers */ +end_write_bio (epd, ret); +return ret; +} +/* * Submit all page(s) of one extent buffer. *
@@ -4355,6 +4447,9 @@ static int submit_eb_page(struct page *page, struct writeback_control *wbc,if (!PagePrivate (page))return 0 ; +if (btrfs_sb (page->mapping->host->i_sb)->sectorsize <PAGE_SIZE ) +return submit_eb_subpage (page, wbc, epd); +spin_lock (&mapping->private_lock);if (!PagePrivate (page)) {spin_unlock (&mapping->private_lock);
Signature · 1 line
2.31.1
Quoted 4 lines · Qu Wenruo
When adding the comments for subpage metadata code, I inserted the comment patch into the wrong position, and then use that patch as a separator between data and metadata write path.
Quoted 8 lines
Thus the submitted metadata write path patchset lacks the real functions to submit subpage metadata write bio.
Qu Wenruo (4):
btrfs: introduceend_bio_subpage_eb_writepage()function
btrfs: introducewrite_one_subpage_eb()function
btrfs: makelock_extent_buffer_for_io()to be subpage compatible
btrfs: introducesubmit_eb_subpage()to submit a subpage metadata page
For the record, the patches have been added to 5.13 queue a few days ago.
Quoted 8 lines · David Sterba
Quoted 7 lines · Qu Wenruo
When adding the comments for subpage metadata code, I inserted the comment patch into the wrong position, and then use that patch as a separator between data and metadata write path.
Thus the submitted metadata write path patchset lacks the real functions to submit subpage metadata write bio.
Quoted 9 lines
Quoted 5 lines
Qu Wenruo (4):
btrfs: introduceend_bio_subpage_eb_writepage()function
btrfs: introducewrite_one_subpage_eb()function
btrfs: makelock_extent_buffer_for_io()to be subpage compatible
btrfs: introducesubmit_eb_subpage()to submit a subpage metadata pageFor the record, the patches have been added to 5.13 queue a few days ago.
Josef had some comments on them, most of them are just related to introducing a new subpage specific endio function other than reusing the existing one.
So I guess if I go that direction, I should just add new patches as a refactor?
Thanks,
Qu
Quoted 17 lines · Qu Wenruo
Quoted 14 lines · David Sterba
Quoted 12 lines · Qu Wenruo
When adding the comments for subpage metadata code, I inserted the comment patch into the wrong position, and then use that patch as a separator between data and metadata write path.
Thus the submitted metadata write path patchset lacks the real functions to submit subpage metadata write bio.
Qu Wenruo (4):
btrfs: introduceend_bio_subpage_eb_writepage()function
btrfs: introducewrite_one_subpage_eb()function
btrfs: makelock_extent_buffer_for_io()to be subpage compatible
btrfs: introducesubmit_eb_subpage()to submit a subpage metadata page
Quoted 7 lines
Quoted 3 lines
For the record, the patches have been added to 5.13 queue a few days ago.
Josef had some comments on them, most of them are just related to introducing a new subpage specific endio function other than reusing the existing one.
I haven't seen any comments for that patchset.
Quoted 2 lines
So I guess if I go that direction, I should just add new patches as a refactor?
Yes please.
Quoted 22 lines · David Sterba
Quoted 21 lines · Qu Wenruo
Quoted 17 lines · David Sterba
Quoted 4 lines · Qu Wenruo
When adding the comments for subpage metadata code, I inserted the comment patch into the wrong position, and then use that patch as a separator between data and metadata write path.
Quoted 8 lines
Thus the submitted metadata write path patchset lacks the real functions to submit subpage metadata write bio.
Qu Wenruo (4):
btrfs: introduceend_bio_subpage_eb_writepage()function
btrfs: introducewrite_one_subpage_eb()function
btrfs: makelock_extent_buffer_for_io()to be subpage compatible
btrfs: introducesubmit_eb_subpage()to submit a subpage metadata pageFor the record, the patches have been added to 5.13 queue a few days ago.
Quoted 5 lines
Quoted 3 lines
Josef had some comments on them, most of them are just related to introducing a new subpage specific endio function other than reusing the existing one.
I haven't seen any comments for that patchset.
It's for the big subpage RW patchset, which includes the four missing patches.
Thanks,
Qu
Quoted 6 lines
Quoted 3 lines
So I guess if I go that direction, I should just add new patches as a refactor?
Yes please.