btrfs: migrate IO path to folios
cover btrfs: migrate IO path to folios
One critical problem I hit the most during my initial higher order folios tests are, incorrect access to the pages which conflicts with the page flag policy.
Since folio flags are only set to certain pages according to their policies (PF_ANY, PF_HEAD, PF_ONLY_HEAD, PF_NO_TAIL, PF_NO_COMPOUND and the most weird on PF_SECOND), setting page flags violating the policy would immedate lead to VM_BUG_ON().
Thus no matter if we go compound page or folio, we can not go the page-by-page iteration helpers that easily.
One of the hot spots which can lead to VM_BUG_ON()s are the endio helpers.
So this patch would:
- Make metadata set/get helpers to utilize folio interfaces
- Make subpage code to accept folios directly
This is to avoid btrfs_page_*() helpers to accept page pointers, which is another hot spot which uses page pointer a lot.
- Migrate btrfs bio endio functions to utilize bio_for_each_folio_all() This completely removes the ability to direct access page pointers. Although we still need some extra folio_page(folio, 0) calls to keep compatible with existing helpers.
And since we're here, also fix the choas of btrfs endio functions' naming scheme, now it would always be:
end_bbio_<target>_(read|write)
The <target> can be:
- data
For non-compressed and non-encoeded operations
- meta
- compressed
For compressed IO and encoded IO.
And since compressed IO path is utilizing unmapped pages (pages without an address_space), thus they don't touch the page flags. This makes compressed IO path a very good test bed for the initial introduction of higher order folio.
Qu Wenruo (3):
btrfs: migrate get_eb_page_index() and get_eb_offset_in_page() to
folios
btrfs: migrate subpage code to folio interfaces
btrfs: migrate various btrfs end io functions to folios
fs/btrfs/accessors.c | 80 ++++---- fs/btrfs/compression.c | 15 +- fs/btrfs/ctree.c | 13 +- fs/btrfs/defrag.c | 3 +- fs/btrfs/disk-io.c | 4 +- fs/btrfs/extent_io.c | 361 +++++++++++++++++++----------------- fs/btrfs/extent_io.h | 40 ++-- fs/btrfs/file.c | 13 +- fs/btrfs/free-space-cache.c | 4 +- fs/btrfs/inode.c | 34 ++-- fs/btrfs/ordered-data.c | 5 +- fs/btrfs/reflink.c | 6 +- fs/btrfs/relocation.c | 5 +- fs/btrfs/subpage.c | 304 +++++++++++++++--------------- fs/btrfs/subpage.h | 74 ++++---- 15 files changed, 499 insertions(+), 462 deletions(-)
Signature · 1 line
2.43.0
PATCH 1/3 btrfs: migrate get_eb_page_index() and get_eb_offset_in_page() to folios
These two functions are still using the old page based code, which is not going to handle larger folios at all.
The migration itself is going to involve the following changes:
- PAGE_SIZE -> folio_size()
- PAGE_SHIFT -> folio_shift()
- get_eb_page_index() -> get_eb_folio_index()
- get_eb_offset_in_page() -> get_eb_offset_in_folio()
And since we're going to support larger folios, although above straight convert is good enough, this patch would add extra comments in the involved functions to explain why the same single line code can now cover 3 cases:
- folio_size == PAGE_SIZE, sectorsize == PAGE_SIZE, nodesize >= PAGE_SIZE The common, non-subpage case with per-page folio.
- folio_size > PAGE_SIZE, sectorsize == PAGE_SIZE, nodesize >= PAGE_SIZE The incoming larger folio, non-subpage case.
- folio_size == PAGE_SIZE, sectorsize < PAGE_SIZE, nodesize < PAGE_SIZE The existing subpage case, we won't larger folio anyway.
--- fs/btrfs/accessors.c | 80 +++++++++++++++------------ fs/btrfs/ctree.c | 13 ++--- fs/btrfs/disk-io.c | 2 +- fs/btrfs/extent_io.c | 126 ++++++++++++++++++++++--------------------- fs/btrfs/extent_io.h | 40 +++++++++----- 5 files changed, 145 insertions(+), 116 deletions(-)
fs/btrfs/accessors.c +46 -34
@@ -59,29 +59,32 @@ void btrfs_init_map_token(struct btrfs_map_token *token, struct extent_buffer *e u##bits btrfs_get_token_##bits(struct btrfs_map_token *token, \ const void *ptr, unsigned long off) \ { \ + struct extent_buffer *eb = token->eb; \ const unsigned long member_offset = (unsigned long)ptr + off; \ - const unsigned long idx = get_eb_page_index(member_offset); \ - const unsigned long oip = get_eb_offset_in_page(token->eb, \ - member_offset); \ + const unsigned long idx = get_eb_folio_index(eb, member_offset);\ + const unsigned long oil = get_eb_offset_in_folio(eb, \ + member_offset);\ + const int unit_size = folio_size(eb->folios[0]); \ + const int unit_shift = folio_shift(eb->folios[0]); \ const int size = sizeof(u##bits); \ u8 lebytes[sizeof(u##bits)]; \ - const int part = PAGE_SIZE - oip; \ + const int part = unit_size - oil; \ \ ASSERT(token); \ ASSERT(token->kaddr); \ ASSERT(check_setget_bounds(token->eb, ptr, off, size)); \ if (token->offset <= member_offset && \ - member_offset + size <= token->offset + PAGE_SIZE) { \ - return get_unaligned_le##bits(token->kaddr + oip); \ + member_offset + size <= token->offset + unit_size) { \ + return get_unaligned_le##bits(token->kaddr + oil); \ } \ token->kaddr = folio_address(token->eb->folios[idx]); \ - token->offset = idx << PAGE_SHIFT; \ - if (INLINE_EXTENT_BUFFER_PAGES == 1 || oip + size <= PAGE_SIZE ) \ - return get_unaligned_le##bits(token->kaddr + oip); \ + token->offset = idx << unit_shift; \ + if (INLINE_EXTENT_BUFFER_PAGES == 1 || oil + size <= unit_size) \ + return get_unaligned_le##bits(token->kaddr + oil); \ \ - memcpy(lebytes, token->kaddr + oip, part); \ + memcpy(lebytes, token->kaddr + oil, part); \ token->kaddr = folio_address(token->eb->folios[idx + 1]); \ - token->offset = (idx + 1) << PAGE_SHIFT; \ + token->offset = (idx + 1) << unit_shift; \ memcpy(lebytes + part, token->kaddr, size - part); \ return get_unaligned_le##bits(lebytes); \ } \
@@ -89,18 +92,20 @@ u##bits btrfs_get_##bits(const struct extent_buffer *eb, \ const void *ptr, unsigned long off) \ { \const unsigned long member_offset = (unsigned long )ptr + off; \ -const unsigned long oip =get_eb_offset_in_page (eb, member_offset); \ -const unsigned long idx =get_eb_page_index (member_offset); \ +const unsigned long idx =get_eb_folio_index (eb, member_offset);\ +const unsigned long oil =get_eb_offset_in_folio (eb, \ + member_offset);\ +const int unit_size =folio_size (eb->folios[0 ]); \char *kaddr =folio_address (eb->folios[idx]); \ const int size = sizeof(u##bits); \ - const int part = PAGE_SIZE - oip; \ + const int part = unit_size - oil; \ u8 lebytes[sizeof(u##bits)]; \ \ ASSERT(check_setget_bounds(eb, ptr, off, size)); \ - if (INLINE_EXTENT_BUFFER_PAGES == 1 || oip + size <= PAGE_SIZE) \ - return get_unaligned_le##bits(kaddr + oip); \ + if (INLINE_EXTENT_BUFFER_PAGES == 1 || oil + size <= unit_size) \ + return get_unaligned_le##bits(kaddr + oil); \ \ - memcpy(lebytes, kaddr + oip, part); \ + memcpy(lebytes, kaddr + oil, part); \ kaddr = folio_address(eb->folios[idx + 1]); \ memcpy(lebytes + part, kaddr, size - part); \ return get_unaligned_le##bits(lebytes); \
@@ -109,53 +114,60 @@ void btrfs_set_token_##bits(struct btrfs_map_token *token, \ const void *ptr, unsigned long off, \ u##bits val) \ { \ + struct extent_buffer *eb = token->eb; \ const unsigned long member_offset = (unsigned long)ptr + off; \ - const unsigned long idx = get_eb_page_index(member_offset); \ - const unsigned long oip = get_eb_offset_in_page(token->eb, \ - member_offset); \ + const unsigned long idx = get_eb_folio_index(eb, member_offset);\ + const unsigned long oil = get_eb_offset_in_folio(eb, \ + member_offset);\ + const int unit_size = folio_size(eb->folios[0]); \ + const int unit_shift = folio_shift(eb->folios[0]); \ const int size = sizeof(u##bits); \ u8 lebytes[sizeof(u##bits)]; \ - const int part = PAGE_SIZE - oip; \ + const int part = unit_size - oil; \ \ ASSERT(token); \ ASSERT(token->kaddr); \ ASSERT(check_setget_bounds(token->eb, ptr, off, size)); \ if (token->offset <= member_offset && \ - member_offset + size <= token->offset + PAGE_SIZE) { \ - put_unaligned_le##bits(val, token->kaddr + oip); \ + member_offset + size <= token->offset + unit_size) { \ + put_unaligned_le##bits(val, token->kaddr + oil); \ return; \ } \ token->kaddr = folio_address(token->eb->folios[idx]); \ - token->offset = idx << PAGE_SHIFT; \ - if (INLINE_EXTENT_BUFFER_PAGES == 1 || oip + size <= PAGE_SIZE) { \ - put_unaligned_le##bits(val, token->kaddr + oip); \ + token->offset = idx << unit_shift; \ + if (INLINE_EXTENT_BUFFER_PAGES == 1 || \ + oil + size <= unit_size) { \ + put_unaligned_le##bits(val, token->kaddr + oil); \ return; \ } \ put_unaligned_le##bits(val, lebytes); \ - memcpy(token->kaddr + oip, lebytes, part); \ + memcpy(token->kaddr + oil, lebytes, part); \ token->kaddr = folio_address(token->eb->folios[idx + 1]); \ - token->offset = (idx + 1) << PAGE_SHIFT; \ + token->offset = (idx + 1) << unit_shift; \ memcpy(token->kaddr, lebytes + part, size - part); \ } \ void btrfs_set_##bits(const struct extent_buffer *eb, void *ptr, \ unsigned long off, u##bits val) \ { \ const unsigned long member_offset = (unsigned long)ptr + off; \ - const unsigned long oip = get_eb_offset_in_page(eb, member_offset); \ - const unsigned long idx = get_eb_page_index(member_offset); \ + const unsigned long idx = get_eb_folio_index(eb, member_offset);\ + const unsigned long oil = get_eb_offset_in_folio(eb, \ + member_offset);\ + const int unit_size = folio_size(eb->folios[0]); \ char *kaddr = folio_address(eb->folios[idx]); \ const int size = sizeof(u##bits); \ - const int part = PAGE_SIZE - oip; \ + const int part = unit_size - oil; \ u8 lebytes[sizeof(u##bits)]; \ \ ASSERT(check_setget_bounds(eb, ptr, off, size)); \ - if (INLINE_EXTENT_BUFFER_PAGES == 1 || oip + size <= PAGE_SIZE) { \ - put_unaligned_le##bits(val, kaddr + oip); \ + if (INLINE_EXTENT_BUFFER_PAGES == 1 || \ + oil + size <= unit_size) { \ + put_unaligned_le##bits(val, kaddr + oil); \ return; \ } \ \ put_unaligned_le##bits(val, lebytes); \ - memcpy(kaddr + oip, lebytes, part); \ + memcpy(kaddr + oil, lebytes, part); \ kaddr = folio_address(eb->folios[idx + 1]); \ memcpy(kaddr, lebytes + part, size - part); \ }
fs/btrfs/ctree.c +7 -6
@@ -820,7 +820,8 @@ int btrfs_bin_search(struct extent_buffer *eb, int first_slot, }while (low < high) { -unsigned long oip; +const int unit_size =folio_size (eb->folios[0 ]); +unsigned long oil;unsigned long offset;struct btrfs_disk_key *tmp;struct btrfs_disk_key unaligned;
@@ -828,14 +829,14 @@ int btrfs_bin_search(struct extent_buffer *eb, int first_slot, mid = (low + high) /2 ; offset = p + mid * item_size; - oip =offset_in_page (offset); + oil =get_eb_offset_in_folio (eb, offset); -if (oip + key_size <=PAGE_SIZE ) { -const unsigned long idx =get_eb_page_index (offset); +if (oil + key_size <= unit_size) { +const unsigned long idx =get_eb_folio_index (eb, offset);char *kaddr =folio_address (eb->folios[idx]); - oip =get_eb_offset_in_page (eb, offset); - tmp = (struct btrfs_disk_key *)(kaddr + oip); + oil =get_eb_offset_in_folio (eb, offset); + tmp = (struct btrfs_disk_key *)(kaddr + oil); }else {read_extent_buffer (eb, &unaligned, offset, key_size); tmp = &unaligned;
fs/btrfs/disk-io.c +1 -1
@@ -395,7 +395,7 @@ int btrfs_validate_extent_buffer(struct extent_buffer *eb,csum_tree_block (eb, result); header_csum =folio_address (eb->folios[0 ]) + -get_eb_offset_in_page (eb, offsetof(struct btrfs_header , csum)); +get_eb_offset_in_folio (eb, offsetof(struct btrfs_header , csum));if (memcmp (result, header_csum, csum_size) !=0 ) { btrfs_warn_rl(fs_info,
fs/btrfs/extent_io.c +64 -62
303 lines not on this page · raw message
fs/btrfs/extent_io.h +27 -13
@@ -121,29 +121,43 @@ struct btrfs_eb_write_context {* * Will handle both sectorsize == PAGE_SIZE and sectorsize < PAGE_SIZE cases. */ -static inline size_t get_eb_offset_in_page (const struct extent_buffer *eb, -unsigned long offset) +static inline size_t get_eb_offset_in_folio (const struct extent_buffer *eb, +unsigned long offset) {/* -* For sectorsize == PAGE_SIZE case, eb->start will always be aligned -* to PAGE_SIZE, thus adding it won't cause any difference. +* 1) sectorsize == PAGE_SIZE and nodesize >= PAGE_SIZE case +* 1.1) One large folio covering the whole eb +* The eb->start is aligned to folio size, thus adding it +* won't cause any difference. +* 1.2) Several page sized folios +* The eb->start is aligned to folio (page) size, thus +* adding it won't cause any difference. * -* For sectorsize < PAGE_SIZE, we must only read the data that belongs -* to the eb, thus we have to take the eb->start into consideration. +* 2) sectorsize < PAGE_SIZE and nodesize < PAGE_SIZE case +* In this case there would only be one page sized folio, and there +* may be several different extent buffers in the page/folio. +* We need to add eb->start to properly access the offset inside +* that eb. */ -return offset_in_page (offset + eb->start); +return offset_in_folio (eb->folios[0 ], offset + eb->start); } -static inline unsigned long get_eb_page_index (unsigned long offset) +static inline unsigned long get_eb_folio_index (const struct extent_buffer *eb, +unsigned long offset) {/* -* For sectorsize == PAGE_SIZE case, plain >> PAGE_SHIFT is enough. +* 1) sectorsize == PAGE_SIZE and nodesize >= PAGE_SIZE case +* 1.1) One large folio covering the whole eb. +* the folio_shift would be large enough to always make us +* return 0 as index. +* 1.2) Several page sized folios +* The folio_shift() would be PAGE_SHIFT, giving us the correct +* index. * -* For sectorsize < PAGE_SIZE case, we only support 64K PAGE_SIZE, -* and have ensured that all tree blocks are contained in one page, -* thus we always get index == 0. +* 2) sectorsize < PAGE_SIZE and nodesize < PAGE_SIZE case +* The folio would only be page sized, and always give us 0 as index. */ -return offset >>PAGE_SHIFT ; +return offset >>folio_shift (eb->folios[0 ]); }/*
Signature · 1 line
2.43.0
PATCH 2/3 btrfs: migrate subpage code to folio interfaces
In reply to Qu WenruoAlthough subpage itself is conflicting with higher folio folio, since subpage (sectorsize < PAGE_SIZE and nodesize < PAGE_SIZE) means we will never need higher order folio, there is a hidden pitfall:
- btrfs_page_*() helpers
Those helpers are an abstraction to handle both subpage and non-subpage cases, which means we're going to pass pages pointers to those helpers.
And since those helpers are shared between data and metadata paths, it's unavoidable to let them to handle folios, including higher order folios).
Meanwhile for true subpage case, we should only have a single page backed folios anyway, thus add a new ASSERT() for btrfs_subpage_assert() to ensure that.
Also since those helpers are shared between both data and metadata, add some extra ASSERT()s for data path to make sure we only get single page backed folio for now.
--- fs/btrfs/compression.c | 7 +- fs/btrfs/defrag.c | 3 +- fs/btrfs/disk-io.c | 2 +- fs/btrfs/extent_io.c | 97 ++++++------ fs/btrfs/file.c | 13 +- fs/btrfs/free-space-cache.c | 4 +- fs/btrfs/inode.c | 34 ++-- fs/btrfs/ordered-data.c | 5 +- fs/btrfs/reflink.c | 6 +- fs/btrfs/relocation.c | 5 +- fs/btrfs/subpage.c | 304 +++++++++++++++++------------------- fs/btrfs/subpage.h | 74 ++++----- 12 files changed, 279 insertions(+), 275 deletions(-)
fs/btrfs/compression.c +4 -3
@@ -306,8 +306,8 @@ static noinline void end_compressed_writeback(const struct compressed_bio *cb)for (i =0 ; i < ret; i++) {struct folio *folio = fbatch.folios[i]; -btrfs_page_clamp_clear_writeback (fs_info, &folio->page, - cb->start, cb->len); +btrfs_folio_clamp_clear_writeback (fs_info, folio, + cb->start, cb->len); }folio_batch_release (&fbatch); }
@@ -541,7 +541,8 @@ static noinline int add_ra_bio_pages(struct inode *inode,* subpage::readers and to unlock the page. */ if (fs_info->sectorsize <PAGE_SIZE ) -btrfs_subpage_start_reader (fs_info, page, cur, add_size); +btrfs_subpage_start_reader (fs_info,page_folio (page), + cur, add_size);put_page (page); cur += add_size; }
fs/btrfs/defrag.c +2 -1
@@ -1189,7 +1189,8 @@ static int defrag_one_locked_target(struct btrfs_inode *inode,/* Update the page status */ for (i = start_index - first_index; i <= last_index - first_index; i++) {ClearPageChecked (pages[i]); -btrfs_page_clamp_set_dirty (fs_info, pages[i], start, len); +btrfs_folio_clamp_set_dirty (fs_info,page_folio (pages[i]), + start, len); }btrfs_delalloc_release_extents (inode, len);extent_changeset_free (data_reserved);
fs/btrfs/disk-io.c +1 -1
@@ -284,7 +284,7 @@ blk_status_t btree_csum_one_bio(struct btrfs_bio *bbio)if (WARN_ON_ONCE (found_start != eb->start))return BLK_STS_IOERR ; -if (WARN_ON (!btrfs_page_test_uptodate (fs_info,folio_page (eb->folios[0 ],0 ), +if (WARN_ON (!btrfs_folio_test_uptodate (fs_info, eb->folios[0 ], eb->start, eb->len)))return BLK_STS_IOERR ;
fs/btrfs/extent_io.c +51 -46
281 lines not on this page · raw message
fs/btrfs/file.c +8 -5
@@ -111,8 +111,8 @@ static void btrfs_drop_pages(struct btrfs_fs_info *fs_info,* accessed as prepare_pages should have marked them accessed * in prepare_pages via find_or_create_page() */ -btrfs_page_clamp_clear_checked (fs_info, pages[i], block_start, - block_len); +btrfs_folio_clamp_clear_checked (fs_info,page_folio (pages[i]), + block_start, block_len);unlock_page (pages[i]);put_page (pages[i]); }
@@ -168,9 +168,12 @@ int btrfs_dirty_pages(struct btrfs_inode *inode, struct page **pages,for (i =0 ; i < num_pages; i++) {struct page *p = pages[i]; -btrfs_page_clamp_set_uptodate (fs_info, p, start_pos, num_bytes); -btrfs_page_clamp_clear_checked (fs_info, p, start_pos, num_bytes); -btrfs_page_clamp_set_dirty (fs_info, p, start_pos, num_bytes); +btrfs_folio_clamp_set_uptodate (fs_info,page_folio (p), + start_pos, num_bytes); +btrfs_folio_clamp_clear_checked (fs_info,page_folio (p), + start_pos, num_bytes); +btrfs_folio_clamp_set_dirty (fs_info,page_folio (p), + start_pos, num_bytes); }/*
fs/btrfs/free-space-cache.c +2 -2
@@ -439,8 +439,8 @@ static void io_ctl_drop_pages(struct btrfs_io_ctl *io_ctl)for (i =0 ; i < io_ctl->num_pages; i++) {if (io_ctl->pages[i]) { -btrfs_page_clear_checked (io_ctl->fs_info, - io_ctl->pages[i], +btrfs_folio_clear_checked (io_ctl->fs_info, +page_folio (io_ctl->pages[i]),page_offset (io_ctl->pages[i]),PAGE_SIZE );unlock_page (io_ctl->pages[i]);
fs/btrfs/inode.c +20 -14
@@ -456,8 +456,8 @@ static inline void btrfs_cleanup_ordered_extents(struct btrfs_inode *inode,* range, then btrfs_mark_ordered_io_finished() will handle * the ordered extent accounting for the range. */ -btrfs_page_clamp_clear_ordered (inode->root->fs_info, page, - offset, bytes); +btrfs_folio_clamp_clear_ordered (inode->root->fs_info, +page_folio (page), offset, bytes);put_page (page); }
@@ -2802,7 +2802,7 @@ static void btrfs_writepage_fixup_worker(struct btrfs_work *work) PAGE_SIZE, !ret);clear_page_dirty_for_io (page); } -btrfs_page_clear_checked (fs_info, page, page_start,PAGE_SIZE ); +btrfs_folio_clear_checked (fs_info,page_folio (page), page_start,PAGE_SIZE );unlock_page (page);put_page (page);kfree (fixup);
@@ -2857,7 +2857,7 @@ int btrfs_writepage_cow_fixup(struct page *page)* page->mapping outside of the page lock. */ ihold (inode); -btrfs_page_set_checked (fs_info, page,page_offset (page),PAGE_SIZE ); +btrfs_folio_set_checked (fs_info,page_folio (page),page_offset (page),PAGE_SIZE );get_page (page);btrfs_init_work (&fixup->work, btrfs_writepage_fixup_worker,NULL ); fixup->page = page;
@@ -4776,9 +4776,10 @@ int btrfs_truncate_block(struct btrfs_inode *inode, loff_t from, loff_t len,memzero_page (page, (block_start -page_offset (page)) + offset, len); } -btrfs_page_clear_checked (fs_info, page, block_start, - block_end +1 - block_start); -btrfs_page_set_dirty (fs_info, page, block_start, block_end +1 - block_start); +btrfs_folio_clear_checked (fs_info,page_folio (page), block_start, + block_end +1 - block_start); +btrfs_folio_set_dirty (fs_info,page_folio (page), block_start, + block_end +1 - block_start);unlock_extent (io_tree, block_start, block_end, &cached_state);if (only_release_metadata)
@@ -8005,7 +8006,7 @@ static void btrfs_invalidate_folio(struct folio *folio, size_t offset, page_end);ASSERT (range_end +1 - cur <U32_MAX ); range_len = range_end +1 - cur; -if (!btrfs_page_test_ordered (fs_info, &folio->page, cur, range_len)) { +if (!btrfs_folio_test_ordered (fs_info, folio, cur, range_len)) {/* * If Ordered (Private2) is cleared, it means endio has * already been executed for the range.
@@ -8014,7 +8015,7 @@ static void btrfs_invalidate_folio(struct folio *folio, size_t offset,*/ goto next; } -btrfs_page_clear_ordered (fs_info, &folio->page, cur, range_len); +btrfs_folio_clear_ordered (fs_info, folio, cur, range_len);/* * IO on this page will never be started, so we need to account
@@ -8084,7 +8085,7 @@ static void btrfs_invalidate_folio(struct folio *folio, size_t offset,* did something wrong. */ ASSERT (!folio_test_ordered (folio)); -btrfs_page_clear_checked (fs_info, &folio->page,folio_pos (folio),folio_size (folio)); +btrfs_folio_clear_checked (fs_info, folio,folio_pos (folio),folio_size (folio));if (!inode_evicting)__btrfs_release_folio (folio,GFP_NOFS );clear_page_extent_mapped (&folio->page);
@@ -8108,6 +8109,7 @@ static void btrfs_invalidate_folio(struct folio *folio, size_t offset,vm_fault_t btrfs_page_mkwrite (struct vm_fault *vmf) {struct page *page = vmf->page; +struct folio *folio =page_folio (page);struct inode *inode =file_inode (vmf->vma->vm_file);struct btrfs_fs_info *fs_info =btrfs_sb (inode->i_sb);struct extent_io_tree *io_tree = &BTRFS_I (inode)->io_tree;
@@ -8124,6 +8126,8 @@ vm_fault_t btrfs_page_mkwrite(struct vm_fault *vmf)u64 page_end;u64 end; +ASSERT (folio_order (folio) ==0 ); + reserved_space =PAGE_SIZE ;sb_start_pagefault (inode->i_sb);
@@ -8227,9 +8231,9 @@ vm_fault_t btrfs_page_mkwrite(struct vm_fault *vmf)if (zero_start !=PAGE_SIZE )memzero_page (page, zero_start,PAGE_SIZE - zero_start); -btrfs_page_clear_checked (fs_info, page, page_start,PAGE_SIZE ); -btrfs_page_set_dirty (fs_info, page, page_start, end +1 - page_start); -btrfs_page_set_uptodate (fs_info, page, page_start, end +1 - page_start); +btrfs_folio_clear_checked (fs_info, folio, page_start,PAGE_SIZE ); +btrfs_folio_set_dirty (fs_info, folio, page_start, end +1 - page_start); +btrfs_folio_set_uptodate (fs_info, folio, page_start, end +1 - page_start);btrfs_set_inode_last_sub_trans (BTRFS_I (inode));
@@ -9800,7 +9804,9 @@ void btrfs_set_range_writeback(struct btrfs_inode *inode, u64 start, u64 end) page =find_get_page (inode->vfs_inode.i_mapping, index);ASSERT (page);/* Pages should be in the extent_io_tree */ -btrfs_page_set_writeback (fs_info, page, start, len); +/* This is for data, which doesn't yet support larger folio. */ +ASSERT (folio_order (page_folio (page)) ==0 ); +btrfs_folio_set_writeback (fs_info,page_folio (page), start, len);put_page (page); index++; }
fs/btrfs/ordered-data.c +3 -2
@@ -322,9 +322,10 @@ static bool can_finish_ordered_extent(struct btrfs_ordered_extent *ordered,* * If there's no such bit, we need to skip to next range. */ -if (!btrfs_page_test_ordered (fs_info, page, file_offset, len)) +if (!btrfs_folio_test_ordered (fs_info,page_folio (page), + file_offset, len))return false; -btrfs_page_clear_ordered (fs_info, page, file_offset, len); +btrfs_folio_clear_ordered (fs_info,page_folio (page), file_offset, len); }/* Now we're fine to update the accounting. */
fs/btrfs/reflink.c +3 -3
@@ -141,9 +141,9 @@ static int copy_inline_to_page(struct btrfs_inode *inode,if (datal < block_size)memzero_page (page, datal, block_size - datal); -btrfs_page_set_uptodate (fs_info, page, file_offset, block_size); -btrfs_page_clear_checked (fs_info, page, file_offset, block_size); -btrfs_page_set_dirty (fs_info, page, file_offset, block_size); +btrfs_folio_set_uptodate (fs_info,page_folio (page), file_offset, block_size); +btrfs_folio_clear_checked (fs_info,page_folio (page), file_offset, block_size); +btrfs_folio_set_dirty (fs_info,page_folio (page), file_offset, block_size); out_unlock:if (page) {unlock_page (page);
fs/btrfs/relocation.c +3 -2
@@ -2895,7 +2895,7 @@ static noinline_for_stack int prealloc_file_extent_cluster(* will re-read the whole page anyway. */ if (page) { -btrfs_subpage_clear_uptodate (fs_info, page, i_size, +btrfs_subpage_clear_uptodate (fs_info,page_folio (page), i_size,round_up (i_size,PAGE_SIZE ) - i_size);unlock_page (page);put_page (page);
@@ -3070,7 +3070,8 @@ static int relocate_one_page(struct inode *inode, struct file_ra_state *ra, clamped_len); goto release_page; } -btrfs_page_set_dirty (fs_info, page, clamped_start, clamped_len); +btrfs_folio_set_dirty (fs_info,page_folio (page), + clamped_start, clamped_len);/* * Set the boundary if it's inside the page.
fs/btrfs/subpage.c +145 -159
635 lines not on this page · raw message
fs/btrfs/subpage.h +37 -37
111 lines not on this page · raw message
Signature · 1 line
2.43.0
PATCH 3/3 btrfs: migrate various btrfs end io functions to folios
In reply to Qu WenruoIf we still go the old page based iterator functions, like bio_for_each_segment_all(), we can hit middle pages of a folio (compound page).
In that case if we set any page flag on those middle pages, we can easily trigger VM_BUG_ON(), as for compound page flags, they should follow their flag policies (normally only set on leading or tail pages).
To avoid such problem in the future full folio migration, here we do:
- Change from bio_for_each_segment_all() to bio_for_each_folio_all() This completely removes the ability to access the middle page.
- Add extra ASSERT()s for data read/write paths To ensure we only get single paged folio for data now.
- Rename those end io functions to follow a certain schema
* end_bbio_compressed_read()
* end_bbio_compressed_write()
These two endio function doesn't set any page flags, as they use pages not mapped to any address space.
They can be very good candidates for higher order folio testing.
And they are shared between compression and encoded IO.
* end_bbio_data_read()
* end_bbio_data_write()
* end_bbio_meta_read()
* end_bbio_meta_write()
The old function names are a disaster:
- end_bio_extent_writepage()
- end_bio_extent_readpage()
- extent_buffer_write_end_io()
- extent_buffer_read_end_io()
They share no schema on where the "end_*io" string should be, nor can be confusing just using "extent_buffer" and "extent" to distinguish data and metadata paths.
--- fs/btrfs/compression.c | 8 +-- fs/btrfs/extent_io.c | 148 +++++++++++++++++++++-------------------- 2 files changed, 80 insertions(+), 76 deletions(-)
fs/btrfs/compression.c +4 -4
@@ -264,7 +264,7 @@ void btrfs_free_compr_page(struct page *page)put_page (page); } -static void end_compressed_bio_read (struct btrfs_bio *bbio) +static void end_bbio_comprssed_read (struct btrfs_bio *bbio) {struct compressed_bio *cb =to_compressed_bio (bbio);blk_status_t status = bbio->bio.bi_status;
@@ -337,7 +337,7 @@ static void btrfs_finish_compressed_write_work(struct work_struct *work)* This also calls the writeback end hooks for the file pages so that metadata * and checksums can be updated in the file. */ -static void end_compressed_bio_write (struct btrfs_bio *bbio) +static void end_bbio_comprssed_write (struct btrfs_bio *bbio) {struct compressed_bio *cb =to_compressed_bio (bbio);struct btrfs_fs_info *fs_info = bbio->inode->root->fs_info;
@@ -384,7 +384,7 @@ void btrfs_submit_compressed_write(struct btrfs_ordered_extent *ordered, cb =alloc_compressed_bio (inode, ordered->file_offset,REQ_OP_WRITE | write_flags, - end_compressed_bio_write); + end_bbio_comprssed_write); cb->start = ordered->file_offset; cb->len = ordered->num_bytes; cb->compressed_pages = compressed_pages;
@@ -589,7 +589,7 @@ void btrfs_submit_compressed_read(struct btrfs_bio *bbio) compressed_len = em->block_len; cb =alloc_compressed_bio (inode, file_offset,REQ_OP_READ , - end_compressed_bio_read); + end_bbio_comprssed_read); cb->start = em->orig_start; em_len = em->len;
fs/btrfs/extent_io.c +76 -72
286 lines not on this page · raw message
Signature · 1 line
2.43.0