Skip to content

btrfs: preparation patches for the incoming metadata folio conversion

Qu Wenruo linux-btrfs 3 versions – In review

Get series with b4
b4 am [email protected]

Versions newest first · linked by author and title

  1. v3 Latest8 patches 8 repliesIn review
  2. v26 patches 22 repliesSuperseded
  3. v16 patches 2 repliesSuperseded

Changes in v3 from the cover letter

- Fix an undefined behavior bug in memcpy_extent_buffer() Unlike the name, memcpy_extent_buffer() needs to handle overlapping ranges, thus it calls copy_pages() which do overlap checks and switch to memmove() when needed.

Patches in v3 No applies recorded yet

#Subject and filesSizeRepliesApplied
1/8btrfs: tests: enhance extent buffer bitmap testsfs/btrfs/tests/extent-io-tests.c+109 -530 replies
2/8btrfs: tests: add self tests for extent buffer memory operationsfs/btrfs/tests/extent-io-tests.c+147 -00 replies
3/8btrfs: refactor extent buffer bitmaps operationsfs/btrfs/extent_io.c+67 -740 replies
4/8btrfs: use write_extent_buffer() to implement write_extent_buffer_*id()fs/btrfs/extent_io.c fs/btrfs/extent_io.h+16 -250 replies
5/8btrfs: refactor main loop in copy_extent_buffer_full()fs/btrfs/extent_io.c+9 -140 replies
6/8btrfs: copy all pages at once at the end of btrfs_clone_extent_buffer()fs/btrfs/extent_io.c+1 -10 replies
7/8btrfs: refactor main loop in memcpy_extent_buffer()fs/btrfs/extent_io.c+30 -260 replies
8/8btrfs: refactor main loop in memmove_extent_buffer()fs/btrfs/extent_io.c+17 -310 replies

Reviews trailers per patch

5 of 8 patches have a Reviewed-by or Acked-by.

PatchSweet Tea DorminyDavid Sterba
1/8R-b R-b
2/8
3/8R-b R-b
4/8R-b R-b
5/8R-b R-b
6/8R-b R-b
7/8
8/8

Trailers given in replies are filled; those carried in the patches from earlier versions are outlined with dashes.

Read the v3 thread

Cover letter [PATCH v3 0/8] btrfs: preparation patches for the incoming metadata folio conversion

[CHANGELOG]
v2:
- Define write_extent_buffer_fsid/chunk_tree_uuid() as inline helpers

v3:
- Fix an undefined behavior bug in memcpy_extent_buffer() Unlike the name, memcpy_extent_buffer() needs to handle overlapping ranges, thus it calls copy_pages() which do overlap checks and switch to memmove() when needed.

Here we introduce __write_extent_buffer() which allows us to switch to go memmove() if needed.

- Also refactor memmove_extent_buffer()
Since we have __write_extent_buffer() which can go memmove(), it's not hard to refactor memmove_extent_buffer().

But there is still a pitfall that we have to handle double page boundaries as the old behavior, explained in the last patch.

- Add selftests on extent buffer memory operations
I have failed too many times refactoring memmove_extent_buffer(), the wasted time should be a memorial for my stupidity.

[BACKGROUND]

Recently I'm checking on the feasibility on converting metadata handling to go a folio based solution.

The best part of using a single folio for metadata is, we can get rid of the complexity of cross-page handling, everything would be just a single memory operation on a continuous memory range.

[PITFALLS]

One of the biggest problem for metadata folio conversion is, we still need the current page based solution (or folios with order 0) as a fallback solution when we can not get a high order folio.

In that case, there would be a hell to handle the four different combinations (folio/folio, folio/page, page/folio, page/page) for extent buffer helpers involving two extent buffers.

Although there are some new ideas on how to handle metadata memory (e.g. go full vmallocated memory), reducing the open-coded memory handling for metadata should always be a good start point.

[OBJECTIVE]

So this patchset is the preparation to reduce direct page operations for metadata.

The patchset would do this mostly by concentrating the operations to use the common helper, write_extent_buffer() and read_extent_buffer().

For bitmap operations it's much complex, thus this patchset refactor it completely to go a 3 part solution:

- Handle the first byte
- Handle the byte aligned ranges
- Handle the last byte

This needs more complex testing (which I failed several times during development) to prevent regression, thus extent buffer bitmap selftests have been enhanced to catch all those new possible corner cases.

The same applies to memcpy_extent_buffer() and memmove_extent_buffer(). There are several pitfalls:

- memcpy_extent_buffer() name is not accurate
Unlike plain memcpy(), memcpy_extent_buffer() needs to handle overlapping ranges.

- memmove_extent_buffer() must handle double page boundaries Explained in the last patch, thus its refactor can not go the same direction as memcpy_extent_buffer()

With too many times spent on debugging memmove_extent_buffer(), a new selftest is added to prevent regression.

Qu Wenruo (8):
btrfs: tests: enhance extent buffer bitmap tests
btrfs: tests: add self tests for extent buffer memory operations
btrfs: refactor extent buffer bitmaps operations
btrfs: use write_extent_buffer() to implement
write_extent_buffer_*id()
btrfs: refactor main loop in copy_extent_buffer_full()
btrfs: copy all pages at once at the end of
btrfs_clone_extent_buffer()
btrfs: refactor main loop in memcpy_extent_buffer()
btrfs: refactor main loop in memmove_extent_buffer()

 fs/btrfs/extent_io.c             | 292 +++++++++++++----------------
 fs/btrfs/extent_io.h             |  19 +-
 fs/btrfs/tests/extent-io-tests.c | 309 +++++++++++++++++++++++++------
 3 files changed, 396 insertions(+), 224 deletions(-)
Signature · 1 line
2.41.0