Skip to content

btrfs: preparation patches for subpage support

Qu Wenruo linux-btrfs oe-kbuild-all 3 versions – In review

Get series with b4
b4 am [email protected]

Versions newest first · linked by author and title

  1. v3 Latest15 patches 3 repliesIn review
  2. v224 patches 42 repliesSuperseded
  3. v11 patch 1 replySuperseded

Changes in v3 from the cover letter

- Rebased to latest misc-next
Now only 15 patches to submit.

- Add two new patches to address u32 and u64 problems
The root problem is the on-disk format is abusing u64 for its length. We have to draw a line between where we should convert to u32. Currently for bio_offset and extent_len, we can safely use u32. Just to be extra safe, added more ASSERT() for this.

- Put BTRFS_MAX_METADATA_BLOCKSIZE into uapi
To avoid circle including "ctree.h"

- Add more changelog for the patch enabling subpage scrub

Patches in v3 No applies recorded yet

#Subject and filesSizeRepliesApplied
1/15btrfs: rename bio_offset of extent_submit_bio_start_t to opt_file_offsetfs/btrfs/disk-io.c fs/btrfs/disk-io.h fs/btrfs/extent_io.h +1 more+15 -162 replies
2/15btrfs: pass bio_offset to check_data_csum() directlyfs/btrfs/ctree.h fs/btrfs/extent_io.c fs/btrfs/extent_io.h +1 more+42 -260 replies
3/15btrfs: inode: make btrfs_verify_data_csum() follow sector sizefs/btrfs/inode.c+15 -30 replies
4/15btrfs: extent_io: extract the btree page submission code into its own helper functionfs/btrfs/extent_io.c+75 -470 replies
5/15btrfs: extent_io: calculate inline extent buffer page size based on page sizefs/btrfs/ctree.h fs/btrfs/extent_io.c fs/btrfs/extent_io.h +1 more+7 -140 replies
6/15btrfs: extent_io: don't allow tree block to cross page boundary for subpage supportfs/btrfs/extent_io.c+7 -00 replies
7/15btrfs: extent_io: update num_extent_pages() to support subpage sized extent bufferfs/btrfs/extent_io.h+8 -20 replies
8/15btrfs: handle sectorsize < PAGE_SIZE case for extent buffer accessorsfs/btrfs/ctree.c fs/btrfs/ctree.h fs/btrfs/extent_io.c +1 more+85 -380 replies
9/15btrfs: file-item: remove the btrfs_find_ordered_sum() call in btrfs_lookup_bio_sums()fs/btrfs/file-item.c fs/btrfs/ordered-data.c fs/btrfs/ordered-data.h+11 -510 replies
10/15btrfs: file-item: refactor btrfs_lookup_bio_sums() to handle out-of-order bvecsfs/btrfs/compression.c fs/btrfs/ctree.h fs/btrfs/file-item.c +1 more+173 -910 replies
11/15btrfs: scrub: reduce the width for extent_len/stripe_len from 64 bits to 32 bitsfs/btrfs/scrub.c+31 -230 replies
12/15btrfs: scrub: always allocate one full page for one sector for RAID56fs/btrfs/scrub.c+16 -50 replies
13/15btrfs: scrub: support subpage tree block scrubfs/btrfs/scrub.c+9 -40 replies
14/15btrfs: scrub: support subpage data scrubfs/btrfs/scrub.c+7 -30 replies
15/15btrfs: scrub: allow scrub to work with subpage sectorsizefs/btrfs/scrub.c+0 -80 replies

Reviews trailers per patch

6 of 15 patches have a Reviewed-by or Acked-by.

PatchJohannes ThumshirnNikolay BorisovJosef Bacik
1/15
2/15R-b R-b R-b
3/15R-b
4/15R-b
5/15R-b R-b
6/15
7/15R-b
8/15
9/15R-b
10/15
11/15
12/15
13/15
14/15
15/15

Trailers given in replies are filled; those carried in the patches from earlier versions are outlined with dashes. Scroll the table sideways for all 3 reviewers.

Read the v3 thread

Cover letter [PATCH v3 00/15] btrfs: preparation patches for subpage support

This is the rebased preparation branch for all patches not yet merged into misc-next.

It can be fetched from github (with experimental sector aligned data write support)
https://github.com/adam900710/linux/tree/subpage

This patchset includes all the unmerged preparation patches for subpage support.

The patchset is sent without the main core for subpage support, as myself has proven that, big patchset bombarding won't really make reviewers happy, but only make the author happy (for a very short time).

Thanks for the hard work from David, there are only 15 patches unmerged. (With 2 new small patches to address u32 u64 problem)

Patch 01~02: bio_offset related fixes. Make bio_offset to be u32.
Patch 03: Refactor metadata submission for later metadata write
support.
Patch 04~08: Metadata related refactor.
Patch 09~10: Data related refactor
Patch 11~15: Scrub related refactor and cleanup

For the scrub patch, there was a discussion with David, about whether we should use sector size as the unit for metadata scrub.

His idea is, sector size should be the minimal unit for DATA, not metadata. This indicates there is a undefined "minimal unit" of access.

But my argument is, sector size is the minimal unit for all btrfs access, current btrfs has an undefined "data size", and that "data size" must equal to sectorsize for current btrfs implementation.

Thus for "data size" < nodesize case, we should first add support for "data size" > sectorsize first.

Thus I kept the scrub patch untouched, since IMHO sector size is still the minimal unit to access, thus iterating using sectorsize is completely sane.

Changelog:
v1:
- Separate prep patches from the huge subpage patchset

- Rebased to misc-next

- Add more commit message for patch "btrfs: extent_io: remove the extent_start/extent_len for end_bio_extent_readpage()" With one runtime example to explain why we are doing the same thing.

- Fix the assert_spin_lock() usage
What we really want is lockdep_assert_held()

- Re-iterate the reason why some extent io tests are invalid This is especially important since later patches will reduce extent_buffer::pages[] to bare minimal, killing the ability to handle certain invalid extent buffers.

- Use sectorsize_bits for division
During the convert, we should only use sectorsize_bits for division, this solves the hassle on 32bit system to do division. But we should not use sectorsize_bits no brain, as bit shift is not straight forward as multiple/division.

- Address the comments for btrfs_lookup_bio_sums() cleanup patchset From naming to macro usages, all of those comments should further improve the readability.

v2:
- Remove new extent_io tree features
Now we won't utilize extent io tree for subpage support, thus new features along with some aggressive refactor is no longer needed.

- Reduce extent_io tree operations to reduce endio time latency Although extent_io tree can do a lot of things like page status, but it has obvious overhead, namingly search btree.
So keep the original behavior by only calling extent_io operation in a big extent, to reduce latency

v3:
- Rebased to latest misc-next
Now only 15 patches to submit.

- Add two new patches to address u32 and u64 problems
The root problem is the on-disk format is abusing u64 for its length. We have to draw a line between where we should convert to u32. Currently for bio_offset and extent_len, we can safely use u32. Just to be extra safe, added more ASSERT() for this.

- Put BTRFS_MAX_METADATA_BLOCKSIZE into uapi
To avoid circle including "ctree.h"

- Add more changelog for the patch enabling subpage scrub

Qu Wenruo (15):
btrfs: rename bio_offset of extent_submit_bio_start_t to
opt_file_offset
btrfs: pass bio_offset to check_data_csum() directly
btrfs: inode: make btrfs_verify_data_csum() follow sector size
btrfs: extent_io: extract the btree page submission code into its own
helper function
btrfs: extent_io: calculate inline extent buffer page size based on
page size
btrfs: extent_io: don't allow tree block to cross page boundary for
subpage support
btrfs: extent_io: update num_extent_pages() to support subpage sized
extent buffer
btrfs: handle sectorsize < PAGE_SIZE case for extent buffer accessors
btrfs: file-item: remove the btrfs_find_ordered_sum() call in
btrfs_lookup_bio_sums()
btrfs: file-item: refactor btrfs_lookup_bio_sums() to handle
out-of-order bvecs
btrfs: scrub: reduce the width for extent_len/stripe_len from 64 bits
to 32 bits
btrfs: scrub: always allocate one full page for one sector for RAID56
btrfs: scrub: support subpage tree block scrub
btrfs: scrub: support subpage data scrub
btrfs: scrub: allow scrub to work with subpage sectorsize

 fs/btrfs/compression.c          |   5 +-
 fs/btrfs/ctree.c                |   3 +-
 fs/btrfs/ctree.h                |  48 ++++--
 fs/btrfs/disk-io.c              |  17 +-
 fs/btrfs/disk-io.h              |   2 +-
 fs/btrfs/extent_io.c            | 232 +++++++++++++++++-----------
 fs/btrfs/extent_io.h            |  18 ++-
 fs/btrfs/file-item.c            | 266 +++++++++++++++++++++-----------
 fs/btrfs/inode.c                |  61 +++++---
 fs/btrfs/ordered-data.c         |  44 ------
 fs/btrfs/ordered-data.h         |   2 -
 fs/btrfs/scrub.c                | 102 +++++++-----
 fs/btrfs/struct-funcs.c         |  18 ++-
 include/uapi/linux/btrfs_tree.h |   4 +
 14 files changed, 496 insertions(+), 326 deletions(-)
Signature · 1 line
2.29.2