Skip to content

btrfs: read repair/direct I/O improvements

Omar Sandoval linux-btrfs 2 versions – In review

Get series with b4
b4 am [email protected]

Versions newest first · linked by author and title

  1. v2 Latest15 patches 21 repliesIn review
  2. v115 patches 53 repliesSuperseded

Patches in v2 No applies recorded yet

#Subject and filesSizeRepliesApplied
1/15block: add bio_for_each_bvec_all().clang-format Documentation/block/biovecs.rst include/linux/bio.h+11 -01 reply
2/15btrfs: fix error handling when submitting direct I/O biofs/btrfs/inode.c+3 -31 reply
3/15btrfs: fix double __endio_write_update_ordered in direct I/Ofs/btrfs/inode.c+66 -1084 replies
4/15btrfs: look at full bi_io_vec for repair decisionfs/btrfs/extent_io.c fs/btrfs/extent_io.h+30 -81 reply
5/15btrfs: don't do repair validation for checksum errorsfs/btrfs/extent_io.c+8 -01 reply
6/15btrfs: clarify btrfs_lookup_bio_sums documentationfs/btrfs/file-item.c+7 -42 replies
7/15btrfs: rename __readpage_endio_check to check_data_csumfs/btrfs/inode.c+11 -110 replies
8/15btrfs: make btrfs_check_repairable() staticfs/btrfs/extent_io.c fs/btrfs/extent_io.h+4 -60 replies
9/15btrfs: kill btrfs_dio_private->privatefs/btrfs/btrfs_inode.h fs/btrfs/inode.c+0 -21 reply
10/15btrfs: convert btrfs_dio_private->pending_bios to refcount_tfs/btrfs/btrfs_inode.h fs/btrfs/inode.c+11 -71 reply
11/15btrfs: put direct I/O checksums in btrfs_dio_private instead of biofs/btrfs/btrfs_inode.h fs/btrfs/inode.c+34 -392 replies
12/15btrfs: get rid of one layer of bios in direct I/Ofs/btrfs/btrfs_inode.h fs/btrfs/inode.c+65 -1362 replies
13/15btrfs: simplify direct I/O read repairfs/btrfs/extent_io.c fs/btrfs/inode.c+44 -2262 replies
14/15btrfs: get rid of endio_repair_workersfs/btrfs/ctree.h fs/btrfs/disk-io.c fs/btrfs/disk-io.h +1 more+2 -100 replies
15/15btrfs: unify buffered and direct I/O read repairfs/btrfs/extent_io.c fs/btrfs/extent_io.h fs/btrfs/inode.c+82 -1681 reply

Reviews trailers per patch

All 15 patches have a Reviewed-by or Acked-by.

PatchNikolay BorisovJosef BacikJohannes Thumshirn
1/15R-b
2/15R-b R-b R-b
3/15R-b R-b
4/15R-b
5/15R-b R-b
6/15R-b R-b R-b
7/15R-b R-b R-b
8/15R-b R-b R-b
9/15R-b R-b R-b
10/15R-b R-b R-b
11/15R-b R-b R-b
12/15R-b
13/15R-b R-b
14/15R-b
15/15R-b

Trailers given in replies are filled; those carried in the patches from earlier versions are outlined with dashes. Scroll the table sideways for all 3 reviewers.

Read the v2 thread

Cover letter [PATCH v2 00/15] btrfs: read repair/direct I/O improvements

From: Omar Sandoval <[email protected]>

Hi,

This is version 2 of my series of fixes, cleanups, and improvements to direct I/O and read repair. It's preparation for adding read repair to my RWF_ENCODED series [1], but it can go in independently.

Patch 1 adds a new bio helper needed for patch 4. Patches 2 and 3 are direct I/O error handling fixes. Patch 4 is a buffered read repair fix. Patch 5 is a buffered read repair improvement. Patches 6-9 are trivial cleanups. Patch 10 converts direct I/O to use refcount_t, which would've helped catch the bug fixed by patch 2. Patches 11-14 drastically simplify the direct I/O code. Patch 15 unifies buffered and direct I/O read repair, which also makes direct I/O repair actually do validation for large failed reads instead of rewriting the whole thing.

Overall, this is net about -350 lines of code and actually makes direct I/O more functional.

Note that this series causes btrfs/142 to fail. This is a bug in the test, as it assumes that direct I/O doesn't do read validation. This is fixed by the fstests patch "btrfs/14{2,3}: use dm-dust instead of fail_make_request" which I sent up yesterday [2].

Jens and Christoph are cc'd for patches 1 and 4. Instead of looking at the bio internals like I did in v1, I added a new bio_for_each_bvec_all() helper.

Changes from v1 [3]:

* Added patch 1 with bio_for_each_bvec_all() helper
* Dropped patch 8 which moved struct definition
* Fixed performance regression [4] in patch 13 caused by accidentally making all direct I/O submission asynchronous
* Fixed uninitialized assert in patch 12
* Fixed misplaced assert in btrfs_check_read_dio_bio in patch 13
* Added reviewed-bys
* Refactored btrfs_submit_direct() and btrfs_submit_direct_hook() in patch 2 (I didn't add Nikolay's reviewed-by to that one because the new patch looks fairly different from patch 1 in v1)
* Rewrapped csum calculations in patch 10 for easier readability
* Clarified several commit messages and comments

This series is based on misc-next.

Thanks!

1: https://lore.kernel.org/linux-fsdevel/[email protected]/
2: https://lore.kernel.org/fstests/d992390752c612acd0893ee3db929e77affded2b.1586983958.git.osandov@fb.com/
3: https://lore.kernel.org/linux-btrfs/[email protected]/
4: https://lore.kernel.org/lkml/20200331090145.GH11705@shao2-debian/

Omar Sandoval (15):
block: add bio_for_each_bvec_all()
btrfs: fix error handling when submitting direct I/O bio
btrfs: fix double __endio_write_update_ordered in direct I/O
btrfs: look at full bi_io_vec for repair decision
btrfs: don't do repair validation for checksum errors
btrfs: clarify btrfs_lookup_bio_sums documentation
btrfs: rename __readpage_endio_check to check_data_csum
btrfs: make btrfs_check_repairable() static
btrfs: kill btrfs_dio_private->private
btrfs: convert btrfs_dio_private->pending_bios to refcount_t
btrfs: put direct I/O checksums in btrfs_dio_private instead of bio
btrfs: get rid of one layer of bios in direct I/O
btrfs: simplify direct I/O read repair
btrfs: get rid of endio_repair_workers
btrfs: unify buffered and direct I/O read repair

 .clang-format                   |   1 +
 Documentation/block/biovecs.rst |   2 +
 fs/btrfs/btrfs_inode.h          |  26 +-
 fs/btrfs/ctree.h                |   1 -
 fs/btrfs/disk-io.c              |   8 +-
 fs/btrfs/disk-io.h              |   1 -
 fs/btrfs/extent_io.c            | 152 ++++---
 fs/btrfs/extent_io.h            |  19 +-
 fs/btrfs/file-item.c            |  11 +-
 fs/btrfs/inode.c                | 745 ++++++++------------------------
 include/linux/bio.h             |   8 +
 11 files changed, 312 insertions(+), 662 deletions(-)
Signature · 1 line
2.26.1