move the low-level btrfs_bio code into a separate file v3
Hi all,
this small series creates a new bio.c file (and a bio.h header for it) to contain all the "storage" layer code below btrfs_submit_bio. The amount of code sitting below btrfs_submit_bio will grow a lot with the "consolidate btrfs checksumming, repair and bio splitting" series, so this pure code move series triest to prepare for that by making sure we have a neat file to add it to.
Changes since v2:
- rebased against the rcu_string changes in misc-next
Changes since v1:
- rebased to the latest misc-next branch
- added a new patch to move struct btrfs_tree_parent_check into a new header to invoide include hell
PATCH 1/3 btrfs: move struct btrfs_tree_parent_check out of disk-io.h
Move struct btrfs_tree_parent_check out of disk-io.h so that volumes.h an various .c files don't have to include disk-io.h just for it.
--- fs/btrfs/backref.c | 1 + fs/btrfs/disk-io.h | 30 +----------------------------- fs/btrfs/parent-check.h | 36 ++++++++++++++++++++++++++++++++++++ fs/btrfs/print-tree.c | 1 + fs/btrfs/qgroup.c | 1 + fs/btrfs/tree-mod-log.c | 1 + fs/btrfs/volumes.h | 2 +- 7 files changed, 42 insertions(+), 30 deletions(-) create mode 100644 fs/btrfs/parent-check.h
fs/btrfs/backref.c +1 -0
@@ -19,6 +19,7 @@#include "accessors.h" #include "extent-tree.h" #include "relocation.h" +#include "parent-check.h" /* Just arbitrary numbers so we can be sure one of these happened. */ #define BACKREF_FOUND_SHARED 6
fs/btrfs/disk-io.h +1 -29
@@ -25,37 +25,9 @@ static inline u64 btrfs_sb_offset(int mirror)return BTRFS_SUPER_INFO_OFFSET ; } -/* All the extra info needed to verify the parentness of a tree block. */ -struct btrfs_tree_parent_check { -/* -* The owner check against the tree block. -* -* Can be 0 to skip the owner check. -*/ -u64 owner_root; - -/* -* Expected transid, can be 0 to skip the check, but such skip -* should only be utlized for backref walk related code. -*/ -u64 transid; - -/* -* The expected first key. -* -* This check can be skipped if @has_first_key is false, such skip -* can happen for case where we don't have the parent node key, -* e.g. reading the tree root, doing backref walk. -*/ -struct btrfs_key first_key; -bool has_first_key; - -/* The expected level. Should always be set. */ -u8 level; -}; -struct btrfs_device ;struct btrfs_fs_devices ; +struct btrfs_tree_parent_check ;void btrfs_check_leaked_roots (struct btrfs_fs_info *fs_info);void btrfs_init_fs_info (struct btrfs_fs_info *fs_info);
fs/btrfs/parent-check.h new file +36 -0
@@ -0,0 +1,36 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +#ifndef BTRFS_PARENT_CHECK_H +#define BTRFS_PARENT_CHECK_H + +#include <uapi/linux/btrfs_tree.h> + +/* All the extra info needed to verify the parentness of a tree block. */ +struct btrfs_tree_parent_check { +/* +* The owner check against the tree block. +* +* Can be 0 to skip the owner check. +*/ +u64 owner_root; + +/* +* Expected transid, can be 0 to skip the check, but such skip +* should only be utlized for backref walk related code. +*/ +u64 transid; + +/* +* The expected first key. +* +* This check can be skipped if @has_first_key is false, such skip +* can happen for case where we don't have the parent node key, +* e.g. reading the tree root, doing backref walk. +*/ +struct btrfs_key first_key; +bool has_first_key; + +/* The expected level. Should always be set. */ +u8 level; +}; + +#endif /* BTRFS_PARENT_CHECK_H */
fs/btrfs/print-tree.c +1 -0
@@ -8,6 +8,7 @@#include "disk-io.h" #include "print-tree.h" #include "accessors.h" +#include "parent-check.h" struct root_name_map {u64 id;
fs/btrfs/qgroup.c +1 -0
@@ -28,6 +28,7 @@#include "accessors.h" #include "extent-tree.h" #include "root-tree.h" +#include "parent-check.h" /* * Helpers to access qgroup reservation
fs/btrfs/tree-mod-log.c +1 -0
@@ -5,6 +5,7 @@#include "disk-io.h" #include "fs.h" #include "accessors.h" +#include "parent-check.h" struct tree_mod_root {u64 logical;
fs/btrfs/volumes.h +1 -1
@@ -11,7 +11,7 @@#include <linux/btrfs.h> #include "async-thread.h" #include "messages.h" -#include "disk-io.h" +#include "parent-check.h" #include "rcu-string.h" #define BTRFS_MAX_DATA_CHUNK_SIZE (10ULL * SZ_1G)
Signature · 1 line
2.30.2
Re: [PATCH 1/3] btrfs: move struct btrfs_tree_parent_check out of disk-io.h
Quoted 6 lines · Christoph Hellwig
Move struct
btrfs_tree_parent_checkout ofdisk-io.hso thatvolumes.han various .c files don't have to includedisk-io.hjust for it.Signed-off-by Christoph Hellwig <[email protected]>Reviewed-by Johannes Thumshirn <[email protected]>Reviewed-by Qu Wenruo <[email protected]>
Thanks,
Josef
Re: [PATCH 1/3] btrfs: move struct btrfs_tree_parent_check out of disk-io.h
In reply to Christoph HellwigQuoted 2 lines · Christoph Hellwig
Move struct
btrfs_tree_parent_checkout ofdisk-io.hso thatvolumes.han various .c files don't have to includedisk-io.hjust for it.
Splitting that from disk-io.h makes sense but why creating a new file that has just the one structure? We have the tree-checker.h that seems like a place for various checks so I'll move it there.
Re: [PATCH 1/3] btrfs: move struct btrfs_tree_parent_check out of disk-io.h
Quoted 6 lines · David Sterba
Quoted 2 lines · Christoph Hellwig
Move struct
btrfs_tree_parent_checkout ofdisk-io.hso thatvolumes.han various .c files don't have to includedisk-io.hjust for it.Splitting that from
disk-io.hmakes sense but why creating a new file that has just the one structure?
Because there is no other header where it obviously fits.
Quoted 2 lines
We have the
tree-checker.hthat seems
like a place for various checks so I'll move it there.
Despite the similar naming there's actually no overlap between the functionality offered in tree-checker.h and these uses of struct btrfs_tree_parent_check at all.
Re: [PATCH 1/3] btrfs: move struct btrfs_tree_parent_check out of disk-io.h
Quoted 8 lines · Christoph Hellwig
Quoted 6 lines · David Sterba
Quoted 2 lines · Christoph Hellwig
Move struct
btrfs_tree_parent_checkout ofdisk-io.hso thatvolumes.han various .c files don't have to includedisk-io.hjust for it.Splitting that from
disk-io.hmakes sense but why creating a new file that has just the one structure?
Quoted 8 lines
Because there is no other header where it obviously fits.
Quoted 2 lines
We have the
tree-checker.hthat seems
like a place for various checks so I'll move it there.Despite the similar naming there's actually no overlap between the functionality offered in
tree-checker.hand these uses of structbtrfs_tree_parent_checkat all.
That's not true, the overlap is quite clear. Tree checker verifies metadata blocks at read time from the available items in the leaves and links between the parent/child nodes is another extension of that. Functions like validate_extent_buffer are in disk-io.c but they are the to tree checker functionality.
PATCH 2/3 btrfs: split the bio submission path into a separate file
In reply to Christoph HellwigThe code used by btrfs_submit_bio only interacts with the rest of volumes.c through __btrfs_map_block (which itself is a more generic version of two exported helpers) and does not really have anything to do with volumes.c. Create a new bio.c file and a bio.h header going along with it for the btrfs_bio-based storage layer, which will grow even more going forward.
Also update the file with my copyright notice given that a large part of the moved code was written or rewritten by me.
--- fs/btrfs/Makefile | 2 +- fs/btrfs/bio.c | 291 ++++++++++++++++++++++++++++++++++++++++ fs/btrfs/bio.h | 126 ++++++++++++++++++ fs/btrfs/compression.c | 2 +- fs/btrfs/ctree.c | 1 + fs/btrfs/disk-io.c | 2 +- fs/btrfs/extent-tree.c | 1 + fs/btrfs/extent_io.c | 2 +- fs/btrfs/file-item.c | 2 +- fs/btrfs/inode.c | 2 +- fs/btrfs/relocation.c | 1 + fs/btrfs/super.c | 2 +- fs/btrfs/tree-log.c | 1 + fs/btrfs/volumes.c | 296 +---------------------------------------- fs/btrfs/volumes.h | 111 +--------------- 15 files changed, 438 insertions(+), 404 deletions(-) create mode 100644 fs/btrfs/bio.c create mode 100644 fs/btrfs/bio.h
fs/btrfs/Makefile +1 -1
@@ -31,7 +31,7 @@ btrfs-y += super.o ctree.o extent-tree.o print-tree.o root-tree.o dir-item.o \ backref.o ulist.o qgroup.o send.o dev-replace.o raid56.o \ uuid-tree.o props.o free-space-tree.o tree-checker.o space-info.o \ block-rsv.o delalloc-space.o block-group.o discard.o reflink.o \ - subpage.o tree-mod-log.o extent-io-tree.o fs.o messages.o + subpage.o tree-mod-log.o extent-io-tree.o fs.o messages.o bio.o btrfs-$(CONFIG_BTRFS_FS_POSIX_ACL) += acl.o btrfs-$(CONFIG_BTRFS_FS_CHECK_INTEGRITY) += check-integrity.o
fs/btrfs/bio.c new file +291 -0
291 lines not on this page · raw message
fs/btrfs/bio.h new file +126 -0
126 lines not on this page · raw message
fs/btrfs/compression.c +1 -1
@@ -27,7 +27,7 @@#include "disk-io.h" #include "transaction.h" #include "btrfs_inode.h" -#include "volumes.h" +#include "bio.h" #include "ordered-data.h" #include "compression.h" #include "extent_io.h"
fs/btrfs/ctree.c +1 -0
@@ -22,6 +22,7 @@#include "accessors.h" #include "extent-tree.h" #include "relocation.h" +#include "parent-check.h" static struct kmem_cache *btrfs_path_cachep;
fs/btrfs/disk-io.c +1 -1
@@ -23,7 +23,7 @@#include "disk-io.h" #include "transaction.h" #include "btrfs_inode.h" -#include "volumes.h" +#include "bio.h" #include "print-tree.h" #include "locking.h" #include "tree-log.h"
fs/btrfs/extent-tree.c +1 -0
@@ -42,6 +42,7 @@#include "root-tree.h" #include "file-item.h" #include "orphan.h" +#include "parent-check.h" #undef SCRAMBLE_DELAYED_REFS
fs/btrfs/extent_io.c +1 -1
@@ -20,7 +20,7 @@#include "extent_map.h" #include "ctree.h" #include "btrfs_inode.h" -#include "volumes.h" +#include "bio.h" #include "check-integrity.h" #include "locking.h" #include "rcu-string.h"
fs/btrfs/file-item.c +1 -1
@@ -14,7 +14,7 @@#include "ctree.h" #include "disk-io.h" #include "transaction.h" -#include "volumes.h" +#include "bio.h" #include "print-tree.h" #include "compression.h" #include "fs.h"
fs/btrfs/inode.c +1 -1
@@ -43,7 +43,7 @@#include "ordered-data.h" #include "xattr.h" #include "tree-log.h" -#include "volumes.h" +#include "bio.h" #include "compression.h" #include "locking.h" #include "free-space-cache.h"
fs/btrfs/relocation.c +1 -0
@@ -35,6 +35,7 @@#include "file-item.h" #include "relocation.h" #include "super.h" +#include "parent-check.h" /* * Relocation overview
fs/btrfs/super.c +1 -1
@@ -35,7 +35,7 @@#include "print-tree.h" #include "props.h" #include "xattr.h" -#include "volumes.h" +#include "bio.h" #include "export.h" #include "compression.h" #include "rcu-string.h"
fs/btrfs/tree-log.c +1 -0
@@ -29,6 +29,7 @@#include "file-item.h" #include "file.h" #include "orphan.h" +#include "parent-check.h" #define MAX_CONFLICT_INODES 10
fs/btrfs/volumes.c +5 -291
338 lines not on this page · raw message
fs/btrfs/volumes.h +5 -106
145 lines not on this page · raw message
Signature · 1 line
2.30.2
PATCH 3/3 btrfs: move repair_io_failure to bio.c
In reply to Christoph Hellwigrepair_io_failure ties directly into all the glory low-level details of mapping a bio with a logic address to the actual physical location. Move it right below btrfs_submit_bio to keep all the related logic together.
Also move btrfs_repair_eb_io_failure to its caller in disk-io.c now that repair_io_failure is available in a header.
--- fs/btrfs/bio.c | 90 +++++++++++++++++++++++++++++++++ fs/btrfs/disk-io.c | 24 +++++++++ fs/btrfs/extent_io.c | 117 +------------------------------------------ fs/btrfs/extent_io.h | 1 - 4 files changed, 116 insertions(+), 116 deletions(-)
fs/btrfs/bio.c +90 -0
@@ -276,6 +276,96 @@ void btrfs_submit_bio(struct btrfs_fs_info *fs_info, struct bio *bio, int mirror } } +/* +* Submit a repair write. +* +* This bypasses btrfs_submit_bio deliberately, as that writes all copies in a +* RAID setup. Here we only want to write the one bad copy, so we do the +* mapping ourselves and submit the bio directly. +* +* The I/O is issued sychronously to block the repair read completion from +* freeing the bio. +*/ +int btrfs_repair_io_failure (struct btrfs_fs_info *fs_info,u64 ino,u64 start, +u64 length,u64 logical,struct page *page, +unsigned int pg_offset,int mirror_num) +{ +struct btrfs_device *dev; +struct bio_vec bvec; +struct bio bio; +u64 map_length =0 ; +u64 sector; +struct btrfs_io_context *bioc =NULL ; +int ret =0 ; + +ASSERT (!(fs_info->sb->s_flags &SB_RDONLY )); +BUG_ON (!mirror_num); + +if (btrfs_repair_one_zone (fs_info, logical)) +return 0 ; + + map_length = length; + +/* +* Avoid races with device replace and make sure our bioc has devices +* associated to its stripes that don't go away while we are doing the +* read repair operation. +*/ +btrfs_bio_counter_inc_blocked (fs_info); +if (btrfs_is_parity_mirror (fs_info, logical, length)) { +/* +* Note that we don't use BTRFS_MAP_WRITE because it's supposed +* to update all raid stripes, but here we just want to correct +* bad stripe, thus BTRFS_MAP_READ is abused to only get the bad +* stripe's dev and sector. +*/ + ret =btrfs_map_block (fs_info,BTRFS_MAP_READ , logical, + &map_length, &bioc,0 ); +if (ret) + goto out_counter_dec; +ASSERT (bioc->mirror_num ==1 ); + }else { + ret =btrfs_map_block (fs_info,BTRFS_MAP_WRITE , logical, + &map_length, &bioc, mirror_num); +if (ret) + goto out_counter_dec; +BUG_ON (mirror_num != bioc->mirror_num); + } + + sector = bioc->stripes[bioc->mirror_num -1 ].physical >>9 ; + dev = bioc->stripes[bioc->mirror_num -1 ].dev; +btrfs_put_bioc (bioc); + +if (!dev || !dev->bdev || + !test_bit (BTRFS_DEV_STATE_WRITEABLE , &dev->dev_state)) { + ret = -EIO ; + goto out_counter_dec; + } + +bio_init (&bio, dev->bdev, &bvec,1 ,REQ_OP_WRITE |REQ_SYNC ); + bio.bi_iter.bi_sector = sector; +__bio_add_page (&bio, page, length, pg_offset); + +btrfsic_check_bio (&bio); + ret =submit_bio_wait (&bio); +if (ret) { +/* try to remap that extent elsewhere? */ +btrfs_dev_stat_inc_and_print (dev,BTRFS_DEV_STAT_WRITE_ERRS ); + goto out_bio_uninit; + } + +btrfs_info_rl_in_rcu (fs_info, +"read error corrected: ino %llu off %llu (dev %s sector %llu)" , + ino, start,btrfs_dev_name (dev), sector); + ret =0 ; + +out_bio_uninit: +bio_uninit (&bio); +out_counter_dec: +btrfs_bio_counter_dec (fs_info); +return ret; +} +int __init btrfs_bioset_init (void ) { if (bioset_init(&btrfs_bioset, BIO_POOL_SIZE,
fs/btrfs/disk-io.c +24 -0
@@ -255,6 +255,30 @@ int btrfs_verify_level_key(struct extent_buffer *eb, int level,return ret; } +static int btrfs_repair_eb_io_failure (const struct extent_buffer *eb, +int mirror_num) +{ +struct btrfs_fs_info *fs_info = eb->fs_info; +u64 start = eb->start; +int i, num_pages =num_extent_pages (eb); +int ret =0 ; + +if (sb_rdonly (fs_info->sb)) +return -EROFS ; + +for (i =0 ; i < num_pages; i++) { +struct page *p = eb->pages[i]; + + ret =btrfs_repair_io_failure (fs_info,0 , start,PAGE_SIZE , + start, p, start -page_offset (p), mirror_num); +if (ret) +break ; + start +=PAGE_SIZE ; + } + +return ret; +} +/* * helper to read a given tree block, doing retries as required when * the checksums don't match and we have alternate mirrors to try.
fs/btrfs/extent_io.c +2 -115
135 lines not on this page · raw message
fs/btrfs/extent_io.h +0 -1
@@ -245,7 +245,6 @@ int extent_invalidate_folio(struct extent_io_tree *tree,int btrfs_alloc_page_array (unsigned int nr_pages,struct page **page_array);void end_extent_writepage (struct page *page,int err,u64 start,u64 end); -int btrfs_repair_eb_io_failure (const struct extent_buffer *eb,int mirror_num);/* * When IO fails, either with EIO or csum verification fails, we
Signature · 1 line
2.30.2
Quoted 9 lines · Christoph Hellwig
Hi all,
this small series creates a new
bio.cfile (and abio.hheader for it) to contain all the "storage" layer code belowbtrfs_submit_bio. The amount of code sitting belowbtrfs_submit_biowill grow a lot with the "consolidate btrfs checksumming, repair and bio splitting" series, so this pure code move series triest to prepare for that by making sure we have a neat file to add it to.
Quoted 7 lines
Changes since v2:
- rebased against thercu_stringchanges in misc-nextChanges since v1:
- rebased to the latest misc-next branch
- added a new patch to move structbtrfs_tree_parent_checkinto a new header to invoide include hell
Added to misc-next, thanks.