Skip to contentSkip to conversation

move the low-level btrfs_bio code into a separate file (resent)

PATCH In review Christoph Hellwig 4 participants 9 messages Sep 12, 2022 – Sep 22, 2022 Btrfs file system

lore
Share & tools
lore
#0 lore raw

[resending because I forgot to Cc the btrfs list, sorry]

Hi all,

this small series creates a new bio.c file (and a bio.h header for it) to contain all the "storage" layer code below btrfs_submit_bio. The amount of code sitting below btrfs_submit_bio will grow a lot with the "consolidate btrfs checksumming, repair and bio splitting" series, so this pure code move series triest to prepare for that by making sure we have a neat file to add it to.

#1 lore raw

PATCH 1/2 btrfs: split the bio submission path into a separate file

The code used by btrfs_submit_bio only interacts with the rest of volumes.c through __btrfs_map_block (which itself is a more generic version of two exported helpers) and does not really have anything to do with volumes.c. Create a new bio.c file and a bio.h header going along with it for the btrfs_bio-based storage layer, which will grow even more going forward.

Also update the file with my copyright notice given that a large part of the moved code was written or rewritten by me.

---
 fs/btrfs/Makefile      |   2 +-
 fs/btrfs/bio.c         | 290 ++++++++++++++++++++++++++++++++++++++++
 fs/btrfs/bio.h         | 103 ++++++++++++++
 fs/btrfs/compression.c |   2 +-
 fs/btrfs/disk-io.c     |   2 +-
 fs/btrfs/extent_io.c   |   2 +-
 fs/btrfs/file-item.c   |   2 +-
 fs/btrfs/inode.c       |   2 +-
 fs/btrfs/super.c       |   2 +-
 fs/btrfs/volumes.c     | 296 +----------------------------------------
 fs/btrfs/volumes.h     |  91 +------------
 11 files changed, 410 insertions(+), 384 deletions(-)
 create mode 100644 fs/btrfs/bio.c
 create mode 100644 fs/btrfs/bio.h
fs/btrfs/Makefile +1 -1
index 99f9995670ea3..f28ec049ae419 100644
@@ -31,7 +31,7 @@ btrfs-y += super.o ctree.o extent-tree.o print-tree.o root-tree.o dir-item.o \
 	   backref.o ulist.o qgroup.o send.o dev-replace.o raid56.o \
 	   uuid-tree.o props.o free-space-tree.o tree-checker.o space-info.o \
 	   block-rsv.o delalloc-space.o block-group.o discard.o reflink.o \
-	   subpage.o tree-mod-log.o
+	   subpage.o tree-mod-log.o bio.o
 
 btrfs-$(CONFIG_BTRFS_FS_POSIX_ACL) += acl.o
 btrfs-$(CONFIG_BTRFS_FS_CHECK_INTEGRITY) += check-integrity.o
fs/btrfs/bio.c new file +290 -0

290 lines not on this page · raw message

fs/btrfs/bio.h new file +103 -0

103 lines not on this page · raw message

fs/btrfs/compression.c +1 -1
index 1c77de3239bc4..4f9dc4115401f 100644
@@ -24,7 +24,7 @@
 #include "disk-io.h"
 #include "transaction.h"
 #include "btrfs_inode.h"
-#include "volumes.h"
+#include "bio.h"
 #include "ordered-data.h"
 #include "compression.h"
 #include "extent_io.h"
fs/btrfs/disk-io.c +1 -1
index f2b875b93ddfb..e3419d2d32295 100644
@@ -23,7 +23,7 @@
 #include "disk-io.h"
 #include "transaction.h"
 #include "btrfs_inode.h"
-#include "volumes.h"
+#include "bio.h"
 #include "print-tree.h"
 #include "locking.h"
 #include "tree-log.h"
fs/btrfs/extent_io.c +1 -1
index cea7d09c2dc1d..a16442eadb02c 100644
@@ -20,7 +20,7 @@
 #include "extent_map.h"
 #include "ctree.h"
 #include "btrfs_inode.h"
-#include "volumes.h"
+#include "bio.h"
 #include "check-integrity.h"
 #include "locking.h"
 #include "rcu-string.h"
fs/btrfs/file-item.c +1 -1
index 29999686d234c..b0cda6ac12bd6 100644
@@ -13,7 +13,7 @@
 #include "ctree.h"
 #include "disk-io.h"
 #include "transaction.h"
-#include "volumes.h"
+#include "bio.h"
 #include "print-tree.h"
 #include "compression.h"
 
fs/btrfs/inode.c +1 -1
index 10849db7f3a58..a98257ea6e907 100644
@@ -43,7 +43,7 @@
 #include "ordered-data.h"
 #include "xattr.h"
 #include "tree-log.h"
-#include "volumes.h"
+#include "bio.h"
 #include "compression.h"
 #include "locking.h"
 #include "free-space-cache.h"
fs/btrfs/super.c +1 -1
index eb0ae7e396ef4..50de5abf74446 100644
@@ -34,7 +34,7 @@
 #include "print-tree.h"
 #include "props.h"
 #include "xattr.h"
-#include "volumes.h"
+#include "bio.h"
 #include "export.h"
 #include "compression.h"
 #include "rcu-string.h"
fs/btrfs/volumes.c +5 -291

338 lines not on this page · raw message

fs/btrfs/volumes.h +5 -86

115 lines not on this page · raw message

Signature · 1 line
2.30.2
Reviewed-by on v1 1/2
#2 lore raw

Re: [PATCH 1/2] btrfs: split the bio submission path into a separate file

fs/btrfs/bio.c line 5

Quoted 5 lines · Christoph Hellwig
fs/btrfs/bio.c new file +5 -0
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Copyright (C) 2007 Oracle.  All rights reserved.
+ * Copyright (C) 2022 Christoph Hellwig.
+ */

IIRC David does try to get rid of all the per-company copyright statements for new files.

Other than that, I'm fine with it.

#3 lore raw

Re: [PATCH 1/2] btrfs: split the bio submission path into a separate file

Quoted 9 lines · Johannes Thumshirn

fs/btrfs/bio.c line 5

Quoted 5 lines · Christoph Hellwig
fs/btrfs/bio.c new file +5 -0
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Copyright (C) 2007 Oracle.  All rights reserved.
+ * Copyright (C) 2022 Christoph Hellwig.
+ */

IIRC David does try to get rid of all the per-company copyright statements for new files.

We just had that discussion in another thread - there really is no basis for getting rid of them. In fact talking to lawyers, most of them thing we should have more of them, not less.

#4 lore raw

Re: [PATCH 1/2] btrfs: split the bio submission path into a separate file

Quoted 11 lines · Christoph Hellwig
Quoted 9 lines · Johannes Thumshirn

fs/btrfs/bio.c line 5

Quoted 5 lines · Christoph Hellwig
fs/btrfs/bio.c new file +5 -0
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Copyright (C) 2007 Oracle.  All rights reserved.
+ * Copyright (C) 2022 Christoph Hellwig.
+ */

IIRC David does try to get rid of all the per-company copyright statements for new files.

Quoted 3 lines

We just had that discussion in another thread - there really is no basis for getting rid of them. In fact talking to lawyers, most of them thing we should have more of them, not less.

There seems to be no clear consensus that would satisfy lawyers (track complete copyright information) and practical point of view (it's in the git metadata). The problem not only I see taking this patch as an example is that your (c) name now appears in file containing changes from several other people whose names are not mentioned (but yes the original copyright is preserved).

I don't want to speculate what would this mean from the legal POV and I think we've heard each other's arguments. If there's a recommended practice coming "from above" as linux project, like it was with the SPDX, ok let everybody add their copyright notices. But until then I won't take such patches.

#5 lore raw

Re: [PATCH 1/2] btrfs: split the bio submission path into a separate file

Yeah, I've got the message that btrfs wants to remain the special and badly opinionated mess it is, and I'll stop bothering you with patches in the future.

#6 lore raw

PATCH 2/2 btrfs: move repair_io_failure to bio.c

In reply to Christoph Hellwig

repair_io_failure ties directly into all the glory low-level details of mapping a bio with a logic address to the actual physical location. Move it right below btrfs_submit_bio to keep all the related logic together.

Also move btrfs_repair_eb_io_failure to its caller in disk-io.c now that repair_io_failure is available in a header.

---
 fs/btrfs/bio.c       |  91 +++++++++++++++++++++++++++++++++
 fs/btrfs/disk-io.c   |  24 +++++++++
 fs/btrfs/extent_io.c | 118 +------------------------------------------
 fs/btrfs/extent_io.h |   1 -
 4 files changed, 117 insertions(+), 117 deletions(-)
fs/btrfs/bio.c +91 -0
index 2bf17274caf75..440c022b29039 100644
@@ -275,6 +275,97 @@ void btrfs_submit_bio(struct btrfs_fs_info *fs_info, struct bio *bio, int mirror
 	}
 }
 
+/*
+ * Submit a repair write.
+ *
+ * This bypasses btrfs_submit_bio deliberately, as that writes all copies in a
+ * RAID setup.  Here we only want to write the one bad copy, so we do the
+ * mapping ourselves and submit the bio directly.
+ *
+ * The I/O is issued sychronously to block the repair read completion from
+ * freeing the bio.
+ */
+int btrfs_repair_io_failure(struct btrfs_fs_info *fs_info, u64 ino, u64 start,
+			    u64 length, u64 logical, struct page *page,
+			    unsigned int pg_offset, int mirror_num)
+{
+	struct btrfs_device *dev;
+	struct bio_vec bvec;
+	struct bio bio;
+	u64 map_length = 0;
+	u64 sector;
+	struct btrfs_io_context *bioc = NULL;
+	int ret = 0;
+
+	ASSERT(!(fs_info->sb->s_flags & SB_RDONLY));
+	BUG_ON(!mirror_num);
+
+	if (btrfs_repair_one_zone(fs_info, logical))
+		return 0;
+
+	map_length = length;
+
+	/*
+	 * Avoid races with device replace and make sure our bioc has devices
+	 * associated to its stripes that don't go away while we are doing the
+	 * read repair operation.
+	 */
+	btrfs_bio_counter_inc_blocked(fs_info);
+	if (btrfs_is_parity_mirror(fs_info, logical, length)) {
+		/*
+		 * Note that we don't use BTRFS_MAP_WRITE because it's supposed
+		 * to update all raid stripes, but here we just want to correct
+		 * bad stripe, thus BTRFS_MAP_READ is abused to only get the bad
+		 * stripe's dev and sector.
+		 */
+		ret = btrfs_map_block(fs_info, BTRFS_MAP_READ, logical,
+				      &map_length, &bioc, 0);
+		if (ret)
+			goto out_counter_dec;
+		ASSERT(bioc->mirror_num == 1);
+	} else {
+		ret = btrfs_map_block(fs_info, BTRFS_MAP_WRITE, logical,
+				      &map_length, &bioc, mirror_num);
+		if (ret)
+			goto out_counter_dec;
+		BUG_ON(mirror_num != bioc->mirror_num);
+	}
+
+	sector = bioc->stripes[bioc->mirror_num - 1].physical >> 9;
+	dev = bioc->stripes[bioc->mirror_num - 1].dev;
+	btrfs_put_bioc(bioc);
+
+	if (!dev || !dev->bdev ||
+	    !test_bit(BTRFS_DEV_STATE_WRITEABLE, &dev->dev_state)) {
+		ret = -EIO;
+		goto out_counter_dec;
+	}
+
+	bio_init(&bio, dev->bdev, &bvec, 1, REQ_OP_WRITE | REQ_SYNC);
+	bio.bi_iter.bi_sector = sector;
+	__bio_add_page(&bio, page, length, pg_offset);
+
+	btrfsic_check_bio(&bio);
+	ret = submit_bio_wait(&bio);
+	if (ret) {
+		/* try to remap that extent elsewhere? */
+		btrfs_dev_stat_inc_and_print(dev, BTRFS_DEV_STAT_WRITE_ERRS);
+		goto out_bio_uninit;
+	}
+
+	btrfs_info_rl_in_rcu(fs_info,
+		"read error corrected: ino %llu off %llu (dev %s sector %llu)",
+				  ino, start,
+				  rcu_str_deref(dev->name), sector);
+	ret = 0;
+
+out_bio_uninit:
+	bio_uninit(&bio);
+out_counter_dec:
+	btrfs_bio_counter_dec(fs_info);
+	return ret;
+}
+
 int __init btrfs_bioset_init(void)
 {
 	if (bioset_init(&btrfs_bioset, BIO_POOL_SIZE,
fs/btrfs/disk-io.c +24 -0
index e3419d2d32295..17b34213e7fe1 100644
@@ -249,6 +249,30 @@ int btrfs_verify_level_key(struct extent_buffer *eb, int level,
 	return ret;
 }
 
+static int btrfs_repair_eb_io_failure(const struct extent_buffer *eb,
+				      int mirror_num)
+{
+	struct btrfs_fs_info *fs_info = eb->fs_info;
+	u64 start = eb->start;
+	int i, num_pages = num_extent_pages(eb);
+	int ret = 0;
+
+	if (sb_rdonly(fs_info->sb))
+		return -EROFS;
+
+	for (i = 0; i < num_pages; i++) {
+		struct page *p = eb->pages[i];
+
+		ret = btrfs_repair_io_failure(fs_info, 0, start, PAGE_SIZE,
+				start, p, start - page_offset(p), mirror_num);
+		if (ret)
+			break;
+		start += PAGE_SIZE;
+	}
+
+	return ret;
+}
+
 /*
  * helper to read a given tree block, doing retries as required when
  * the checksums don't match and we have alternate mirrors to try.
fs/btrfs/extent_io.c +2 -116

136 lines not on this page · raw message

fs/btrfs/extent_io.h +0 -1
index 69a86ae6fd508..e653e64598bf7 100644
@@ -243,7 +243,6 @@ void extent_clear_unlock_delalloc(struct btrfs_inode *inode, u64 start, u64 end,
 int btrfs_alloc_page_array(unsigned int nr_pages, struct page **page_array);
 
 void end_extent_writepage(struct page *page, int err, u64 start, u64 end);
-int btrfs_repair_eb_io_failure(const struct extent_buffer *eb, int mirror_num);
 
 /*
  * When IO fails, either with EIO or csum verification fails, we
Signature · 1 line
2.30.2
Reviewed-by
#8 lore raw
In reply to Christoph Hellwig
Quoted 10 lines · Christoph Hellwig

[resending because I forgot to Cc the btrfs list, sorry]

Hi all,

this small series creates a new bio.c file (and a bio.h header for it) to contain all the "storage" layer code below btrfs_submit_bio. The amount of code sitting below btrfs_submit_bio will grow a lot with the "consolidate btrfs checksumming, repair and bio splitting" series, so this pure code move series triest to prepare for that by making sure we have a neat file to add it to.

You can add

to the series, thanks,

Josef