Skip to contentSkip to conversation

The remaining extent_io.c split code

PATCH In review Josef Bacik 2 participants 7 messages Sep 24, 2019 – Sep 27, 2019 Btrfs file system

lore
Share & tools
lore
#0 lore raw

cover The remaining extent_io.c split code

Hopefully all of it makes it this time, if you want you can pull from

git://git.kernel.org/pub/scm/linux/kernel/git/josef/btrfs-next.git \
extent-io-rearranging

I've rebased on top of misc-next and fixed up all the conflicts. Thanks,

Josef

#1 lore raw

PATCH 2/4 btrfs: separate out the extent buffer init code

We want to move this into it's own file, so separate out the init/exit code for setting up the extent_buffer cache.

---
 fs/btrfs/extent_io.c | 36 +++++++++++++++++++-----------------
 fs/btrfs/extent_io.h |  2 ++
 fs/btrfs/super.c     |  9 ++++++++-
 3 files changed, 29 insertions(+), 18 deletions(-)
fs/btrfs/extent_io.c +19 -17
index 2e40ce6d0452..07ef107f0e6c 100644
@@ -136,32 +136,30 @@ static int __must_check flush_write_bio(struct extent_page_data *epd)
 
 int __init extent_io_init(void)
 {
-	extent_buffer_cache = kmem_cache_create("btrfs_extent_buffer",
-			sizeof(struct extent_buffer), 0,
-			SLAB_MEM_SPREAD, NULL);
-	if (!extent_buffer_cache)
-		return -ENOMEM;
-
 	if (bioset_init(&btrfs_bioset, BIO_POOL_SIZE,
 			offsetof(struct btrfs_io_bio, bio),
 			BIOSET_NEED_BVECS))
-		goto free_buffer_cache;
+		return -ENOMEM;
 
-	if (bioset_integrity_create(&btrfs_bioset, BIO_POOL_SIZE))
-		goto free_bioset;
+	if (bioset_integrity_create(&btrfs_bioset, BIO_POOL_SIZE)) {
+		bioset_exit(&btrfs_bioset);
+		return -ENOMEM;
+	}
 
 	return 0;
+}
 
-free_bioset:
-	bioset_exit(&btrfs_bioset);
-
-free_buffer_cache:
-	kmem_cache_destroy(extent_buffer_cache);
-	extent_buffer_cache = NULL;
-	return -ENOMEM;
+int __init extent_buffer_init(void)
+{
+	extent_buffer_cache = kmem_cache_create("btrfs_extent_buffer",
+			sizeof(struct extent_buffer), 0,
+			SLAB_MEM_SPREAD, NULL);
+	if (!extent_buffer_cache)
+		return -ENOMEM;
+	return 0;
 }
 
-void __cold extent_io_exit(void)
+void __cold extent_buffer_exit(void)
 {
 	btrfs_extent_buffer_leak_debug_check();
 
@@ -171,6 +169,10 @@ void __cold extent_io_exit(void)
 	 */
 	rcu_barrier();
 	kmem_cache_destroy(extent_buffer_cache);
+}
+
+void __cold extent_io_exit(void)
+{
 	bioset_exit(&btrfs_bioset);
 }
 
fs/btrfs/extent_io.h +2 -0
index e22045cef89b..e2246956e544 100644
@@ -332,4 +332,6 @@ bool find_lock_delalloc_range(struct inode *inode,
 #endif
 struct extent_buffer *alloc_test_extent_buffer(struct btrfs_fs_info *fs_info,
 					       u64 start);
+int __init extent_buffer_init(void);
+void __cold extent_buffer_exit(void);
 #endif
fs/btrfs/super.c +8 -1
index 843015b9a11e..7207bb39f236 100644
@@ -2360,10 +2360,14 @@ static int __init init_btrfs_fs(void)
 	if (err)
 		goto free_cachep;
 
-	err = extent_state_cache_init();
+	err = extent_buffer_init();
 	if (err)
 		goto free_extent_io;
 
+	err = extent_state_cache_init();
+	if (err)
+		goto free_extent_buffer;
+
 	err = extent_map_init();
 	if (err)
 		goto free_extent_state_cache;
@@ -2428,6 +2432,8 @@ static int __init init_btrfs_fs(void)
 	extent_map_exit();
 free_extent_state_cache:
 	extent_state_cache_exit();
+free_extent_buffer:
+	extent_buffer_exit();
 free_extent_io:
 	extent_io_exit();
 free_cachep:
@@ -2449,6 +2455,7 @@ static void __exit exit_btrfs_fs(void)
 	ordered_data_exit();
 	extent_map_exit();
 	extent_state_cache_exit();
+	extent_buffer_exit();
 	extent_io_exit();
 	btrfs_interface_exit();
 	btrfs_end_io_wq_exit();
Signature · 1 line
2.21.0
#2 lore raw

PATCH 3/4 btrfs: migrate the extent_buffer code out of extent-io.h

In reply to Josef Bacik

There's two main things we do with extent buffers, we mess with the buffers themselves, and then we do IO on those buffers. Separate out the actual extent buffer manipulation code from extent_io since they are unrelated.

---
 fs/btrfs/ctree.h         |   2 +-
 fs/btrfs/disk-io.h       |   2 +
 fs/btrfs/extent-buffer.h | 152 +++++++++++++++++++++++++++++++++++++++
 fs/btrfs/extent_io.h     | 148 +-------------------------------------
 4 files changed, 157 insertions(+), 147 deletions(-)
 create mode 100644 fs/btrfs/extent-buffer.h
fs/btrfs/ctree.h +1 -1
index 5e7ff169683c..c3a7df64db32 100644
@@ -29,7 +29,7 @@
 #include <linux/refcount.h>
 #include <linux/crc32c.h>
 #include "extent-io-tree.h"
-#include "extent_io.h"
+#include "extent-buffer.h"
 #include "extent_map.h"
 #include "async-thread.h"
 #include "block-rsv.h"
fs/btrfs/disk-io.h +2 -0
index a6958103d87e..affb45b1ed82 100644
@@ -6,6 +6,8 @@
 #ifndef BTRFS_DISK_IO_H
 #define BTRFS_DISK_IO_H
 
+#include "extent_io.h"
+
 #define BTRFS_SUPER_INFO_OFFSET SZ_64K
 #define BTRFS_SUPER_INFO_SIZE 4096
 
fs/btrfs/extent-buffer.h new file +152 -0

152 lines not on this page · raw message

fs/btrfs/extent_io.h +2 -146

193 lines not on this page · raw message

Signature · 1 line
2.21.0
#3 lore raw

PATCH 4/4 btrfs: move the extent-buffer code

In reply to Josef Bacik

This moves the extent buffer code into its own file.

---
 fs/btrfs/Makefile        |    3 +-
 fs/btrfs/extent-buffer.c | 1268 ++++++++++++++++++++++++++++++++++++++
 fs/btrfs/extent_io.c     | 1256 -------------------------------------
 3 files changed, 1270 insertions(+), 1257 deletions(-)
 create mode 100644 fs/btrfs/extent-buffer.c
fs/btrfs/Makefile +2 -1
index 7ce7202f5fe8..4d669c1d09fa 100644
@@ -11,7 +11,8 @@ btrfs-y += super.o ctree.o extent-tree.o print-tree.o root-tree.o dir-item.o \
 	   compression.o delayed-ref.o relocation.o delayed-inode.o scrub.o \
 	   reada.o backref.o ulist.o qgroup.o send.o dev-replace.o raid56.o \
 	   uuid-tree.o props.o free-space-tree.o tree-checker.o space-info.o \
-	   block-rsv.o delalloc-space.o block-group.o extent-io-tree.o
+	   block-rsv.o delalloc-space.o block-group.o extent-io-tree.o \
+	   extent-buffer.o
 
 btrfs-$(CONFIG_BTRFS_FS_POSIX_ACL) += acl.o
 btrfs-$(CONFIG_BTRFS_FS_CHECK_INTEGRITY) += check-integrity.o
fs/btrfs/extent-buffer.c new file +1268 -0

1,268 lines not on this page · raw message

fs/btrfs/extent_io.c +0 -1256

1,291 lines not on this page · raw message

Signature · 1 line
2.21.0
#4 lore raw
In reply to Josef Bacik
Quoted 4 lines · Josef Bacik

Hopefully all of it makes it this time, if you want you can pull from

git://git.kernel.org/pub/scm/linux/kernel/git/josef/btrfs-next.git \
extent-io-rearranging

The size of the exported patch 1/4 is 109066 bytes and the diff itself is incomprehensible to even see what code moves where and what is new.

I'm still thinking if this is a good idea to apply a monster patch, even it's just moving code around. The previous series splitting extent-tree.c were better so I'd rather take that approach again. Some of the functions belong logically together and won't break compilation and would actually make it possible for a human to review.

#5 lore raw
Quoted 6 lines · David Sterba
Quoted 4 lines · Josef Bacik

Hopefully all of it makes it this time, if you want you can pull from

git://git.kernel.org/pub/scm/linux/kernel/git/josef/btrfs-next.git \
extent-io-rearranging

Quoted 8 lines

The size of the exported patch 1/4 is 109066 bytes and the diff itself is incomprehensible to even see what code moves where and what is new.

I'm still thinking if this is a good idea to apply a monster patch, even it's just moving code around. The previous series splitting extent-tree.c were better so I'd rather take that approach again. Some of the functions belong logically together and won't break compilation and would actually make it possible for a human to review.

extent-tree.c was way different because a bunch of things had to be renamed and exported. Also extent-tree.c got split into many more files, so there was less bulk being moved around. extent_io.c is different because basically everything is exported already, so it's really just move definitions, move code. I literally just split vim'ed, cut and paste between the two files.

Things like this are going to be impossible to review. It's a one time pain for more readability and better decisions down the road. I did my best to keep the logical changes to their own patch, but the fact is we have _a lot_ of code for each of these different logical things. I can turn it into 45 patches moving one function at a time, but who's going to go and check each individual patch?

Josef

#6 lore raw
Quoted 22 lines · Josef Bacik
Quoted 6 lines · David Sterba
Quoted 4 lines · Josef Bacik

Hopefully all of it makes it this time, if you want you can pull from

git://git.kernel.org/pub/scm/linux/kernel/git/josef/btrfs-next.git \
extent-io-rearranging

Quoted 8 lines

The size of the exported patch 1/4 is 109066 bytes and the diff itself is incomprehensible to even see what code moves where and what is new.

I'm still thinking if this is a good idea to apply a monster patch, even it's just moving code around. The previous series splitting extent-tree.c were better so I'd rather take that approach again. Some of the functions belong logically together and won't break compilation and would actually make it possible for a human to review.

extent-tree.c was way different because a bunch of things had to be renamed and exported. Also extent-tree.c got split into many more files, so there was less bulk being moved around. extent_io.c is different because basically everything is exported already, so it's really just move definitions, move code. I literally just split vim'ed, cut and paste between the two files.

Quoted 5 lines

Things like this are going to be impossible to review. It's a one time pain for more readability and better decisions down the road. I did my best to keep the logical changes to their own patch, but the fact is we have _a lot_ of code for each of these different logical things. I can turn it into 45 patches moving one function at a time, but who's going to go and check each individual patch?

I'm not talking about 45 patches but somethig like 2-4. One of the patches in extent-tree split has 65K, and I basically redone it (and several others) by hand due to small changes in the original code. This counts as a review, even though the patch was large, but with the difference that I was able to do that in one go per patch. The 100K scares me from the beginning, thus my question.

If you say that 'this is going to be impossible to review' then, well we can't merge it in that form. And from past experience, monster patches are never one time pain and come up as a reminder of low patch standards. The bad thing is that it's never the patch author who has to suffer.

The idea for split is to do eg. the following groups:

- extent leak detection functions
- state bit manipulation functions (set_extent_bit etc)
- other io tree functions

I'm looking for an achievable long-term strategy to do those large code movements and find the balance between writing the patches and reviewing them.