Skip to contentSkip to conversation

btrfs: bug fixes for the tree mod log and small refactorings

PATCH In review [email protected] 6 participants 20 messages Mar 11, 2021 – Mar 16, 2021 Btrfs file system

lore
Share & tools
lore
#0 lore raw

cover btrfs: bug fixes for the tree mod log and small refactorings

From: Filipe Manana <[email protected]>

This patchset fixes a couple bugs, in the two first patches, with the tree mod log code. The remaining patches just move all that code into a separate file, since it's quite large and ctree.c is huge as well, and do some small refactorings and cleanups.

One of the bugs in particular, has been hit frequently by Zygo, hitting a BUG_ON().

Filipe Manana (9):
btrfs: fix race when cloning extent buffer during rewind of an old
root
btrfs: always pin deleted leaves when there are active tree mod log
users
btrfs: move the tree mod log code into its own file
btrfs: use booleans where appropriate for the tree mod log functions
btrfs: use a bit to track the existence of tree mod log users
btrfs: use the new bit BTRFS_FS_TREE_MOD_LOG_USERS at
btrfs_free_tree_block()
btrfs: remove unnecessary leaf check at btrfs_tree_mod_log_free_eb()
btrfs: add and use helper to get lowest sequence number for the tree
mod log
btrfs: update debug message when checking seq number of a delayed ref

 fs/btrfs/Makefile       |   2 +-
 fs/btrfs/backref.c      |  33 +-
 fs/btrfs/ctree.c        | 954 ++--------------------------------------
 fs/btrfs/ctree.h        |  20 +-
 fs/btrfs/delayed-ref.c  |  31 +-
 fs/btrfs/extent-tree.c  |  21 +-
 fs/btrfs/qgroup.c       |   9 +-
 fs/btrfs/tree-mod-log.c | 912 ++++++++++++++++++++++++++++++++++++++
 fs/btrfs/tree-mod-log.h |  53 +++
 9 files changed, 1056 insertions(+), 979 deletions(-)
 create mode 100644 fs/btrfs/tree-mod-log.c
 create mode 100644 fs/btrfs/tree-mod-log.h
Signature · 1 line
2.28.0
#1 lore raw

PATCH 1/9 btrfs: fix race when cloning extent buffer during rewind of an old root

From: Filipe Manana <[email protected]>

While resolving backreferences, as part of a logical ino ioctl call or fiemap, we can end up hitting a BUG_ON() when replaying tree mod log operations of a root, triggering a stack trace like the following:

  ------------[ cut here ]------------
  kernel BUG at fs/btrfs/ctree.c:1210!
  invalid opcode: 0000 [#1] SMP KASAN PTI
  CPU: 1 PID: 19054 Comm: crawl_335 Tainted: G        W         5.11.0-2d11c0084b02-misc-next+ #89
  Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.12.0-1 04/01/2014
  RIP: 0010:__tree_mod_log_rewind+0x3b1/0x3c0
  Code: 05 48 8d 74 10 (...)
  RSP: 0018:ffffc90001eb70b8 EFLAGS: 00010297
  RAX: 0000000000000000 RBX: ffff88812344e400 RCX: ffffffffb28933b6
  RDX: 0000000000000007 RSI: dffffc0000000000 RDI: ffff88812344e42c
  RBP: ffffc90001eb7108 R08: 1ffff11020b60a20 R09: ffffed1020b60a20
  R10: ffff888105b050f9 R11: ffffed1020b60a1f R12: 00000000000000ee
  R13: ffff8880195520c0 R14: ffff8881bc958500 R15: ffff88812344e42c
  FS:  00007fd1955e8700(0000) GS:ffff8881f5600000(0000) knlGS:0000000000000000
  CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
  CR2: 00007efdb7928718 CR3: 000000010103a006 CR4: 0000000000170ee0
  Call Trace:
   btrfs_search_old_slot+0x265/0x10d0
   ? lock_acquired+0xbb/0x600
   ? btrfs_search_slot+0x1090/0x1090
   ? free_extent_buffer.part.61+0xd7/0x140
   ? free_extent_buffer+0x13/0x20
   resolve_indirect_refs+0x3e9/0xfc0
   ? lock_downgrade+0x3d0/0x3d0
   ? __kasan_check_read+0x11/0x20
   ? add_prelim_ref.part.11+0x150/0x150
   ? lock_downgrade+0x3d0/0x3d0
   ? __kasan_check_read+0x11/0x20
   ? lock_acquired+0xbb/0x600
   ? __kasan_check_write+0x14/0x20
   ? do_raw_spin_unlock+0xa8/0x140
   ? rb_insert_color+0x30/0x360
   ? prelim_ref_insert+0x12d/0x430
   find_parent_nodes+0x5c3/0x1830
   ? resolve_indirect_refs+0xfc0/0xfc0
   ? lock_release+0xc8/0x620
   ? fs_reclaim_acquire+0x67/0xf0
   ? lock_acquire+0xc7/0x510
   ? lock_downgrade+0x3d0/0x3d0
   ? lockdep_hardirqs_on_prepare+0x160/0x210
   ? lock_release+0xc8/0x620
   ? fs_reclaim_acquire+0x67/0xf0
   ? lock_acquire+0xc7/0x510
   ? poison_range+0x38/0x40
   ? unpoison_range+0x14/0x40
   ? trace_hardirqs_on+0x55/0x120
   btrfs_find_all_roots_safe+0x142/0x1e0
   ? find_parent_nodes+0x1830/0x1830
   ? btrfs_inode_flags_to_xflags+0x50/0x50
   iterate_extent_inodes+0x20e/0x580
   ? tree_backref_for_extent+0x230/0x230
   ? lock_downgrade+0x3d0/0x3d0
   ? read_extent_buffer+0xdd/0x110
   ? lock_downgrade+0x3d0/0x3d0
   ? __kasan_check_read+0x11/0x20
   ? lock_acquired+0xbb/0x600
   ? __kasan_check_write+0x14/0x20
   ? _raw_spin_unlock+0x22/0x30
   ? __kasan_check_write+0x14/0x20
   iterate_inodes_from_logical+0x129/0x170
   ? iterate_inodes_from_logical+0x129/0x170
   ? btrfs_inode_flags_to_xflags+0x50/0x50
   ? iterate_extent_inodes+0x580/0x580
   ? __vmalloc_node+0x92/0xb0
   ? init_data_container+0x34/0xb0
   ? init_data_container+0x34/0xb0
   ? kvmalloc_node+0x60/0x80
   btrfs_ioctl_logical_to_ino+0x158/0x230
   btrfs_ioctl+0x205e/0x4040
   ? __might_sleep+0x71/0xe0
   ? btrfs_ioctl_get_supported_features+0x30/0x30
   ? getrusage+0x4b6/0x9c0
   ? __kasan_check_read+0x11/0x20
   ? lock_release+0xc8/0x620
   ? __might_fault+0x64/0xd0
   ? lock_acquire+0xc7/0x510
   ? lock_downgrade+0x3d0/0x3d0
   ? lockdep_hardirqs_on_prepare+0x210/0x210
   ? lockdep_hardirqs_on_prepare+0x210/0x210
   ? __kasan_check_read+0x11/0x20
   ? do_vfs_ioctl+0xfc/0x9d0
   ? ioctl_file_clone+0xe0/0xe0
   ? lock_downgrade+0x3d0/0x3d0
   ? lockdep_hardirqs_on_prepare+0x210/0x210
   ? __kasan_check_read+0x11/0x20
   ? lock_release+0xc8/0x620
   ? __task_pid_nr_ns+0xd3/0x250
   ? lock_acquire+0xc7/0x510
   ? __fget_files+0x160/0x230
   ? __fget_light+0xf2/0x110
   __x64_sys_ioctl+0xc3/0x100
   do_syscall_64+0x37/0x80
   entry_SYSCALL_64_after_hwframe+0x44/0xa9
  RIP: 0033:0x7fd1976e2427
  Code: 00 00 90 48 8b 05 (...)
  RSP: 002b:00007fd1955e5cf8 EFLAGS: 00000246 ORIG_RAX: 0000000000000010
  RAX: ffffffffffffffda RBX: 00007fd1955e5f40 RCX: 00007fd1976e2427
  RDX: 00007fd1955e5f48 RSI: 00000000c038943b RDI: 0000000000000004
  RBP: 0000000001000000 R08: 0000000000000000 R09: 00007fd1955e6120
  R10: 0000557835366b00 R11: 0000000000000246 R12: 0000000000000004
  R13: 00007fd1955e5f48 R14: 00007fd1955e5f40 R15: 00007fd1955e5ef8
  Modules linked in:
  ---[ end trace ec8931a1c36e57be ]---

(gdb) l *(__tree_mod_log_rewind+0x3b1)
0xffffffff81893521 is in __tree_mod_log_rewind (fs/btrfs/ctree.c:1210).

  1205                     * the modification. as we're going backwards, we do the
  1206                     * opposite of each operation here.
  1207                     */

1208 switch (tm->op) {
1209 case MOD_LOG_KEY_REMOVE_WHILE_FREEING:
1210 BUG_ON(tm->slot < n);
1211 fallthrough;
1212 case MOD_LOG_KEY_REMOVE_WHILE_MOVING:
1213 case MOD_LOG_KEY_REMOVE:
1214 btrfs_set_node_key(eb, &tm->key, tm->slot);

Here's what happens to hit that BUG_ON():

1) We have one tree mod log user (through fiemap or the logical ino ioctl), with a sequence number of 1, so we have fs_info->tree_mod_seq == 1;

2) Another task is at ctree.c:balance_level() and we have eb X currently as the root of the tree, and we promote its single child, eb Y, as the new root.

Then, at ctree.c:balance_level(), we call:

      tree_mod_log_insert_root(eb X, eb Y, 1);

3) At tree_mod_log_insert_root() we create tree mod log elements for each slot of eb X, of operation type MOD_LOG_KEY_REMOVE_WHILE_FREEING each with a ->logical pointing to ebX->start. These are placed in an array named tm_list.
Lets assume there are N elements (N pointers in eb X);

4) Then, still at tree_mod_log_insert_root(), we create a tree mod log element of operation type MOD_LOG_ROOT_REPLACE, ->logical set to ebY->start, ->old_root.logical set to ebX->start, ->old_root.level set to the level of eb X and ->generation set to the generation of eb X;

5) Then tree_mod_log_insert_root() calls tree_mod_log_free_eb() with tm_list as argument. After that, tree_mod_log_free_eb() calls __tree_mod_log_insert() for each member of tm_list in reverse order, from highest slot in eb X, slot N - 1, to slot 0 of eb X;

6) __tree_mod_log_insert() sets the sequence number of each given tree mod log operation - it increments fs_info->tree_mod_seq and sets fs_info->tree_mod_seq as the sequence number of the given tree mod log operation.

This means that for the tm_list created at tree_mod_log_insert_root(), the element corresponding to slot 0 of eb X has the highest sequence number (1 + N), and the element corresponding to the last slot has the lowest sequence number (2);

7) Then, after inserting tm_list's elements into the tree mod log rbtree, the MOD_LOG_ROOT_REPLACE element is inserted, which gets the highest sequence number, which is N + 2;

8) Back to ctree.c:balance_level(), we free eb X by calling btrfs_free_tree_block() on it. Because eb X was created in the current transaction, has no other references and writeback did not happen for it, we add it back to the free space cache/tree;

9) Later some other task T allocates the metadata extent from eb X, since it is marked as free space in the space cache/tree, and uses it as a node for some other btree;

10) The tree mod log user task calls btrfs_search_old_slot(), which calls get_old_root(), and finally that calls __tree_mod_log_oldest_root() with time_seq == 1 and eb_root == eb Y;

11) First iteration of the while loop finds the tree mod log element with sequence number N + 2, for the logical address of eb Y and of type MOD_LOG_ROOT_REPLACE;

12) Because the operation type is MOD_LOG_ROOT_REPLACE, we don't break out

    of the loop, and set root_logical to point to tm->old_root.logical
    which corresponds to the logical address of eb X;

13) On the next iteration of the while loop, the call to tree_mod_log_search_oldest() returns the smallest tree mod log element for the logical address of eb X, which has a sequence number of 2, an operation type of MOD_LOG_KEY_REMOVE_WHILE_FREEING and corresponds to the old slot N - 1 of eb X (eb X had N items in it before being freed);

14) We then break out of the while loop and return the tree mod log operation of type MOD_LOG_ROOT_REPLACE (eb Y), and not the one for slot N - 1 of eb X, to get_old_root();

15) At get_old_root(), we process the MOD_LOG_ROOT_REPLACE operation and set "logical" to the logical address of eb X, which was the old root. We then call tree_mod_log_search() passing it the logical address of eb X and time_seq == 1;

16) Then before calling tree_mod_log_search(), task T adds a key to eb X, which results in adding a tree mod log operation of type MOD_LOG_KEY_ADD to the tree mod log - this is done at ctree.c:insert_ptr() - but after adding the tree mod log operation and before updating the number of items in eb X from 0 to 1...

17) The task at get_old_root() calls tree_mod_log_search() and gets the tree mod log operation of type MOD_LOG_KEY_ADD just added by task T. Then it enters the following if branch:

if (old_root && tm && tm->op != MOD_LOG_KEY_REMOVE_WHILE_FREEING) {
(...)
} (...)

Calls read_tree_block() for eb X, which gets a reference on eb X but does not lock it - task T has it locked.
Then it clones eb X while it has nritems set to 0 in its header, before task T sets nritems to 1 in eb X's header. From hereupon we use the clone of eb X which no other task has access to;

18) Then we call __tree_mod_log_rewind(), passing it the MOD_LOG_KEY_ADD mod log operation we just got from tree_mod_log_search() in the previous step and the cloned version of eb X;

19) At __tree_mod_log_rewind(), we set the local variable "n" to the number of items set in eb X's clone, which is 0. Then we enter the while loop, and in its first iteration we process the MOD_LOG_KEY_ADD operation, which just decrements "n" from 0 to (u32)-1, since "n" is declared with a type of u32. At the end of this iteration we call rb_next() to find the next tree mod log operation for eb X, that gives us the mod log operation of type MOD_LOG_KEY_REMOVE_WHILE_FREEING, for slot 0, with a sequence number of N + 1 (steps 3 to 6);

20) Then we go back to the top of the while loop and trigger the following

    BUG_ON():

        (...)
        switch (tm->op) {
        case MOD_LOG_KEY_REMOVE_WHILE_FREEING:
                 BUG_ON(tm->slot < n);
                 fallthrough;
        (...)

    Because "n" has a value of (u32)-1 (4294967295) and tm->slot is 0.

Fix this by taking a read lock on the extent buffer before cloning it at ctree.c:get_old_root(). This should be done regardless of the extent buffer having been freed and reused, as a concurrent task might be modifying it (while holding a write lock on it).

Fixes 834328a8493079 ("Btrfs: tree mod log's old roots could still be part of the tree")
---
 fs/btrfs/ctree.c | 2 ++
 1 file changed, 2 insertions(+)
fs/btrfs/ctree.c +2 -0
index f33e69f02230..19d7b81cee55 100644
@@ -1365,7 +1365,9 @@ get_old_root(struct btrfs_root *root, u64 time_seq)
 				   "failed to read tree block %llu from get_old_root",
 				   logical);
 		} else {
+			btrfs_tree_read_lock(old);
 			eb = btrfs_clone_extent_buffer(old);
+			btrfs_tree_read_unlock(old);
 			free_extent_buffer(old);
 		}
 	} else if (old_root) {
Signature · 1 line
2.28.0
#2 lore raw

PATCH 2/9 btrfs: always pin deleted leaves when there are active tree mod log users

In reply to [email protected]

From: Filipe Manana <[email protected]>

When freeing a tree block we may end up adding its extent back to the free space cache/tree, as long as there are no more references for it, it was created in the current transaction and writeback for it never happened. This is generally fine, however when we have tree mod log operations it can result in inconsistent versions of a btree after unwinding extent buffers with the recorded tree mod log operations.

This is because:

* We only log operations for nodes (adding and removing key/pointers), for leaves we don't do anything;

* This means that we can log a MOD_LOG_KEY_REMOVE_WHILE_FREEING operation for a node that points to a leaf that was deleted;

* Before we apply the logged operation to unwind a node, we can have that leaf's extent allocated again, either as a node or as a leaf, and possibly for another btree. This is possible if the leaf was created in the current transaction and writeback for it never started, in which case btrfs_free_tree_block() returns its extent back to the free space cache/tree;

* Then, before applying the tree mod log operation, some task allocates the metadata extent just freed before, and uses it either as a leaf or as a node for some btree (can be the same or another one, it does not matter);

* After applying the MOD_LOG_KEY_REMOVE_WHILE_FREEING operation we now get the target node with an item pointing to the metadata extent that now has content different from what it had before the leaf was deleted. It might now belong to a different btree and be a node and not a leaf anymore.

As a consequence, the results of searches after the unwinding can be unpredictable and produce unexpected results.

So make sure we pin extent buffers corresponding to leaves when there are tree mod log users.

---
 fs/btrfs/extent-tree.c | 23 ++++++++++++++++++++++-
 1 file changed, 22 insertions(+), 1 deletion(-)
fs/btrfs/extent-tree.c +22 -1
index 5e228d6ad63f..2482b26b1971 100644
@@ -3310,6 +3310,7 @@ void btrfs_free_tree_block(struct btrfs_trans_handle *trans,
 
 	if (last_ref && btrfs_header_generation(buf) == trans->transid) {
 		struct btrfs_block_group *cache;
+		bool must_pin = false;
 
 		if (root->root_key.objectid != BTRFS_TREE_LOG_OBJECTID) {
 			ret = check_ref_cleanup(trans, buf->start);
@@ -3327,7 +3328,27 @@ void btrfs_free_tree_block(struct btrfs_trans_handle *trans,
 			goto out;
 		}
 
-		if (btrfs_is_zoned(fs_info)) {
+		/*
+		 * If this is a leaf and there are tree mod log users, we may
+		 * have recorded mod log operations that point to this leaf.
+		 * So we must make sure no one reuses this leaf's extent before
+		 * mod log operations are applied to a node, otherwise after
+		 * rewinding a node using the mod log operations we get an
+		 * inconsistent btree, as the leaf's extent may now be used as
+		 * a node or leaf for another different btree.
+		 * We are safe from races here because at this point no other
+		 * node or root points to this extent buffer, so if after this
+		 * check a new tree mod log user joins, it will not be able to
+		 * find a node pointing to this leaf and record operations that
+		 * point to this leaf.
+		 */
+		if (btrfs_header_level(buf) == 0) {
+			read_lock(&fs_info->tree_mod_log_lock);
+			must_pin = !list_empty(&fs_info->tree_mod_seq_list);
+			read_unlock(&fs_info->tree_mod_log_lock);
+		}
+
+		if (must_pin || btrfs_is_zoned(fs_info)) {
 			btrfs_redirty_list_add(trans->transaction, buf);
 			pin_down_extent(trans, cache, buf->start, buf->len, 1);
 			btrfs_put_block_group(cache);
Signature · 1 line
2.28.0
#6 lore raw

PATCH 3/9 btrfs: move the tree mod log code into its own file

In reply to [email protected]

From: Filipe Manana <[email protected]>

The tree modification log, which records modifications done to btrees, is quite large and currently spread all over ctree.c, which is a huge file already.

To make things better organized, move all that code into its own separate source and header files. Functions and definitions that are used outside of the module (mostly by ctree.c) are renamed so that they start with a "btrfs_" prefix. Everything else remains unchanged.

This makes it easier to go over the tree modification log code everytime I need to go read it to fix a bug.

---
 fs/btrfs/Makefile       |   2 +-
 fs/btrfs/backref.c      |  33 +-
 fs/btrfs/ctree.c        | 956 ++--------------------------------------
 fs/btrfs/ctree.h        |  17 -
 fs/btrfs/delayed-ref.c  |   9 +-
 fs/btrfs/qgroup.c       |   9 +-
 fs/btrfs/tree-mod-log.c | 889 +++++++++++++++++++++++++++++++++++++
 fs/btrfs/tree-mod-log.h |  52 +++
 8 files changed, 1006 insertions(+), 961 deletions(-)
 create mode 100644 fs/btrfs/tree-mod-log.c
 create mode 100644 fs/btrfs/tree-mod-log.h
fs/btrfs/Makefile +1 -1
index b634c42115ea..fdba34b10a98 100644
@@ -28,7 +28,7 @@ btrfs-y += super.o ctree.o extent-tree.o print-tree.o root-tree.o dir-item.o \
 	   reada.o backref.o ulist.o qgroup.o send.o dev-replace.o raid56.o \
 	   uuid-tree.o props.o free-space-tree.o tree-checker.o space-info.o \
 	   block-rsv.o delalloc-space.o block-group.o discard.o reflink.o \
-	   subpage.o
+	   subpage.o tree-mod-log.o
 
 btrfs-$(CONFIG_BTRFS_FS_POSIX_ACL) += acl.o
 btrfs-$(CONFIG_BTRFS_FS_CHECK_INTEGRITY) += check-integrity.o
fs/btrfs/backref.c +17 -16
index f47c1528eb9a..117d423fdb93 100644
@@ -14,6 +14,7 @@
 #include "delayed-ref.h"
 #include "locking.h"
 #include "misc.h"
+#include "tree-mod-log.h"
 
 /* Just an arbitrary number so we can be sure this happened */
 #define BACKREF_FOUND_SHARED 6
@@ -452,7 +453,7 @@ static int add_all_parents(struct btrfs_root *root, struct btrfs_path *path,
 	if (path->slots[0] >= btrfs_header_nritems(eb) ||
 	    is_shared_data_backref(preftrees, eb->start) ||
 	    ref->root_id != btrfs_header_owner(eb)) {
-		if (time_seq == SEQ_LAST)
+		if (time_seq == BTRFS_SEQ_LAST)
 			ret = btrfs_next_leaf(root, path);
 		else
 			ret = btrfs_next_old_leaf(root, path, time_seq);
@@ -476,7 +477,7 @@ static int add_all_parents(struct btrfs_root *root, struct btrfs_path *path,
 		if (slot == 0 &&
 		    (is_shared_data_backref(preftrees, eb->start) ||
 		     ref->root_id != btrfs_header_owner(eb))) {
-			if (time_seq == SEQ_LAST)
+			if (time_seq == BTRFS_SEQ_LAST)
 				ret = btrfs_next_leaf(root, path);
 			else
 				ret = btrfs_next_old_leaf(root, path, time_seq);
@@ -514,7 +515,7 @@ static int add_all_parents(struct btrfs_root *root, struct btrfs_path *path,
 			eie = NULL;
 		}
 next:
-		if (time_seq == SEQ_LAST)
+		if (time_seq == BTRFS_SEQ_LAST)
 			ret = btrfs_next_item(root, path);
 		else
 			ret = btrfs_next_old_item(root, path, time_seq);
@@ -574,7 +575,7 @@ static int resolve_indirect_ref(struct btrfs_fs_info *fs_info,
 
 	if (path->search_commit_root)
 		root_level = btrfs_header_level(root->commit_root);
-	else if (time_seq == SEQ_LAST)
+	else if (time_seq == BTRFS_SEQ_LAST)
 		root_level = btrfs_header_level(root->node);
 	else
 		root_level = btrfs_old_root_level(root, time_seq);
@@ -605,7 +606,7 @@ static int resolve_indirect_ref(struct btrfs_fs_info *fs_info,
 	    search_key.offset >= LLONG_MAX)
 		search_key.offset = 0;
 	path->lowest_level = level;
-	if (time_seq == SEQ_LAST)
+	if (time_seq == BTRFS_SEQ_LAST)
 		ret = btrfs_search_slot(NULL, root, &search_key, path, 0, 0);
 	else
 		ret = btrfs_search_old_slot(root, &search_key, path, time_seq);
@@ -1147,8 +1148,8 @@ static int add_keyed_refs(struct btrfs_fs_info *fs_info,
  * indirect refs to their parent bytenr.
  * When roots are found, they're added to the roots list
  *
- * If time_seq is set to SEQ_LAST, it will not search delayed_refs, and behave
- * much like trans == NULL case, the difference only lies in it will not
+ * If time_seq is set to BTRFS_SEQ_LAST, it will not search delayed_refs, and
+ * behave much like trans == NULL case, the difference only lies in it will not
  * commit root.
  * The special case is for qgroup to search roots in commit_transaction().
  *
@@ -1199,7 +1200,7 @@ static int find_parent_nodes(struct btrfs_trans_handle *trans,
 		path->skip_locking = 1;
 	}
 
-	if (time_seq == SEQ_LAST)
+	if (time_seq == BTRFS_SEQ_LAST)
 		path->skip_locking = 1;
 
 	/*
@@ -1217,9 +1218,9 @@ static int find_parent_nodes(struct btrfs_trans_handle *trans,
 
 #ifdef CONFIG_BTRFS_FS_RUN_SANITY_TESTS
 	if (trans && likely(trans->type != __TRANS_DUMMY) &&
-	    time_seq != SEQ_LAST) {
+	    time_seq != BTRFS_SEQ_LAST) {
 #else
-	if (trans && time_seq != SEQ_LAST) {
+	if (trans && time_seq != BTRFS_SEQ_LAST) {
 #endif
 		/*
 		 * look if there are updates for this ref queued and lock the
@@ -1527,7 +1528,7 @@ int btrfs_check_shared(struct btrfs_root *root, u64 inum, u64 bytenr,
 	struct btrfs_trans_handle *trans;
 	struct ulist_iterator uiter;
 	struct ulist_node *node;
-	struct seq_list elem = SEQ_LIST_INIT(elem);
+	struct btrfs_seq_list elem = BTRFS_SEQ_LIST_INIT(elem);
 	int ret = 0;
 	struct share_check shared = {
 		.root_objectid = root->root_key.objectid,
@@ -1953,7 +1954,7 @@ int iterate_extent_inodes(struct btrfs_fs_info *fs_info,
 	struct ulist *roots = NULL;
 	struct ulist_node *ref_node = NULL;
 	struct ulist_node *root_node = NULL;
-	struct seq_list tree_mod_seq_elem = SEQ_LIST_INIT(tree_mod_seq_elem);
+	struct btrfs_seq_list seq_elem = BTRFS_SEQ_LIST_INIT(seq_elem);
 	struct ulist_iterator ref_uiter;
 	struct ulist_iterator root_uiter;
 
@@ -1971,12 +1972,12 @@ int iterate_extent_inodes(struct btrfs_fs_info *fs_info,
 	}
 
 	if (trans)
-		btrfs_get_tree_mod_seq(fs_info, &tree_mod_seq_elem);
+		btrfs_get_tree_mod_seq(fs_info, &seq_elem);
 	else
 		down_read(&fs_info->commit_root_sem);
 
 	ret = btrfs_find_all_leafs(trans, fs_info, extent_item_objectid,
-				   tree_mod_seq_elem.seq, &refs,
+				   seq_elem.seq, &refs,
 				   &extent_item_pos, ignore_offset);
 	if (ret)
 		goto out;
@@ -1984,7 +1985,7 @@ int iterate_extent_inodes(struct btrfs_fs_info *fs_info,
 	ULIST_ITER_INIT(&ref_uiter);
 	while (!ret && (ref_node = ulist_next(refs, &ref_uiter))) {
 		ret = btrfs_find_all_roots_safe(trans, fs_info, ref_node->val,
-						tree_mod_seq_elem.seq, &roots,
+						seq_elem.seq, &roots,
 						ignore_offset);
 		if (ret)
 			break;
@@ -2007,7 +2008,7 @@ int iterate_extent_inodes(struct btrfs_fs_info *fs_info,
 	free_leaf_list(refs);
 out:
 	if (trans) {
-		btrfs_put_tree_mod_seq(fs_info, &tree_mod_seq_elem);
+		btrfs_put_tree_mod_seq(fs_info, &seq_elem);
 		btrfs_end_transaction(trans);
 	} else {
 		up_read(&fs_info->commit_root_sem);
fs/btrfs/ctree.c +37 -919

1,106 lines not on this page · raw message

fs/btrfs/ctree.h +0 -17
index ae8901aec4f1..50208673bd55 100644
@@ -500,16 +500,6 @@ struct btrfs_discard_ctl {
 	atomic64_t discard_bytes_saved;
 };
 
-/* delayed seq elem */
-struct seq_list {
-	struct list_head list;
-	u64 seq;
-};
-
-#define SEQ_LIST_INIT(name)	{ .list = LIST_HEAD_INIT((name).list), .seq = 0 }
-
-#define SEQ_LAST	((u64)-1)
-
 enum btrfs_orphan_cleanup_state {
 	ORPHAN_CLEANUP_STARTED	= 1,
 	ORPHAN_CLEANUP_DONE	= 2,
@@ -2946,13 +2936,6 @@ static inline void btrfs_clear_sb_rdonly(struct super_block *sb)
 	clear_bit(BTRFS_FS_STATE_RO, &btrfs_sb(sb)->fs_state);
 }
 
-/* tree mod log functions from ctree.c */
-u64 btrfs_get_tree_mod_seq(struct btrfs_fs_info *fs_info,
-			   struct seq_list *elem);
-void btrfs_put_tree_mod_seq(struct btrfs_fs_info *fs_info,
-			    struct seq_list *elem);
-int btrfs_old_root_level(struct btrfs_root *root, u64 time_seq);
-
 /* root-item.c */
 int btrfs_add_root_ref(struct btrfs_trans_handle *trans, u64 root_id,
 		       u64 ref_id, u64 dirid, u64 sequence, const char *name,
fs/btrfs/delayed-ref.c +5 -4
index 63be7d01a9a3..d87472a68bf4 100644
@@ -11,6 +11,7 @@
 #include "transaction.h"
 #include "qgroup.h"
 #include "space-info.h"
+#include "tree-mod-log.h"
 
 struct kmem_cache *btrfs_delayed_ref_head_cachep;
 struct kmem_cache *btrfs_delayed_tree_ref_cachep;
@@ -496,10 +497,10 @@ void btrfs_merge_delayed_refs(struct btrfs_trans_handle *trans,
 
 	read_lock(&fs_info->tree_mod_log_lock);
 	if (!list_empty(&fs_info->tree_mod_seq_list)) {
-		struct seq_list *elem;
+		struct btrfs_seq_list *elem;
 
 		elem = list_first_entry(&fs_info->tree_mod_seq_list,
-					struct seq_list, list);
+					struct btrfs_seq_list, list);
 		seq = elem->seq;
 	}
 	read_unlock(&fs_info->tree_mod_log_lock);
@@ -517,13 +518,13 @@ void btrfs_merge_delayed_refs(struct btrfs_trans_handle *trans,
 
 int btrfs_check_delayed_seq(struct btrfs_fs_info *fs_info, u64 seq)
 {
-	struct seq_list *elem;
+	struct btrfs_seq_list *elem;
 	int ret = 0;
 
 	read_lock(&fs_info->tree_mod_log_lock);
 	if (!list_empty(&fs_info->tree_mod_seq_list)) {
 		elem = list_first_entry(&fs_info->tree_mod_seq_list,
-					struct seq_list, list);
+					struct btrfs_seq_list, list);
 		if (seq >= elem->seq) {
 			btrfs_debug(fs_info,
 				"holding back delayed_ref %#x.%x, lowest is %#x.%x",
fs/btrfs/qgroup.c +5 -4
index 1b7d3cadc39e..9d8d1ac86962 100644
@@ -23,6 +23,7 @@
 #include "qgroup.h"
 #include "block-group.h"
 #include "sysfs.h"
+#include "tree-mod-log.h"
 
 /* TODO XXX FIXME
  *  - subvol delete -> delete when ref goes to 0? delete limits also?
@@ -2631,12 +2632,12 @@ int btrfs_qgroup_account_extents(struct btrfs_trans_handle *trans)
 					record->data_rsv,
 					BTRFS_QGROUP_RSV_DATA);
 			/*
-			 * Use SEQ_LAST as time_seq to do special search, which
-			 * doesn't lock tree or delayed_refs and search current
-			 * root. It's safe inside commit_transaction().
+			 * Use BTRFS_SEQ_LAST as time_seq to do special search,
+			 * which doesn't lock tree or delayed_refs and search
+			 * current root. It's safe inside commit_transaction().
 			 */
 			ret = btrfs_find_all_roots(trans, fs_info,
-				record->bytenr, SEQ_LAST, &new_roots, false);
+				record->bytenr, BTRFS_SEQ_LAST, &new_roots, false);
 			if (ret < 0)
 				goto cleanup;
 			if (qgroup_to_skip) {
fs/btrfs/tree-mod-log.c new file +889 -0

889 lines not on this page · raw message

fs/btrfs/tree-mod-log.h new file +52 -0

52 lines not on this page · raw message

Signature · 1 line
2.28.0
#10 lore raw

PATCH 4/9 btrfs: use booleans where appropriate for the tree mod log functions

In reply to [email protected]

From: Filipe Manana <[email protected]>

Several functions of the tree modification log use integers as booleans, so change them to use booleans instead, making their use more clear.

---
 fs/btrfs/ctree.c        |  6 +++---
 fs/btrfs/tree-mod-log.c | 42 ++++++++++++++++++++---------------------
 fs/btrfs/tree-mod-log.h |  2 +-
 3 files changed, 25 insertions(+), 25 deletions(-)
fs/btrfs/ctree.c +3 -3
index 2c6e525b3458..1b0f28f69cb3 100644
@@ -500,7 +500,7 @@ static noinline int __btrfs_cow_block(struct btrfs_trans_handle *trans,
 			parent_start = buf->start;
 
 		atomic_inc(&cow->refs);
-		ret = btrfs_tree_mod_log_insert_root(root->node, cow, 1);
+		ret = btrfs_tree_mod_log_insert_root(root->node, cow, true);
 		BUG_ON(ret < 0);
 		rcu_assign_pointer(root->node, cow);
 
@@ -958,7 +958,7 @@ static noinline int balance_level(struct btrfs_trans_handle *trans,
 			goto enospc;
 		}
 
-		ret = btrfs_tree_mod_log_insert_root(root->node, child, 1);
+		ret = btrfs_tree_mod_log_insert_root(root->node, child, true);
 		BUG_ON(ret < 0);
 		rcu_assign_pointer(root->node, child);
 
@@ -2480,7 +2480,7 @@ static noinline int insert_new_root(struct btrfs_trans_handle *trans,
 	btrfs_mark_buffer_dirty(c);
 
 	old = root->node;
-	ret = btrfs_tree_mod_log_insert_root(root->node, c, 0);
+	ret = btrfs_tree_mod_log_insert_root(root->node, c, false);
 	BUG_ON(ret < 0);
 	rcu_assign_pointer(root->node, c);
 
fs/btrfs/tree-mod-log.c +21 -21

131 lines not on this page · raw message

fs/btrfs/tree-mod-log.h +1 -1
index 15e7a4c4bb14..2f7396ea57ae 100644
@@ -30,7 +30,7 @@ void btrfs_put_tree_mod_seq(struct btrfs_fs_info *fs_info,
 			    struct btrfs_seq_list *elem);
 int btrfs_tree_mod_log_insert_root(struct extent_buffer *old_root,
 				   struct extent_buffer *new_root,
-				   int log_removal);
+				   bool log_removal);
 int btrfs_tree_mod_log_insert_key(struct extent_buffer *eb, int slot,
 				  enum btrfs_mod_log_op op, gfp_t flags);
 int btrfs_tree_mod_log_free_eb(struct extent_buffer *eb);
Signature · 1 line
2.28.0
#12 lore raw

PATCH 5/9 btrfs: use a bit to track the existence of tree mod log users

In reply to [email protected]

From: Filipe Manana <[email protected]>

The tree modification log functions are called very frequently, basically they are called everytime a btree is modified (a pointer added or removed to a node, a new root for a btree is set, etc). Because of that, to avoid heavy lock contention on the lock that protects the list of tree mod log users, we have checks that test the emptiness of the list with a full memory barrier before the checks, so that when there are no tree mod log users we avoid taking the lock.

Replace the memory barrier and list emptiness check with a test for a new bit set at fs_info->flags. This bit is used to indicate when there are tree mod log users, set whenever a user is added to the list and cleared when the last user is removed from the list. This makes the intention a bit more obvious and possibly more efficient (assuming test_bit() may be cheaper than a full memory barrier on some architectures).

---
 fs/btrfs/ctree.h        |  3 +++
 fs/btrfs/tree-mod-log.c | 11 ++++++-----
 2 files changed, 9 insertions(+), 5 deletions(-)
fs/btrfs/ctree.h +3 -0
index 50208673bd55..90184ee2f832 100644
@@ -580,6 +580,9 @@ enum {
 
 	/* Indicate that we can't trust the free space tree for caching yet */
 	BTRFS_FS_FREE_SPACE_TREE_UNTRUSTED,
+
+	/* Indicate whether there are any tree modification log users. */
+	BTRFS_FS_TREE_MOD_LOG_USERS,
 };
 
 /*
fs/btrfs/tree-mod-log.c +6 -5
index f2a6da1b2903..b912b82c36c9 100644
@@ -60,6 +60,7 @@ u64 btrfs_get_tree_mod_seq(struct btrfs_fs_info *fs_info,
 	if (!elem->seq) {
 		elem->seq = btrfs_inc_tree_mod_seq(fs_info);
 		list_add_tail(&elem->list, &fs_info->tree_mod_seq_list);
+		set_bit(BTRFS_FS_TREE_MOD_LOG_USERS, &fs_info->flags);
 	}
 	write_unlock(&fs_info->tree_mod_log_lock);
 
@@ -83,7 +84,9 @@ void btrfs_put_tree_mod_seq(struct btrfs_fs_info *fs_info,
 	list_del(&elem->list);
 	elem->seq = 0;
 
-	if (!list_empty(&fs_info->tree_mod_seq_list)) {
+	if (list_empty(&fs_info->tree_mod_seq_list)) {
+		clear_bit(BTRFS_FS_TREE_MOD_LOG_USERS, &fs_info->flags);
+	} else {
 		struct btrfs_seq_list *first;
 
 		first = list_first_entry(&fs_info->tree_mod_seq_list,
@@ -166,8 +169,7 @@ static noinline int tree_mod_log_insert(struct btrfs_fs_info *fs_info,
 static inline bool tree_mod_dont_log(struct btrfs_fs_info *fs_info,
 				    struct extent_buffer *eb)
 {
-	smp_mb();
-	if (list_empty(&(fs_info)->tree_mod_seq_list))
+	if (!test_bit(BTRFS_FS_TREE_MOD_LOG_USERS, &fs_info->flags))
 		return true;
 	if (eb && btrfs_header_level(eb) == 0)
 		return true;
@@ -185,8 +187,7 @@ static inline bool tree_mod_dont_log(struct btrfs_fs_info *fs_info,
 static inline bool tree_mod_need_log(const struct btrfs_fs_info *fs_info,
 				    struct extent_buffer *eb)
 {
-	smp_mb();
-	if (list_empty(&(fs_info)->tree_mod_seq_list))
+	if (!test_bit(BTRFS_FS_TREE_MOD_LOG_USERS, &fs_info->flags))
 		return false;
 	if (eb && btrfs_header_level(eb) == 0)
 		return false;
Signature · 1 line
2.28.0
#15 lore raw

PATCH 6/9 btrfs: use the new bit BTRFS_FS_TREE_MOD_LOG_USERS at btrfs_free_tree_block()

In reply to [email protected]

From: Filipe Manana <[email protected]>

Instead of exposing implementation details of the tree mod log to check if there are active tree mod log users at btrfs_free_tree_block(), use the new bit BTRFS_FS_TREE_MOD_LOG_USERS for fs_info->flags instead. This way extent-tree.c does not need to known about any of the internals of the tree mod log and avoids taking a lock unnecessarily as well.

---
 fs/btrfs/extent-tree.c | 8 +++-----
 1 file changed, 3 insertions(+), 5 deletions(-)
fs/btrfs/extent-tree.c +3 -5
index 2482b26b1971..7a28314189b4 100644
@@ -3342,11 +3342,9 @@ void btrfs_free_tree_block(struct btrfs_trans_handle *trans,
 		 * find a node pointing to this leaf and record operations that
 		 * point to this leaf.
 		 */
-		if (btrfs_header_level(buf) == 0) {
-			read_lock(&fs_info->tree_mod_log_lock);
-			must_pin = !list_empty(&fs_info->tree_mod_seq_list);
-			read_unlock(&fs_info->tree_mod_log_lock);
-		}
+		if (btrfs_header_level(buf) == 0 &&
+		    test_bit(BTRFS_FS_TREE_MOD_LOG_USERS, &fs_info->flags))
+			must_pin = true;
 
 		if (must_pin || btrfs_is_zoned(fs_info)) {
 			btrfs_redirty_list_add(trans->transaction, buf);
Signature · 1 line
2.28.0
#16 lore raw

PATCH 7/9 btrfs: remove unnecessary leaf check at btrfs_tree_mod_log_free_eb()

In reply to [email protected]

From: Filipe Manana <[email protected]>

At btrfs_tree_mod_log_free_eb() we check if we are dealing with a leaf, and if so, return immediately and do nothing. However this check can be removed, because after it we call tree_mod_need_log(), which returns false when given an extent buffer that corresponds to a leaf.

So just remove the leaf check and pass the extent buffer to tree_mod_need_log().

---
 fs/btrfs/tree-mod-log.c | 5 +----
 1 file changed, 1 insertion(+), 4 deletions(-)
fs/btrfs/tree-mod-log.c +1 -4
index b912b82c36c9..c854cebf27f6 100644
@@ -554,10 +554,7 @@ int btrfs_tree_mod_log_free_eb(struct extent_buffer *eb)
 	int i;
 	int ret = 0;
 
-	if (btrfs_header_level(eb) == 0)
-		return 0;
-
-	if (!tree_mod_need_log(eb->fs_info, NULL))
+	if (!tree_mod_need_log(eb->fs_info, eb))
 		return 0;
 
 	nritems = btrfs_header_nritems(eb);
Signature · 1 line
2.28.0
#17 lore raw

PATCH 8/9 btrfs: add and use helper to get lowest sequence number for the tree mod log

In reply to [email protected]

From: Filipe Manana <[email protected]>

There are two places outside the tree mod log module that extract the lowest sequence number of the tree mod log. These places end up duplicating code and open coding the logic and internal implementation details of the tree mod log. So add a helper to the tree mod log module and header that returns the lowest sequence number or 0 if there aren't any tree mod log users at the moment.

---
 fs/btrfs/delayed-ref.c  | 31 ++++++++-----------------------
 fs/btrfs/tree-mod-log.c | 25 +++++++++++++++++++++++++
 fs/btrfs/tree-mod-log.h |  1 +
 3 files changed, 34 insertions(+), 23 deletions(-)
fs/btrfs/delayed-ref.c +8 -23
index d87472a68bf4..7a3131cbf1fb 100644
@@ -495,16 +495,7 @@ void btrfs_merge_delayed_refs(struct btrfs_trans_handle *trans,
 	if (head->is_data)
 		return;
 
-	read_lock(&fs_info->tree_mod_log_lock);
-	if (!list_empty(&fs_info->tree_mod_seq_list)) {
-		struct btrfs_seq_list *elem;
-
-		elem = list_first_entry(&fs_info->tree_mod_seq_list,
-					struct btrfs_seq_list, list);
-		seq = elem->seq;
-	}
-	read_unlock(&fs_info->tree_mod_log_lock);
-
+	seq = btrfs_tree_mod_log_lowest_seq(fs_info);
 again:
 	for (node = rb_first_cached(&head->ref_tree); node;
 	     node = rb_next(node)) {
@@ -518,23 +509,17 @@ void btrfs_merge_delayed_refs(struct btrfs_trans_handle *trans,
 
 int btrfs_check_delayed_seq(struct btrfs_fs_info *fs_info, u64 seq)
 {
-	struct btrfs_seq_list *elem;
+	u64 min_seq = btrfs_tree_mod_log_lowest_seq(fs_info);
 	int ret = 0;
 
-	read_lock(&fs_info->tree_mod_log_lock);
-	if (!list_empty(&fs_info->tree_mod_seq_list)) {
-		elem = list_first_entry(&fs_info->tree_mod_seq_list,
-					struct btrfs_seq_list, list);
-		if (seq >= elem->seq) {
-			btrfs_debug(fs_info,
-				"holding back delayed_ref %#x.%x, lowest is %#x.%x",
-				(u32)(seq >> 32), (u32)seq,
-				(u32)(elem->seq >> 32), (u32)elem->seq);
-			ret = 1;
-		}
+	if (min_seq != 0 && seq >= min_seq) {
+		btrfs_debug(fs_info,
+			    "holding back delayed_ref %#x.%x, lowest is %#x.%x",
+			    (u32)(seq >> 32), (u32)seq,
+			    (u32)(min_seq >> 32), (u32)min_seq);
+		ret = 1;
 	}
 
-	read_unlock(&fs_info->tree_mod_log_lock);
 	return ret;
 }
 
fs/btrfs/tree-mod-log.c +25 -0
index c854cebf27f6..b2083b575e21 100644
@@ -885,3 +885,28 @@ int btrfs_old_root_level(struct btrfs_root *root, u64 time_seq)
 	return level;
 }
 
+/*
+ * Return the lowest sequence number in the tree modification log.
+ *
+ * @fs_info:	the filesystem context
+ *
+ * Returns the sequence number of the oldest tree modification log user, which
+ * corresponds to the lowest sequence number of all existing users. If there are
+ * no users it returns 0.
+ */
+u64 btrfs_tree_mod_log_lowest_seq(struct btrfs_fs_info *fs_info)
+{
+	u64 ret = 0;
+
+	read_lock(&fs_info->tree_mod_log_lock);
+	if (!list_empty(&fs_info->tree_mod_seq_list)) {
+		struct btrfs_seq_list *elem;
+
+		elem = list_first_entry(&fs_info->tree_mod_seq_list,
+					struct btrfs_seq_list, list);
+		ret = elem->seq;
+	}
+	read_unlock(&fs_info->tree_mod_log_lock);
+
+	return ret;
+}
fs/btrfs/tree-mod-log.h +1 -0
index 2f7396ea57ae..d218f8a661bc 100644
@@ -48,5 +48,6 @@ int btrfs_tree_mod_log_eb_copy(struct extent_buffer *dst,
 int btrfs_tree_mod_log_insert_move(struct extent_buffer *eb,
 				   int dst_slot, int src_slot,
 				   int nr_items);
+u64 btrfs_tree_mod_log_lowest_seq(struct btrfs_fs_info *fs_info);
 
 #endif /* BTRFS_TREE_MOD_LOG_H */
Signature · 1 line
2.28.0
#18 lore raw

PATCH 9/9 btrfs: update debug message when checking seq number of a delayed ref

In reply to [email protected]

From: Filipe Manana <[email protected]>

We used to encode two different numbers in the tree mod log counter used for sequence numbers, one in the upper 32 bits and the other one in the lower 32 bits. However that is no longer the case, we stopped doing that since commit fcebe4562dec83 ("Btrfs: rework qgroup accounting").

So update the debug message at btrfs_check_delayed_seq to stop extracting the two 32 bits counters and print instead the 64 bits sequence numbers.

---
 fs/btrfs/delayed-ref.c | 5 ++---
 1 file changed, 2 insertions(+), 3 deletions(-)
fs/btrfs/delayed-ref.c +2 -3
index 7a3131cbf1fb..8c1d78befb5b 100644
@@ -514,9 +514,8 @@ int btrfs_check_delayed_seq(struct btrfs_fs_info *fs_info, u64 seq)
 
 	if (min_seq != 0 && seq >= min_seq) {
 		btrfs_debug(fs_info,
-			    "holding back delayed_ref %#x.%x, lowest is %#x.%x",
-			    (u32)(seq >> 32), (u32)seq,
-			    (u32)(min_seq >> 32), (u32)min_seq);
+			    "holding back delayed_ref %llu, lowest is %llu",
+			    seq, min_seq);
 		ret = 1;
 	}
 
Signature · 1 line
2.28.0
#7 lore raw

Re: [PATCH 3/9] btrfs: move the tree mod log code into its own file

In reply to [email protected]
Thank you for the patch! Perhaps something to improve:

Hi,

Thank you for the patch! Perhaps something to improve:

[auto build test WARNING on v5.12-rc2]
[also build test WARNING on next-20210311]
[cannot apply to kdave/for-next]
[If your patch is applied to the wrong git tree, kindly drop us a note. And when submitting patch, we suggest to use '--base' as documented in
https://git-scm.com/docs/git-format-patch]

url:    https://github.com/0day-ci/linux/commits/fdmanana-kernel-org/btrfs-bug-fixes-for-the-tree-mod-log-and-small-refactorings/20210311-223429
base:    a38fd8748464831584a19438cbb3082b5a2dab15
config: x86_64-randconfig-m001-20210311 (attached as .config)
compiler: gcc-9 (Debian 9.3.0-22) 9.3.0

If you fix the issue, kindly add following tag as appropriate

smatch warnings:
fs/btrfs/tree-mod-log.c:286 btrfs_tree_mod_log_insert_move() warn: missing error code 'ret' fs/btrfs/tree-mod-log.c:519 btrfs_tree_mod_log_eb_copy() warn: missing error code 'ret' fs/btrfs/tree-mod-log.c:577 btrfs_tree_mod_log_free_eb() warn: missing error code 'ret'

vim +/ret +286 fs/btrfs/tree-mod-log.c

246
247 int btrfs_tree_mod_log_insert_move(struct extent_buffer *eb,
248 int dst_slot, int src_slot,
249 int nr_items)
250 {
251 struct tree_mod_elem *tm = NULL;
252 struct tree_mod_elem **tm_list = NULL;
253 int ret = 0;
254 int i;
255 int locked = 0;
256
257 if (!tree_mod_need_log(eb->fs_info, eb))
258 return 0;
259
260 tm_list = kcalloc(nr_items, sizeof(struct tree_mod_elem *), GFP_NOFS);
261 if (!tm_list)
262 return -ENOMEM;
263
264 tm = kzalloc(sizeof(*tm), GFP_NOFS);
265 if (!tm) {
266 ret = -ENOMEM;
267 goto free_tms;
268 }
269
270 tm->logical = eb->start;
271 tm->slot = src_slot;
272 tm->move.dst_slot = dst_slot;
273 tm->move.nr_items = nr_items;
274 tm->op = BTRFS_MOD_LOG_MOVE_KEYS;
275
276 for (i = 0; i + dst_slot < src_slot && i < nr_items; i++) {
277 tm_list[i] = alloc_tree_mod_elem(eb, i + dst_slot,
278 BTRFS_MOD_LOG_KEY_REMOVE_WHILE_MOVING, GFP_NOFS);
279 if (!tm_list[i]) {
280 ret = -ENOMEM;
281 goto free_tms;
282 }
283 }
284
285 if (tree_mod_dont_log(eb->fs_info, eb))

 > 286			goto free_tms;

287 locked = 1;
288
289 /*
290 * When we override something during the move, we log these removals.
291 * This can only happen when we move towards the beginning of the
292 * buffer, i.e. dst_slot < src_slot.
293 */
294 for (i = 0; i + dst_slot < src_slot && i < nr_items; i++) {
295 ret = tree_mod_log_insert(eb->fs_info, tm_list[i]);
296 if (ret)
297 goto free_tms;
298 }
299
300 ret = tree_mod_log_insert(eb->fs_info, tm);
301 if (ret)
302 goto free_tms;
303 write_unlock(&eb->fs_info->tree_mod_log_lock);
304 kfree(tm_list);
305
306 return 0;
307 free_tms:
308 for (i = 0; i < nr_items; i++) {
309 if (tm_list[i] && !RB_EMPTY_NODE(&tm_list[i]->node))
310 rb_erase(&tm_list[i]->node, &eb->fs_info->tree_mod_log);
311 kfree(tm_list[i]);
312 }
313 if (locked)
314 write_unlock(&eb->fs_info->tree_mod_log_lock);
315 kfree(tm_list);
316 kfree(tm);
317
318 return ret;
319 }
320

---
0-DAY CI Kernel Test Service, Intel Corporation
https://lists.01.org/hyperkitty/list/kbuild-all(a)lists.01.org

#8 lore raw

Re: [PATCH 3/9] btrfs: move the tree mod log code into its own file

Quoted 17 lines · kernel test robot

Hi,

Thank you for the patch! Perhaps something to improve:

[auto build test WARNING on v5.12-rc2]
[also build test WARNING on next-20210311]
[cannot apply to kdave/for-next]
[If your patch is applied to the wrong git tree, kindly drop us a note. And when submitting patch, we suggest to use '--base' as documented in
https://git-scm.com/docs/git-format-patch]

url:    https://github.com/0day-ci/linux/commits/fdmanana-kernel-org/btrfs-bug-fixes-for-the-tree-mod-log-and-small-refactorings/20210311-223429
base:    a38fd8748464831584a19438cbb3082b5a2dab15
config: x86_64-randconfig-m001-20210311 (attached as .config)
compiler: gcc-9 (Debian 9.3.0-22) 9.3.0
Quoted 7 lines

If you fix the issue, kindly add following tag as appropriate

smatch warnings:
fs/btrfs/tree-mod-log.c:286 btrfs_tree_mod_log_insert_move() warn: missing error code 'ret' fs/btrfs/tree-mod-log.c:519 btrfs_tree_mod_log_eb_copy() warn: missing error code 'ret' fs/btrfs/tree-mod-log.c:577 btrfs_tree_mod_log_free_eb() warn: missing error code 'ret'

No, those are all ok.
For all cases, 'ret' was initialized to 0 when declared, and that's what we want to return when tree_mod_dont_log() returns true.

Quoted 82 lines

vim +/ret +286 fs/btrfs/tree-mod-log.c

   246
   247  int btrfs_tree_mod_log_insert_move(struct extent_buffer *eb,
   248                                     int dst_slot, int src_slot,
   249                                     int nr_items)
   250  {
   251          struct tree_mod_elem *tm = NULL;
   252          struct tree_mod_elem **tm_list = NULL;
   253          int ret = 0;
   254          int i;
   255          int locked = 0;
   256
   257          if (!tree_mod_need_log(eb->fs_info, eb))
   258                  return 0;
   259
   260          tm_list = kcalloc(nr_items, sizeof(struct tree_mod_elem *), GFP_NOFS);
   261          if (!tm_list)
   262                  return -ENOMEM;
   263
   264          tm = kzalloc(sizeof(*tm), GFP_NOFS);
   265          if (!tm) {
   266                  ret = -ENOMEM;
   267                  goto free_tms;
   268          }
   269
   270          tm->logical = eb->start;
   271          tm->slot = src_slot;
   272          tm->move.dst_slot = dst_slot;
   273          tm->move.nr_items = nr_items;
   274          tm->op = BTRFS_MOD_LOG_MOVE_KEYS;
   275
   276          for (i = 0; i + dst_slot < src_slot && i < nr_items; i++) {
   277                  tm_list[i] = alloc_tree_mod_elem(eb, i + dst_slot,
   278                      BTRFS_MOD_LOG_KEY_REMOVE_WHILE_MOVING, GFP_NOFS);
   279                  if (!tm_list[i]) {
   280                          ret = -ENOMEM;
   281                          goto free_tms;
   282                  }
   283          }
   284
   285          if (tree_mod_dont_log(eb->fs_info, eb))
Quoted 1 line

286 goto free_tms;

   287          locked = 1;
   288
   289          /*
   290           * When we override something during the move, we log these removals.
   291           * This can only happen when we move towards the beginning of the
   292           * buffer, i.e. dst_slot < src_slot.
   293           */
   294          for (i = 0; i + dst_slot < src_slot && i < nr_items; i++) {
   295                  ret = tree_mod_log_insert(eb->fs_info, tm_list[i]);
   296                  if (ret)
   297                          goto free_tms;
   298          }
   299
   300          ret = tree_mod_log_insert(eb->fs_info, tm);
   301          if (ret)
   302                  goto free_tms;
   303          write_unlock(&eb->fs_info->tree_mod_log_lock);
   304          kfree(tm_list);
   305
   306          return 0;
   307  free_tms:
   308          for (i = 0; i < nr_items; i++) {
   309                  if (tm_list[i] && !RB_EMPTY_NODE(&tm_list[i]->node))
   310                          rb_erase(&tm_list[i]->node, &eb->fs_info->tree_mod_log);
   311                  kfree(tm_list[i]);
   312          }
   313          if (locked)
   314                  write_unlock(&eb->fs_info->tree_mod_log_lock);
   315          kfree(tm_list);
   316          kfree(tm);
   317
   318          return ret;
   319  }
   320

---
0-DAY CI Kernel Test Service, Intel Corporation
https://lists.01.org/hyperkitty/list/kbuild-all(a)lists.01.org

Reviewed-by on v1 3/9
#9 lore raw

Re: [PATCH 3/9] btrfs: move the tree mod log code into its own file

In reply to [email protected]
Quoted 11 lines · On 11/3/21 10:31 pm, [email protected] wrote:

From: Filipe Manana <[email protected]>

The tree modification log, which records modifications done to btrees, is quite large and currently spread all over ctree.c, which is a huge file already.

To make things better organized, move all that code into its own separate source and header files. Functions and definitions that are used outside of the module (mostly by ctree.c) are renamed so that they start with a "btrfs_" prefix. Everything else remains unchanged.

Quoted 4 lines

This makes it easier to go over the tree modification log code everytime I need to go read it to fix a bug.

Reviewed-by on v1 4/9
#11 lore raw

Re: [PATCH 4/9] btrfs: use booleans where appropriate for the tree mod log functions

In reply to [email protected]
Quoted 6 lines · On 11/3/21 10:31 pm, [email protected] wrote:

From: Filipe Manana <[email protected]>

Several functions of the tree modification log use integers as booleans, so change them to use booleans instead, making their use more clear.

#13 lore raw

Re: [PATCH 5/9] btrfs: use a bit to track the existence of tree mod log users

In reply to [email protected]

Hi,

It will be more easy to backport for heavy i/o load if we put this patch before '[PATCH 3/9] btrfs: move the tree mod log code into its own file' ?

And this patch can be merged into one with the following
'[PATCH 6/9] btrfs: use the new bit BTRFS_FS_TREE_MOD_LOG_USERS at btrfs_free_tree_block()'?

Best Regards
Wang Yugui ([email protected])
2021/03/13

Quoted 82 lines

From: Filipe Manana <[email protected]>

The tree modification log functions are called very frequently, basically they are called everytime a btree is modified (a pointer added or removed to a node, a new root for a btree is set, etc). Because of that, to avoid heavy lock contention on the lock that protects the list of tree mod log users, we have checks that test the emptiness of the list with a full memory barrier before the checks, so that when there are no tree mod log users we avoid taking the lock.

Replace the memory barrier and list emptiness check with a test for a new bit set at fs_info->flags. This bit is used to indicate when there are tree mod log users, set whenever a user is added to the list and cleared when the last user is removed from the list. This makes the intention a bit more obvious and possibly more efficient (assuming test_bit() may be cheaper than a full memory barrier on some architectures).

---
 fs/btrfs/ctree.h        |  3 +++
 fs/btrfs/tree-mod-log.c | 11 ++++++-----
 2 files changed, 9 insertions(+), 5 deletions(-)
fs/btrfs/ctree.h +3 -0
index 50208673bd55..90184ee2f832 100644
@@ -580,6 +580,9 @@ enum {
 
 	/* Indicate that we can't trust the free space tree for caching yet */
 	BTRFS_FS_FREE_SPACE_TREE_UNTRUSTED,
+
+	/* Indicate whether there are any tree modification log users. */
+	BTRFS_FS_TREE_MOD_LOG_USERS,
 };
 
 /*
fs/btrfs/tree-mod-log.c +6 -5
index f2a6da1b2903..b912b82c36c9 100644
@@ -60,6 +60,7 @@ u64 btrfs_get_tree_mod_seq(struct btrfs_fs_info *fs_info,
 	if (!elem->seq) {
 		elem->seq = btrfs_inc_tree_mod_seq(fs_info);
 		list_add_tail(&elem->list, &fs_info->tree_mod_seq_list);
+		set_bit(BTRFS_FS_TREE_MOD_LOG_USERS, &fs_info->flags);
 	}
 	write_unlock(&fs_info->tree_mod_log_lock);
 
@@ -83,7 +84,9 @@ void btrfs_put_tree_mod_seq(struct btrfs_fs_info *fs_info,
 	list_del(&elem->list);
 	elem->seq = 0;
 
-	if (!list_empty(&fs_info->tree_mod_seq_list)) {
+	if (list_empty(&fs_info->tree_mod_seq_list)) {
+		clear_bit(BTRFS_FS_TREE_MOD_LOG_USERS, &fs_info->flags);
+	} else {
 		struct btrfs_seq_list *first;
 
 		first = list_first_entry(&fs_info->tree_mod_seq_list,
@@ -166,8 +169,7 @@ static noinline int tree_mod_log_insert(struct btrfs_fs_info *fs_info,
 static inline bool tree_mod_dont_log(struct btrfs_fs_info *fs_info,
 				    struct extent_buffer *eb)
 {
-	smp_mb();
-	if (list_empty(&(fs_info)->tree_mod_seq_list))
+	if (!test_bit(BTRFS_FS_TREE_MOD_LOG_USERS, &fs_info->flags))
 		return true;
 	if (eb && btrfs_header_level(eb) == 0)
 		return true;
@@ -185,8 +187,7 @@ static inline bool tree_mod_dont_log(struct btrfs_fs_info *fs_info,
 static inline bool tree_mod_need_log(const struct btrfs_fs_info *fs_info,
 				    struct extent_buffer *eb)
 {
-	smp_mb();
-	if (list_empty(&(fs_info)->tree_mod_seq_list))
+	if (!test_bit(BTRFS_FS_TREE_MOD_LOG_USERS, &fs_info->flags))
 		return false;
 	if (eb && btrfs_header_level(eb) == 0)
 		return false;
Signature · 1 line
2.28.0
#14 lore raw

Re: [PATCH 5/9] btrfs: use a bit to track the existence of tree mod log users

Quoted 5 lines · Wang Yugui

Hi,

It will be more easy to backport for heavy i/o load if we put this patch before '[PATCH 3/9] btrfs: move the tree mod log code into its own file' ?

Only bug fixes and fixes for serious performance regressions are backported to stable kernels.

This patch is neither of them. It's a cleanup.
It mentions that replacing the memory barriers with test_bit() may provide a slight benefit. If it does it's such a micro optimization that I doubt it causes any observable effect on any workload. Even if it does, which I seriously doubt, it's still not a candidate for stable backports.

Quoted 3 lines

And this patch can be merged into one with the following
'[PATCH 6/9] btrfs: use the new bit BTRFS_FS_TREE_MOD_LOG_USERS at btrfs_free_tree_block()'?

They do the same thing, but for slightly different cases. So I prefer to keep them separate for ease of review.

Quoted 89 lines

Best Regards
Wang Yugui ([email protected])
2021/03/13

Quoted 82 lines

From: Filipe Manana <[email protected]>

The tree modification log functions are called very frequently, basically they are called everytime a btree is modified (a pointer added or removed to a node, a new root for a btree is set, etc). Because of that, to avoid heavy lock contention on the lock that protects the list of tree mod log users, we have checks that test the emptiness of the list with a full memory barrier before the checks, so that when there are no tree mod log users we avoid taking the lock.

Replace the memory barrier and list emptiness check with a test for a new bit set at fs_info->flags. This bit is used to indicate when there are tree mod log users, set whenever a user is added to the list and cleared when the last user is removed from the list. This makes the intention a bit more obvious and possibly more efficient (assuming test_bit() may be cheaper than a full memory barrier on some architectures).

---
 fs/btrfs/ctree.h        |  3 +++
 fs/btrfs/tree-mod-log.c | 11 ++++++-----
 2 files changed, 9 insertions(+), 5 deletions(-)
fs/btrfs/ctree.h +3 -0
index 50208673bd55..90184ee2f832 100644
@@ -580,6 +580,9 @@ enum {
 
      /* Indicate that we can't trust the free space tree for caching yet */
      BTRFS_FS_FREE_SPACE_TREE_UNTRUSTED,
+
+     /* Indicate whether there are any tree modification log users. */
+     BTRFS_FS_TREE_MOD_LOG_USERS,
 };
 
 /*
fs/btrfs/tree-mod-log.c +6 -6
index f2a6da1b2903..b912b82c36c9 100644
@@ -60,6 +60,7 @@ u64 btrfs_get_tree_mod_seq(struct btrfs_fs_info *fs_info,
      if (!elem->seq) {
              elem->seq = btrfs_inc_tree_mod_seq(fs_info);
              list_add_tail(&elem->list, &fs_info->tree_mod_seq_list);
+             set_bit(BTRFS_FS_TREE_MOD_LOG_USERS, &fs_info->flags);
      }
      write_unlock(&fs_info->tree_mod_log_lock);
 
@@ -83,7 +84,9 @@ void btrfs_put_tree_mod_seq(struct btrfs_fs_info *fs_info,
      list_del(&elem->list);
      elem->seq = 0;
 
-     if (!list_empty(&fs_info->tree_mod_seq_list)) {
+     if (list_empty(&fs_info->tree_mod_seq_list)) {
+             clear_bit(BTRFS_FS_TREE_MOD_LOG_USERS, &fs_info->flags);
+     } else {
              struct btrfs_seq_list *first;
 
              first = list_first_entry(&fs_info->tree_mod_seq_list,
@@ -166,8 +169,7 @@ static noinline int tree_mod_log_insert(struct btrfs_fs_info *fs_info,
 static inline bool tree_mod_dont_log(struct btrfs_fs_info *fs_info,
                                  struct extent_buffer *eb)
 {
-     smp_mb();
-     if (list_empty(&(fs_info)->tree_mod_seq_list))
+     if (!test_bit(BTRFS_FS_TREE_MOD_LOG_USERS, &fs_info->flags))
              return true;
      if (eb && btrfs_header_level(eb) == 0)
              return true;
@@ -185,8 +187,7 @@ static inline bool tree_mod_dont_log(struct btrfs_fs_info *fs_info,
 static inline bool tree_mod_need_log(const struct btrfs_fs_info *fs_info,
                                  struct extent_buffer *eb)
 {
-     smp_mb();
-     if (list_empty(&(fs_info)->tree_mod_seq_list))
+     if (!test_bit(BTRFS_FS_TREE_MOD_LOG_USERS, &fs_info->flags))
              return false;
      if (eb && btrfs_header_level(eb) == 0)
              return false;
--

2.28.0

#3 lore raw

Re: [PATCH 2/9] btrfs: always pin deleted leaves when there are active tree mod log users

In reply to [email protected]
Quoted 77 lines · On Thu, Mar 11, 2021 at 02:31:06PM +0000, [email protected] wrote:

From: Filipe Manana <[email protected]>

When freeing a tree block we may end up adding its extent back to the free space cache/tree, as long as there are no more references for it, it was created in the current transaction and writeback for it never happened. This is generally fine, however when we have tree mod log operations it can result in inconsistent versions of a btree after unwinding extent buffers with the recorded tree mod log operations.

This is because:

* We only log operations for nodes (adding and removing key/pointers), for leaves we don't do anything;

* This means that we can log a MOD_LOG_KEY_REMOVE_WHILE_FREEING operation for a node that points to a leaf that was deleted;

* Before we apply the logged operation to unwind a node, we can have that leaf's extent allocated again, either as a node or as a leaf, and possibly for another btree. This is possible if the leaf was created in the current transaction and writeback for it never started, in which case btrfs_free_tree_block() returns its extent back to the free space cache/tree;

* Then, before applying the tree mod log operation, some task allocates the metadata extent just freed before, and uses it either as a leaf or as a node for some btree (can be the same or another one, it does not matter);

* After applying the MOD_LOG_KEY_REMOVE_WHILE_FREEING operation we now get the target node with an item pointing to the metadata extent that now has content different from what it had before the leaf was deleted. It might now belong to a different btree and be a node and not a leaf anymore.

As a consequence, the results of searches after the unwinding can be unpredictable and produce unexpected results.

So make sure we pin extent buffers corresponding to leaves when there are tree mod log users.

---
 fs/btrfs/extent-tree.c | 23 ++++++++++++++++++++++-
 1 file changed, 22 insertions(+), 1 deletion(-)
fs/btrfs/extent-tree.c +15 -1
index 5e228d6ad63f..2482b26b1971 100644
@@ -3310,6 +3310,7 @@ void btrfs_free_tree_block(struct btrfs_trans_handle *trans,
 
 	if (last_ref && btrfs_header_generation(buf) == trans->transid) {
 		struct btrfs_block_group *cache;
+		bool must_pin = false;
 
 		if (root->root_key.objectid != BTRFS_TREE_LOG_OBJECTID) {
 			ret = check_ref_cleanup(trans, buf->start);
@@ -3327,7 +3328,27 @@ void btrfs_free_tree_block(struct btrfs_trans_handle *trans,
 			goto out;
 		}
 
-		if (btrfs_is_zoned(fs_info)) {
+		/*
+		 * If this is a leaf and there are tree mod log users, we may
+		 * have recorded mod log operations that point to this leaf.
+		 * So we must make sure no one reuses this leaf's extent before
+		 * mod log operations are applied to a node, otherwise after
+		 * rewinding a node using the mod log operations we get an
+		 * inconsistent btree, as the leaf's extent may now be used as
+		 * a node or leaf for another different btree.
+		 * We are safe from races here because at this point no other
+		 * node or root points to this extent buffer, so if after this
+		 * check a new tree mod log user joins, it will not be able to
+		 * find a node pointing to this leaf and record operations that
+		 * point to this leaf.
+		 */

fs/btrfs/extent-tree.c void btrfs_free_tree_block(struct btrfs_trans_handle *trans, · line 3354

Quoted 10 lines
fs/btrfs/extent-tree.c +7 -0
+		if (btrfs_header_level(buf) == 0) {
+			read_lock(&fs_info->tree_mod_log_lock);
+			must_pin = !list_empty(&fs_info->tree_mod_seq_list);
+			read_unlock(&fs_info->tree_mod_log_lock);
+		}
+
+		if (must_pin || btrfs_is_zoned(fs_info)) {
 			btrfs_redirty_list_add(trans->transaction, buf);
 			pin_down_extent(trans, cache, buf->start, buf->len, 1);
 			btrfs_put_block_group(cache);

This has been added in d3575156f662 ("btrfs: zoned: redirty released extent buffers") 5.12-rc1, so it is a regression but otherwise it sounds like it's not related only to zoned mode. I'm not sure if this is relevant for older stable trees because of missing btrfs_redirty_list_add, possibly with some tweaks. Please let me know, thanks.

#4 lore raw

Re: [PATCH 2/9] btrfs: always pin deleted leaves when there are active tree mod log users

Quoted 88 lines · David Sterba
Quoted 86 lines · On Thu, Mar 11, 2021 at 02:31:06PM +0000, [email protected] wrote:

From: Filipe Manana <[email protected]>

When freeing a tree block we may end up adding its extent back to the free space cache/tree, as long as there are no more references for it, it was created in the current transaction and writeback for it never happened. This is generally fine, however when we have tree mod log operations it can result in inconsistent versions of a btree after unwinding extent buffers with the recorded tree mod log operations.

This is because:

* We only log operations for nodes (adding and removing key/pointers), for leaves we don't do anything;

* This means that we can log a MOD_LOG_KEY_REMOVE_WHILE_FREEING operation for a node that points to a leaf that was deleted;

* Before we apply the logged operation to unwind a node, we can have that leaf's extent allocated again, either as a node or as a leaf, and possibly for another btree. This is possible if the leaf was created in the current transaction and writeback for it never started, in which case btrfs_free_tree_block() returns its extent back to the free space cache/tree;

* Then, before applying the tree mod log operation, some task allocates the metadata extent just freed before, and uses it either as a leaf or as a node for some btree (can be the same or another one, it does not matter);

* After applying the MOD_LOG_KEY_REMOVE_WHILE_FREEING operation we now get the target node with an item pointing to the metadata extent that now has content different from what it had before the leaf was deleted. It might now belong to a different btree and be a node and not a leaf anymore.

As a consequence, the results of searches after the unwinding can be unpredictable and produce unexpected results.

So make sure we pin extent buffers corresponding to leaves when there are tree mod log users.

---
 fs/btrfs/extent-tree.c | 23 ++++++++++++++++++++++-
 1 file changed, 22 insertions(+), 1 deletion(-)
fs/btrfs/extent-tree.c +22 -1
index 5e228d6ad63f..2482b26b1971 100644
@@ -3310,6 +3310,7 @@ void btrfs_free_tree_block(struct btrfs_trans_handle *trans,
 
      if (last_ref && btrfs_header_generation(buf) == trans->transid) {
              struct btrfs_block_group *cache;
+             bool must_pin = false;
 
              if (root->root_key.objectid != BTRFS_TREE_LOG_OBJECTID) {
                      ret = check_ref_cleanup(trans, buf->start);
@@ -3327,7 +3328,27 @@ void btrfs_free_tree_block(struct btrfs_trans_handle *trans,
                      goto out;
              }
 
-             if (btrfs_is_zoned(fs_info)) {
+             /*
+              * If this is a leaf and there are tree mod log users, we may
+              * have recorded mod log operations that point to this leaf.
+              * So we must make sure no one reuses this leaf's extent before
+              * mod log operations are applied to a node, otherwise after
+              * rewinding a node using the mod log operations we get an
+              * inconsistent btree, as the leaf's extent may now be used as
+              * a node or leaf for another different btree.
+              * We are safe from races here because at this point no other
+              * node or root points to this extent buffer, so if after this
+              * check a new tree mod log user joins, it will not be able to
+              * find a node pointing to this leaf and record operations that
+              * point to this leaf.
+              */
+             if (btrfs_header_level(buf) == 0) {
+                     read_lock(&fs_info->tree_mod_log_lock);
+                     must_pin = !list_empty(&fs_info->tree_mod_seq_list);
+                     read_unlock(&fs_info->tree_mod_log_lock);
+             }
+
+             if (must_pin || btrfs_is_zoned(fs_info)) {
                      btrfs_redirty_list_add(trans->transaction, buf);
                      pin_down_extent(trans, cache, buf->start, buf->len, 1);
Quoted 8 lines

fs/btrfs/extent-tree.c void btrfs_free_tree_block(struct btrfs_trans_handle *trans, · line 3354

Quoted 1 line
fs/btrfs/extent-tree.c +0 -0
                      btrfs_put_block_group(cache);

This has been added in d3575156f662 ("btrfs: zoned: redirty released extent buffers") 5.12-rc1, so it is a regression but otherwise it sounds like it's not related only to zoned mode. I'm not sure if this is relevant for older stable trees because of missing btrfs_redirty_list_add, possibly with some tweaks. Please let me know, thanks.

It's not related to zoned filesystems at all.
I just happened to reuse that if branch for the zoned case because it does the same thing I needed to do, and the call to btrfs_redirty_list_add() does nothing for the non-zoned case, so it's safe.

I.e., it's the same as adding the following instead:

if (must_pin) {
pin_down_extent(trans, cache, buf->start, buf->len, 1); btrfs_put_block_group(cache);
goto out;
}

I opted for the shorter version by ORing the two cases.

But yes, for pre 5.12-rc1 the patch will not apply and the above code would be needed.

Thanks.

#5 lore raw

Re: [PATCH 2/9] btrfs: always pin deleted leaves when there are active tree mod log users

Quoted 106 lines · Filipe Manana
Quoted 88 lines · David Sterba
Quoted 86 lines · On Thu, Mar 11, 2021 at 02:31:06PM +0000, [email protected] wrote:

From: Filipe Manana <[email protected]>

When freeing a tree block we may end up adding its extent back to the free space cache/tree, as long as there are no more references for it, it was created in the current transaction and writeback for it never happened. This is generally fine, however when we have tree mod log operations it can result in inconsistent versions of a btree after unwinding extent buffers with the recorded tree mod log operations.

This is because:

* We only log operations for nodes (adding and removing key/pointers), for leaves we don't do anything;

* This means that we can log a MOD_LOG_KEY_REMOVE_WHILE_FREEING operation for a node that points to a leaf that was deleted;

* Before we apply the logged operation to unwind a node, we can have that leaf's extent allocated again, either as a node or as a leaf, and possibly for another btree. This is possible if the leaf was created in the current transaction and writeback for it never started, in which case btrfs_free_tree_block() returns its extent back to the free space cache/tree;

* Then, before applying the tree mod log operation, some task allocates the metadata extent just freed before, and uses it either as a leaf or as a node for some btree (can be the same or another one, it does not matter);

* After applying the MOD_LOG_KEY_REMOVE_WHILE_FREEING operation we now get the target node with an item pointing to the metadata extent that now has content different from what it had before the leaf was deleted. It might now belong to a different btree and be a node and not a leaf anymore.

As a consequence, the results of searches after the unwinding can be unpredictable and produce unexpected results.

So make sure we pin extent buffers corresponding to leaves when there are tree mod log users.

---
 fs/btrfs/extent-tree.c | 23 ++++++++++++++++++++++-
 1 file changed, 22 insertions(+), 1 deletion(-)
fs/btrfs/extent-tree.c +22 -1
index 5e228d6ad63f..2482b26b1971 100644
@@ -3310,6 +3310,7 @@ void btrfs_free_tree_block(struct btrfs_trans_handle *trans,
 
      if (last_ref && btrfs_header_generation(buf) == trans->transid) {
              struct btrfs_block_group *cache;
+             bool must_pin = false;
 
              if (root->root_key.objectid != BTRFS_TREE_LOG_OBJECTID) {
                      ret = check_ref_cleanup(trans, buf->start);
@@ -3327,7 +3328,27 @@ void btrfs_free_tree_block(struct btrfs_trans_handle *trans,
                      goto out;
              }
 
-             if (btrfs_is_zoned(fs_info)) {
+             /*
+              * If this is a leaf and there are tree mod log users, we may
+              * have recorded mod log operations that point to this leaf.
+              * So we must make sure no one reuses this leaf's extent before
+              * mod log operations are applied to a node, otherwise after
+              * rewinding a node using the mod log operations we get an
+              * inconsistent btree, as the leaf's extent may now be used as
+              * a node or leaf for another different btree.
+              * We are safe from races here because at this point no other
+              * node or root points to this extent buffer, so if after this
+              * check a new tree mod log user joins, it will not be able to
+              * find a node pointing to this leaf and record operations that
+              * point to this leaf.
+              */
+             if (btrfs_header_level(buf) == 0) {
+                     read_lock(&fs_info->tree_mod_log_lock);
+                     must_pin = !list_empty(&fs_info->tree_mod_seq_list);
+                     read_unlock(&fs_info->tree_mod_log_lock);
+             }
+
+             if (must_pin || btrfs_is_zoned(fs_info)) {
                      btrfs_redirty_list_add(trans->transaction, buf);
                      pin_down_extent(trans, cache, buf->start, buf->len, 1);
Quoted 8 lines

fs/btrfs/extent-tree.c void btrfs_free_tree_block(struct btrfs_trans_handle *trans, · line 3354

Quoted 1 line
fs/btrfs/extent-tree.c +0 -0
                      btrfs_put_block_group(cache);

This has been added in d3575156f662 ("btrfs: zoned: redirty released extent buffers") 5.12-rc1, so it is a regression but otherwise it sounds like it's not related only to zoned mode. I'm not sure if this is relevant for older stable trees because of missing btrfs_redirty_list_add, possibly with some tweaks. Please let me know, thanks.

It's not related to zoned filesystems at all.
I just happened to reuse that if branch for the zoned case because it does the same thing I needed to do, and the call to btrfs_redirty_list_add() does nothing for the non-zoned case, so it's safe.

I.e., it's the same as adding the following instead:

Quoted 7 lines

if (must_pin) {
pin_down_extent(trans, cache, buf->start, buf->len, 1); btrfs_put_block_group(cache);
goto out;
}

I opted for the shorter version by ORing the two cases.

FYI the redirtying is one thing we promised Josef to get rid in the zoned ASAP. We just can't get it to work properly yet.

#19 lore raw
In reply to [email protected]
Quoted 6 lines · On Thu, Mar 11, 2021 at 02:31:04PM +0000, [email protected] wrote:

From: Filipe Manana <[email protected]>

This patchset fixes a couple bugs, in the two first patches, with the tree mod log code. The remaining patches just move all that code into a separate file, since it's quite large and ctree.c is huge as well, and do some small refactorings and cleanups.

Great, thanks. Patchset moved from for-next to misc-next.