Patch "Btrfs: fix regression running delayed references when using qgroups" has been added to the 4.3-stable tree
Subsystems (2)
This is a note to let you know that I've just added the patch titled
Btrfs: fix regression running delayed references when using qgroups
to the 4.3-stable tree which can be found at:
http://www.kernel.org/git/?p=linux/kernel/git/stable/stable-queue.git;a=summary
The filename of the patch is:
0002-4.3.2-Btrfs-fix-regression-running-delayed-references-when.patch
and it can be found in the queue-4.3 subdirectory.
If you, or anyone else, feels it should not be added to the stable tree, please let <[email protected]> know about it.
Quoted 1 line
From
276b67714b35e9f6ce23c4bc0c750ad007e5a52cMon Sep 17 00:00:00 2001
From: Filipe Manana <[email protected]>
Date: Fri, 23 Oct 2015 07:52:54 +0100
Subject: Btrfs: fix regression running delayed references when using qgroups
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit
From: Filipe Manana <[email protected]>
commit b06c4bf5c874a57254b197f53ddf588e7a24a2bf upstream.
In the kernel 4.2 merge window we had a big changes to the implementation of delayed references and qgroups which made the no_quota field of delayed references not used anymore. More specifically the no_quota field is not used anymore as of:
commit 0ed4792af0e8 ("btrfs: qgroup: Switch to new extent-oriented qgroup mechanism.")
Leaving the no_quota field actually prevents delayed references from getting merged, which in turn cause the following BUG_ON(), at fs/btrfs/extent-tree.c, to be hit when qgroups are enabled:
static int run_delayed_tree_ref(...)
{
(...)
BUG_ON(node->ref_mod != 1);
(...)
}This happens on a scenario like the following:
1) Ref1 bytenr X, action = BTRFS_ADD_DELAYED_REF, no_quota = 1, added.
2) Ref2 bytenr X, action = BTRFS_DROP_DELAYED_REF, no_quota = 0, added.
It's not merged with Ref1 because Ref1->no_quota != Ref2->no_quota.
3) Ref3 bytenr X, action = BTRFS_ADD_DELAYED_REF, no_quota = 1, added. It's not merged with the reference at the tail of the list of refs for bytenr X because the reference at the tail, Ref2 is incompatible due to Ref2->no_quota != Ref3->no_quota.
4) Ref4 bytenr X, action = BTRFS_DROP_DELAYED_REF, no_quota = 0, added. It's not merged with the reference at the tail of the list of refs for bytenr X because the reference at the tail, Ref3 is incompatible due to Ref3->no_quota != Ref4->no_quota.
5) We run delayed references, trigger merging of delayed references,
through __btrfs_run_delayed_refs() -> btrfs_merge_delayed_refs().
6) Ref1 and Ref3 are merged as Ref1->no_quota = Ref3->no_quota and all other conditions are satisfied too. So Ref1 gets a ref_mod value of 2.
7) Ref2 and Ref4 are merged as Ref2->no_quota = Ref4->no_quota and all other conditions are satisfied too. So Ref2 gets a ref_mod value of 2.
8) Ref1 and Ref2 aren't merged, because they have different values for their no_quota field.
9) Delayed reference Ref1 is picked for running (select_delayed_ref()
always prefers references with an action == BTRFS_ADD_DELAYED_REF).
So run_delayed_tree_ref() is called for Ref1 which triggers the
BUG_ON because Ref1->red_mod != 1 (equals 2).So fix this by removing the no_quota field, as it's not used anymore as of commit 0ed4792af0e8 ("btrfs: qgroup: Switch to new extent-oriented qgroup mechanism.").
The use of no_quota was also buggy in at least two places:
1) At delayed-refs.c:btrfs_add_delayed_tree_ref() - we were setting no_quota to 0 instead of 1 when the following condition was true:
is_fstree(ref_root) || !fs_info->quota_enabled
2) At extent-tree.c:__btrfs_inc_extent_ref() - we were attempting to reset a node's no_quota when the condition "!is_fstree(root_objectid) || !root->fs_info->quota_enabled" was true but we did it only in an unused local stack variable, that is, we never reset the no_quota value in the node itself.
This fixes the remainder of problems several people have been having when running delayed references, mostly while a balance is running in parallel, on a 4.2+ kernel.
Very special thanks to Stéphane Lesimple for helping debugging this issue and testing this fix on his multi terabyte filesystem (which took more than one day to balance alone, plus fsck, etc).
Also, this fixes deadlock issue when using the clone ioctl with qgroups enabled, as reported by Elias Probst in the mailing list. The deadlock happens because after calling btrfs_insert_empty_item we have our path holding a write lock on a leaf of the fs/subvol tree and then before releasing the path we called check_ref() which did backref walking, when qgroups are enabled, and tried to read lock the same leaf. The trace for this case is the following:
INFO: task systemd-nspawn:6095 blocked for more than 120 seconds.
(...)
Call Trace:
[<ffffffff86999201>] schedule+0x74/0x83
[<ffffffff863ef64c>] btrfs_tree_read_lock+0xc0/0xea
[<ffffffff86137ed7>] ? wait_woken+0x74/0x74
[<ffffffff8639f0a7>] btrfs_search_old_slot+0x51a/0x810
[<ffffffff863a129b>] btrfs_next_old_leaf+0xdf/0x3ce
[<ffffffff86413a00>] ? ulist_add_merge+0x1b/0x127
[<ffffffff86411688>] __resolve_indirect_refs+0x62a/0x667
[<ffffffff863ef546>] ? btrfs_clear_lock_blocking_rw+0x78/0xbe
[<ffffffff864122d3>] find_parent_nodes+0xaf3/0xfc6
[<ffffffff86412838>] __btrfs_find_all_roots+0x92/0xf0
[<ffffffff864128f2>] btrfs_find_all_roots+0x45/0x65
[<ffffffff8639a75b>] ? btrfs_get_tree_mod_seq+0x2b/0x88
[<ffffffff863e852e>] check_ref+0x64/0xc4
[<ffffffff863e9e01>] btrfs_clone+0x66e/0xb5d
[<ffffffff863ea77f>] btrfs_ioctl_clone+0x48f/0x5bb
[<ffffffff86048a68>] ? native_sched_clock+0x28/0x77
[<ffffffff863ed9b0>] btrfs_ioctl+0xabc/0x25cb
(...)The problem goes away by eleminating check_ref(), which no longer is needed as its purpose was to get a value for the no_quota field of a delayed reference (this patch removes the no_quota field as mentioned earlier).
--- fs/btrfs/ctree.h | 4 +-- fs/btrfs/delayed-ref.c | 28 ++++++---------------- fs/btrfs/delayed-ref.h | 7 +---- fs/btrfs/extent-tree.c | 45 ++++++++++++----------------------- fs/btrfs/file.c | 10 +++---- fs/btrfs/inode.c | 4 +-- fs/btrfs/ioctl.c | 62 ------------------------------------------------- fs/btrfs/relocation.c | 16 +++++------- fs/btrfs/tree-log.c | 2 - 9 files changed, 44 insertions(+), 134 deletions(-)
fs/btrfs/ctree.h +2 -2
@@ -3398,7 +3398,7 @@ int btrfs_set_disk_extent_flags(struct bint btrfs_free_extent (struct btrfs_trans_handle *trans,struct btrfs_root *root,u64 bytenr,u64 num_bytes,u64 parent,u64 root_objectid, -u64 owner,u64 offset,int no_quota); +u64 owner,u64 offset);int btrfs_free_reserved_extent (struct btrfs_root *root,u64 start,u64 len,int delalloc);
@@ -3411,7 +3411,7 @@ int btrfs_finish_extent_commit(struct btint btrfs_inc_extent_ref (struct btrfs_trans_handle *trans,struct btrfs_root *root,u64 bytenr,u64 num_bytes,u64 parent, -u64 root_objectid,u64 owner,u64 offset,int no_quota); +u64 root_objectid,u64 owner,u64 offset);int btrfs_start_dirty_block_groups (struct btrfs_trans_handle *trans,struct btrfs_root *root);
fs/btrfs/delayed-ref.c +8 -20
@@ -220,7 +220,7 @@ static bool merge_ref(struct btrfs_transif (seq && next->seq >= seq) goto next; -if (next->type != ref->type || next->no_quota != ref->no_quota) +if (next->type != ref->type) goto next; if ((ref->type == BTRFS_TREE_BLOCK_REF_KEY ||
@@ -405,8 +405,7 @@ add_delayed_ref_tail_merge(struct btrfs_ exist = list_entry(href->ref_list.prev, struct btrfs_delayed_ref_node, list);/* No need to compare bytenr nor is_head */ -if (exist->type != ref->type || exist->no_quota != ref->no_quota || - exist->seq != ref->seq) +if (exist->type != ref->type || exist->seq != ref->seq) goto add_tail; if ((exist->type == BTRFS_TREE_BLOCK_REF_KEY ||
@@ -637,7 +636,7 @@ add_delayed_tree_ref(struct btrfs_fs_infstruct btrfs_delayed_ref_head *head_ref, struct btrfs_delayed_ref_node *ref, u64 bytenr, u64 num_bytes, u64 parent, u64 ref_root, int level, - int action, int no_quota) + int action) { struct btrfs_delayed_tree_ref *full_ref;struct btrfs_delayed_ref_root *delayed_refs;
@@ -659,7 +658,6 @@ add_delayed_tree_ref(struct btrfs_fs_inf ref->action = action; ref->is_head =0 ; ref->in_tree =1 ; - ref->no_quota = no_quota; ref->seq = seq; full_ref =btrfs_delayed_node_to_tree_ref (ref);
@@ -692,7 +690,7 @@ add_delayed_data_ref(struct btrfs_fs_infstruct btrfs_delayed_ref_head *head_ref, struct btrfs_delayed_ref_node *ref, u64 bytenr, u64 num_bytes, u64 parent, u64 ref_root, u64 owner, - u64 offset, int action, int no_quota) + u64 offset, int action) { struct btrfs_delayed_data_ref *full_ref;struct btrfs_delayed_ref_root *delayed_refs;
@@ -715,7 +713,6 @@ add_delayed_data_ref(struct btrfs_fs_inf ref->action = action; ref->is_head =0 ; ref->in_tree =1 ; - ref->no_quota = no_quota; ref->seq = seq; full_ref =btrfs_delayed_node_to_data_ref (ref);
@@ -746,17 +743,13 @@ int btrfs_add_delayed_tree_ref(struct btstruct btrfs_trans_handle *trans, u64 bytenr, u64 num_bytes, u64 parent, u64 ref_root, int level, int action, - struct btrfs_delayed_extent_op *extent_op, - int no_quota) + struct btrfs_delayed_extent_op *extent_op) { struct btrfs_delayed_tree_ref *ref;struct btrfs_delayed_ref_head *head_ref;struct btrfs_delayed_ref_root *delayed_refs;struct btrfs_qgroup_extent_record *record =NULL ; -if (!is_fstree (ref_root) || !fs_info->quota_enabled) - no_quota =0 ; -BUG_ON (extent_op && extent_op->is_data); ref =kmem_cache_alloc (btrfs_delayed_tree_ref_cachep,GFP_NOFS );if (!ref)
@@ -785,8 +778,7 @@ int btrfs_add_delayed_tree_ref(struct bt bytenr, num_bytes, action, 0);add_delayed_tree_ref (fs_info, trans, head_ref, &ref->node, bytenr, - num_bytes, parent, ref_root, level, action, - no_quota); + num_bytes, parent, ref_root, level, action);spin_unlock (&delayed_refs->lock);return 0 ;
@@ -807,17 +799,13 @@ int btrfs_add_delayed_data_ref(struct bt u64 bytenr, u64 num_bytes, u64 parent, u64 ref_root, u64 owner, u64 offset, int action, - struct btrfs_delayed_extent_op *extent_op, - int no_quota) + struct btrfs_delayed_extent_op *extent_op) { struct btrfs_delayed_data_ref *ref;struct btrfs_delayed_ref_head *head_ref;struct btrfs_delayed_ref_root *delayed_refs;struct btrfs_qgroup_extent_record *record =NULL ; -if (!is_fstree (ref_root) || !fs_info->quota_enabled) - no_quota =0 ; -BUG_ON (extent_op && !extent_op->is_data); ref =kmem_cache_alloc (btrfs_delayed_data_ref_cachep,GFP_NOFS );if (!ref)
@@ -853,7 +841,7 @@ int btrfs_add_delayed_data_ref(struct btadd_delayed_data_ref (fs_info, trans, head_ref, &ref->node, bytenr, num_bytes, parent, ref_root, owner, offset, - action, no_quota); + action);spin_unlock (&delayed_refs->lock);return 0 ;
fs/btrfs/delayed-ref.h +2 -5
@@ -68,7 +68,6 @@ struct btrfs_delayed_ref_node { unsigned int action:8; unsigned int type:8; - unsigned int no_quota:1;/* is this node still in the rbtree? */ unsigned int is_head:1; unsigned int in_tree:1;
@@ -233,15 +232,13 @@ int btrfs_add_delayed_tree_ref(struct btstruct btrfs_trans_handle *trans, u64 bytenr, u64 num_bytes, u64 parent, u64 ref_root, int level, int action, - struct btrfs_delayed_extent_op *extent_op, - int no_quota); + struct btrfs_delayed_extent_op *extent_op);int btrfs_add_delayed_data_ref (struct btrfs_fs_info *fs_info,struct btrfs_trans_handle *trans,u64 bytenr,u64 num_bytes,u64 parent,u64 ref_root,u64 owner,u64 offset,int action, -struct btrfs_delayed_extent_op *extent_op, -int no_quota); +struct btrfs_delayed_extent_op *extent_op); int btrfs_add_delayed_extent_op(struct btrfs_fs_info *fs_info, struct btrfs_trans_handle *trans, u64 bytenr, u64 num_bytes,
fs/btrfs/extent-tree.c +16 -29
161 lines not on this page · raw message
fs/btrfs/file.c +5 -5
@@ -855,7 +855,7 @@ next_slot: disk_bytenr, num_bytes,0 , root->root_key.objectid, new_key.objectid, - start - extent_offset, 1); + start - extent_offset);BUG_ON (ret);/* -ENOMEM */ } key.offset = start;
@@ -933,7 +933,7 @@ delete_extent_item: disk_bytenr, num_bytes,0 , root->root_key.objectid, key.objectid, key.offset - - extent_offset, 0); + extent_offset);BUG_ON (ret);/* -ENOMEM */ inode_sub_bytes (inode, extent_end - key.offset);
@@ -1212,7 +1212,7 @@ again: ret =btrfs_inc_extent_ref (trans, root, bytenr, num_bytes,0 , root->root_key.objectid, - ino, orig_offset,1 ); + ino, orig_offset);BUG_ON (ret);/* -ENOMEM */ if (split == start) {
@@ -1239,7 +1239,7 @@ again: del_nr++; ret =btrfs_free_extent (trans, root, bytenr, num_bytes,0 , root->root_key.objectid, - ino, orig_offset,0 ); + ino, orig_offset);BUG_ON (ret);/* -ENOMEM */ } other_start =0 ;
@@ -1256,7 +1256,7 @@ again: del_nr++; ret =btrfs_free_extent (trans, root, bytenr, num_bytes,0 , root->root_key.objectid, - ino, orig_offset,0 ); + ino, orig_offset);BUG_ON (ret);/* -ENOMEM */ }if (del_nr ==0 ) {
fs/btrfs/inode.c +2 -2
@@ -2579,7 +2579,7 @@ again: ret =btrfs_inc_extent_ref (trans, root, new->bytenr, new->disk_len,0 , backref->root_id, backref->inum, - new->file_pos,0 );/* start - extent_offset */ + new->file_pos);/* start - extent_offset */ if (ret) {btrfs_abort_transaction (trans, root, ret); goto out_free_path;
@@ -4521,7 +4521,7 @@ delete: ret =btrfs_free_extent (trans, root, extent_start, extent_num_bytes,0 ,btrfs_header_owner (leaf), - ino, extent_offset,0 ); + ino, extent_offset);BUG_ON (ret);if (btrfs_should_throttle_delayed_refs (trans, root)) btrfs_async_run_delayed_refs(root,
fs/btrfs/ioctl.c +1 -61
93 lines not on this page · raw message
fs/btrfs/relocation.c +7 -9
@@ -1716,7 +1716,7 @@ int replace_file_extents(struct btrfs_tr ret =btrfs_inc_extent_ref (trans, root, new_bytenr, num_bytes, parent,btrfs_header_owner (leaf), - key.objectid, key.offset,1 ); + key.objectid, key.offset);if (ret) {btrfs_abort_transaction (trans, root, ret);break ;
@@ -1724,7 +1724,7 @@ int replace_file_extents(struct btrfs_tr ret =btrfs_free_extent (trans, root, bytenr, num_bytes, parent,btrfs_header_owner (leaf), - key.objectid, key.offset,1 ); + key.objectid, key.offset);if (ret) {btrfs_abort_transaction (trans, root, ret);break ;
@@ -1900,23 +1900,21 @@ again: ret =btrfs_inc_extent_ref (trans, src, old_bytenr, blocksize, path->nodes[level]->start, - src->root_key.objectid, level -1 ,0 , -1 ); + src->root_key.objectid, level -1 ,0 );BUG_ON (ret); ret =btrfs_inc_extent_ref (trans, dest, new_bytenr, blocksize,0 , dest->root_key.objectid, level -1 , -0 ,1 ); +0 );BUG_ON (ret); ret =btrfs_free_extent (trans, src, new_bytenr, blocksize, path->nodes[level]->start, - src->root_key.objectid, level -1 ,0 , -1 ); + src->root_key.objectid, level -1 ,0 );BUG_ON (ret); ret =btrfs_free_extent (trans, dest, old_bytenr, blocksize,0 , dest->root_key.objectid, level -1 , -0 ,1 ); +0 );BUG_ON (ret);btrfs_unlock_up_safe (path,0 );
@@ -2745,7 +2743,7 @@ static int do_relocation(struct btrfs_tr node->eb->start, blocksize, upper->eb->start,btrfs_header_owner (upper->eb), - node->level, 0, 1); + node->level, 0);BUG_ON (ret); ret =btrfs_drop_subtree (trans, root, eb, upper->eb);
fs/btrfs/tree-log.c +1 -1
@@ -691,7 +691,7 @@ static noinline int replay_one_extent(st ret =btrfs_inc_extent_ref (trans, root, ins.objectid, ins.offset,0 , root->root_key.objectid, - key->objectid, offset,0 ); + key->objectid, offset);if (ret) goto out; }else {
Patches currently in stable-queue which might be from [email protected] are
queue-4.3/btrfs-fix-file-corruption-and-data-loss-after-cloning-inline-extents.patch queue-4.3/btrfs-fix-truncation-of-compressed-and-inlined-extents.patch queue-4.3/btrfs-fix-regression-when-running-delayed-references.patch queue-4.3/btrfs-fix-race-leading-to-bug_on-when-running-delalloc-for-nodatacow.patch queue-4.3/btrfs-fix-race-leading-to-incorrect-item-deletion-when-dropping-extents.patch queue-4.3/btrfs-fix-race-when-listing-an-inode-s-xattrs.patch queue-4.3/0002-4.3.2-Btrfs-fix-regression-running-delayed-references-when.patch queue-4.3/btrfs-fix-resending-received-snapshot-with-parent.patch