Drop some mis-uses of READA
cover Drop some mis-uses of READA
In debugging Zygo's huge commit delays I noticed we were burning a bunch of time doing READA in cases where we don't need to. The way READA works in btrfs is we'll load up adjacent nodes and leaves as we walk down. This is useful for operations where we're going to be reading sequentially across the tree.
But for delayed refs we're looking up one bytenr, and then another one which could be elsewhere in the tree. With large enough extent trees this results in a lot of unneeded latency.
The same applies to build_backref_tree, but that's even worse because we're looking up backrefs, which are essentially randomly spread out across the extent root. Thanks,
Josef
PATCH 1/2 btrfs: do not use READA for running delayed refs
READA will generate a lot of extra reads for adjacent nodes, but when running delayed refs we have no idea if the next ref is going to be adjacent or not, so this potentially just generates a lot of extra IO. To make matters worse each ref is truly just looking for one item, it doesn't generally search forward, so we simply don't need it here.
--- fs/btrfs/extent-tree.c | 4 ---- 1 file changed, 4 deletions(-)
fs/btrfs/extent-tree.c +0 -4
@@ -1469,7 +1469,6 @@ static int __btrfs_inc_extent_ref(struct btrfs_trans_handle *trans,if (!path)return -ENOMEM ; - path->reada =READA_FORWARD ; path->leave_spinning =1 ;/* this will setup the path even if it fails to insert the back ref */ ret = insert_inline_extent_backref(trans, path, bytenr, num_bytes,
@@ -1494,7 +1493,6 @@ static int __btrfs_inc_extent_ref(struct btrfs_trans_handle *trans,btrfs_mark_buffer_dirty (leaf);btrfs_release_path (path); - path->reada =READA_FORWARD ; path->leave_spinning =1 ;/* now insert the actual backref */ ret = insert_extent_backref(trans, path, bytenr, parent, root_objectid,
@@ -1604,7 +1602,6 @@ static int run_delayed_extent_op(struct btrfs_trans_handle *trans, } again: - path->reada =READA_FORWARD ; path->leave_spinning =1 ; ret =btrfs_search_slot (trans, fs_info->extent_root, &key, path,0 ,1 );if (ret <0 ) {
@@ -2999,7 +2996,6 @@ static int __btrfs_free_extent(struct btrfs_trans_handle *trans,if (!path)return -ENOMEM ; - path->reada =READA_FORWARD ; path->leave_spinning =1 ; is_data = owner_objectid >=BTRFS_FIRST_FREE_OBJECTID ;
Signature · 1 line
2.24.1
Re: [PATCH 1/2] btrfs: do not use READA for running delayed refs
Quoted 7 lines · Josef Bacik
READA will generate a lot of extra reads for adjacent nodes, but when running delayed refs we have no idea if the next ref is going to be adjacent or not, so this potentially just generates a lot of extra IO. To make matters worse each ref is truly just looking for one item, it doesn't generally search forward, so we simply don't need it here.
Signed-off-by Josef Bacik <[email protected]>
Thanks,
Qu
Quoted 41 lines
--- fs/btrfs/extent-tree.c | 4 ---- 1 file changed, 4 deletions(-)fs/btrfs/extent-tree.c +0
-4@@ -1469,7 +1469,6 @@ static int __btrfs_inc_extent_ref(struct btrfs_trans_handle *trans,if (!path)return -ENOMEM ; - path->reada =READA_FORWARD ; path->leave_spinning =1 ;/* this will setup the path even if it fails to insert the back ref */ ret = insert_inline_extent_backref(trans, path, bytenr, num_bytes,@@ -1494,7 +1493,6 @@ static int __btrfs_inc_extent_ref(struct btrfs_trans_handle *trans,btrfs_mark_buffer_dirty (leaf);btrfs_release_path (path); - path->reada =READA_FORWARD ; path->leave_spinning =1 ;/* now insert the actual backref */ ret = insert_extent_backref(trans, path, bytenr, parent, root_objectid,@@ -1604,7 +1602,6 @@ static int run_delayed_extent_op(struct btrfs_trans_handle *trans, } again: - path->reada =READA_FORWARD ; path->leave_spinning =1 ; ret =btrfs_search_slot (trans, fs_info->extent_root, &key, path,0 ,1 );if (ret <0 ) {@@ -2999,7 +2996,6 @@ static int __btrfs_free_extent(struct btrfs_trans_handle *trans,if (!path)return -ENOMEM ; - path->reada =READA_FORWARD ; path->leave_spinning =1 ; is_data = owner_objectid >=BTRFS_FIRST_FREE_OBJECTID ;
PATCH 2/2 btrfs: do not READA in build_backref_tree
In reply to Josef BacikHere we are just searching down to the bytenr we're building the backref tree for, and all of it's paths to the roots. These bytenrs are not guaranteed to be anywhere near each other, so the READA just generates extra latency.
--- fs/btrfs/relocation.c | 2 -- 1 file changed, 2 deletions(-)
fs/btrfs/relocation.c +0 -2
@@ -759,8 +759,6 @@ struct backref_node *build_backref_tree(struct reloc_control *rc, err = -ENOMEM ; goto out; } - path1->reada =READA_FORWARD ; - path2->reada =READA_FORWARD ; node =alloc_backref_node (cache);if (!node) {
Signature · 1 line
2.24.1
Re: [PATCH 2/2] btrfs: do not READA in build_backref_tree
Quoted 6 lines · Josef Bacik
Here we are just searching down to the bytenr we're building the backref tree for, and all of it's paths to the roots. These bytenrs are not guaranteed to be anywhere near each other, so the READA just generates extra latency.
Signed-off-by Josef Bacik <[email protected]>
Thanks,
Qu
Quoted 18 lines
--- fs/btrfs/relocation.c | 2 -- 1 file changed, 2 deletions(-)fs/btrfs/relocation.c +0
-2@@ -759,8 +759,6 @@ struct backref_node *build_backref_tree(struct reloc_control *rc, err = -ENOMEM ; goto out; } - path1->reada =READA_FORWARD ; - path2->reada =READA_FORWARD ; node =alloc_backref_node (cache);if (!node) {
Quoted 5 lines · Josef Bacik
In debugging Zygo's huge commit delays I noticed we were burning a bunch of time doing READA in cases where we don't need to. The way READA works in btrfs is we'll load up adjacent nodes and leaves as we walk down. This is useful for operations where we're going to be reading sequentially across the tree.
Quoted 7 lines
But for delayed refs we're looking up one bytenr, and then another one which could be elsewhere in the tree. With large enough extent trees this results in a lot of unneeded latency.
The same applies to
build_backref_tree, but that's even worse because we're looking up backrefs, which are essentially randomly spread out across the extent root. Thanks,
There are quite some other locations abusing READA.
E.g. btrfs_read_block_groups(), where we're just searching for block group items. There is no guarantee that next block group item is in next a few leaves.
I guess it's a good time to review all READA abuse. Or would you mind me to do that?
Thanks,
Qu
Quoted 3 lines
Josef
Quoted 16 lines · Qu Wenruo
Quoted 5 lines · Josef Bacik
In debugging Zygo's huge commit delays I noticed we were burning a bunch of time doing READA in cases where we don't need to. The way READA works in btrfs is we'll load up adjacent nodes and leaves as we walk down. This is useful for operations where we're going to be reading sequentially across the tree.
Quoted 7 lines
But for delayed refs we're looking up one bytenr, and then another one which could be elsewhere in the tree. With large enough extent trees this results in a lot of unneeded latency.
The same applies to
build_backref_tree, but that's even worse because we're looking up backrefs, which are essentially randomly spread out across the extent root. Thanks,
Quoted 8 lines
There are quite some other locations abusing READA.
E.g.
btrfs_read_block_groups(), where we're just searching for block group items. There is no guarantee that next block group item is in next a few leaves.I guess it's a good time to review all READA abuse. Or would you mind me to do that?
If you find some clear example where the items are scattered over the tree then yes. For the rest it would be good to put a comment that the readahead really helps. Thanks.
Quoted 5 lines · Josef Bacik
In debugging Zygo's huge commit delays I noticed we were burning a bunch of time doing READA in cases where we don't need to. The way READA works in btrfs is we'll load up adjacent nodes and leaves as we walk down. This is useful for operations where we're going to be reading sequentially across the tree.
Quoted 7 lines
But for delayed refs we're looking up one bytenr, and then another one which could be elsewhere in the tree. With large enough extent trees this results in a lot of unneeded latency.
The same applies to
build_backref_tree, but that's even worse because we're looking up backrefs, which are essentially randomly spread out across the extent root. Thanks,
Makes sense, I'll add it to misc-next. Thanks.