btrfs: prefetch chunk tree leaves at mount
David Sterba
maintainerPATCH 1/1 btrfs: prefetch chunk tree leaves at mount
The whole chunk tree is read at mount time so we can utilize readahead to get the tree blocks to memory before we read the items. The idea is from Robbie, but instead of updating search slot readahead, this patch implements the chunk tree readahead manually from nodes on level 1.
We've decided to do specific readahead optimizations and then unify them under a common API so we don't break everything by changing the search slot readahead logic.
Higher chunk trees grow on large filesystems (many terabytes), and prefetching just level 1 seems to be sufficient. Provided example was from a 200TiB filesystem with chunk tree level 2.
--- fs/btrfs/volumes.c | 23 +++++++++++++++++++++++ 1 file changed, 23 insertions(+)
fs/btrfs/volumes.c +23 -0
@@ -7013,6 +7013,19 @@ bool btrfs_check_rw_degradable(struct btrfs_fs_info *fs_info,return ret; } +void readahead_tree_node_children (struct extent_buffer *node) +{ +int i; +const int nr_items =btrfs_header_nritems (node); + +for (i =0 ; i < nr_items; i++) { +u64 start; + + start =btrfs_node_blockptr (node, i); +readahead_tree_block (node->fs_info, start); + } +} +int btrfs_read_chunk_tree (struct btrfs_fs_info *fs_info) {struct btrfs_root *root = fs_info->chunk_root;
@@ -7023,6 +7036,7 @@ int btrfs_read_chunk_tree(struct btrfs_fs_info *fs_info)int ret;int slot;u64 total_dev =0 ; +u64 last_ra_node =0 ; path =btrfs_alloc_path ();if (!path)
@@ -7048,6 +7062,8 @@ int btrfs_read_chunk_tree(struct btrfs_fs_info *fs_info)if (ret <0 ) goto error;while (1 ) { +struct extent_buffer *node; + leaf = path->nodes[0 ]; slot = path->slots[0 ];if (slot >=btrfs_header_nritems (leaf)) {
@@ -7058,6 +7074,13 @@ int btrfs_read_chunk_tree(struct btrfs_fs_info *fs_info) goto error;break ; } + node = path->nodes[1 ]; +if (node) { +if (last_ra_node != node->start) { +readahead_tree_node_children (node); + last_ra_node = node->start; + } + }btrfs_item_key_to_cpu (leaf, &found_key, slot);if (found_key.type ==BTRFS_DEV_ITEM_KEY ) {struct btrfs_dev_item *dev_item;
Signature · 1 line
2.25.0
Quoted 9 lines · David Sterba
The whole chunk tree is read at mount time so we can utilize readahead to get the tree blocks to memory before we read the items. The idea is from Robbie, but instead of updating search slot readahead, this patch implements the chunk tree readahead manually from nodes on level 1.
We've decided to do specific readahead optimizations and then unify them under a common API so we don't break everything by changing the search slot readahead logic.
So this is just for now, and then will be replaced with a bigger rework of the readahead logic?
Quoted 52 lines
Higher chunk trees grow on large filesystems (many terabytes), and prefetching just level 1 seems to be sufficient. Provided example was from a 200TiB filesystem with chunk tree level 2.
CC Robbie Ko <[email protected]>Signed-off-by David Sterba <[email protected]>--- fs/btrfs/volumes.c | 23 +++++++++++++++++++++++ 1 file changed, 23 insertions(+)fs/btrfs/volumes.c +16
-0@@ -7013,6 +7013,19 @@ bool btrfs_check_rw_degradable(struct btrfs_fs_info *fs_info,return ret; } +void readahead_tree_node_children (struct extent_buffer *node) +{ +int i; +const int nr_items =btrfs_header_nritems (node); + +for (i =0 ; i < nr_items; i++) { +u64 start; + + start =btrfs_node_blockptr (node, i); +readahead_tree_block (node->fs_info, start); + } +} +int btrfs_read_chunk_tree (struct btrfs_fs_info *fs_info) {struct btrfs_root *root = fs_info->chunk_root;@@ -7023,6 +7036,7 @@ int btrfs_read_chunk_tree(struct btrfs_fs_info *fs_info)int ret;int slot;u64 total_dev =0 ; +u64 last_ra_node =0 ; path =btrfs_alloc_path ();if (!path)@@ -7048,6 +7062,8 @@ int btrfs_read_chunk_tree(struct btrfs_fs_info *fs_info)if (ret <0 ) goto error;while (1 ) { +struct extent_buffer *node; + leaf = path->nodes[0 ]; slot = path->slots[0 ];if (slot >=btrfs_header_nritems (leaf)) {
fs/btrfs/volumes.c int btrfs_read_chunk_tree(struct btrfs_fs_info *fs_info) · line 7083
Quoted 10 lines
fs/btrfs/volumes.c +7
-0@@ -7058,6 +7074,13 @@ int btrfs_read_chunk_tree(struct btrfs_fs_info *fs_info) goto error;break ; } + node = path->nodes[1 ]; +if (node) { +if (last_ra_node != node->start) { +readahead_tree_node_children (node); + last_ra_node = node->start; + } + }
We're doing a read search, path->nodes[1] won't be read locked here, so this isn't technically safe. I realize that nobody else is going to be messing with stuff here, so maybe just a comment. Thanks,
Josef
Quoted 6 lines · Josef Bacik
Quoted 5 lines · David Sterba
The whole chunk tree is read at mount time so we can utilize readahead to get the tree blocks to memory before we read the items. The idea is from Robbie, but instead of updating search slot readahead, this patch implements the chunk tree readahead manually from nodes on level 1.
Quoted 6 lines
Quoted 3 lines
We've decided to do specific readahead optimizations and then unify them under a common API so we don't break everything by changing the search slot readahead logic.
So this is just for now, and then will be replaced with a bigger rework of the readahead logic?
Yes. Validating the readahead optimization just for the chunk tree is easy and safe. Once we have the readahead logic updated, switching it should be straightforward, I'm expecting one of the RA modes to be like the one for the chunk tree.
Quoted 58 lines
Quoted 58 lines
Higher chunk trees grow on large filesystems (many terabytes), and prefetching just level 1 seems to be sufficient. Provided example was from a 200TiB filesystem with chunk tree level 2.
CC Robbie Ko <[email protected]>Signed-off-by David Sterba <[email protected]>--- fs/btrfs/volumes.c | 23 +++++++++++++++++++++++ 1 file changed, 23 insertions(+)fs/btrfs/volumes.c +19
-0@@ -7013,6 +7013,19 @@ bool btrfs_check_rw_degradable(struct btrfs_fs_info *fs_info,return ret; } +void readahead_tree_node_children (struct extent_buffer *node) +{ +int i; +const int nr_items =btrfs_header_nritems (node); + +for (i =0 ; i < nr_items; i++) { +u64 start; + + start =btrfs_node_blockptr (node, i); +readahead_tree_block (node->fs_info, start); + } +} +int btrfs_read_chunk_tree (struct btrfs_fs_info *fs_info) {struct btrfs_root *root = fs_info->chunk_root;@@ -7023,6 +7036,7 @@ int btrfs_read_chunk_tree(struct btrfs_fs_info *fs_info)int ret;int slot;u64 total_dev =0 ; +u64 last_ra_node =0 ; path =btrfs_alloc_path ();if (!path)@@ -7048,6 +7062,8 @@ int btrfs_read_chunk_tree(struct btrfs_fs_info *fs_info)if (ret <0 ) goto error;while (1 ) { +struct extent_buffer *node; + leaf = path->nodes[0 ]; slot = path->slots[0 ];if (slot >=btrfs_header_nritems (leaf)) {@@ -7058,6 +7074,13 @@ int btrfs_read_chunk_tree(struct btrfs_fs_info *fs_info) goto error;break ; } + node = path->nodes[1 ]; +if (node) { +if (last_ra_node != node->start) {
Quoted 8 lines
fs/btrfs/volumes.c int btrfs_read_chunk_tree(struct btrfs_fs_info *fs_info) · line 7083
Quoted 4 lines
fs/btrfs/volumes.c +4
-0+readahead_tree_node_children (node); + last_ra_node = node->start; + } + }We're doing a read search, path->nodes[1] won't be read locked here, so this isn't technically safe. I realize that nobody else is going to be messing with stuff here, so maybe just a comment. Thanks,
Right, for the mount path we skip locking for some other structures too so I guess as long as it's documented it's acceptable.
David Sterba
maintainerPATCH v2 1/1 btrfs: prefetch chunk tree leaves at mount
In reply to David SterbaThe whole chunk tree is read at mount time so we can utilize readahead to get the tree blocks to memory before we read the items. The idea is from Robbie, but instead of updating search slot readahead, this patch implements the chunk tree readahead manually from nodes on level 1.
We've decided to do specific readahead optimizations and then unify them under a common API so we don't break everything by changing the search slot readahead logic.
Higher chunk trees grow on large filesystems (many terabytes), and prefetching just level 1 seems to be sufficient. Provided example was from a 200TiB filesystem with chunk tree level 2.
---
v2:
- make readahead_tree_node_children static
- add comment about node locks
fs/btrfs/volumes.c | 27 +++++++++++++++++++++++++++ 1 file changed, 27 insertions(+)
fs/btrfs/volumes.c +27 -0
@@ -7013,6 +7013,19 @@ bool btrfs_check_rw_degradable(struct btrfs_fs_info *fs_info,return ret; } +static void readahead_tree_node_children (struct extent_buffer *node) +{ +int i; +const int nr_items =btrfs_header_nritems (node); + +for (i =0 ; i < nr_items; i++) { +u64 start; + + start =btrfs_node_blockptr (node, i); +readahead_tree_block (node->fs_info, start); + } +} +int btrfs_read_chunk_tree (struct btrfs_fs_info *fs_info) {struct btrfs_root *root = fs_info->chunk_root;
@@ -7023,6 +7036,7 @@ int btrfs_read_chunk_tree(struct btrfs_fs_info *fs_info)int ret;int slot;u64 total_dev =0 ; +u64 last_ra_node =0 ; path =btrfs_alloc_path ();if (!path)
@@ -7048,6 +7062,8 @@ int btrfs_read_chunk_tree(struct btrfs_fs_info *fs_info)if (ret <0 ) goto error;while (1 ) { +struct extent_buffer *node; + leaf = path->nodes[0 ]; slot = path->slots[0 ];if (slot >=btrfs_header_nritems (leaf)) {
@@ -7058,6 +7074,17 @@ int btrfs_read_chunk_tree(struct btrfs_fs_info *fs_info) goto error;break ; } +/* +* The nodes on level 1 are not locked but we don't need to do +* that during mount time as nothing else can access the tree +*/ + node = path->nodes[1 ]; +if (node) { +if (last_ra_node != node->start) { +readahead_tree_node_children (node); + last_ra_node = node->start; + } + }btrfs_item_key_to_cpu (leaf, &found_key, slot);if (found_key.type ==BTRFS_DEV_ITEM_KEY ) {struct btrfs_dev_item *dev_item;
Signature · 1 line
2.25.0
Quoted 9 lines · David Sterba
The whole chunk tree is read at mount time so we can utilize readahead to get the tree blocks to memory before we read the items. The idea is from Robbie, but instead of updating search slot readahead, this patch implements the chunk tree readahead manually from nodes on level 1.
We've decided to do specific readahead optimizations and then unify them under a common API so we don't break everything by changing the search slot readahead logic.
Quoted 6 lines
Higher chunk trees grow on large filesystems (many terabytes), and prefetching just level 1 seems to be sufficient. Provided example was from a 200TiB filesystem with chunk tree level 2.
CC Robbie Ko <[email protected]>Signed-off-by David Sterba <[email protected]>
Thanks,
Josef