Skip to contentSkip to conversation

btrfs: qgroup: Don't trace subtree if we're dropping tree reloc tree

PATCH In review Qu Wenruo 3 participants 5 messages Sep 5, 2018 – Sep 6, 2018 Btrfs file system

lore
Share & tools
lore

Qu Wenruo

author
#0 lore raw

PATCH 1/1 btrfs: qgroup: Don't trace subtree if we're dropping tree reloc tree

Tree reloc tree doesn't contribute to qgroup numbers, as we have accounted them at balance time (check replace_path()).

Skip such unneeded subtree trace should reduce some performance overhead.

---
 fs/btrfs/extent-tree.c | 8 +++++++-
 1 file changed, 7 insertions(+), 1 deletion(-)
fs/btrfs/extent-tree.c +7 -1
index de6f75f5547b..4588153f414c 100644
@@ -8643,7 +8643,13 @@ static noinline int do_walk_down(struct btrfs_trans_handle *trans,
 			parent = 0;
 		}
 
-		if (need_account) {
+		/*
+		 * Tree reloc tree doesn't contribute to qgroup numbers, and
+		 * we have already accounted them at merge time (replace_path),
+		 * thus we could skip expensive subtree trace here.
+		 */
+		if (root->root_key.objectid != BTRFS_TREE_RELOC_OBJECTID &&
+		    need_account) {
 			ret = btrfs_qgroup_trace_subtree(trans, next,
 							 generation, level - 1);
 			if (ret) {
Signature · 1 line
2.18.0
#1 lore raw
Quoted 1 line · Qu Wenruo

Tree reloc tree doesn't contribute to qgroup numbers, as we have

I think you can call it just 'reloc tree', I'm fixing that in all changelogs and comments anyway.

Quoted 4 lines

accounted them at balance time (check replace_path()).

Skip such unneeded subtree trace should reduce some performance overhead.

Please provide some numbers or description of the improvement. There are several performance problems caused by qgroups so it would be good to get a better idea how much this patch is going to help. Thanks.

#2 lore raw
Quoted 5 lines · David Sterba
Quoted 1 line · Qu Wenruo

Tree reloc tree doesn't contribute to qgroup numbers, as we have

I think you can call it just 'reloc tree', I'm fixing that in all changelogs and comments anyway.

But there is another tree called data reloc tree.
That why I'm sticking to tree reloc tree to distinguish from data reloc tree.

Quoted 9 lines
Quoted 5 lines

accounted them at balance time (check replace_path()).

Skip such unneeded subtree trace should reduce some performance overhead.

Please provide some numbers or description of the improvement. There are several performance problems caused by qgroups so it would be good to get a better idea how much this patch is going to help. Thanks.

That's the problem.
For my internal test, with 3000+ tree blocks, metadata balance could save about 1~2%.
But according to dump-tree, the tree layout is almost the worst case scenario, just one metadata block group owns all the tree blocks.

To get a real world scenario, I need a file with hundreds GB or even several TB and populate it with a good amount of inline files and enough CoW to fragment the metadata usage.
Which I don't have such free space.

Anyone who is still struggling with balance + quota, any test data is appreciated.

Thanks,
Qu

#3 lore raw
Quoted 4 lines · Qu Wenruo
Quoted 1 line · David Sterba

On Wed, Sep 05, 2018 at 01:03:39PM +0800, Qu Wenruo wrote:

Quoted 8 lines
Quoted 4 lines
Quoted 1 line

Tree reloc tree doesn't contribute to qgroup numbers, as we have

I think you can call it just 'reloc tree', I'm fixing that in all changelogs and comments anyway.

But there is another tree called data reloc tree.
That why I'm sticking to tree reloc tree to distinguish from data reloc tree.

So call it 'reloc tree' and 'data reloc tree', the naming of BTRFS_TREE_RELOC_OBJECTID does not follow the other tree naming and I don't know if the inention was to name it after the 'tree log' scheme or the 'extent tree'. Or if there was an intention behind the naming at all.

If this is going to bother us too much, I won't mind renaming it to BTRFS_RELOC_TREE_OBJECTID everywhere. For consistency.

Quoted 9 lines
Quoted 8 lines
Quoted 4 lines

accounted them at balance time (check replace_path()).

Skip such unneeded subtree trace should reduce some performance overhead.

Please provide some numbers or description of the improvement. There are several performance problems caused by qgroups so it would be good to get a better idea how much this patch is going to help. Thanks.

Quoted 5 lines

That's the problem.
For my internal test, with 3000+ tree blocks, metadata balance could save about 1~2%.
But according to dump-tree, the tree layout is almost the worst case scenario, just one metadata block group owns all the tree blocks.

If you do such test, it's also a good example for the changelog. It describes the worst case and this information can be used to prepare testing environment on large data samples.

#4 lore raw
Quoted 14 lines · David Sterba
Quoted 4 lines · Qu Wenruo
Quoted 1 line · David Sterba

On Wed, Sep 05, 2018 at 01:03:39PM +0800, Qu Wenruo wrote:

Quoted 8 lines
Quoted 4 lines
Quoted 1 line

Tree reloc tree doesn't contribute to qgroup numbers, as we have

I think you can call it just 'reloc tree', I'm fixing that in all changelogs and comments anyway.

But there is another tree called data reloc tree.
That why I'm sticking to tree reloc tree to distinguish from data reloc tree.

Quoted 9 lines

So call it 'reloc tree' and 'data reloc tree', the naming of BTRFS_TREE_RELOC_OBJECTID does not follow the other tree naming and I don't know if the inention was to name it after the 'tree log' scheme or the 'extent tree'. Or if there was an intention behind the naming at all.

If this is going to bother us too much, I won't mind renaming it to BTRFS_RELOC_TREE_OBJECTID everywhere. For consistency.

I'll go reloc tree for later reference.

Renaming makes sense, although I'd like to change its name to something more distinguish (without the reloc part).

For now, I'm fine using reloc tree.

Quoted 15 lines
Quoted 9 lines
Quoted 8 lines
Quoted 4 lines

accounted them at balance time (check replace_path()).

Skip such unneeded subtree trace should reduce some performance overhead.

Please provide some numbers or description of the improvement. There are several performance problems caused by qgroups so it would be good to get a better idea how much this patch is going to help. Thanks.

Quoted 5 lines

That's the problem.
For my internal test, with 3000+ tree blocks, metadata balance could save about 1~2%.
But according to dump-tree, the tree layout is almost the worst case scenario, just one metadata block group owns all the tree blocks.

Quoted 3 lines

If you do such test, it's also a good example for the changelog. It describes the worst case and this information can be used to prepare testing environment on large data samples.

Already preparing the real world case (great thanks for Fujitsu guys providing 1TB storage for the test).

I'll provide the data along after the test is done.

Thanks,
Qu