btrfs: qgroup: Fix the wrong target io_tree when freeing reserved data space
PATCH 1/2 btrfs: qgroup: Fix the wrong target io_tree when freeing reserved data space
[BUG]
Under the follow case with qgroup enabled, if some error happened after we have reserved delalloc space, then in error handling path, we could cause qgroup data space leakage:
From btrfs_truncate_block() in inode.c:
ret = btrfs_delalloc_reserve_space(inode, &data_reserved, block_start, blocksize); if (ret) goto out;
again:
page = find_or_create_page(mapping, index, mask);
if (!page) {
btrfs_delalloc_release_space(inode, data_reserved,
block_start, blocksize, true);
btrfs_delalloc_release_extents(BTRFS_I(inode), blocksize, true);
ret = -ENOMEM;
goto out;
}[CAUSE]
In above case, btrfs_delalloc_reserve_space() will call btrfs_qgroup_reserve_data() and mark the io_tree range with EXTENT_QGROUP_RESERVED flag.
In the error handling path, btrfs_delalloc_release_space() calls btrfs_qgroup_free_data() which should clear EXTENT_QGROUP_RESERVED flag and reduce the reserved data space accroding to the cleared range.
However due to a completion bug, btrfs_qgroup_free_data() will clear EXTENT_QGROUP_RESERVED flag in BTRFS_I(inode)->io_failure_tree, other than the correct BTRFS_I(inode)->io_tree.
Since io_failure_tree is never marked with that flag, btrfs_qgroup_free_data() will not free any data reserved space at all, causing a leakage.
All of such error handling cases can only be triggered some errors not from qgroup, so regular EDQUOT error won't trigger the bug. Normally we need error injection to trigger such bug.
[FIX]
Fix the wrong target io_tree.
--- fs/btrfs/qgroup.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
fs/btrfs/qgroup.c +1 -1
@@ -3492,7 +3492,7 @@ static int qgroup_free_reserved_data(struct inode *inode,* EXTENT_QGROUP_RESERVED, we won't double free. * So not need to rush. */ - ret =clear_record_extent_bits (&BTRFS_I (inode)->io_failure_tree, + ret =clear_record_extent_bits (&BTRFS_I (inode)->io_tree, free_start, free_start + free_len -1 ,EXTENT_QGROUP_RESERVED , &changeset);if (ret <0 )
Signature · 1 line
2.23.0
PATCH 2/2 btrfs: qgroup: Fix reserved data space leak if we have multiple reserve calls
[BUG]
The following script can cause btrfs qgroup data space leak:
mkfs.btrfs -f $dev
mount $dev -o nospace_cache $mnt
btrfs subv create $mnt/subv
btrfs quota en $mnt
btrfs quota rescan -w $mnt
btrfs qgroup limit 128m $mnt/subv
for (( i = 0; i < 3; i++)); do
# Create 3 64M holes for latter fallocate to fail truncate -s 192m $mnt/subv/file
xfs_io -c "pwrite 64m 4k" $mnt/subv/file > /dev/null xfs_io -c "pwrite 128m 4k" $mnt/subv/file > /dev/null sync
# it's supposed to fail, and each failure will leak at least 64M # data space
xfs_io -f -c "falloc 0 192m" $mnt/subv/file &> /dev/null rm $mnt/subv/file
sync
done
# Shouldn't fail after we removed the file
xfs_io -f -c "falloc 0 64m" $mnt/subv/file
[CAUSE]
Btrfs qgroup data reserve code allows multiple reserve happen on a single extent_changeset:
The only usage is in btrfs_fallocate():
struct extent_changeset *data_reserved = NULL; btrfs_qgroup_reserve_data(inode, &data_reserved, range_start, range_len); ... btrfs_qgroup_reserve_data(inode, &data_reserved, new_range_start, new_range_len); extent_changeset_free(data_reserved);
However in btrfs_qgroup_reserve_data(), if one of the call failed, it will cleanup all reserved space.
The cleanup itself is OK, but it only cleans up all EXTENT_QGROUP_RESERVED flag, forget to release the reserved bytes.
So if multiple btrfs_qgroup_reserve_data() get called, and the last one failed, then previously reserved data space will get leaked.
And due to the fact that EXTENT_QGROUP_RESERVED flag is cleaned correctly, btrfs_qgroup_check_reserved_leak() won't catch the leakage.
[FIX]
Also free previously reserved data bytes when btrfs_qgroup_reserve_data fails.
--- fs/btrfs/qgroup.c | 3 +++ 1 file changed, 3 insertions(+)
fs/btrfs/qgroup.c +3 -0
@@ -3448,6 +3448,9 @@ int btrfs_qgroup_reserve_data(struct inode *inode,while ((unode =ulist_next (&reserved->range_changed, &uiter)))clear_extent_bit (&BTRFS_I (inode)->io_tree, unode->val, unode->aux,EXTENT_QGROUP_RESERVED ,0 ,0 ,NULL ); +/* Also free data bytes of already reserved one */ +btrfs_qgroup_free_refroot (root->fs_info, root->root_key.objectid, + orig_reserved,BTRFS_QGROUP_RSV_DATA );extent_changeset_release (reserved);return ret; }
Signature · 1 line
2.23.0
Re: [PATCH 2/2] btrfs: qgroup: Fix reserved data space leak if we have multiple reserve calls
Quoted 25 lines · Qu Wenruo
[BUG]
The following script can cause btrfs qgroup data space leak:mkfs.btrfs -f $dev
mount $dev -onospace_cache$mntbtrfs subv create $mnt/subv
btrfs quota en $mnt
btrfs quota rescan -w $mnt
btrfs qgroup limit 128m $mnt/subvfor (( i = 0; i < 3; i++)); do
# Create 3 64M holes for latter fallocate to fail truncate -s 192m $mnt/subv/file
xfs_io-c "pwrite 64m 4k" $mnt/subv/file >/dev/nullxfs_io-c "pwrite 128m 4k" $mnt/subv/file >/dev/nullsync# it's supposed to fail, and each failure will leak at least 64M # data space
xfs_io-f -c "falloc 0 192m" $mnt/subv/file &>/dev/nullrm $mnt/subv/file
sync
done
Quoted 5 lines
# Shouldn't fail after we removed the file
xfs_io-f -c "falloc 0 64m" $mnt/subv/file[CAUSE]
Btrfs qgroup data reserve code allows multiple reserve happen on a
^
reservations to happen
Quoted 10 lines
single
extent_changeset:The only usage is in
btrfs_fallocate():struct extent_changeset *data_reserved = NULL; btrfs_qgroup_reserve_data(inode, &data_reserved, range_start, range_len); ... btrfs_qgroup_reserve_data(inode, &data_reserved, new_range_start, new_range_len); extent_changeset_free(data_reserved);
I take it you refer to the while() loop in btrfs_fallocate. The code above is really just a _VERY_ condensed version. extent_changeset_free is at the end of the function. Instead of putting random lines of code just explicitly state it, something along the lines of:
"The only such pattern is in btrfs_fallocate in the main while loop in that function".
Quoted 10 lines
However in
btrfs_qgroup_reserve_data(), if one of the call failed, it > will cleanup all reserved space.
The cleanup itself is OK, but it only cleans up allEXTENT_QGROUP_RESERVEDflag, forget to release the reserved bytes.So if multiple
btrfs_qgroup_reserve_data()get called, and the last one failed, then previously reserved data space will get leaked.And due to the fact that
EXTENT_QGROUP_RESERVEDflag is cleaned correctly,btrfs_qgroup_check_reserved_leak()won't catch the leakage.
How about rephraing the above 3 paragraphs along the lines of:
"btrfs_qgroup_reserve_data's error handling has a bug in that on error it clears all ranges in the io_tree with EXTENT_QGROUP_RESERVED flag and doesn't free the reserved bytes. This behavior has a two fold effect:
1. Clearing EXTENT_QGROUP_RESERVED ranges preventsbtrfs_qgroup_check_reserved_leak to catch the leakage.
2. Leak the previously reserved data bytes.
The bug manifests when N calls to btrfs_qgroup_reserve_data are made and the last one fails, leaking space allocated in the previous ones. "
Quoted 26 lines
[FIX]
Also free previously reserved data bytes whenbtrfs_qgroup_reserve_datafails.Fixes 524725537023 ("btrfs: qgroup: Introduce btrfs_qgroup_reserve_data function")Signed-off-by Qu Wenruo <[email protected]>--- fs/btrfs/qgroup.c | 3 +++ 1 file changed, 3 insertions(+)fs/btrfs/qgroup.c +3
-0@@ -3448,6 +3448,9 @@ int btrfs_qgroup_reserve_data(struct inode *inode,while ((unode =ulist_next (&reserved->range_changed, &uiter)))clear_extent_bit (&BTRFS_I (inode)->io_tree, unode->val, unode->aux,EXTENT_QGROUP_RESERVED ,0 ,0 ,NULL ); +/* Also free data bytes of already reserved one */ +btrfs_qgroup_free_refroot (root->fs_info, root->root_key.objectid, + orig_reserved,BTRFS_QGROUP_RSV_DATA );extent_changeset_release (reserved);return ret; }
Quoted 27 lines · Qu Wenruo
[BUG]
Under the follow case with qgroup enabled, if some error happened after we have reserved delalloc space, then in error handling path, we could cause qgroup data space leakage:From
btrfs_truncate_block()ininode.c:ret = btrfs_delalloc_reserve_space(inode, &data_reserved, block_start, blocksize); if (ret) goto out;again:
page = find_or_create_page(mapping, index, mask); if (!page) { btrfs_delalloc_release_space(inode, data_reserved, block_start, blocksize, true); btrfs_delalloc_release_extents(BTRFS_I(inode), blocksize, true); ret = -ENOMEM; goto out; }[CAUSE]
In above case,btrfs_delalloc_reserve_space()will callbtrfs_qgroup_reserve_data()and mark theio_treerange withEXTENT_QGROUP_RESERVEDflag.
Quoted 7 lines
In the error handling path,
btrfs_delalloc_release_space()callsbtrfs_qgroup_free_data()which should clearEXTENT_QGROUP_RESERVEDflag and reduce the reserved data space accroding to the cleared range.However due to a completion bug,
btrfs_qgroup_free_data()will clearEXTENT_QGROUP_RESERVEDflag in BTRFS_I(inode)->io_failure_tree, other than the correct BTRFS_I(inode)->io_tree.
This is a bit confusing because the error is actually in qgroup_free_reserved_data, which is called from __btrfs_qgroup_release_data. But in the latter function there is also a call to clear_record_extent_bits with the correct tree. Just fix the function name by using qgroup_free_reserved_data.
Quoted 5 lines
Since
io_failure_treeis never marked with that flag,btrfs_qgroup_free_data()will not free any data reserved space at all, causing a leakage.All of such error handling cases can only be triggered some errors not
I take it you meant:
This error handling can only be triggered by errors outside of qgroup e.g. EDQUOT can't triger the bug?
The first part of the sentence is hard to parse.
Quoted 27 lines
from qgroup, so regular EDQUOT error won't trigger the bug. Normally we need error injection to trigger such bug.
[FIX]
Fix the wrong targetio_tree.Reported-by Josef Bacik <[email protected]>Fixes bc42bda22345 ("btrfs: qgroup: Fix qgroup reserved space underflow by only freeing reserved ranges")Signed-off-by Qu Wenruo <[email protected]>--- fs/btrfs/qgroup.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)fs/btrfs/qgroup.c +1
-1@@ -3492,7 +3492,7 @@ static int qgroup_free_reserved_data(struct inode *inode,* EXTENT_QGROUP_RESERVED, we won't double free. * So not need to rush. */ - ret =clear_record_extent_bits (&BTRFS_I (inode)->io_failure_tree, + ret =clear_record_extent_bits (&BTRFS_I (inode)->io_tree, free_start, free_start + free_len -1 ,EXTENT_QGROUP_RESERVED , &changeset);if (ret <0 )
Quoted 38 lines · Nikolay Borisov
Quoted 27 lines · Qu Wenruo
[BUG]
Under the follow case with qgroup enabled, if some error happened after we have reserved delalloc space, then in error handling path, we could cause qgroup data space leakage:From
btrfs_truncate_block()ininode.c:ret = btrfs_delalloc_reserve_space(inode, &data_reserved, block_start, blocksize); if (ret) goto out;again:
page = find_or_create_page(mapping, index, mask); if (!page) { btrfs_delalloc_release_space(inode, data_reserved, block_start, blocksize, true); btrfs_delalloc_release_extents(BTRFS_I(inode), blocksize, true); ret = -ENOMEM; goto out; }[CAUSE]
In above case,btrfs_delalloc_reserve_space()will callbtrfs_qgroup_reserve_data()and mark theio_treerange withEXTENT_QGROUP_RESERVEDflag.Quoted 7 lines
In the error handling path,
btrfs_delalloc_release_space()callsbtrfs_qgroup_free_data()which should clearEXTENT_QGROUP_RESERVEDflag and reduce the reserved data space accroding to the cleared range.However due to a completion bug,
btrfs_qgroup_free_data()will clearEXTENT_QGROUP_RESERVEDflag in BTRFS_I(inode)->io_failure_tree, other than the correct BTRFS_I(inode)->io_tree.
Quoted 5 lines
This is a bit confusing because the error is actually in
qgroup_free_reserved_data, which is called from__btrfs_qgroup_release_data. But in the latter function there is also a call toclear_record_extent_bitswith the correct tree. Just fix the function name by usingqgroup_free_reserved_data.
Right, I ignored some caller here, as the caller chain is not only dependent on btrfs_qgroup_free_data() but also on the parameter. E.g. only when reserved is non-null we go qgroup_free_reserved_data().
Quoted 11 lines
Quoted 6 lines
Since
io_failure_treeis never marked with that flag,btrfs_qgroup_free_data()will not free any data reserved space at all, causing a leakage.All of such error handling cases can only be triggered some errors not
I take it you meant:
This error handling can only be triggered by errors outside of qgroup e.g. EDQUOT can't triger the bug?
Right.
I'll change it too something like "such leakage can only be triggered by errors outside of qgroup."
Thanks,
Qu
Quoted 30 lines
The first part of the sentence is hard to parse.
Quoted 27 lines
from qgroup, so regular EDQUOT error won't trigger the bug. Normally we need error injection to trigger such bug.
[FIX]
Fix the wrong targetio_tree.Reported-by Josef Bacik <[email protected]>Fixes bc42bda22345 ("btrfs: qgroup: Fix qgroup reserved space underflow by only freeing reserved ranges")Signed-off-by Qu Wenruo <[email protected]>--- fs/btrfs/qgroup.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)fs/btrfs/qgroup.c +1
-1@@ -3492,7 +3492,7 @@ static int qgroup_free_reserved_data(struct inode *inode,* EXTENT_QGROUP_RESERVED, we won't double free. * So not need to rush. */ - ret =clear_record_extent_bits (&BTRFS_I (inode)->io_failure_tree, + ret =clear_record_extent_bits (&BTRFS_I (inode)->io_tree, free_start, free_start + free_len -1 ,EXTENT_QGROUP_RESERVED , &changeset);if (ret <0 )