fstests: btrfs: test snapshot creation with existing qgroup
PATCH 1/1 fstests: btrfs: test snapshot creation with existing qgroup
[BUG]
There is a sysbot regression report about transaction abort during snapshot creation, which is caused by the new timing of qgroup creation and too strict error check.
[FIX]
The proper fix is already submitted, with the title "btrfs: do not abort transaction if there is already an existing qgroup".
[TEST]
The new test case would reproduce the regression by:
- Create a subvolume and a snapshot of it
- Record the subvolumeid of the snapshot
- Re-create the fs
Since btrfs won't reuse the subvolume id, we have to re-create the fs.
- Enable quota and create a qgroup with the same subvolumeid
- Create a subvolume and a snapshot of it
For unpatched and affected kernel (thankfully no release is affected), the snapshot creation would fail due to aborted transaction.
- Make sure the subvolume id doesn't change for the snapshot There is one very hacky attempt to fix it by avoiding using the subvolume id, which is completely wrong and would be caught by this extra check.
--- tests/btrfs/303 | 80 +++++++++++++++++++++++++++++++++++++++++++++ tests/btrfs/303.out | 2 ++ 2 files changed, 82 insertions(+) create mode 100755 tests/btrfs/303 create mode 100644 tests/btrfs/303.out
tests/btrfs/303 new file +80 -0
@@ -0,0 +1,80 @@ +#! /bin/bash +# SPDX-License-Identifier: GPL-2.0 +# Copyright (C) 2023 SUSE Linux Products GmbH. All Rights Reserved. +# +# FS QA Test 303 +# +# A regression test to make sure snapshot creation won't cause transaction +# abort if there is already an existing qgroup. +# +. ./common/preamble +_begin_fstest auto quick qgroup + +. ./common/filter + +_supported_fs btrfs +_require_scratch + +_fixed_by_kernel_commit xxxxxxxxxxxx \ + "btrfs: do not abort transaction if there is already an existing qgroup" + +_scratch_mkfs >> $seqres.full 2>&1 || _fail "mkfs failed" +_scratch_mount + +# Create the first subvolume and get its id. +# This subvolume id should not change no matter if there is an existing +# qgroup for it. +$BTRFS_UTIL_PROG subvolume create "$SCRATCH_MNT/subvol" >> $seqres.full +$BTRFS_UTIL_PROG subvolume snapshot "$SCRATCH_MNT/subvol" \ + "$SCRATCH_MNT/snapshot">> $seqres.full + +init_subvolid=$(_btrfs_get_subvolid "$SCRATCH_MNT" "snapshot") + +if [ -z "$init_subvolid" ]; then + _fail "Unable to get the subvolid of the first snapshot" +fi + +echo "Subvolumeid: ${init_subvolid}" >> $seqres.full + +_scratch_unmount + +# Re-create the fs, as btrfs won't reuse the subvolume id. +_scratch_mkfs >> $seqres.full 2>&1 || _fail "2nd mkfs failed" +_scratch_mount + +$BTRFS_UTIL_PROG quota enable "$SCRATCH_MNT" >> $seqres.full +$BTRFS_UTIL_PROG quota rescan -w "$SCRATCH_MNT" >> $seqres.full + +# Create a qgroup for the first subvolume, this would make the later +# subvolume creation to find an existing qgroup, and abort transaction. +$BTRFS_UTIL_PROG qgroup create 0/"$init_subvolid" "$SCRATCH_MNT" >> $seqres.full +sync + +# Now create the first snapshot, which should have the same subvolid no matter +# if the quota is enabled. +$BTRFS_UTIL_PROG subvolume create "$SCRATCH_MNT/subvol" >> $seqres.full +$BTRFS_UTIL_PROG subvolume snapshot "$SCRATCH_MNT/subvol" \ + "$SCRATCH_MNT/snapshot">> $seqres.full + +# Either the snapshot create failed and transaction is aborted thus no +# snapshot here, or we should be able to create the snapshot. +new_subvolid=$(_btrfs_get_subvolid "$SCRATCH_MNT" "snapshot") + +echo "Subvolumeid: ${new_subvolid}" >> $seqres.full + +if [ -z "$new_subvolid" ]; then + _fail "Unable to get the subvolid of the first snapshot" +fi + +# Make sure the subvolumeid for the first snapshot didn't change. +if [ "$new_subvolid" -ne "$init_subvolid" ]; then + _fail "Subvolumeid for the first snapshot changed, has ${new_subvolid} expect ${init_subvolid}" +fi + +_scratch_unmount + +echo "Silence is golden" + +# success, all done +status=0 +exit
tests/btrfs/303.out new file +2 -0
@@ -0,0 +1,2 @@ +QA output created by 303 +Silence is golden
Signature · 1 line
2.42.0
Quoted 46 lines · Qu Wenruo
[BUG]
There is a sysbot regression report about transaction abort during snapshot creation, which is caused by the new timing of qgroup creation and too strict error check.[FIX]
The proper fix is already submitted, with the title "btrfs: do not abort transaction if there is already an existing qgroup".[TEST]
The new test case would reproduce the regression by:- Create a subvolume and a snapshot of it
- Record the subvolumeid of the snapshot
- Re-create the fs
Since btrfs won't reuse the subvolume id, we have to re-create the fs.- Enable quota and create a qgroup with the same subvolumeid
- Create a subvolume and a snapshot of it
For unpatched and affected kernel (thankfully no release is affected), the snapshot creation would fail due to aborted transaction.- Make sure the subvolume id doesn't change for the snapshot There is one very hacky attempt to fix it by avoiding using the subvolume id, which is completely wrong and would be caught by this extra check.
Signed-off-by Qu Wenruo <[email protected]>--- tests/btrfs/303 | 80 +++++++++++++++++++++++++++++++++++++++++++++ tests/btrfs/303.out | 2 ++ 2 files changed, 82 insertions(+) create mode 100755 tests/btrfs/303 create mode 100644 tests/btrfs/303.outtests/btrfs/303 new file +1
-0@@ -0,0 +1,80 @@ +#! /bin/bash
tests/btrfs/303 line 11
Quoted 10 lines
tests/btrfs/303 new file +10
-0+# SPDX-License-Identifier: GPL-2.0 +# Copyright (C) 2023 SUSE Linux Products GmbH. All Rights Reserved. +# +# FS QA Test 303 +# +# A regression test to make sure snapshot creation won't cause transaction +# abort if there is already an existing qgroup. +# +. ./common/preamble +_begin_fstest auto quick qgroup
Also 'snapshot' and 'subvol' groups.
Quoted 30 lines
tests/btrfs/303 new file +30
-0+ +. ./common/filter + +_supported_fs btrfs +_require_scratch + +_fixed_by_kernel_commit xxxxxxxxxxxx \ + "btrfs: do not abort transaction if there is already an existing qgroup" + +_scratch_mkfs >> $seqres.full 2>&1 || _fail "mkfs failed" +_scratch_mount + +# Create the first subvolume and get its id. +# This subvolume id should not change no matter if there is an existing +# qgroup for it. +$BTRFS_UTIL_PROG subvolume create "$SCRATCH_MNT/subvol" >> $seqres.full +$BTRFS_UTIL_PROG subvolume snapshot "$SCRATCH_MNT/subvol" \ + "$SCRATCH_MNT/snapshot">> $seqres.full + +init_subvolid=$(_btrfs_get_subvolid "$SCRATCH_MNT" "snapshot") + +if [ -z "$init_subvolid" ]; then + _fail "Unable to get the subvolid of the first snapshot" +fi + +echo "Subvolumeid: ${init_subvolid}" >> $seqres.full + +_scratch_unmount + +# Re-create the fs, as btrfs won't reuse the subvolume id.
tests/btrfs/303 line 51
Quoted 10 lines
tests/btrfs/303 new file +10
-0+_scratch_mkfs >> $seqres.full 2>&1 || _fail "2nd mkfs failed" +_scratch_mount + +$BTRFS_UTIL_PROG quota enable "$SCRATCH_MNT" >> $seqres.full +$BTRFS_UTIL_PROG quota rescan -w "$SCRATCH_MNT" >> $seqres.full + +# Create a qgroup for the first subvolume, this would make the later +# subvolume creation to find an existing qgroup, and abort transaction. +$BTRFS_UTIL_PROG qgroup create 0/"$init_subvolid" "$SCRATCH_MNT" >> $seqres.full +sync
This sync is not needed. An unpatched kernel still fails, and a patched kernel passes this test without the sync.
Also, please always comment on why a sync is needed. In this case it can be removed because it's redundant.
Quoted 13 lines
tests/btrfs/303 new file +13
-0+ +# Now create the first snapshot, which should have the same subvolid no matter +# if the quota is enabled. +$BTRFS_UTIL_PROG subvolume create "$SCRATCH_MNT/subvol" >> $seqres.full +$BTRFS_UTIL_PROG subvolume snapshot "$SCRATCH_MNT/subvol" \ + "$SCRATCH_MNT/snapshot">> $seqres.full + +# Either the snapshot create failed and transaction is aborted thus no +# snapshot here, or we should be able to create the snapshot. +new_subvolid=$(_btrfs_get_subvolid "$SCRATCH_MNT" "snapshot") + +echo "Subvolumeid: ${new_subvolid}" >> $seqres.full +
tests/btrfs/303 line 74
Quoted 10 lines
tests/btrfs/303 new file +10
-0+if [ -z "$new_subvolid" ]; then + _fail "Unable to get the subvolid of the first snapshot" +fi + +# Make sure the subvolumeid for the first snapshot didn't change. +if [ "$new_subvolid" -ne "$init_subvolid" ]; then + _fail "Subvolumeid for the first snapshot changed, has ${new_subvolid} expect ${init_subvolid}" +fi + +_scratch_unmount
This explicit unmount is not needed, the fstests framework automatically does that.
Otherwise it looks fine, thanks.
Quoted 18 lines
tests/btrfs/303 new file +6
-0+ +echo "Silence is golden" + +# success, all done +status=0 +exittests/btrfs/303.out new file +2
-1@@ -0,0 +1,2 @@ +QA output created by 303 +Silence is golden --2.42.0
Quoted 104 lines · Filipe Manana
Quoted 46 lines · Qu Wenruo
[BUG]
There is a sysbot regression report about transaction abort during snapshot creation, which is caused by the new timing of qgroup creation and too strict error check.[FIX]
The proper fix is already submitted, with the title "btrfs: do not abort transaction if there is already an existing qgroup".[TEST]
The new test case would reproduce the regression by:- Create a subvolume and a snapshot of it
- Record the subvolumeid of the snapshot
- Re-create the fs
Since btrfs won't reuse the subvolume id, we have to re-create the fs.- Enable quota and create a qgroup with the same subvolumeid
- Create a subvolume and a snapshot of it
For unpatched and affected kernel (thankfully no release is affected), the snapshot creation would fail due to aborted transaction.- Make sure the subvolume id doesn't change for the snapshot There is one very hacky attempt to fix it by avoiding using the subvolume id, which is completely wrong and would be caught by this extra check.
Signed-off-by Qu Wenruo <[email protected]>--- tests/btrfs/303 | 80 +++++++++++++++++++++++++++++++++++++++++++++ tests/btrfs/303.out | 2 ++ 2 files changed, 82 insertions(+)create mode 100755 tests/btrfs/303
create mode 100644 tests/btrfs/303.outtests/btrfs/303 new file +1
-0@@ -0,0 +1,80 @@ +#! /bin/bashtests/btrfs/303 line 11
Quoted 10 lines
tests/btrfs/303 new file +10
-0+# SPDX-License-Identifier: GPL-2.0 +# Copyright (C) 2023 SUSE Linux Products GmbH. All Rights Reserved. +# +# FS QA Test 303 +# +# A regression test to make sure snapshot creation won't cause transaction +# abort if there is already an existing qgroup. +# +. ./common/preamble +_begin_fstest auto quick qgroupAlso 'snapshot' and 'subvol' groups.
Quoted 30 lines
tests/btrfs/303 new file +30
-0+ +. ./common/filter + +_supported_fs btrfs +_require_scratch + +_fixed_by_kernel_commit xxxxxxxxxxxx \ + "btrfs: do not abort transaction if there is already an existing qgroup" + +_scratch_mkfs >> $seqres.full 2>&1 || _fail "mkfs failed" +_scratch_mount + +# Create the first subvolume and get its id. +# This subvolume id should not change no matter if there is an existing +# qgroup for it. +$BTRFS_UTIL_PROG subvolume create "$SCRATCH_MNT/subvol" >> $seqres.full +$BTRFS_UTIL_PROG subvolume snapshot "$SCRATCH_MNT/subvol" \ + "$SCRATCH_MNT/snapshot">> $seqres.full + +init_subvolid=$(_btrfs_get_subvolid "$SCRATCH_MNT" "snapshot") + +if [ -z "$init_subvolid" ]; then + _fail "Unable to get the subvolid of the first snapshot" +fi + +echo "Subvolumeid: ${init_subvolid}" >> $seqres.full + +_scratch_unmount + +# Re-create the fs, as btrfs won't reuse the subvolume id.tests/btrfs/303 line 51
Quoted 10 lines
tests/btrfs/303 new file +10
-0+_scratch_mkfs >> $seqres.full 2>&1 || _fail "2nd mkfs failed" +_scratch_mount + +$BTRFS_UTIL_PROG quota enable "$SCRATCH_MNT" >> $seqres.full +$BTRFS_UTIL_PROG quota rescan -w "$SCRATCH_MNT" >> $seqres.full + +# Create a qgroup for the first subvolume, this would make the later +# subvolume creation to find an existing qgroup, and abort transaction. +$BTRFS_UTIL_PROG qgroup create 0/"$init_subvolid" "$SCRATCH_MNT" >> $seqres.full +syncThis sync is not needed. An unpatched kernel still fails, and a patched kernel passes this test without the sync.
Quoted 2 lines
Also, please always comment on why a sync is needed. In this case it can be removed because it's redundant.
Would address all comments, but here I'm a little curious about the "sync" comment principle.
I totally understand for this particular case the "sync" is unnecessary, the qgroup item doesn't need to be committed, thus it's totally reasonable to remove this sync.
But I'm wondering is there any other reasons why we should avoid unnecessary "sync"s?
Like slowing down the test or just to improve our awareness and avoid sync-happy guys?
Thanks,
Qu
Quoted 49 lines
Quoted 14 lines
tests/btrfs/303 new file +13
-0+ +# Now create the first snapshot, which should have the same subvolid no matter +# if the quota is enabled. +$BTRFS_UTIL_PROG subvolume create "$SCRATCH_MNT/subvol" >> $seqres.full +$BTRFS_UTIL_PROG subvolume snapshot "$SCRATCH_MNT/subvol" \ + "$SCRATCH_MNT/snapshot">> $seqres.full + +# Either the snapshot create failed and transaction is aborted thus no +# snapshot here, or we should be able to create the snapshot. +new_subvolid=$(_btrfs_get_subvolid "$SCRATCH_MNT" "snapshot") + +echo "Subvolumeid: ${new_subvolid}" >> $seqres.full +tests/btrfs/303 line 74
Quoted 10 lines
tests/btrfs/303 new file +10
-0+if [ -z "$new_subvolid" ]; then + _fail "Unable to get the subvolid of the first snapshot" +fi + +# Make sure the subvolumeid for the first snapshot didn't change. +if [ "$new_subvolid" -ne "$init_subvolid" ]; then + _fail "Subvolumeid for the first snapshot changed, has ${new_subvolid} expect ${init_subvolid}" +fi + +_scratch_unmountThis explicit unmount is not needed, the fstests framework automatically does that.
Otherwise it looks fine, thanks.
Quoted 18 lines
tests/btrfs/303 new file +6
-0+ +echo "Silence is golden" + +# success, all done +status=0 +exittests/btrfs/303.out new file +2
-1@@ -0,0 +1,2 @@ +QA output created by 303 +Silence is golden --2.42.0
Quoted 114 lines · Qu Wenruo
Quoted 104 lines · Filipe Manana
Quoted 46 lines · Qu Wenruo
[BUG]
There is a sysbot regression report about transaction abort during snapshot creation, which is caused by the new timing of qgroup creation and too strict error check.[FIX]
The proper fix is already submitted, with the title "btrfs: do not abort transaction if there is already an existing qgroup".[TEST]
The new test case would reproduce the regression by:- Create a subvolume and a snapshot of it
- Record the subvolumeid of the snapshot
- Re-create the fs
Since btrfs won't reuse the subvolume id, we have to re-create the fs.- Enable quota and create a qgroup with the same subvolumeid
- Create a subvolume and a snapshot of it
For unpatched and affected kernel (thankfully no release is affected), the snapshot creation would fail due to aborted transaction.- Make sure the subvolume id doesn't change for the snapshot There is one very hacky attempt to fix it by avoiding using the subvolume id, which is completely wrong and would be caught by this extra check.
Signed-off-by Qu Wenruo <[email protected]>--- tests/btrfs/303 | 80 +++++++++++++++++++++++++++++++++++++++++++++ tests/btrfs/303.out | 2 ++ 2 files changed, 82 insertions(+)create mode 100755 tests/btrfs/303
create mode 100644 tests/btrfs/303.outtests/btrfs/303 new file +1
-0@@ -0,0 +1,80 @@ +#! /bin/bashtests/btrfs/303 line 11
Quoted 10 lines
tests/btrfs/303 new file +10
-0+# SPDX-License-Identifier: GPL-2.0 +# Copyright (C) 2023 SUSE Linux Products GmbH. All Rights Reserved. +# +# FS QA Test 303 +# +# A regression test to make sure snapshot creation won't cause transaction +# abort if there is already an existing qgroup. +# +. ./common/preamble +_begin_fstest auto quick qgroupAlso 'snapshot' and 'subvol' groups.
Quoted 30 lines
tests/btrfs/303 new file +30
-0+ +. ./common/filter + +_supported_fs btrfs +_require_scratch + +_fixed_by_kernel_commit xxxxxxxxxxxx \ + "btrfs: do not abort transaction if there is already an existing qgroup" + +_scratch_mkfs >> $seqres.full 2>&1 || _fail "mkfs failed" +_scratch_mount + +# Create the first subvolume and get its id. +# This subvolume id should not change no matter if there is an existing +# qgroup for it. +$BTRFS_UTIL_PROG subvolume create "$SCRATCH_MNT/subvol" >> $seqres.full +$BTRFS_UTIL_PROG subvolume snapshot "$SCRATCH_MNT/subvol" \ + "$SCRATCH_MNT/snapshot">> $seqres.full + +init_subvolid=$(_btrfs_get_subvolid "$SCRATCH_MNT" "snapshot") + +if [ -z "$init_subvolid" ]; then + _fail "Unable to get the subvolid of the first snapshot" +fi + +echo "Subvolumeid: ${init_subvolid}" >> $seqres.full + +_scratch_unmount + +# Re-create the fs, as btrfs won't reuse the subvolume id.tests/btrfs/303 line 51
Quoted 10 lines
tests/btrfs/303 new file +10
-0+_scratch_mkfs >> $seqres.full 2>&1 || _fail "2nd mkfs failed" +_scratch_mount + +$BTRFS_UTIL_PROG quota enable "$SCRATCH_MNT" >> $seqres.full +$BTRFS_UTIL_PROG quota rescan -w "$SCRATCH_MNT" >> $seqres.full + +# Create a qgroup for the first subvolume, this would make the later +# subvolume creation to find an existing qgroup, and abort transaction. +$BTRFS_UTIL_PROG qgroup create 0/"$init_subvolid" "$SCRATCH_MNT" >> $seqres.full +syncThis sync is not needed. An unpatched kernel still fails, and a patched kernel passes this test without the sync.
Quoted 2 lines
Also, please always comment on why a sync is needed. In this case it can be removed because it's redundant.
Would address all comments, but here I'm a little curious about the "sync" comment principle.
Quoted 6 lines
I totally understand for this particular case the "sync" is unnecessary, the qgroup item doesn't need to be committed, thus it's totally reasonable to remove this sync.
But I'm wondering is there any other reasons why we should avoid unnecessary "sync"s?
It's a principle of clarity and simplicity.
Adding a comment like "Make the qgroup item persisted in the qgroup btree, commit the current transaction",
makes it clear why the sync is needed, what it accomplishes.
It makes the test easier to read for other people, and easier to maintain.
In this case it's a simplicity principle also because if it's not needed, it's better to not be there, otherwise it just makes it confusing and longer than necessary.
Quoted 2 lines
Like slowing down the test or just to improve our awareness and avoid sync-happy guys?
My comment is not about performance.
I run tests in a VM dedicated to that, so adding a sync will at most increase run time by a few milliseconds - that's insignificant and therefore I don't care.
All I care about is that the test is clear and has no unnecessary steps - be it a sync or something else.
Thanks.
Quoted 52 lines
Thanks,
QuQuoted 49 lines
Quoted 14 lines
tests/btrfs/303 new file +13
-0+ +# Now create the first snapshot, which should have the same subvolid no matter +# if the quota is enabled. +$BTRFS_UTIL_PROG subvolume create "$SCRATCH_MNT/subvol" >> $seqres.full +$BTRFS_UTIL_PROG subvolume snapshot "$SCRATCH_MNT/subvol" \ + "$SCRATCH_MNT/snapshot">> $seqres.full + +# Either the snapshot create failed and transaction is aborted thus no +# snapshot here, or we should be able to create the snapshot. +new_subvolid=$(_btrfs_get_subvolid "$SCRATCH_MNT" "snapshot") + +echo "Subvolumeid: ${new_subvolid}" >> $seqres.full +tests/btrfs/303 line 74
Quoted 10 lines
tests/btrfs/303 new file +10
-0+if [ -z "$new_subvolid" ]; then + _fail "Unable to get the subvolid of the first snapshot" +fi + +# Make sure the subvolumeid for the first snapshot didn't change. +if [ "$new_subvolid" -ne "$init_subvolid" ]; then + _fail "Subvolumeid for the first snapshot changed, has ${new_subvolid} expect ${init_subvolid}" +fi + +_scratch_unmountThis explicit unmount is not needed, the fstests framework automatically does that.
Otherwise it looks fine, thanks.
Quoted 18 lines
tests/btrfs/303 new file +6
-0+ +echo "Silence is golden" + +# success, all done +status=0 +exittests/btrfs/303.out new file +2
-1@@ -0,0 +1,2 @@ +QA output created by 303 +Silence is golden --2.42.0