btrfs-progs: bug fixes exposed during delayed chunk items insertion
cover btrfs-progs: bug fixes exposed during delayed chunk items insertion
I'm recently working on a prototype on seed device sprout at mkfs time, and one core feature for it is delayed chunk item insertion.
Currently the delayed chunk item insertion is done, and mkfs sprout prototype is pretty close to finish.
But during the development, I exposed some bugs in error handling path, especially the 2nd one is exposed during testing, and it's there for a while, affecting existing code base.
Currently my delayed chunk item insertion can survive all mkfs and fsck tests, so I don't expect more bugs exposed, thus it's time for the pure bug fixes first.
Qu Wenruo (2):
btrfs-progs: fix a memory leak when starting a transaction on fs with
error
btrfs-progs: fix an error path which can lead to empty device list
kernel-shared/transaction.c | 6 ++++-- kernel-shared/volumes.c | 15 +++++++++++++++ 2 files changed, 19 insertions(+), 2 deletions(-)
Signature · 1 line
2.35.1
PATCH 2/2 btrfs-progs: fix an error path which can lead to empty device list
[BUG]
With the incoming delayed chunk item insertion feature, there is a super weird failure at mkfs/022:
====== RUN CHECK ./mkfs.btrfs -f --rootdir tmp.KnKpP5 -d dup -b 350M tests/test.img
...
Checksum: crc32c
Number of devices: 0
Devices:
ID SIZE PATHNote the "Number of devices: 0" line, this means our fs_info->fs_devices->devices list is empty.
And since our rw device list is empty, we won't finish the mkfs with proper superblock magic, and cause later btrfs check to fail.
[CAUSE]
Although the failure is only triggered by the incoming delayed chunk item insertion feature, the bug itself is here for a while.
In btrfs_alloc_chunk(), we move rw devices to our @private_devs list first, then in create_chunk(), we move it back to our rw devices list.
This dance is pretty dangerous, epsecially if btrfs_alloc_dev_extent() failed inside create_chunk(), and current profile have multiple stripes (including DUP), we will exit create_chunk() directly, without moving the remaining devices in @private_devs list back to @dev_list.
Furthermore, btrfs_alloc_chunk() is expected to return -ENOSPC, as we call btrfs_alloc_chunk() to pre-allocate chunks, and ignore the -ENOSPC error if it's just a pre-allocation failure.
This existing error path can lead to the empty rw list seen above.
[FIX]
After create_chunk(), unconditionally move all devices in @private_devs back to rw device list.
And add extra check to make sure our rw device list is never empty after a chunk allocation attempt.
--- kernel-shared/volumes.c | 15 +++++++++++++++ 1 file changed, 15 insertions(+)
kernel-shared/volumes.c +15 -0
@@ -1559,6 +1559,21 @@ again: } ret =create_chunk (trans, info, &ctl, &private_devs); + +/* +* This can happen if above create_chunk() failed, we need to move all +* devices back to dev_list. +*/ +while (!list_empty (&private_devs)) { + device = list_entry(private_devs.next, struct btrfs_device, + dev_list); +list_move (&device->dev_list, dev_list); + } +/* +* All private devs moved back to @dev_list, now dev_list should not be +* empty. +*/ +ASSERT (!list_empty (dev_list)); *start = ctl.start; *num_bytes = ctl.num_bytes;
Signature · 1 line
2.35.1
PATCH 1/2 btrfs-progs: fix a memory leak when starting a transaction on fs with error
In reply to Qu WenruoFunction btrfs_start_transaction() will allocate the memory unconditionally, but if the fs has an aborted transaction we don't free the allocated memory but return error directly.
Fix it by only allocate the new memory after the transaction_aborted check.
--- kernel-shared/transaction.c | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-)
kernel-shared/transaction.c +4 -2
@@ -25,13 +25,15 @@ struct btrfs_trans_handle* btrfs_start_transaction(struct btrfs_root *root, int num_blocks) {struct btrfs_fs_info *fs_info = root->fs_info; -struct btrfs_trans_handle *h =kzalloc (sizeof (*h),GFP_NOFS ); - +struct btrfs_trans_handle *h; +if (fs_info->transaction_aborted)return ERR_PTR (-EROFS ); + h =kzalloc (sizeof (*h),GFP_NOFS );if (!h)return ERR_PTR (-ENOMEM ); +if (root->commit_root) {error ("commit_root already set when starting transaction" );kfree (h);
Signature · 1 line
2.35.1
Re: [PATCH 1/2] btrfs-progs: fix a memory leak when starting a transaction on fs with error
Quoted 8 lines · Qu Wenruo
Function
btrfs_start_transaction()will allocate the memory unconditionally, but if the fs has an aborted transaction we don't free the allocated memory but return error directly.Fix it by only allocate the new memory after the
transaction_abortedcheck.Signed-off-by Qu Wenruo <[email protected]>
Thanks,
Josef
Re: [PATCH 2/2] btrfs-progs: fix an error path which can lead to empty device list
In reply to Qu WenruoQuoted 35 lines · Qu Wenruo
[BUG]
With the incoming delayed chunk item insertion feature, there is a super weird failure at mkfs/022:====== RUN CHECK ./mkfs.btrfs -f --rootdir tmp.KnKpP5 -d dup -b 350M tests/test.img ... Checksum: crc32c Number of devices: 0 Devices: ID SIZE PATHNote the "Number of devices: 0" line, this means our
fs_info->fs_devices->deviceslist is empty.And since our rw device list is empty, we won't finish the mkfs with proper superblock magic, and cause later btrfs check to fail.
[CAUSE]
Although the failure is only triggered by the incoming delayed chunk item insertion feature, the bug itself is here for a while.In
btrfs_alloc_chunk(), we move rw devices to our @private_devs list first, then increate_chunk(), we move it back to our rw devices list.This dance is pretty dangerous, epsecially if
btrfs_alloc_dev_extent()failed insidecreate_chunk(), and current profile have multiple stripes (including DUP), we will exitcreate_chunk()directly, without moving the remaining devices in @private_devs list back to @dev_list.Furthermore,
btrfs_alloc_chunk()is expected to return-ENOSPC, as we callbtrfs_alloc_chunk()to pre-allocate chunks, and ignore the-ENOSPCerror if it's just a pre-allocation failure.This existing error path can lead to the empty rw list seen above.
Quoted 8 lines
[FIX]
Aftercreate_chunk(), unconditionally move all devices in @private_devs back to rw device list.And add extra check to make sure our rw device list is never empty after a chunk allocation attempt.
Signed-off-by Qu Wenruo <[email protected]>
Thanks,
Josef
Re: [PATCH 1/2] btrfs-progs: fix a memory leak when starting a transaction on fs with error
In reply to Qu WenruoQuoted 24 lines · Qu Wenruo
Function
btrfs_start_transaction()will allocate the memory unconditionally, but if the fs has an aborted transaction we don't free the allocated memory but return error directly.Fix it by only allocate the new memory after the
transaction_abortedcheck.Signed-off-by Qu Wenruo <[email protected]>--- kernel-shared/transaction.c | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-)kernel-shared/transaction.c +2
-2@@ -25,13 +25,15 @@ struct btrfs_trans_handle* btrfs_start_transaction(struct btrfs_root *root, int num_blocks) {struct btrfs_fs_info *fs_info = root->fs_info; -struct btrfs_trans_handle *h =kzalloc (sizeof (*h),GFP_NOFS ); - +struct btrfs_trans_handle *h; +
kernel-shared/transaction.c struct btrfs_trans_handle* btrfs_start_transaction(struct btrfs_root *root, · line 39
Quoted 10 lines
kernel-shared/transaction.c +2
-0if (fs_info->transaction_aborted)return ERR_PTR (-EROFS ); + h =kzalloc (sizeof (*h),GFP_NOFS );if (!h)return ERR_PTR (-ENOMEM ); +if (root->commit_root) {error ("commit_root already set when starting transaction" );kfree (h);
If you are moving allocation of h anyway, why not move it beyond all checks and delete redundant kfree(h)?
Re: [PATCH 1/2] btrfs-progs: fix a memory leak when starting a transaction on fs with error
Quoted 28 lines · Andrei Borzenkov
Quoted 27 lines · Qu Wenruo
Function
btrfs_start_transaction()will allocate the memory unconditionally, but if the fs has an aborted transaction we don't free the allocated memory but return error directly.Fix it by only allocate the new memory after the
transaction_abortedcheck.Signed-off-by Qu Wenruo <[email protected]>--- kernel-shared/transaction.c | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-)kernel-shared/transaction.c +2
-2@@ -25,13 +25,15 @@ struct btrfs_trans_handle* btrfs_start_transaction(struct btrfs_root *root, int num_blocks) {struct btrfs_fs_info *fs_info = root->fs_info; -struct btrfs_trans_handle *h =kzalloc (sizeof (*h),GFP_NOFS ); - +struct btrfs_trans_handle *h; +if (fs_info->transaction_aborted)return ERR_PTR (-EROFS );
Quoted 10 lines
kernel-shared/transaction.c struct btrfs_trans_handle* btrfs_start_transaction(struct btrfs_root *root, · line 39
Quoted 7 lines
kernel-shared/transaction.c +2
-0+ h =kzalloc (sizeof (*h),GFP_NOFS );if (!h)return ERR_PTR (-ENOMEM ); +if (root->commit_root) {error ("commit_root already set when starting transaction" );kfree (h);If you are moving allocation of h anyway, why not move it beyond all checks and delete redundant kfree(h)?
Good idea.
Thanks,
Qu