btrfs: updates to error path for delayed dir index insertion failure
cover btrfs: updates to error path for delayed dir index insertion failure
From: Filipe Manana <[email protected]>
Some updates to the error path for delayed dir index insertion failure, motivated by a recent syzbot report:
https://lore.kernel.org/linux-btrfs/[email protected]/
Details in the changelogs.
Filipe Manana (2):
btrfs: improve error message after failure to add delayed dir index item
btrfs: remove BUG() after failure to insert delayed dir index item
fs/btrfs/delayed-inode.c | 71 ++++++++++++++++++++++++++-------------- 1 file changed, 46 insertions(+), 25 deletions(-)
Signature · 1 line
2.40.1
PATCH 1/2 btrfs: improve error message after failure to add delayed dir index item
From: Filipe Manana <[email protected]>
If we fail to add a delayed dir index item because there's already another item with the same index number, we print an error message (and then BUG). However that message isn't very helpful to debug anything because we don't know what's the index number and what are the values of index counters in the inode and its delayed inode (index_cnt fields of struct btrfs_inode and struct btrfs_delayed_node).
So update the error message to include the index number and counters.
We actually had a recent case where this issue was hit by a syzbot report (see the link below).
--- fs/btrfs/delayed-inode.c | 7 ++++--- 1 file changed, 4 insertions(+), 3 deletions(-)
fs/btrfs/delayed-inode.c +4 -3
@@ -1498,9 +1498,10 @@ int btrfs_insert_delayed_dir_index(struct btrfs_trans_handle *trans, ret =__btrfs_add_delayed_item (delayed_node, delayed_item);if (unlikely (ret)) {btrfs_err (trans->fs_info, -"err add delayed dir index item(name: %.*s) into the insertion tree of the delayed node(root id: %llu, inode id: %llu, errno: %d)" , - name_len, name, delayed_node->root->root_key.objectid, - delayed_node->inode_id, ret); +"error adding delayed dir index item, name: %.*s, index: %llu, root: %llu, dir: %llu, dir->index_cnt: %llu, delayed_node->index_cnt: %llu, error: %d" , + name_len, name, index,btrfs_root_id (delayed_node->root), + delayed_node->inode_id, dir->index_cnt, + delayed_node->index_cnt, ret);BUG (); }mutex_unlock (&delayed_node->mutex);
Signature · 1 line
2.40.1
PATCH 2/2 btrfs: remove BUG() after failure to insert delayed dir index item
In reply to [email protected]From: Filipe Manana <[email protected]>
Instead of calling BUG() when we fail to insert a delayed dir index item into the delayed node's tree, we can just release all the resources we have allocated/acquired before and return the error to the caller. This is fine because all existing call chains undo anything they have done before calling btrfs_insert_delayed_dir_index() or BUG_ON (when creating pending snapshots in the transaction commit path).
So remove the BUG() call and do proper error handling.
This relates to a syzbot report linked below, but does not fix it because it only prevents hitting a BUG(), it does not fix the issue where somehow we attempt to use twice the same index number for different index items.
--- fs/btrfs/delayed-inode.c | 72 +++++++++++++++++++++++++--------------- 1 file changed, 46 insertions(+), 26 deletions(-)
fs/btrfs/delayed-inode.c +46 -26
@@ -1413,7 +1413,29 @@ void btrfs_balance_delayed_items(struct btrfs_fs_info *fs_info)btrfs_wq_run_delayed_node (delayed_root, fs_info,BTRFS_DELAYED_BATCH ); } -/* Will return 0 or -ENOMEM */ +static void btrfs_release_dir_index_item_space (struct btrfs_trans_handle *trans) +{ +struct btrfs_fs_info *fs_info = trans->fs_info; +const u64 bytes =btrfs_calc_insert_metadata_size (fs_info,1 ); + +if (test_bit (BTRFS_FS_LOG_RECOVERING , &fs_info->flags)) +return ; + +/* +* Adding the new dir index item does not require touching another +* leaf, so we can release 1 unit of metadata that was previously +* reserved when starting the transaction. This applies only to +* the case where we had a transaction start and excludes the +* transaction join case (when replaying log trees). +*/ +trace_btrfs_space_reservation (fs_info,"transaction" , + trans->transid, bytes,0 ); +btrfs_block_rsv_release (fs_info, trans->block_rsv, bytes,NULL ); +ASSERT (trans->bytes_reserved >= bytes); + trans->bytes_reserved -= bytes; +} + +/* Will return 0, -ENOMEM or -EEXIST (index number collision, unexpected). */ int btrfs_insert_delayed_dir_index(struct btrfs_trans_handle *trans, const char *name, int name_len, struct btrfs_inode *dir,
@@ -1455,6 +1477,27 @@ int btrfs_insert_delayed_dir_index(struct btrfs_trans_handle *trans,mutex_lock (&delayed_node->mutex); +/* +* First attempt to insert the delayed item. This is to make the error +* handling path simpler in case we fail (-EEXIST). There's no risk of +* any other task coming in and running the delayed item before we do +* the metadata space reservation below, because we are holding the +* delayed node's mutex and that mutex must also be locked before the +* node's delayed items can be run. +*/ + ret =__btrfs_add_delayed_item (delayed_node, delayed_item); +if (unlikely (ret)) { +btrfs_err (trans->fs_info, +"error adding delayed dir index item, name: %.*s, index: %llu, root: %llu, dir: %llu, dir->index_cnt: %llu, delayed_node->index_cnt: %llu, error: %d" , + name_len, name, index,btrfs_root_id (delayed_node->root), + delayed_node->inode_id, dir->index_cnt, + delayed_node->index_cnt, ret); +mutex_unlock (&delayed_node->mutex); +btrfs_release_delayed_item (delayed_item); +btrfs_release_dir_index_item_space (trans); + goto release_node; + } +if (delayed_node->index_item_leaves ==0 || delayed_node->curr_index_batch_size + data_len > leaf_data_size) { delayed_node->curr_index_batch_size = data_len;
@@ -1478,31 +1521,8 @@ int btrfs_insert_delayed_dir_index(struct btrfs_trans_handle *trans, } delayed_node->index_item_leaves++; - }else if (!test_bit (BTRFS_FS_LOG_RECOVERING , &fs_info->flags)) { -const u64 bytes =btrfs_calc_insert_metadata_size (fs_info,1 ); - -/* -* Adding the new dir index item does not require touching another -* leaf, so we can release 1 unit of metadata that was previously -* reserved when starting the transaction. This applies only to -* the case where we had a transaction start and excludes the -* transaction join case (when replaying log trees). -*/ -trace_btrfs_space_reservation (fs_info,"transaction" , - trans->transid, bytes,0 ); -btrfs_block_rsv_release (fs_info, trans->block_rsv, bytes,NULL ); -ASSERT (trans->bytes_reserved >= bytes); - trans->bytes_reserved -= bytes; - } - - ret =__btrfs_add_delayed_item (delayed_node, delayed_item); -if (unlikely (ret)) { -btrfs_err (trans->fs_info, -"error adding delayed dir index item, name: %.*s, index: %llu, root: %llu, dir: %llu, dir->index_cnt: %llu, delayed_node->index_cnt: %llu, error: %d" , - name_len, name, index,btrfs_root_id (delayed_node->root), - delayed_node->inode_id, dir->index_cnt, - delayed_node->index_cnt, ret); -BUG (); + }else { +btrfs_release_dir_index_item_space (trans); }mutex_unlock (&delayed_node->mutex);
Signature · 1 line
2.40.1
v2 cover btrfs: updates to error path for delayed dir index insertion failure
In reply to [email protected]From: Filipe Manana <[email protected]>
Some updates to the error path for delayed dir index insertion failure, motivated by a recent syzbot report:
https://lore.kernel.org/linux-btrfs/[email protected]/
Details in the changelogs.
v2: Fixed error path in patch 2 to release delayed item before unlocking
the delayed node. Added patch 3 to prevent such mistakes in the future.
Filipe Manana (3):
btrfs: improve error message after failure to add delayed dir index item
btrfs: remove BUG() after failure to insert delayed dir index item
btrfs: assert delayed node locked when removing delayed item
fs/btrfs/delayed-inode.c | 85 ++++++++++++++++++++++++++-------------- 1 file changed, 55 insertions(+), 30 deletions(-)
Signature · 1 line
2.40.1
PATCH v2 1/3 btrfs: improve error message after failure to add delayed dir index item
From: Filipe Manana <[email protected]>
If we fail to add a delayed dir index item because there's already another item with the same index number, we print an error message (and then BUG). However that message isn't very helpful to debug anything because we don't know what's the index number and what are the values of index counters in the inode and its delayed inode (index_cnt fields of struct btrfs_inode and struct btrfs_delayed_node).
So update the error message to include the index number and counters.
We actually had a recent case where this issue was hit by a syzbot report (see the link below).
--- fs/btrfs/delayed-inode.c | 7 ++++--- 1 file changed, 4 insertions(+), 3 deletions(-)
fs/btrfs/delayed-inode.c +4 -3
@@ -1498,9 +1498,10 @@ int btrfs_insert_delayed_dir_index(struct btrfs_trans_handle *trans, ret =__btrfs_add_delayed_item (delayed_node, delayed_item);if (unlikely (ret)) {btrfs_err (trans->fs_info, -"err add delayed dir index item(name: %.*s) into the insertion tree of the delayed node(root id: %llu, inode id: %llu, errno: %d)" , - name_len, name, delayed_node->root->root_key.objectid, - delayed_node->inode_id, ret); +"error adding delayed dir index item, name: %.*s, index: %llu, root: %llu, dir: %llu, dir->index_cnt: %llu, delayed_node->index_cnt: %llu, error: %d" , + name_len, name, index,btrfs_root_id (delayed_node->root), + delayed_node->inode_id, dir->index_cnt, + delayed_node->index_cnt, ret);BUG (); }mutex_unlock (&delayed_node->mutex);
Signature · 1 line
2.40.1
Re: [PATCH v2 1/3] btrfs: improve error message after failure to add delayed dir index item
Quoted 9 lines · On 2023/8/28 16:06, [email protected] wrote:
From: Filipe Manana <[email protected]>
If we fail to add a delayed dir index item because there's already another item with the same index number, we print an error message (and then BUG). However that message isn't very helpful to debug anything because we don't know what's the index number and what are the values of index counters in the inode and its delayed inode (
index_cntfields of structbtrfs_inodeand structbtrfs_delayed_node).
Quoted 7 lines
So update the error message to include the index number and counters.
We actually had a recent case where this issue was hit by a syzbot report (see the link below).
Thanks,
Qu
Quoted 22 lines
--- fs/btrfs/delayed-inode.c | 7 ++++--- 1 file changed, 4 insertions(+), 3 deletions(-)fs/btrfs/delayed-inode.c +4
-3@@ -1498,9 +1498,10 @@ int btrfs_insert_delayed_dir_index(struct btrfs_trans_handle *trans, ret =__btrfs_add_delayed_item (delayed_node, delayed_item);if (unlikely (ret)) {btrfs_err (trans->fs_info, -"err add delayed dir index item(name: %.*s) into the insertion tree of the delayed node(root id: %llu, inode id: %llu, errno: %d)" , - name_len, name, delayed_node->root->root_key.objectid, - delayed_node->inode_id, ret); +"error adding delayed dir index item, name: %.*s, index: %llu, root: %llu, dir: %llu, dir->index_cnt: %llu, delayed_node->index_cnt: %llu, error: %d" , + name_len, name, index,btrfs_root_id (delayed_node->root), + delayed_node->inode_id, dir->index_cnt, + delayed_node->index_cnt, ret);BUG (); }mutex_unlock (&delayed_node->mutex);
PATCH v2 2/3 btrfs: remove BUG() after failure to insert delayed dir index item
In reply to [email protected]From: Filipe Manana <[email protected]>
Instead of calling BUG() when we fail to insert a delayed dir index item into the delayed node's tree, we can just release all the resources we have allocated/acquired before and return the error to the caller. This is fine because all existing call chains undo anything they have done before calling btrfs_insert_delayed_dir_index() or BUG_ON (when creating pending snapshots in the transaction commit path).
So remove the BUG() call and do proper error handling.
This relates to a syzbot report linked below, but does not fix it because it only prevents hitting a BUG(), it does not fix the issue where somehow we attempt to use twice the same index number for different index items.
--- fs/btrfs/delayed-inode.c | 74 +++++++++++++++++++++++++--------------- 1 file changed, 47 insertions(+), 27 deletions(-)
fs/btrfs/delayed-inode.c +47 -27
@@ -1413,7 +1413,29 @@ void btrfs_balance_delayed_items(struct btrfs_fs_info *fs_info)btrfs_wq_run_delayed_node (delayed_root, fs_info,BTRFS_DELAYED_BATCH ); } -/* Will return 0 or -ENOMEM */ +static void btrfs_release_dir_index_item_space (struct btrfs_trans_handle *trans) +{ +struct btrfs_fs_info *fs_info = trans->fs_info; +const u64 bytes =btrfs_calc_insert_metadata_size (fs_info,1 ); + +if (test_bit (BTRFS_FS_LOG_RECOVERING , &fs_info->flags)) +return ; + +/* +* Adding the new dir index item does not require touching another +* leaf, so we can release 1 unit of metadata that was previously +* reserved when starting the transaction. This applies only to +* the case where we had a transaction start and excludes the +* transaction join case (when replaying log trees). +*/ +trace_btrfs_space_reservation (fs_info,"transaction" , + trans->transid, bytes,0 ); +btrfs_block_rsv_release (fs_info, trans->block_rsv, bytes,NULL ); +ASSERT (trans->bytes_reserved >= bytes); + trans->bytes_reserved -= bytes; +} + +/* Will return 0, -ENOMEM or -EEXIST (index number collision, unexpected). */ int btrfs_insert_delayed_dir_index(struct btrfs_trans_handle *trans, const char *name, int name_len, struct btrfs_inode *dir,
@@ -1455,6 +1477,27 @@ int btrfs_insert_delayed_dir_index(struct btrfs_trans_handle *trans,mutex_lock (&delayed_node->mutex); +/* +* First attempt to insert the delayed item. This is to make the error +* handling path simpler in case we fail (-EEXIST). There's no risk of +* any other task coming in and running the delayed item before we do +* the metadata space reservation below, because we are holding the +* delayed node's mutex and that mutex must also be locked before the +* node's delayed items can be run. +*/ + ret =__btrfs_add_delayed_item (delayed_node, delayed_item); +if (unlikely (ret)) { +btrfs_err (trans->fs_info, +"error adding delayed dir index item, name: %.*s, index: %llu, root: %llu, dir: %llu, dir->index_cnt: %llu, delayed_node->index_cnt: %llu, error: %d" , + name_len, name, index,btrfs_root_id (delayed_node->root), + delayed_node->inode_id, dir->index_cnt, + delayed_node->index_cnt, ret); +btrfs_release_delayed_item (delayed_item); +btrfs_release_dir_index_item_space (trans); +mutex_unlock (&delayed_node->mutex); + goto release_node; + } +if (delayed_node->index_item_leaves ==0 || delayed_node->curr_index_batch_size + data_len > leaf_data_size) { delayed_node->curr_index_batch_size = data_len;
@@ -1472,37 +1515,14 @@ int btrfs_insert_delayed_dir_index(struct btrfs_trans_handle *trans,* impossible. */ if (WARN_ON (ret)) { -mutex_unlock (&delayed_node->mutex);btrfs_release_delayed_item (delayed_item); +mutex_unlock (&delayed_node->mutex); goto release_node; } delayed_node->index_item_leaves++; - }else if (!test_bit (BTRFS_FS_LOG_RECOVERING , &fs_info->flags)) { -const u64 bytes =btrfs_calc_insert_metadata_size (fs_info,1 ); - -/* -* Adding the new dir index item does not require touching another -* leaf, so we can release 1 unit of metadata that was previously -* reserved when starting the transaction. This applies only to -* the case where we had a transaction start and excludes the -* transaction join case (when replaying log trees). -*/ -trace_btrfs_space_reservation (fs_info,"transaction" , - trans->transid, bytes,0 ); -btrfs_block_rsv_release (fs_info, trans->block_rsv, bytes,NULL ); -ASSERT (trans->bytes_reserved >= bytes); - trans->bytes_reserved -= bytes; - } - - ret =__btrfs_add_delayed_item (delayed_node, delayed_item); -if (unlikely (ret)) { -btrfs_err (trans->fs_info, -"error adding delayed dir index item, name: %.*s, index: %llu, root: %llu, dir: %llu, dir->index_cnt: %llu, delayed_node->index_cnt: %llu, error: %d" , - name_len, name, index,btrfs_root_id (delayed_node->root), - delayed_node->inode_id, dir->index_cnt, - delayed_node->index_cnt, ret); -BUG (); + }else { +btrfs_release_dir_index_item_space (trans); }mutex_unlock (&delayed_node->mutex);
Signature · 1 line
2.40.1
Re: [PATCH v2 2/3] btrfs: remove BUG() after failure to insert delayed dir index item
Quoted 37 lines · On 2023/8/28 16:06, [email protected] wrote:
From: Filipe Manana <[email protected]>
Instead of calling
BUG()when we fail to insert a delayed dir index item into the delayed node's tree, we can just release all the resources we have allocated/acquired before and return the error to the caller. This is fine because all existing call chains undo anything they have done before callingbtrfs_insert_delayed_dir_index()orBUG_ON(when creating pending snapshots in the transaction commit path).So remove the
BUG()call and do proper error handling.This relates to a syzbot report linked below, but does not fix it because it only prevents hitting a
BUG(), it does not fix the issue where somehow we attempt to use twice the same index number for different index items.Signed-off-by Filipe Manana <[email protected]>--- fs/btrfs/delayed-inode.c | 74 +++++++++++++++++++++++++--------------- 1 file changed, 47 insertions(+), 27 deletions(-)fs/btrfs/delayed-inode.c +7
-1@@ -1413,7 +1413,29 @@ void btrfs_balance_delayed_items(struct btrfs_fs_info *fs_info)btrfs_wq_run_delayed_node (delayed_root, fs_info,BTRFS_DELAYED_BATCH ); } -/* Will return 0 or -ENOMEM */ +static void btrfs_release_dir_index_item_space (struct btrfs_trans_handle *trans) +{ +struct btrfs_fs_info *fs_info = trans->fs_info; +const u64 bytes =btrfs_calc_insert_metadata_size (fs_info,1 ); + +if (test_bit (BTRFS_FS_LOG_RECOVERING , &fs_info->flags)) +return ;
fs/btrfs/delayed-inode.c void btrfs_balance_delayed_items(struct btrfs_fs_info *fs_info) · line 1432
Quoted 10 lines
fs/btrfs/delayed-inode.c +10
-0+ +/* +* Adding the new dir index item does not require touching another +* leaf, so we can release 1 unit of metadata that was previously +* reserved when starting the transaction. This applies only to +* the case where we had a transaction start and excludes the +* transaction join case (when replaying log trees). +*/ +trace_btrfs_space_reservation (fs_info,"transaction" , + trans->transid, bytes,0 );
I know this is from the old code, but shouldn't we use "-bytes" instead?
Otherwise looks fine to me.
Thanks,
Qu
Quoted 77 lines
fs/btrfs/delayed-inode.c +29
-26+btrfs_block_rsv_release (fs_info, trans->block_rsv, bytes,NULL ); +ASSERT (trans->bytes_reserved >= bytes); + trans->bytes_reserved -= bytes; +} + +/* Will return 0, -ENOMEM or -EEXIST (index number collision, unexpected). */ int btrfs_insert_delayed_dir_index(struct btrfs_trans_handle *trans, const char *name, int name_len, struct btrfs_inode *dir,@@ -1455,6 +1477,27 @@ int btrfs_insert_delayed_dir_index(struct btrfs_trans_handle *trans,mutex_lock (&delayed_node->mutex); +/* +* First attempt to insert the delayed item. This is to make the error +* handling path simpler in case we fail (-EEXIST). There's no risk of +* any other task coming in and running the delayed item before we do +* the metadata space reservation below, because we are holding the +* delayed node's mutex and that mutex must also be locked before the +* node's delayed items can be run. +*/ + ret =__btrfs_add_delayed_item (delayed_node, delayed_item); +if (unlikely (ret)) { +btrfs_err (trans->fs_info, +"error adding delayed dir index item, name: %.*s, index: %llu, root: %llu, dir: %llu, dir->index_cnt: %llu, delayed_node->index_cnt: %llu, error: %d" , + name_len, name, index,btrfs_root_id (delayed_node->root), + delayed_node->inode_id, dir->index_cnt, + delayed_node->index_cnt, ret); +btrfs_release_delayed_item (delayed_item); + btrfs_release_dir_index_item_space(trans); > + mutex_unlock(&delayed_node->mutex); + goto release_node; + } +if (delayed_node->index_item_leaves ==0 || delayed_node->curr_index_batch_size + data_len > leaf_data_size) { delayed_node->curr_index_batch_size = data_len;@@ -1472,37 +1515,14 @@ int btrfs_insert_delayed_dir_index(struct btrfs_trans_handle *trans,* impossible. */ if (WARN_ON (ret)) { -mutex_unlock (&delayed_node->mutex);btrfs_release_delayed_item (delayed_item); +mutex_unlock (&delayed_node->mutex); goto release_node; } delayed_node->index_item_leaves++; - }else if (!test_bit (BTRFS_FS_LOG_RECOVERING , &fs_info->flags)) { -const u64 bytes =btrfs_calc_insert_metadata_size (fs_info,1 ); - -/* -* Adding the new dir index item does not require touching another -* leaf, so we can release 1 unit of metadata that was previously -* reserved when starting the transaction. This applies only to -* the case where we had a transaction start and excludes the -* transaction join case (when replaying log trees). -*/ -trace_btrfs_space_reservation (fs_info,"transaction" , - trans->transid, bytes,0 ); -btrfs_block_rsv_release (fs_info, trans->block_rsv, bytes,NULL ); -ASSERT (trans->bytes_reserved >= bytes); - trans->bytes_reserved -= bytes; - } - - ret =__btrfs_add_delayed_item (delayed_node, delayed_item); -if (unlikely (ret)) { -btrfs_err (trans->fs_info, -"error adding delayed dir index item, name: %.*s, index: %llu, root: %llu, dir: %llu, dir->index_cnt: %llu, delayed_node->index_cnt: %llu, error: %d" , - name_len, name, index,btrfs_root_id (delayed_node->root), - delayed_node->inode_id, dir->index_cnt, - delayed_node->index_cnt, ret); -BUG (); + }else { +btrfs_release_dir_index_item_space (trans); }mutex_unlock (&delayed_node->mutex);
Re: [PATCH v2 2/3] btrfs: remove BUG() after failure to insert delayed dir index item
Quoted 45 lines · Qu Wenruo
Quoted 41 lines · On 2023/8/28 16:06, [email protected] wrote:
From: Filipe Manana <[email protected]>
Instead of calling
BUG()when we fail to insert a delayed dir index item into the delayed node's tree, we can just release all the resources we have allocated/acquired before and return the error to the caller. This is fine because all existing call chains undo anything they have done before callingbtrfs_insert_delayed_dir_index()orBUG_ON(when creating pending snapshots in the transaction commit path).So remove the
BUG()call and do proper error handling.This relates to a syzbot report linked below, but does not fix it because it only prevents hitting a
BUG(), it does not fix the issue where somehow we attempt to use twice the same index number for different index items.Signed-off-by Filipe Manana <[email protected]>--- fs/btrfs/delayed-inode.c | 74 +++++++++++++++++++++++++--------------- 1 file changed, 47 insertions(+), 27 deletions(-)fs/btrfs/delayed-inode.c +11
-1@@ -1413,7 +1413,29 @@ void btrfs_balance_delayed_items(struct btrfs_fs_info *fs_info)btrfs_wq_run_delayed_node (delayed_root, fs_info,BTRFS_DELAYED_BATCH ); } -/* Will return 0 or -ENOMEM */ +static void btrfs_release_dir_index_item_space (struct btrfs_trans_handle *trans) +{ +struct btrfs_fs_info *fs_info = trans->fs_info; +const u64 bytes =btrfs_calc_insert_metadata_size (fs_info,1 ); + +if (test_bit (BTRFS_FS_LOG_RECOVERING , &fs_info->flags)) +return ; + +/* +* Adding the new dir index item does not require touching another +* leaf, so we can release 1 unit of metadata that was previously
Quoted 8 lines
fs/btrfs/delayed-inode.c void btrfs_balance_delayed_items(struct btrfs_fs_info *fs_info) · line 1432
Quoted 6 lines
fs/btrfs/delayed-inode.c +6
-0+* reserved when starting the transaction. This applies only to +* the case where we had a transaction start and excludes the +* transaction join case (when replaying log trees). +*/ +trace_btrfs_space_reservation (fs_info,"transaction" , + trans->transid, bytes,0 );I know this is from the old code, but shouldn't we use "-bytes" instead?
Nop.
The last argument, a value of 0, indicates that it's a space release. Allocations get a value of 1 for that last argument.
Quoted 83 lines
Otherwise looks fine to me.
Thanks,
QuQuoted 77 lines
fs/btrfs/delayed-inode.c +29
-26+btrfs_block_rsv_release (fs_info, trans->block_rsv, bytes,NULL ); +ASSERT (trans->bytes_reserved >= bytes); + trans->bytes_reserved -= bytes; +} + +/* Will return 0, -ENOMEM or -EEXIST (index number collision, unexpected). */ int btrfs_insert_delayed_dir_index(struct btrfs_trans_handle *trans, const char *name, int name_len, struct btrfs_inode *dir,@@ -1455,6 +1477,27 @@ int btrfs_insert_delayed_dir_index(struct btrfs_trans_handle *trans,mutex_lock (&delayed_node->mutex); +/* +* First attempt to insert the delayed item. This is to make the error +* handling path simpler in case we fail (-EEXIST). There's no risk of +* any other task coming in and running the delayed item before we do +* the metadata space reservation below, because we are holding the +* delayed node's mutex and that mutex must also be locked before the +* node's delayed items can be run. +*/ + ret =__btrfs_add_delayed_item (delayed_node, delayed_item); +if (unlikely (ret)) { +btrfs_err (trans->fs_info, +"error adding delayed dir index item, name: %.*s, index: %llu, root: %llu, dir: %llu, dir->index_cnt: %llu, delayed_node->index_cnt: %llu, error: %d" , + name_len, name, index,btrfs_root_id (delayed_node->root), + delayed_node->inode_id, dir->index_cnt, + delayed_node->index_cnt, ret); +btrfs_release_delayed_item (delayed_item); + btrfs_release_dir_index_item_space(trans); > + mutex_unlock(&delayed_node->mutex); + goto release_node; + } +if (delayed_node->index_item_leaves ==0 || delayed_node->curr_index_batch_size + data_len > leaf_data_size) { delayed_node->curr_index_batch_size = data_len;@@ -1472,37 +1515,14 @@ int btrfs_insert_delayed_dir_index(struct btrfs_trans_handle *trans,* impossible. */ if (WARN_ON (ret)) { -mutex_unlock (&delayed_node->mutex);btrfs_release_delayed_item (delayed_item); +mutex_unlock (&delayed_node->mutex); goto release_node; } delayed_node->index_item_leaves++; - }else if (!test_bit (BTRFS_FS_LOG_RECOVERING , &fs_info->flags)) { -const u64 bytes =btrfs_calc_insert_metadata_size (fs_info,1 ); - -/* -* Adding the new dir index item does not require touching another -* leaf, so we can release 1 unit of metadata that was previously -* reserved when starting the transaction. This applies only to -* the case where we had a transaction start and excludes the -* transaction join case (when replaying log trees). -*/ -trace_btrfs_space_reservation (fs_info,"transaction" , - trans->transid, bytes,0 ); -btrfs_block_rsv_release (fs_info, trans->block_rsv, bytes,NULL ); -ASSERT (trans->bytes_reserved >= bytes); - trans->bytes_reserved -= bytes; - } - - ret =__btrfs_add_delayed_item (delayed_node, delayed_item); -if (unlikely (ret)) { -btrfs_err (trans->fs_info, -"error adding delayed dir index item, name: %.*s, index: %llu, root: %llu, dir: %llu, dir->index_cnt: %llu, delayed_node->index_cnt: %llu, error: %d" , - name_len, name, index,btrfs_root_id (delayed_node->root), - delayed_node->inode_id, dir->index_cnt, - delayed_node->index_cnt, ret); -BUG (); + }else { +btrfs_release_dir_index_item_space (trans); }mutex_unlock (&delayed_node->mutex);
Qu Wenruo
Reviewed-byRe: [PATCH v2 2/3] btrfs: remove BUG() after failure to insert delayed dir index item
Quoted 55 lines · Filipe Manana
Quoted 45 lines · Qu Wenruo
Quoted 41 lines · On 2023/8/28 16:06, [email protected] wrote:
From: Filipe Manana <[email protected]>
Instead of calling
BUG()when we fail to insert a delayed dir index item into the delayed node's tree, we can just release all the resources we have allocated/acquired before and return the error to the caller. This is fine because all existing call chains undo anything they have done before callingbtrfs_insert_delayed_dir_index()orBUG_ON(when creating pending snapshots in the transaction commit path).So remove the
BUG()call and do proper error handling.This relates to a syzbot report linked below, but does not fix it because it only prevents hitting a
BUG(), it does not fix the issue where somehow we attempt to use twice the same index number for different index items.Signed-off-by Filipe Manana <[email protected]>--- fs/btrfs/delayed-inode.c | 74 +++++++++++++++++++++++++--------------- 1 file changed, 47 insertions(+), 27 deletions(-)fs/btrfs/delayed-inode.c +11
-1@@ -1413,7 +1413,29 @@ void btrfs_balance_delayed_items(struct btrfs_fs_info *fs_info)btrfs_wq_run_delayed_node (delayed_root, fs_info,BTRFS_DELAYED_BATCH ); } -/* Will return 0 or -ENOMEM */ +static void btrfs_release_dir_index_item_space (struct btrfs_trans_handle *trans) +{ +struct btrfs_fs_info *fs_info = trans->fs_info; +const u64 bytes =btrfs_calc_insert_metadata_size (fs_info,1 ); + +if (test_bit (BTRFS_FS_LOG_RECOVERING , &fs_info->flags)) +return ; + +/* +* Adding the new dir index item does not require touching another +* leaf, so we can release 1 unit of metadata that was previously Quoted 8 lines
fs/btrfs/delayed-inode.c void btrfs_balance_delayed_items(struct btrfs_fs_info *fs_info) · line 1432
Quoted 6 lines
fs/btrfs/delayed-inode.c +6
-0+* reserved when starting the transaction. This applies only to +* the case where we had a transaction start and excludes the +* transaction join case (when replaying log trees). +*/ +trace_btrfs_space_reservation (fs_info,"transaction" , + trans->transid, bytes,0 );I know this is from the old code, but shouldn't we use "-bytes" instead?
Quoted 3 lines
Nop.
The last argument, a value of 0, indicates that it's a space release. Allocations get a value of 1 for that last argument.
Oh, my bad.
Then it looks good to me.
Thanks,
Qu
Quoted 85 lines
Quoted 85 lines
Otherwise looks fine to me.
Thanks,
QuQuoted 77 lines
fs/btrfs/delayed-inode.c +29
-26+btrfs_block_rsv_release (fs_info, trans->block_rsv, bytes,NULL ); +ASSERT (trans->bytes_reserved >= bytes); + trans->bytes_reserved -= bytes; +} + +/* Will return 0, -ENOMEM or -EEXIST (index number collision, unexpected). */ int btrfs_insert_delayed_dir_index(struct btrfs_trans_handle *trans, const char *name, int name_len, struct btrfs_inode *dir,@@ -1455,6 +1477,27 @@ int btrfs_insert_delayed_dir_index(struct btrfs_trans_handle *trans,mutex_lock (&delayed_node->mutex); +/* +* First attempt to insert the delayed item. This is to make the error +* handling path simpler in case we fail (-EEXIST). There's no risk of +* any other task coming in and running the delayed item before we do +* the metadata space reservation below, because we are holding the +* delayed node's mutex and that mutex must also be locked before the +* node's delayed items can be run. +*/ + ret =__btrfs_add_delayed_item (delayed_node, delayed_item); +if (unlikely (ret)) { +btrfs_err (trans->fs_info, +"error adding delayed dir index item, name: %.*s, index: %llu, root: %llu, dir: %llu, dir->index_cnt: %llu, delayed_node->index_cnt: %llu, error: %d" , + name_len, name, index,btrfs_root_id (delayed_node->root), + delayed_node->inode_id, dir->index_cnt, + delayed_node->index_cnt, ret); +btrfs_release_delayed_item (delayed_item); + btrfs_release_dir_index_item_space(trans); > + mutex_unlock(&delayed_node->mutex); + goto release_node; + } +if (delayed_node->index_item_leaves ==0 || delayed_node->curr_index_batch_size + data_len > leaf_data_size) { delayed_node->curr_index_batch_size = data_len;@@ -1472,37 +1515,14 @@ int btrfs_insert_delayed_dir_index(struct btrfs_trans_handle *trans,* impossible. */ if (WARN_ON (ret)) { -mutex_unlock (&delayed_node->mutex);btrfs_release_delayed_item (delayed_item); +mutex_unlock (&delayed_node->mutex); goto release_node; } delayed_node->index_item_leaves++; - }else if (!test_bit (BTRFS_FS_LOG_RECOVERING , &fs_info->flags)) { -const u64 bytes =btrfs_calc_insert_metadata_size (fs_info,1 ); - -/* -* Adding the new dir index item does not require touching another -* leaf, so we can release 1 unit of metadata that was previously -* reserved when starting the transaction. This applies only to -* the case where we had a transaction start and excludes the -* transaction join case (when replaying log trees). -*/ -trace_btrfs_space_reservation (fs_info,"transaction" , - trans->transid, bytes,0 ); -btrfs_block_rsv_release (fs_info, trans->block_rsv, bytes,NULL ); -ASSERT (trans->bytes_reserved >= bytes); - trans->bytes_reserved -= bytes; - } - - ret =__btrfs_add_delayed_item (delayed_node, delayed_item); -if (unlikely (ret)) { -btrfs_err (trans->fs_info, -"error adding delayed dir index item, name: %.*s, index: %llu, root: %llu, dir: %llu, dir->index_cnt: %llu, delayed_node->index_cnt: %llu, error: %d" , - name_len, name, index,btrfs_root_id (delayed_node->root), - delayed_node->inode_id, dir->index_cnt, - delayed_node->index_cnt, ret); -BUG (); + }else { +btrfs_release_dir_index_item_space (trans); }mutex_unlock (&delayed_node->mutex);
PATCH v2 3/3 btrfs: assert delayed node locked when removing delayed item
In reply to [email protected]From: Filipe Manana <[email protected]>
When removing a delayed item, or releasing which will remove it as well, we will modify one of the delayed node's rbtrees and item counter if the delayed item is in one of the rbtrees. This require having the delayed node's mutex locked, otherwise we will race with other tasks modifying the rbtrees and the counter.
This is motivated by a previous version of another patch actually calling btrfs_release_delayed_item() after unlocking the delayed node's mutex and against a delayed item that is in a rbtree.
So assert at __btrfs_remove_delayed_item() that the delayed node's mutex is locked.
--- fs/btrfs/delayed-inode.c | 12 ++++++++---- 1 file changed, 8 insertions(+), 4 deletions(-)
fs/btrfs/delayed-inode.c +8 -4
@@ -412,6 +412,7 @@ static void finish_one_item(struct btrfs_delayed_root *delayed_root)static void __btrfs_remove_delayed_item (struct btrfs_delayed_item *delayed_item) { +struct btrfs_delayed_node *delayed_node = delayed_item->delayed_node;struct rb_root_cached *root;struct btrfs_delayed_root *delayed_root;
@@ -419,18 +420,21 @@ static void __btrfs_remove_delayed_item(struct btrfs_delayed_item *delayed_item)if (RB_EMPTY_NODE (&delayed_item->rb_node))return ; - delayed_root = delayed_item->delayed_node->root->fs_info->delayed_root; +/* If it's in a rbtree, then we need to have delayed node locked. */ +lockdep_assert_held (&delayed_node->mutex); + + delayed_root = delayed_node->root->fs_info->delayed_root;BUG_ON (!delayed_root);if (delayed_item->type ==BTRFS_DELAYED_INSERTION_ITEM ) - root = &delayed_item->delayed_node->ins_root; + root = &delayed_node->ins_root;else - root = &delayed_item->delayed_node->del_root; + root = &delayed_node->del_root;rb_erase_cached (&delayed_item->rb_node, root);RB_CLEAR_NODE (&delayed_item->rb_node); - delayed_item->delayed_node->count--; + delayed_node->count--;finish_one_item (delayed_root); }
Signature · 1 line
2.40.1
Re: [PATCH v2 3/3] btrfs: assert delayed node locked when removing delayed item
Quoted 8 lines · On 2023/8/28 16:06, [email protected] wrote:
From: Filipe Manana <[email protected]>
When removing a delayed item, or releasing which will remove it as well, we will modify one of the delayed node's rbtrees and item counter if the delayed item is in one of the rbtrees. This require having the delayed node's mutex locked, otherwise we will race with other tasks modifying the rbtrees and the counter.
Quoted 8 lines
This is motivated by a previous version of another patch actually calling
btrfs_release_delayed_item()after unlocking the delayed node's mutex and against a delayed item that is in a rbtree.So assert at
__btrfs_remove_delayed_item()that the delayed node's mutex is locked.Signed-off-by Filipe Manana <[email protected]>
Although it would be even better to add lockdep asserts for both __btrfs_add_delayed_item() and __btrfs_remove_delayed_item().
Thanks,
Qu
Quoted 42 lines
--- fs/btrfs/delayed-inode.c | 12 ++++++++---- 1 file changed, 8 insertions(+), 4 deletions(-)fs/btrfs/delayed-inode.c +8
-4@@ -412,6 +412,7 @@ static void finish_one_item(struct btrfs_delayed_root *delayed_root)static void __btrfs_remove_delayed_item (struct btrfs_delayed_item *delayed_item) { +struct btrfs_delayed_node *delayed_node = delayed_item->delayed_node;struct rb_root_cached *root;struct btrfs_delayed_root *delayed_root;@@ -419,18 +420,21 @@ static void __btrfs_remove_delayed_item(struct btrfs_delayed_item *delayed_item)if (RB_EMPTY_NODE (&delayed_item->rb_node))return ; - delayed_root = delayed_item->delayed_node->root->fs_info->delayed_root; +/* If it's in a rbtree, then we need to have delayed node locked. */ +lockdep_assert_held (&delayed_node->mutex); + + delayed_root = delayed_node->root->fs_info->delayed_root;BUG_ON (!delayed_root);if (delayed_item->type ==BTRFS_DELAYED_INSERTION_ITEM ) - root = &delayed_item->delayed_node->ins_root; + root = &delayed_node->ins_root;else - root = &delayed_item->delayed_node->del_root; + root = &delayed_node->del_root;rb_erase_cached (&delayed_item->rb_node, root);RB_CLEAR_NODE (&delayed_item->rb_node); - delayed_item->delayed_node->count--; + delayed_node->count--;finish_one_item (delayed_root); }
Quoted 9 lines · On Mon, Aug 28, 2023 at 09:06:41AM +0100, [email protected] wrote:
From: Filipe Manana <[email protected]>
Some updates to the error path for delayed dir index insertion failure, motivated by a recent syzbot report:
https://lore.kernel.org/linux-btrfs/[email protected]/
Details in the changelogs.
Quoted 7 lines
v2: Fixed error path in patch 2 to release delayed item before unlocking
the delayed node. Added patch 3 to prevent such mistakes in the future.Filipe Manana (3):
btrfs: improve error message after failure to add delayed dir index item
btrfs: removeBUG()after failure to insert delayed dir index item
btrfs: assert delayed node locked when removing delayed item
Added to misc-next, thanks.