ocfs2: split transactions in dio completion to avoid credit exhaustion
Heming Zhao
reviewerv5 cover ocfs2: split transactions in dio completion to avoid credit exhaustion
For easier merging, this patch is based on Joseph's patch [1].
v4->v5:
change update inode size and remove orphan logic.
add above change in commit log.
v3->v4:
Remove [patch 2/2] as the revert operation is incorrect.
v2->v3:
Following the discussion, use 'batch' and 'handle' to control restarting the jbd2 transaction.
v1->v2:
following the review comments, restore the i_size update code in ocfs2_dio_end_io_write().
the runtime of the test script [2].
real 1m49.489s user 0m0.358s sys 0m22.869s
[1]:
https://lore.kernel.org/ocfs2-devel/46yilbaq5z5x6gdfdpoa6lprf6sf3gbxriuku2odje4kx4bovf@jd735cphfutz/T/#t
[2]:
https://lore.kernel.org/ocfs2-devel/[email protected]/T/#mbe2b5f52ee249178e1ad4c76d964de2dc818eb32
Heming Zhao (1):
ocfs2: split transactions in dio completion to avoid credit exhaustion
fs/ocfs2/aops.c | 63 +++++++++++++++++++++++++++---------------------- 1 file changed, 35 insertions(+), 28 deletions(-)
Signature · 1 line
2.43.0
Heming Zhao
reviewerPATCH v5 1/1 ocfs2: split transactions in dio completion to avoid credit exhaustion
During ocfs2 dio operations, JBD2 may report warnings via following call trace:ocfs2_dio_end_io_write
ocfs2_mark_extent_written
ocfs2_change_extent_flag
ocfs2_split_extent
ocfs2_try_to_merge_extent
ocfs2_extend_rotate_transaction
ocfs2_extend_trans
jbd2__journal_restart
start_this_handle
output: JBD2: kworker/6:2 wants too many credits credits:5450 rsv_credits:0 max:5449
To prevent exceeding the credits limit, modify ocfs2_dio_end_io_write() to handle extents in a batch of transaction.
Additionally, relocate ocfs2_del_inode_from_orphan(). The orphan inode should only be removed from the orphan list after the extent tree update is complete. This ensures that if a crash occurs in the middle of extent tree updates, we won't leave stale blocks beyond EOF.
This patch also changes the logic for updating the inode size and removing orphan, making it similar to ext4_dio_write_end_io(). Both operations are performed only when everything looks good.
Finally, thanks to Jans and Joseph for providing the bug fix prototype and suggestions.
--- fs/ocfs2/aops.c | 63 +++++++++++++++++++++++++++---------------------- 1 file changed, 35 insertions(+), 28 deletions(-)
fs/ocfs2/aops.c +35 -28
@@ -37,6 +37,8 @@#include "namei.h" #include "sysfile.h" +#define OCFS2_DIO_MARK_EXTENT_BATCH 200 +static int ocfs2_symlink_get_block (struct inode *inode,sector_t iblock,struct buffer_head *bh_result,int create) {
@@ -2277,7 +2279,7 @@ static int ocfs2_dio_end_io_write(struct inode *inode,struct ocfs2_alloc_context *meta_ac =NULL ;handle_t *handle =NULL ;loff_t end = offset + bytes; -int ret =0 , credits =0 ; +int ret =0 , credits =0 , batch =0 ;ocfs2_init_dealloc_ctxt (&dealloc);
@@ -2294,18 +2296,6 @@ static int ocfs2_dio_end_io_write(struct inode *inode, goto out; } -/* Delete orphan before acquire i_rwsem. */ -if (dwc->dw_orphaned) { -BUG_ON (dwc->dw_writer_pid !=task_pid_nr (current)); - - end = end >i_size_read (inode) ? end :0 ; - - ret =ocfs2_del_inode_from_orphan (osb, inode, di_bh, - !!end, end); -if (ret <0 ) -mlog_errno (ret); - } -down_write (&oi->ip_alloc_sem); di = (struct ocfs2_dinode *)di_bh->b_data;
@@ -2326,24 +2316,25 @@ static int ocfs2_dio_end_io_write(struct inode *inode, credits =ocfs2_calc_extend_credits (inode->i_sb, &di->id2.i_list); - handle =ocfs2_start_trans (osb, credits); -if (IS_ERR (handle)) { - ret =PTR_ERR (handle); -mlog_errno (ret); - goto unlock; - } - ret =ocfs2_journal_access_di (handle,INODE_CACHE (inode), di_bh, -OCFS2_JOURNAL_ACCESS_WRITE ); -if (ret) { -mlog_errno (ret); - goto commit; - } -list_for_each_entry (ue, &dwc->dw_zero_list, ue_node) { +if (!handle) { + handle =ocfs2_start_trans (osb, credits); +if (IS_ERR (handle)) { + ret =PTR_ERR (handle); +mlog_errno (ret); + goto unlock; + } + ret =ocfs2_journal_access_di (handle,INODE_CACHE (inode), di_bh, +OCFS2_JOURNAL_ACCESS_WRITE ); +if (ret) { +mlog_errno (ret); + goto commit; + } + } ret =ocfs2_assure_trans_credits (handle, credits);if (ret <0 ) {mlog_errno (ret); -break ; + goto commit; } ret = ocfs2_mark_extent_written(inode, &et, handle, ue->ue_cpos, 1,
@@ -2351,7 +2342,13 @@ static int ocfs2_dio_end_io_write(struct inode *inode, meta_ac, &dealloc);if (ret <0 ) {mlog_errno (ret); -break ; + goto commit; + } + +if (++batch ==OCFS2_DIO_MARK_EXTENT_BATCH ) { +ocfs2_commit_trans (osb, handle); + handle =NULL ; + batch =0 ; } }
@@ -2360,10 +2357,20 @@ static int ocfs2_dio_end_io_write(struct inode *inode,if (ret <0 )mlog_errno (ret); } + commit:ocfs2_commit_trans (osb, handle); unlock:up_write (&oi->ip_alloc_sem); + +/* everything looks good, let's start the cleanup */ +if (!ret && dwc->dw_orphaned) { +BUG_ON (dwc->dw_writer_pid !=task_pid_nr (current)); + + ret =ocfs2_del_inode_from_orphan (osb, inode, di_bh,0 ,0 ); +if (ret <0 ) +mlog_errno (ret); + }ocfs2_inode_unlock (inode,1 );brelse (di_bh); out:
Signature · 1 line
2.43.0
Heming Zhao
reviewerHi Joseph,
ping.
Does this patch lack your 'reviewed-by', or is Jan's 'reviewed-by' enough?
- Heming
Quoted 155 lines · Heming Zhao
During ocfs2 dio operations, JBD2 may report warnings via following call trace:
ocfs2_dio_end_io_write
ocfs2_mark_extent_written
ocfs2_change_extent_flag
ocfs2_split_extent
ocfs2_try_to_merge_extent
ocfs2_extend_rotate_transaction
ocfs2_extend_trans
jbd2__journal_restart
start_this_handle
output: JBD2: kworker/6:2 wants too many credits credits:5450 rsv_credits:0 max:5449To prevent exceeding the credits limit, modify
ocfs2_dio_end_io_write()to handle extents in a batch of transaction.Additionally, relocate
ocfs2_del_inode_from_orphan(). The orphan inode should only be removed from the orphan list after the extent tree update is complete. This ensures that if a crash occurs in the middle of extent tree updates, we won't leave stale blocks beyond EOF.This patch also changes the logic for updating the inode size and removing orphan, making it similar to
ext4_dio_write_end_io(). Both operations are performed only when everything looks good.Finally, thanks to Jans and Joseph for providing the bug fix prototype and suggestions.
Suggested-by Jan Kara <[email protected]>Suggested-by Joseph Qi <[email protected]>Reviewed-by Jan Kara <[email protected]>Signed-off-by Heming Zhao <[email protected]>--- fs/ocfs2/aops.c | 63 +++++++++++++++++++++++++++---------------------- 1 file changed, 35 insertions(+), 28 deletions(-)fs/ocfs2/aops.c +35
-28@@ -37,6 +37,8 @@#include "namei.h" #include "sysfile.h" +#define OCFS2_DIO_MARK_EXTENT_BATCH 200 +static int ocfs2_symlink_get_block (struct inode *inode,sector_t iblock,struct buffer_head *bh_result,int create) {@@ -2277,7 +2279,7 @@ static int ocfs2_dio_end_io_write(struct inode *inode,struct ocfs2_alloc_context *meta_ac =NULL ;handle_t *handle =NULL ;loff_t end = offset + bytes; -int ret =0 , credits =0 ; +int ret =0 , credits =0 , batch =0 ;ocfs2_init_dealloc_ctxt (&dealloc);@@ -2294,18 +2296,6 @@ static int ocfs2_dio_end_io_write(struct inode *inode, goto out; } -/* Delete orphan before acquire i_rwsem. */ -if (dwc->dw_orphaned) { -BUG_ON (dwc->dw_writer_pid !=task_pid_nr (current)); - - end = end >i_size_read (inode) ? end :0 ; - - ret =ocfs2_del_inode_from_orphan (osb, inode, di_bh, - !!end, end); -if (ret <0 ) -mlog_errno (ret); - } -down_write (&oi->ip_alloc_sem); di = (struct ocfs2_dinode *)di_bh->b_data;@@ -2326,24 +2316,25 @@ static int ocfs2_dio_end_io_write(struct inode *inode, credits =ocfs2_calc_extend_credits (inode->i_sb, &di->id2.i_list); - handle =ocfs2_start_trans (osb, credits); -if (IS_ERR (handle)) { - ret =PTR_ERR (handle); -mlog_errno (ret); - goto unlock; - } - ret =ocfs2_journal_access_di (handle,INODE_CACHE (inode), di_bh, -OCFS2_JOURNAL_ACCESS_WRITE ); -if (ret) { -mlog_errno (ret); - goto commit; - } -list_for_each_entry (ue, &dwc->dw_zero_list, ue_node) { +if (!handle) { + handle =ocfs2_start_trans (osb, credits); +if (IS_ERR (handle)) { + ret =PTR_ERR (handle); +mlog_errno (ret); + goto unlock; + } + ret =ocfs2_journal_access_di (handle,INODE_CACHE (inode), di_bh, +OCFS2_JOURNAL_ACCESS_WRITE ); +if (ret) { +mlog_errno (ret); + goto commit; + } + } ret =ocfs2_assure_trans_credits (handle, credits);if (ret <0 ) {mlog_errno (ret); -break ; + goto commit; } ret = ocfs2_mark_extent_written(inode, &et, handle, ue->ue_cpos, 1,@@ -2351,7 +2342,13 @@ static int ocfs2_dio_end_io_write(struct inode *inode, meta_ac, &dealloc);if (ret <0 ) {mlog_errno (ret); -break ; + goto commit; + } + +if (++batch ==OCFS2_DIO_MARK_EXTENT_BATCH ) { +ocfs2_commit_trans (osb, handle); + handle =NULL ; + batch =0 ; } }@@ -2360,10 +2357,20 @@ static int ocfs2_dio_end_io_write(struct inode *inode,if (ret <0 )mlog_errno (ret); } + commit:ocfs2_commit_trans (osb, handle); unlock:up_write (&oi->ip_alloc_sem); + +/* everything looks good, let's start the cleanup */ +if (!ret && dwc->dw_orphaned) { +BUG_ON (dwc->dw_writer_pid !=task_pid_nr (current)); + + ret =ocfs2_del_inode_from_orphan (osb, inode, di_bh,0 ,0 ); +if (ret <0 ) +mlog_errno (ret); + }ocfs2_inode_unlock (inode,1 );brelse (di_bh); out:Signature · 1 line
2.43.0
Joseph Qi
maintainerQuoted 132 lines · Heming Zhao
During ocfs2 dio operations, JBD2 may report warnings via following call trace:
ocfs2_dio_end_io_write
ocfs2_mark_extent_written
ocfs2_change_extent_flag
ocfs2_split_extent
ocfs2_try_to_merge_extent
ocfs2_extend_rotate_transaction
ocfs2_extend_trans
jbd2__journal_restart
start_this_handle
output: JBD2: kworker/6:2 wants too many credits credits:5450 rsv_credits:0 max:5449To prevent exceeding the credits limit, modify
ocfs2_dio_end_io_write()to handle extents in a batch of transaction.Additionally, relocate
ocfs2_del_inode_from_orphan(). The orphan inode should only be removed from the orphan list after the extent tree update is complete. This ensures that if a crash occurs in the middle of extent tree updates, we won't leave stale blocks beyond EOF.This patch also changes the logic for updating the inode size and removing orphan, making it similar to
ext4_dio_write_end_io(). Both operations are performed only when everything looks good.Finally, thanks to Jans and Joseph for providing the bug fix prototype and suggestions.
Suggested-by Jan Kara <[email protected]>Suggested-by Joseph Qi <[email protected]>Reviewed-by Jan Kara <[email protected]>Signed-off-by Heming Zhao <[email protected]>--- fs/ocfs2/aops.c | 63 +++++++++++++++++++++++++++---------------------- 1 file changed, 35 insertions(+), 28 deletions(-)fs/ocfs2/aops.c +25
-28@@ -37,6 +37,8 @@#include "namei.h" #include "sysfile.h" +#define OCFS2_DIO_MARK_EXTENT_BATCH 200 +static int ocfs2_symlink_get_block (struct inode *inode,sector_t iblock,struct buffer_head *bh_result,int create) {@@ -2277,7 +2279,7 @@ static int ocfs2_dio_end_io_write(struct inode *inode,struct ocfs2_alloc_context *meta_ac =NULL ;handle_t *handle =NULL ;loff_t end = offset + bytes; -int ret =0 , credits =0 ; +int ret =0 , credits =0 , batch =0 ;ocfs2_init_dealloc_ctxt (&dealloc);@@ -2294,18 +2296,6 @@ static int ocfs2_dio_end_io_write(struct inode *inode, goto out; } -/* Delete orphan before acquire i_rwsem. */ -if (dwc->dw_orphaned) { -BUG_ON (dwc->dw_writer_pid !=task_pid_nr (current)); - - end = end >i_size_read (inode) ? end :0 ; - - ret =ocfs2_del_inode_from_orphan (osb, inode, di_bh, - !!end, end); -if (ret <0 ) -mlog_errno (ret); - } -down_write (&oi->ip_alloc_sem); di = (struct ocfs2_dinode *)di_bh->b_data;@@ -2326,24 +2316,25 @@ static int ocfs2_dio_end_io_write(struct inode *inode, credits =ocfs2_calc_extend_credits (inode->i_sb, &di->id2.i_list); - handle =ocfs2_start_trans (osb, credits); -if (IS_ERR (handle)) { - ret =PTR_ERR (handle); -mlog_errno (ret); - goto unlock; - } - ret =ocfs2_journal_access_di (handle,INODE_CACHE (inode), di_bh, -OCFS2_JOURNAL_ACCESS_WRITE ); -if (ret) { -mlog_errno (ret); - goto commit; - } -list_for_each_entry (ue, &dwc->dw_zero_list, ue_node) { +if (!handle) { + handle =ocfs2_start_trans (osb, credits); +if (IS_ERR (handle)) { + ret =PTR_ERR (handle); +mlog_errno (ret); + goto unlock; + } + ret =ocfs2_journal_access_di (handle,INODE_CACHE (inode), di_bh, +OCFS2_JOURNAL_ACCESS_WRITE ); +if (ret) { +mlog_errno (ret); + goto commit; + } + } ret =ocfs2_assure_trans_credits (handle, credits);if (ret <0 ) {mlog_errno (ret); -break ; + goto commit; } ret = ocfs2_mark_extent_written(inode, &et, handle, ue->ue_cpos, 1,@@ -2351,7 +2342,13 @@ static int ocfs2_dio_end_io_write(struct inode *inode, meta_ac, &dealloc);if (ret <0 ) {mlog_errno (ret); -break ; + goto commit; + } + +if (++batch ==OCFS2_DIO_MARK_EXTENT_BATCH ) { +ocfs2_commit_trans (osb, handle); + handle =NULL ; + batch =0 ; } }
fs/ocfs2/aops.c static int ocfs2_dio_end_io_write(struct inode *inode, · line 2362
Quoted 6 lines
fs/ocfs2/aops.c +1
-0@@ -2360,10 +2357,20 @@ static int ocfs2_dio_end_io_write(struct inode *inode,if (ret <0 )mlog_errno (ret); } + commit:ocfs2_commit_trans (osb, handle);
We have to check 'handle' here to prevent NULL pointer dereference. e.g. rightly after OCFS2_DIO_MARK_EXTENT_BATCH round.
And you've missed ocfs2_start_trans() and ocfs2_journal_access_di() before ocfs2_set_inode_size().
Thanks,
Joseph
Quoted 14 lines
fs/ocfs2/aops.c +9
-0unlock:up_write (&oi->ip_alloc_sem); + +/* everything looks good, let's start the cleanup */ +if (!ret && dwc->dw_orphaned) { +BUG_ON (dwc->dw_writer_pid !=task_pid_nr (current)); + + ret =ocfs2_del_inode_from_orphan (osb, inode, di_bh,0 ,0 ); +if (ret <0 ) +mlog_errno (ret); + }ocfs2_inode_unlock (inode,1 );brelse (di_bh); out:
Heming Zhao
reviewerQuoted 134 lines · Joseph Qi
Quoted 131 lines · Heming Zhao
During ocfs2 dio operations, JBD2 may report warnings via following call trace:
ocfs2_dio_end_io_write
ocfs2_mark_extent_written
ocfs2_change_extent_flag
ocfs2_split_extent
ocfs2_try_to_merge_extent
ocfs2_extend_rotate_transaction
ocfs2_extend_trans
jbd2__journal_restart
start_this_handle
output: JBD2: kworker/6:2 wants too many credits credits:5450 rsv_credits:0 max:5449To prevent exceeding the credits limit, modify
ocfs2_dio_end_io_write()to handle extents in a batch of transaction.Additionally, relocate
ocfs2_del_inode_from_orphan(). The orphan inode should only be removed from the orphan list after the extent tree update is complete. This ensures that if a crash occurs in the middle of extent tree updates, we won't leave stale blocks beyond EOF.This patch also changes the logic for updating the inode size and removing orphan, making it similar to
ext4_dio_write_end_io(). Both operations are performed only when everything looks good.Finally, thanks to Jans and Joseph for providing the bug fix prototype and suggestions.
Suggested-by Jan Kara <[email protected]>Suggested-by Joseph Qi <[email protected]>Reviewed-by Jan Kara <[email protected]>Signed-off-by Heming Zhao <[email protected]>--- fs/ocfs2/aops.c | 63 +++++++++++++++++++++++++++---------------------- 1 file changed, 35 insertions(+), 28 deletions(-)fs/ocfs2/aops.c +25
-28@@ -37,6 +37,8 @@#include "namei.h" #include "sysfile.h" +#define OCFS2_DIO_MARK_EXTENT_BATCH 200 +static int ocfs2_symlink_get_block (struct inode *inode,sector_t iblock,struct buffer_head *bh_result,int create) {@@ -2277,7 +2279,7 @@ static int ocfs2_dio_end_io_write(struct inode *inode,struct ocfs2_alloc_context *meta_ac =NULL ;handle_t *handle =NULL ;loff_t end = offset + bytes; -int ret =0 , credits =0 ; +int ret =0 , credits =0 , batch =0 ;ocfs2_init_dealloc_ctxt (&dealloc);@@ -2294,18 +2296,6 @@ static int ocfs2_dio_end_io_write(struct inode *inode, goto out; } -/* Delete orphan before acquire i_rwsem. */ -if (dwc->dw_orphaned) { -BUG_ON (dwc->dw_writer_pid !=task_pid_nr (current)); - - end = end >i_size_read (inode) ? end :0 ; - - ret =ocfs2_del_inode_from_orphan (osb, inode, di_bh, - !!end, end); -if (ret <0 ) -mlog_errno (ret); - } -down_write (&oi->ip_alloc_sem); di = (struct ocfs2_dinode *)di_bh->b_data;@@ -2326,24 +2316,25 @@ static int ocfs2_dio_end_io_write(struct inode *inode, credits =ocfs2_calc_extend_credits (inode->i_sb, &di->id2.i_list); - handle =ocfs2_start_trans (osb, credits); -if (IS_ERR (handle)) { - ret =PTR_ERR (handle); -mlog_errno (ret); - goto unlock; - } - ret =ocfs2_journal_access_di (handle,INODE_CACHE (inode), di_bh, -OCFS2_JOURNAL_ACCESS_WRITE ); -if (ret) { -mlog_errno (ret); - goto commit; - } -list_for_each_entry (ue, &dwc->dw_zero_list, ue_node) { +if (!handle) { + handle =ocfs2_start_trans (osb, credits); +if (IS_ERR (handle)) { + ret =PTR_ERR (handle); +mlog_errno (ret); + goto unlock; + } + ret =ocfs2_journal_access_di (handle,INODE_CACHE (inode), di_bh, +OCFS2_JOURNAL_ACCESS_WRITE ); +if (ret) { +mlog_errno (ret); + goto commit; + } + } ret =ocfs2_assure_trans_credits (handle, credits);if (ret <0 ) {mlog_errno (ret); -break ; + goto commit; } ret = ocfs2_mark_extent_written(inode, &et, handle, ue->ue_cpos, 1,@@ -2351,7 +2342,13 @@ static int ocfs2_dio_end_io_write(struct inode *inode, meta_ac, &dealloc);if (ret <0 ) {mlog_errno (ret); -break ; + goto commit; + } + +if (++batch ==OCFS2_DIO_MARK_EXTENT_BATCH ) { +ocfs2_commit_trans (osb, handle); + handle =NULL ; + batch =0 ; } }
Quoted 10 lines
fs/ocfs2/aops.c static int ocfs2_dio_end_io_write(struct inode *inode, · line 2362
Quoted 7 lines
fs/ocfs2/aops.c +1
-0@@ -2360,10 +2357,20 @@ static int ocfs2_dio_end_io_write(struct inode *inode,if (ret <0 )mlog_errno (ret); } + commit:ocfs2_commit_trans (osb, handle);We have to check 'handle' here to prevent NULL pointer dereference. e.g. rightly after
OCFS2_DIO_MARK_EXTENT_BATCHround.
Good catch.
Quoted 3 lines
And you've missed
ocfs2_start_trans()andocfs2_journal_access_di()beforeocfs2_set_inode_size().
Only the ocfs2_start_trans() call was missing; as access() is started in ocfs2_set_inode_size() via ocfs2_mark_inode_dirty().
I will submit v6 reflecting above changes.
- Heming
Quoted 17 lines
Quoted 16 lines
fs/ocfs2/aops.c +9
-0unlock:up_write (&oi->ip_alloc_sem); + +/* everything looks good, let's start the cleanup */ +if (!ret && dwc->dw_orphaned) { +BUG_ON (dwc->dw_writer_pid !=task_pid_nr (current)); + + ret =ocfs2_del_inode_from_orphan (osb, inode, di_bh,0 ,0 ); +if (ret <0 ) +mlog_errno (ret); + }ocfs2_inode_unlock (inode,1 );brelse (di_bh); out: