ext4: call ext4_error_inode() if jbd2_journal_dirty_metadata() fails
Subsystems (3)
Theodore Ts'o
maintainerPATCH 1/2 ext4: call ext4_error_inode() if jbd2_journal_dirty_metadata() fails
While it's true that errors can only happen if there is a bug in jbd2_journal_dirty_metadata(), if a bug does happen, we need to halt the kernel or remount the file system read-only in order to avoid further data loss. The ext4_journal_abort_handle() function doesn't do any of this, and while it's likely that this call (since it doesn't adjust refcounts) will likely result in the file system eventually deadlocking since the current transaction will never be able to close, it's much cleaner to call let ext4's error handling system deal with this situation.
There's a separate bug here which is that if certain jbd2 errors errors occur and file system is mounted errors=continue, the file system will probably eventually end grind to a halt as described above. But things have been this way in a long time, and usually when we have these sorts of errors it's pretty much a disaster --- and that's why the jbd2 layer aggressively retries memory allocations, which is the most likely cause of these jbd2 errors.
--- fs/ext4/ext4_jbd2.c | 9 +++++++++ 1 file changed, 9 insertions(+)
fs/ext4/ext4_jbd2.c +9 -0
@@ -259,6 +259,15 @@ int __ext4_handle_dirty_metadata(const char *where, unsigned int line,if (WARN_ON_ONCE (err)) {ext4_journal_abort_handle (where, line, __func__, bh, handle, err); +ext4_error_inode (inode, where, line, + bh->b_blocknr, +"journal_dirty_metadata failed: " +"handle type %u started at line %u, " +"credits %u/%u, errcode %d" , + handle->h_type, + handle->h_line_no, + handle->h_requested_credits, + handle->h_buffer_credits, err); } }else {if (inode)
Signature · 1 line
1.8.5.rc3.362.gdf10213
Theodore Ts'o
maintainerPATCH 2/2 jbd2: return ENOSPC in journal_dirty_metadata if a handle runs out of space
If a handle runs out of space, we currently stop the kernel with a BUG. This makes it hard to figure out what might be going on. So return an error of ENOSPC, so we can let the file system layer figure out what is going on, to make it more likely we can get useful debugging information). This should make it easier to debug problems such as the one which was reported by:
https://bugzilla.kernel.org/show_bug.cgi?id=44731
The only two callers of this function are ext4_handle_dirty_metadata() and ocfs2_journal_dirty(). The ocfs2 function will trigger a BUG_ON(), which means there will be no change in behavior. The ext4 function will call ext4_error_inode() which will print the useful debugging information and then handle the situation using ext4's error handling mechanisms (i.e., which might mean halting the kernel or remounting the file system read-only).
Also, since both file systems already call WARN_ON(), drop the WARN_ON from jbd2_journal_dirty_metadata() to avoid two stack traces from being displayed.
--- fs/jbd2/transaction.c | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-)
fs/jbd2/transaction.c +4 -2
@@ -1290,7 +1290,10 @@ int jbd2_journal_dirty_metadata(handle_t *handle, struct buffer_head *bh)* once a transaction -bzzz */ jh->b_modified =1 ; -J_ASSERT_JH (jh, handle->h_buffer_credits >0 ); +if (handle->h_buffer_credits <=0 ) { + ret = -ENOSPC ; + goto out_unlock_bh; + } handle->h_buffer_credits--; }
@@ -1373,7 +1376,6 @@ out_unlock_bh:jbd2_journal_put_journal_head (jh); out:JBUFFER_TRACE (jh,"exit" ); -WARN_ON (ret);/* All errors are bugs, so dump the stack */ return ret; }
Signature · 1 line
1.8.5.rc3.362.gdf10213
Re: [PATCH 2/2] jbd2: return ENOSPC in journal_dirty_metadata if a handle runs out of space
Quoted 17 lines · Ted Tso
If a handle runs out of space, we currently stop the kernel with a BUG. This makes it hard to figure out what might be going on. So return an error of ENOSPC, so we can let the file system layer figure out what is going on, to make it more likely we can get useful debugging information). This should make it easier to debug problems such as the one which was reported by:
https://bugzilla.kernel.org/show_bug.cgi?id=44731The only two callers of this function are
ext4_handle_dirty_metadata()andocfs2_journal_dirty(). The ocfs2 function will trigger aBUG_ON(), which means there will be no change in behavior. The ext4 function will callext4_error_inode()which will print the useful debugging information and then handle the situation using ext4's error handling mechanisms (i.e., which might mean halting the kernel or remounting the file system read-only).
Quoted 6 lines
Also, since both file systems already call
WARN_ON(), drop theWARN_ONfromjbd2_journal_dirty_metadata()to avoid two stack traces from being displayed.Signed-off-by "Theodore Ts'o" <[email protected]>Cc ocfs2-devel at oss.oracle.com
The patch looks good. You can add:
Honza
Quoted 35 lines
--- fs/jbd2/transaction.c | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-)fs/jbd2/transaction.c +4
-2@@ -1290,7 +1290,10 @@ int jbd2_journal_dirty_metadata(handle_t *handle, struct buffer_head *bh)* once a transaction -bzzz */ jh->b_modified =1 ; -J_ASSERT_JH (jh, handle->h_buffer_credits >0 ); +if (handle->h_buffer_credits <=0 ) { + ret = -ENOSPC ; + goto out_unlock_bh; + } handle->h_buffer_credits--; }@@ -1373,7 +1376,6 @@ out_unlock_bh:jbd2_journal_put_journal_head (jh); out:JBUFFER_TRACE (jh,"exit" ); -WARN_ON (ret);/* All errors are bugs, so dump the stack */ return ret; }Signature · 6 lines
1.8.5.rc3.362.gdf10213 -- To unsubscribe from this list: send the line "unsubscribe linux-ext4" in the body of a message to majordomo at vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Signature · 2 lines
Jan Kara <[email protected]> SUSE Labs, CR
Re: [PATCH 2/2] jbd2: return ENOSPC in journal_dirty_metadata if a handle runs out of space
In reply to Theodore Ts'oQuoted 17 lines · Theodore Ts'o
If a handle runs out of space, we currently stop the kernel with a BUG. This makes it hard to figure out what might be going on. So return an error of ENOSPC, so we can let the file system layer figure out what is going on, to make it more likely we can get useful debugging information). This should make it easier to debug problems such as the one which was reported by:
https://bugzilla.kernel.org/show_bug.cgi?id=44731The only two callers of this function are
ext4_handle_dirty_metadata()andocfs2_journal_dirty(). The ocfs2 function will trigger aBUG_ON(), which means there will be no change in behavior. The ext4 function will callext4_error_inode()which will print the useful debugging information and then handle the situation using ext4's error handling mechanisms (i.e., which might mean halting the kernel or remounting the file system read-only).
Quoted 6 lines
Also, since both file systems already call
WARN_ON(), drop theWARN_ONfromjbd2_journal_dirty_metadata()to avoid two stack traces from being displayed.Signed-off-by "Theodore Ts'o" <[email protected]>Cc ocfs2-devel at oss.oracle.com
Quoted 35 lines
--- fs/jbd2/transaction.c | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-)fs/jbd2/transaction.c +4
-2@@ -1290,7 +1290,10 @@ int jbd2_journal_dirty_metadata(handle_t *handle, struct buffer_head *bh)* once a transaction -bzzz */ jh->b_modified =1 ; -J_ASSERT_JH (jh, handle->h_buffer_credits >0 ); +if (handle->h_buffer_credits <=0 ) { + ret = -ENOSPC ; + goto out_unlock_bh; + } handle->h_buffer_credits--; }@@ -1373,7 +1376,6 @@ out_unlock_bh:jbd2_journal_put_journal_head (jh); out:JBUFFER_TRACE (jh,"exit" ); -WARN_ON (ret);/* All errors are bugs, so dump the stack */ return ret; }Signature · 6 lines
1.8.5.rc3.362.gdf10213 -- To unsubscribe from this list: send the line "unsubscribe linux-ext4" in the body of a message to majordomo at vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Quoted 18 lines · Ted Tso
While it's true that errors can only happen if there is a bug in
jbd2_journal_dirty_metadata(), if a bug does happen, we need to halt the kernel or remount the file system read-only in order to avoid further data loss. Theext4_journal_abort_handle()function doesn't do any of this, and while it's likely that this call (since it doesn't adjust refcounts) will likely result in the file system eventually deadlocking since the current transaction will never be able to close, it's much cleaner to call let ext4's error handling system deal with this situation.There's a separate bug here which is that if certain jbd2 errors errors occur and file system is mounted errors=continue, the file system will probably eventually end grind to a halt as described above. But things have been this way in a long time, and usually when we have these sorts of errors it's pretty much a disaster --- and that's why the jbd2 layer aggressively retries memory allocations, which is the most likely cause of these jbd2 errors.
Quoted 2 lines
Signed-off-by "Theodore Ts'o" <[email protected]>
The patch looks good. You can add:
Honza
Quoted 31 lines
--- fs/ext4/ext4_jbd2.c | 9 +++++++++ 1 file changed, 9 insertions(+)fs/ext4/ext4_jbd2.c +9
-0@@ -259,6 +259,15 @@ int __ext4_handle_dirty_metadata(const char *where, unsigned int line,if (WARN_ON_ONCE (err)) {ext4_journal_abort_handle (where, line, __func__, bh, handle, err); +ext4_error_inode (inode, where, line, + bh->b_blocknr, +"journal_dirty_metadata failed: " +"handle type %u started at line %u, " +"credits %u/%u, errcode %d" , + handle->h_type, + handle->h_line_no, + handle->h_requested_credits, + handle->h_buffer_credits, err); } }else {if (inode)Signature · 6 lines
1.8.5.rc3.362.gdf10213 -- To unsubscribe from this list: send the line "unsubscribe linux-ext4" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html
Signature · 2 lines
Jan Kara <[email protected]> SUSE Labs, CR