+ jbd-fix-error-handling-for-checkpoint-io.patch added to -mm tree
Hi Andrew and Jan,
Quoted 4 lines
The patch titled
jbd: fix error handling for checkpoint io
has been added to the -mm tree. Its filename is
jbd-fix-error-handling-for-checkpoint-io.patch
[snip]
Quoted 24 lines
Subject: jbd: fix error handling for checkpoint io
From: Hidehiro Kawai <[email protected]>When a checkpointing IO fails, current JBD code doesn't check the error and continue journaling. This means latest metadata can be lost from both the journal and filesystem.
This patch leaves the failed metadata blocks in the journal space and aborts journaling in the case of
log_do_checkpoint(). To achieve this, we need to do:1. don't remove the failed buffer from the checkpoint list where in the case of
__try_to_free_cp_buf()because it may be released or overwritten by a later transaction
2.log_do_checkpoint()is the last chance, remove the failed buffer from the checkpoint list and abort the journal
3. when checkpointing fails, don't update the journal super block to prevent the journaled contents from being cleaned. For safety, don't updatej_tailandj_tail_sequenceeither
4. when checkpointing fails, notify this error to the ext3 layer so that ext3 don't clear theneeds_recoveryflag, otherwise the journaled contents are ignored and cleaned in the recovery phase
5. if the recovery fails, keep theneeds_recoveryflag
Quoted 3 lines
6. prevent
cleanup_journal_tail()from being called between__journal_drop_transaction()andjournal_abort()(a race issue betweenjournal_flush()and__log_wait_for_space()
When I read the source code again, I noticed the race condition described in 6 doesn't happen. I've thought journal_flush() can invoke log_do_checkpoint() while __log_wait_for_space() is invoking log_do_checkpoint(), but it would be wrong.
First journal_flush() invokes __log_start_commit() and log_wait_commit() pair. After this, there is no running transaction and no starting handle. New handles are also not created because j_barrier_count blocks it. Thus, when journal_flush() invokes log_do_checkpoint(), there is no other process which invokes __log_wait_for_space() and log_do_checkpoint() to get free log space. So invocations of log_do_checkpoint() are always isolated, the race condition doesn't happen.
If my understanding is correct, adding mutex_lock() around log_do_checkpoint() (see bellow) is unneeded.
What do you think about this?
[snip]
Review int journal_flush(journal_t *journal) · line 1375
Quoted 8 lines
(quoted diff) +2
-0@@ -1359,10 +1369,16 @@ int journal_flush(journal_t *journal) spin_lock(&journal->j_list_lock); while (!err && journal->j_checkpoint_transactions != NULL) { spin_unlock(&journal->j_list_lock); + mutex_lock(&journal->j_checkpoint_mutex); err = log_do_checkpoint(journal); + mutex_unlock(&journal->j_checkpoint_mutex); spin_lock(&journal->j_list_lock);
Best regards,
Signature · 3 lines
Hidehiro Kawai Hitachi, Systems Development Laboratory Linux Technology Center
Hello,
Quoted 40 lines · Hidehiro Kawai
Quoted 4 lines
The patch titled
jbd: fix error handling for checkpoint io
has been added to the -mm tree. Its filename is
jbd-fix-error-handling-for-checkpoint-io.patch[snip]
Quoted 24 lines
Subject: jbd: fix error handling for checkpoint io
From: Hidehiro Kawai <[email protected]>When a checkpointing IO fails, current JBD code doesn't check the error and continue journaling. This means latest metadata can be lost from both the journal and filesystem.
This patch leaves the failed metadata blocks in the journal space and aborts journaling in the case of
log_do_checkpoint(). To achieve this, we need to do:1. don't remove the failed buffer from the checkpoint list where in the case of
__try_to_free_cp_buf()because it may be released or overwritten by a later transaction
2.log_do_checkpoint()is the last chance, remove the failed buffer from the checkpoint list and abort the journal
3. when checkpointing fails, don't update the journal super block to prevent the journaled contents from being cleaned. For safety, don't updatej_tailandj_tail_sequenceeither
4. when checkpointing fails, notify this error to the ext3 layer so that ext3 don't clear theneeds_recoveryflag, otherwise the journaled contents are ignored and cleaned in the recovery phase
5. if the recovery fails, keep theneeds_recoveryflagQuoted 3 lines
6. prevent
cleanup_journal_tail()from being called between__journal_drop_transaction()andjournal_abort()(a race issue betweenjournal_flush()and__log_wait_for_space()When I read the source code again, I noticed the race condition described in 6 doesn't happen. I've thought
journal_flush()can invokelog_do_checkpoint()while__log_wait_for_space()is invokinglog_do_checkpoint(), but it would be wrong.
Quoted 8 lines
First
journal_flush()invokes__log_start_commit()andlog_wait_commit()pair. After this, there is no running transaction and no starting handle. New handles are also not created becausej_barrier_countblocks it. Thus, whenjournal_flush()invokeslog_do_checkpoint(), there is no other process which invokes__log_wait_for_space()andlog_do_checkpoint()to get free log space. So invocations oflog_do_checkpoint()are always isolated, the race condition doesn't happen.
I'm not quite following you. j_barrier_count is increased only injournal_lock_updates(). Noone is forced to first call journal_lock_updates() and only after that journal_flush() (although usually it is done that way). So I think taking the j_checkpoint_mutex in journal_flush() is really a good thing to do.
Quoted 6 lines
If my understanding is correct, adding
mutex_lock()aroundlog_do_checkpoint()(see bellow) is unneeded.What do you think about this?
[snip]
Quoted 8 lines
Review int journal_flush(journal_t *journal) · line 1375
Quoted 8 lines
(quoted diff) +2
-0@@ -1359,10 +1369,16 @@ int journal_flush(journal_t *journal) spin_lock(&journal->j_list_lock); while (!err && journal->j_checkpoint_transactions != NULL) { spin_unlock(&journal->j_list_lock); + mutex_lock(&journal->j_checkpoint_mutex); err = log_do_checkpoint(journal); + mutex_unlock(&journal->j_checkpoint_mutex); spin_lock(&journal->j_list_lock);
Honza
Signature · 2 lines
Jan Kara <[email protected]> SUSE Labs, CR