Question on the WriteCache / WriteBarrier FAQ entry
Hello,
I try to fully understand this entry:
"Many drives use a write back cache in order to speed up the performance of writes. However, there are conditions such as power failure when the write cache memory is never flushed to the actual disk. This causes problems for XFS and journaled filesystems in general because they rely on knowing when a write has completed to the disk. They need to know that the log information has made it to disk before allowing metadata to go to disk. When the metadata makes it to disk then the tail of the log can move. So if the writes never make it to the physical disk, then the ordering is violated and the log and metadata can be lost, resulting in filesystem corruption."
I have problems with: "When the metadata makes it to disk then the tail of the log can move". What does that mean exactly?
What I imagine is this: XFS write transaction to its log and the log grows. When writing the meta data changes of a complete transaction XFS removes it from the log. Now when the metadata changes of a transaction has been written completely but the transaction itself has not, it may happen that a transaction is removed from the on disk log before it has been written. But even when this does not happen there are metadata changes on disk that the log doesn't know about.
So there are two situations where unordered writes can make a journalling filesystem corrupt:
1) Metadata make it to disk before the transaction that belongs to them => There are metadata changes that XFS doesn't know about.
2) A transaction might be deleted from the log before it has been written => This leads to a corrupted log.
Is that correct and complete? Please give feedback.
You may use any of my text here to update / clarify the FAQ ;-)
Regards,
Signature · 2 lines
Martin 'Helios' Steigerwald - http://www.Lichtvoll.de GPG: 03B0 0D6C 0040 0710 4AFA B82F 991B EAAC A599 84C7
Quoted 8 lines · Martin Steigerwald
Hello,
I try to fully understand this entry:
"Many drives use a write back cache in order to speed up the performance of writes. However, there are conditions such as power failure when the write cache memory is never flushed to the actual disk. This causes problems for XFS and journaled filesystems in general because they rely on knowing when a write has completed to the disk. They need to know that the log information has made it to disk before allowing metadata to go to disk. When the metadata makes it to disk then the tail of the log can move. So if the writes never make it to the physical disk, then the ordering is violated and the log and metadata can be lost, resulting in filesystem corruption."
I have problems with: "When the metadata makes it to disk then the tail of the log can move". What does that mean exactly?
Too vague for you, was it? ;-))
http://oss.sgi.com/projects/xfs/design_docs/xfsdocs93_pdf/trans.pdf
has a good description in its 7 steps of a transaction.
Quoted 1 line
What I imagine is this: XFS write transaction to its log and the log grows. When writing the meta data changes of a complete transaction XFS removes it from the log.
Pretty much.
The log is a fixed size and it wraps around with each basic block (512 bytes) having an embedded wrap# (or cycle#).
We consider the head of the log as the point where a new transaction can go and the tail of the log as the last transaction whose metadata is still outstanding.
Between the tail and the head are the active items which need to be replayed on log recovery.
When we get an io callback for a metadata write, we remove the metadata items from a list of active items (AIL) and the tail pointer is based on the minimum entry (by log sequence #) in the AIL. So the tail is effectively moved on so that we know we can write over these inactive items in the ondisk log and
can reclaim some space for the new ones to come.
Also we know that on recovery we will not look at this old transaction.
Quoted 1 line
Now when the metadata changes of a transaction has been written completely but the transaction itself has not,
Which should not happen as we don't allow metadata to be written until the associated transaction has made it to disk (see the 7 steps). But if something went wrong and it did happen...
Quoted 1 line
it may happen that a transaction is removed from the on disk log before it has been written.
Things would get confused.
I guess it might try to find the item which is not in the AIL as the AIL gets updated on transaction callback.
Quoted 1 line
But even when this does not happen there are metadata changes on disk that the log doesn't know about.
Yep, it's not good when we expect log replay to do the right thing.
Quoted 4 lines
So there are two situations where unordered writes can make a journalling filesystem corrupt:
1) Metadata make it to disk before the transaction that belongs to them => There are metadata changes that XFS doesn't know about.
Well, that aren't reflected in the log.
Quoted 2 lines
2) A transaction might be deleted from the log before it has been written => This leads to a corrupted log.
A transaction deleted before its metadata has made it to disk, yes. A transaction might be deleted (tail# moved on) because we believe it's metadata has made it to disk when it really hasn't (was in the write cache) in which case we need the transaction if recovery is required and we don't have it.
I don't know if I want to try to go thru all the bad things that can happen and see how the code would handle it.
Cheers,
Tim.