Skip to contentSkip to conversation

hfsplus: introduce journal replay functionality

PATCH Superseded Vyacheslav Dubeyko 2 participants 3 messages Feb 12, 2014 – Feb 24, 2014 VFS

v3 of 4 Superseded Latest is v4 →

lore
Share & tools
lore
#0 lore raw

v3 cover hfsplus: introduce journal replay functionality

Hi,

This is third version of patchset that implements journal replay functionality in HFS+ file system driver.

v2->v3
* Fix unresolvable references to __udivdi3 and __umoddi3 on i386 (Andrew Morton)
* Fix printk mismatch warnings on i386 (Andrew Morton)
* Fix bisection holes (implicit declarations and warnings) on i386 (Andrew Morton)

v1->v2
* Fix error messages incorrectness (Hin-Tak Leung)
* Fix issue with remount "mount -o remount,rw <device> <folder>"
* Rework "force" and "norecovery" mount options support (Hin-Tak Leung)

Technical Note TN1150:
"The purpose of the journal is to ensure that when a group of
related changes are being made, that either all of those changes are actually made, or none of them are made. This is done by gathering up all of the changes, and storing them in a separate place (in the journal). Once the journal copy of the changes is completely written to disk, the changes can actually be written to their normal locations on disk. If a failure happens at that time, the changes can simply be copied from the journal to their normal locations. If a failure happens when the changes are being written to the journal, but before they are marked complete, then all of those changes are ignored."

"A group of related changes is called a transaction. When all of
the changes of a transaction have been written to their normal locations on disk, that transaction has been committed, and is removed from the journal. The journal may contain several transactions. Copying changes from all transactions to their normal locations on disk is called replaying the journal."

"In order to replay the journal, an implementation just loops
over the transactions, copying each individual block in the transaction from the journal to its proper location on the volume. Once those blocks have been flushed to the media (not just the driver!), it may update the journal header to remove the transactions."

"Here are the steps to replay the journal:
1. Read the volume header into variable vhb. The volume may have an HFS wrapper; if so, you will need to use it to determine the location of the volume header.
2. Test the kHFSVolumeJournaledBit in the attributes field of the volume header. If it is not set, there is no journal to replay, and you are done.
3. Read the journal info block from the allocation block number vhb.journalInfoBlock, into variable jib.
4. If kJIJournalNeedsInitMask is set in jib.flags, the journal was never initialized, so there is no journal to replay.
5. Verify that kJIJournalInFSMask is set and kJIJournalOnOtherDeviceMask is clear in jib.flags.
6. Read the journal header at jib.offset bytes from the start of the volume, and place it in variable jhdr.
7. If jhdr.start equals jhdr.end, the journal does not have any transactions, so there is nothing to replay.
8. Set the current offset in the journal (typically a local variable) to the start of the journal buffer, jhdr.start.
9. While jhdr.start does not equal jhdr.end, perform the following steps:
1. Read a block list header of jhdr.blhdr_size bytes from the current offset in the journal into variable blhdr.
2. For each block in bhdr.binfo[1] to bhdr.binfo[blhdr.num_blocks], inclusive, copy bsize bytes from the current offset in the journal to sector bnum on the volume (to byte offset bnum*jdhr.jhdr_size). Remember that jhdr_size is the size of a sector, in bytes.
3. If bhdr.binfo[0].next is zero, you have completed the last block list of the current transaction; set jhdr.start to the current offset in the journal."

It was reworked concept of "force" mount option. It is used to force write access to volumes that are marked as locked, only. This option doesn't work for journaled volumes with non-empty journal. HFS+ drive tries to replay journal and ignores forcing write access in the case of journal replay failure. It is possible to access HFS+ volume in read-only mode only for such case. The "force" option is ignored for the case of "norecovery" option request also.

With the best regards,
Vyacheslav Dubeyko.

---
 Documentation/filesystems/hfsplus.txt |   16 +-
 fs/hfsplus/Makefile                   |    3 +-
 fs/hfsplus/hfsplus_fs.h               |   33 +-
 fs/hfsplus/hfsplus_raw.h              |   53 +-
 fs/hfsplus/journal.c                  | 1221 +++++++++++++++++++++++++++++++++
 fs/hfsplus/options.c                  |   17 +-
 fs/hfsplus/part_tbl.c                 |    4 +-
 fs/hfsplus/super.c                    |  105 ++-
 fs/hfsplus/wrapper.c                  |   28 +-
 9 files changed, 1441 insertions(+), 39 deletions(-)
Signature · 1 line
1.7.9.5
#1 lore raw
Quoted 4 lines · Vyacheslav Dubeyko

Hi,

This is third version of patchset that implements journal replay functionality in HFS+ file system driver.

Causes file system errors after trying to replay journal on iPod Nano 8GB with 4K sector size. Details follow. My implementation "[PATCH]
hfsplus: add journal replay" works fine.

fsck_hfs output:
Verifying volume disk2s2
Checking file system
Checking Journaled HFS Plus volume.
Checking extents overflow file.Checking catalog file. Checking multi-linked files.Checking catalog hierarchy. Checking extended attributes file.
Unused node is not erased (node = 53)
Checking volume bitmap.
Volume bitmap needs minor repair for under-allocation Checking volume information.
Volume header needs minor repairThe volume Sergei Antonov’s iPod was found corrupt and needs to be repaired.
Error: This disk needs to be repaired. Click Repair Disk.

In system log:

Feb 24 10:45:31 sas-MacBook-Pro kernel[0]: jnl: disk2s2: do_jnl_io:
strategy err 0x6
Feb 24 10:45:31 sas-MacBook-Pro kernel[0]: jnl: disk2s2:
end_transaction: only wrote 0 of 147456 bytes to the journal!
Feb 24 10:45:31 sas-MacBook-Pro kernel[0]: hfs: unmount initiated on
Sergei Antonovs iPod on device disk2s2
Feb 24 10:45:31 sas-MacBook-Pro kernel[0]: jnl: disk2s2: close:

journal 0xffffff8015507a80, is invalid. aborting outstanding transactions
--
To unsubscribe from this list: send the line "unsubscribe linux-fsdevel" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html

#2 lore raw
Quoted 10 lines · Sergei Antonov
Quoted 4 lines · Vyacheslav Dubeyko

Hi,

This is third version of patchset that implements journal replay functionality in HFS+ file system driver.

Causes file system errors after trying to replay journal on iPod Nano 8GB with 4K sector size. Details follow. My implementation "[PATCH]
hfsplus: add journal replay" works fine.

If you have found some bug then you can report about it. And I'll fix it after reproducing. But it doesn't mean that it needs to suggest the patch instead of mine. It needs to cooperate. And nobody prevented you from sharing your patch before mine. I think that suggesting a patch after existing of another one is dishonest way and simply cheating.

Yesterday I published fourth version of the patch. Do you have the same result for this version? Could you describe in more details how I can reproduce the issue?

Thanks,
Vyacheslav Dubeyko.

Quoted 23 lines

fsck_hfs output:
Verifying volume disk2s2
Checking file system
Checking Journaled HFS Plus volume.
Checking extents overflow file.Checking catalog file. Checking multi-linked files.Checking catalog hierarchy. Checking extended attributes file.
Unused node is not erased (node = 53)
Checking volume bitmap.
Volume bitmap needs minor repair for under-allocation Checking volume information.
Volume header needs minor repairThe volume Sergei Antonov’s iPod was found corrupt and needs to be repaired.
Error: This disk needs to be repaired. Click Repair Disk.

In system log:

Feb 24 10:45:31 sas-MacBook-Pro kernel[0]: jnl: disk2s2: do_jnl_io:
strategy err 0x6
Feb 24 10:45:31 sas-MacBook-Pro kernel[0]: jnl: disk2s2:
end_transaction: only wrote 0 of 147456 bytes to the journal!
Feb 24 10:45:31 sas-MacBook-Pro kernel[0]: hfs: unmount initiated on
Sergei Antonovs iPod on device disk2s2
Feb 24 10:45:31 sas-MacBook-Pro kernel[0]: jnl: disk2s2: close:
Quoted 6 lines

journal 0xffffff8015507a80, is invalid. aborting outstanding transactions
--
To unsubscribe from this list: send the line "unsubscribe linux-fsdevel" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html

--
To unsubscribe from this list: send the line "unsubscribe linux-fsdevel" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html