Skip to contentSkip to conversation

A few thoughts on hfsplus dev (Re: [PATCH v2] hfsplus: add journal replay)

DISCUSSION Hin-Tak Leung 2 participants 2 messages Apr 4, 2014 – Apr 6, 2014 VFS

lore
Share & tools
lore
#0 lore raw

Hi Sergei,

------------------------------

Quoted 1 line · GMT Sergei Antonov
Quoted 1 line

I am ready for further convincing efforts.

As I said, I am happy to review your work, and I hope you and Vyacheslav can work together. However, I wish to point out that Vyacheslav has a track record going back for a few years of contributing to the hfs+ driver, and I believe there was a patch submitted a few months ago to make him the maintainer. In the hierarchy of linux kernel development, informal as it is, he is still probably one of the people that you need to convince. The way you are going about the matter, it is not contributing to getting your ideas accepted.

Quoted 4 lines
Quoted 4 lines

Though now that Vyacheslav has stopped replying in the thread there is no progress. Two compromises have been suggested by me in the thread, I'll recap them here for you or anyone who wasn't following.
1. Since collecting all transactions requires more memory (up to 12 MB in my tests, but theoretically more), I suggested a threshold. Let's say it will be 1 MB. Once collected data reaches one meg, flush it, free the buffers and go on collecting.
2. A very simple suggestions that makes my logic closer to Vyacheslav's one-by-one transaction processing approach. In case we encounter a corrupted transaction, replay (good) transactions collected so far. Corrupted transaction is a rare thing (caused by cosmic rays!), but partial replay looks sane and not a big change to the code.

...

Quoted 3 lines

Could you clarify "transaction based"? Seriously. Because since there are transactions in HFS+ journal, all replay implementations are transaction-based in a way.

Transactions are what they are. The journal is an ordered list of transactions. Each of them has a checksum for integrity, and is treated as one unit of change. Theorectically, you should apply one after the other in the correct order, until the list is exhausted, or until you encounter the first of one such unit that does not pass sanity checks (the checksum, and other out-of-bound conditions, etc).

How you go about it, or buffering a number of transactions for performance reasons, is up to you (or the implementor), *up to a certain point*. I suggest you supply a patch on top of Vyacheslav's patch set to demonstrate how you want to go about it. It is all rather vague and not constructive going around in circle about words and meaning of words. So I suggest that you show us the code to do it,
how you would like to modify Vyacheslav's to work in your way.

Be really careful about saying things like "all replay implementations". There is no such thing as "all" replay implementations: there is an official specification
- TN1150, and there is an official implementation, that inside xnu and diskdev_cmds from Apple. Anything else is about you making a Sergei's file system that's not quite HFS+.

If you read/write the journal qualitatively different from how Apple does it, you are by definition wrong, even if your implemention of journalling does it faster and "cleverer" than Apple's.

Hin-Tak

#1 lore raw
Quoted 5 lines · Hin-Tak Leung

Hi Sergei,

------------------------------
On Fri, Mar 28, 2014 1:42 PM GMT Sergei Antonov wrote:

Quoted 8 lines
Quoted 1 line
Quoted 1 line

I am ready for further convincing efforts.

As I said, I am happy to review your work, and I hope you and Vyacheslav can work together. However, I wish to point out that Vyacheslav has a track record going back for a few years of contributing to the hfs+ driver, and I believe there was a patch submitted a few months ago to make him the maintainer. In the hierarchy of linux kernel development, informal as it is, he is still probably one of the people that you need to convince. The way you are going about the matter, it is not contributing to getting your ideas accepted.

This saddens me. I thought I just asked questions.

Quoted 32 lines
Quoted 4 lines
Quoted 4 lines

Though now that Vyacheslav has stopped replying in the thread there is no progress. Two compromises have been suggested by me in the thread, I'll recap them here for you or anyone who wasn't following.
1. Since collecting all transactions requires more memory (up to 12 MB in my tests, but theoretically more), I suggested a threshold. Let's say it will be 1 MB. Once collected data reaches one meg, flush it, free the buffers and go on collecting.
2. A very simple suggestions that makes my logic closer to Vyacheslav's one-by-one transaction processing approach. In case we encounter a corrupted transaction, replay (good) transactions collected so far. Corrupted transaction is a rare thing (caused by cosmic rays!), but partial replay looks sane and not a big change to the code.

...

Quoted 3 lines

Could you clarify "transaction based"? Seriously. Because since there are transactions in HFS+ journal, all replay implementations are transaction-based in a way.

Transactions are what they are. The journal is an ordered list of transactions. Each of them has a checksum for integrity, and is treated as one unit of change. Theorectically, you should apply one after the other in the correct order, until the list is exhausted, or until you encounter the first of one such unit that does not pass sanity checks (the checksum, and other out-of-bound conditions, etc).

How you go about it, or buffering a number of transactions for performance reasons, is up to you (or the implementor), *up to a certain point*. I suggest you supply a patch on top of Vyacheslav's patch set to demonstrate how you want to go about it. It is all rather vague and not constructive going around in circle about words and meaning of words. So I suggest that you show us the code to do it,
how you would like to modify Vyacheslav's to work in your way.

Be really careful about saying things like "all replay implementations". There is no such thing as "all" replay implementations: there is an official specification
- TN1150, and there is an official implementation, that inside xnu and diskdev_cmds from Apple. Anything else is about you making a Sergei's file system that's not quite HFS+.

Quoted 3 lines

If you read/write the journal qualitatively different from how Apple does it, you are by definition wrong, even if your implemention of journalling does it faster and "cleverer" than Apple's.

Apple also first collects data from all transactions and only after that writes data to disk.
See http://www.opensource.apple.com/source/xnu/xnu-2422.1.72/bsd/vfs/vfs_journal.c
(search for "coalesce").