Serious problems with HFS+
I've noticed a few problems with HFS+ support in recent kernels on another user's machine running Ubuntu (Warty) running 2.6.8.1-3-powerpc. I'm not in a position to extensively test or fix either of these problem because of the fs tools situation so I'm just passing this on.
First, it reports inappropriate blocks to stat(2). It uses 4096 byte blocks rather than 512 byte blocks which stat callers are expecting. This seriously confuses du(1) (and me, for a bit). Looks like it may be forgetting to set s_blocksize_bits.
Second, if an HFS+ filesystem mounted via Firewire or USB becomes detached, the filesystem appears to continue working just fine. I can find on the entire tree, despite memory pressure. I can even create new files that continue to appear in directory listings! Writes to such files succeed (they're async, of course) and the typical app is none the wiser. It's only when apps attempt to read later that they encounter problems. It turns out that various apps including scp ignore IO errors on read and silently copy zero-filled files to the destination. So I got this report as "why aren't the pictures I took off my camera visible on my website?"
This is obviously a really nasty failure mode. At the very least, open of new files should fail with -EIO. Preferably the fs should force a read-only remount on IO errors. Given that the vast majority of HFS+ filesystems Linux is likely to be used with are on hotpluggable media, I think this FS should be marked EXPERIMENTAL until such integrity problems are addressed.
Having the whole directory tree seemingly pinned in memory is probably something that wants addressing as well.
Signature · 1 line
Mathematics is the supreme nostalgia of our time.
Quoted 2 lines · Matt Mackall
I've noticed a few problems with HFS+ support in recent kernels on another user's machine running Ubuntu (Warty) running 2.6.8.1-3-powerpc.
I also saw some of the things that you reported with Debian proper (sarge). Unfortunately, I haven't been able to use my HFS+ formatted system with Linux for a while, I may do so, if necessary to fix the bugs.
Quoted 2 lines
I'm not in a position to extensively test or fix either of these problem because of the fs tools situation so I'm just passing this on.
I, OTOH, can test fixes on the next weekend, since I have a full backup scheduled for this week.
Quoted 4 lines
First, it reports inappropriate blocks to stat(2). It uses 4096 byte blocks rather than 512 byte blocks which stat callers are expecting. This seriously confuses du(1) (and me, for a bit). Looks like it may be forgetting to set
s_blocksize_bits.
I had not noticed what are the size of the blocks that HFS+ seems to use, but indeed I saw both very confused du and ls outputs. I don't know what may be the cause.
Quoted 2 lines
Second, if an HFS+ filesystem mounted via Firewire or USB becomes detached, the filesystem appears to continue working just fine.
That's the only way that I am using a HFS+ fs (fia Firewire or USB), since the drive in question is in an enclosure.
Quoted 8 lines
I can find on the entire tree, despite memory pressure. I can even create new files that continue to appear in directory listings! Writes to such files succeed (they're async, of course) and the typical app is none the wiser. It's only when apps attempt to read later that they encounter problems. It turns out that various apps including scp ignore IO errors on read and silently copy zero-filled files to the destination. So I got this report as "why aren't the pictures I took off my camera visible on my website?"
I haven't seen this behaviour for lack of testing, but I will try to reproduce it with an HFS+ fs on a file mounted via loopback.
Quoted 6 lines
This is obviously a really nasty failure mode. At the very least, open of new files should fail with
-EIO. Preferably the fs should force a read-only remount on IO errors. Given that the vast majority of HFS+ filesystems Linux is likely to be used with are on hotpluggable media, I think this FS should be marked EXPERIMENTAL until such integrity problems are addressed.
I agree. This is, indeed, very scary.
Just another data point, Rogério.
Signature · 3 lines
=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-= Rogério Brito - [email protected] - http://www.ime.usp.br/~rbrito =-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=
Hi,
Quoted 10 lines · Matt Mackall
I've noticed a few problems with HFS+ support in recent kernels on another user's machine running Ubuntu (Warty) running 2.6.8.1-3-powerpc. I'm not in a position to extensively test or fix either of these problem because of the fs tools situation so I'm just passing this on.
First, it reports inappropriate blocks to stat(2). It uses 4096 byte blocks rather than 512 byte blocks which stat callers are expecting. This seriously confuses du(1) (and me, for a bit). Looks like it may be forgetting to set
s_blocksize_bits.
This should be fixed since 2.6.10.
Quoted 10 lines
Second, if an HFS+ filesystem mounted via Firewire or USB becomes detached, the filesystem appears to continue working just fine. I can find on the entire tree, despite memory pressure. I can even create new files that continue to appear in directory listings! Writes to such files succeed (they're async, of course) and the typical app is none the wiser. It's only when apps attempt to read later that they encounter problems. It turns out that various apps including scp ignore IO errors on read and silently copy zero-filled files to the destination. So I got this report as "why aren't the pictures I took off my camera visible on my website?"
HFS+ metadata is also in the page cache, so as long as everything is cached, HFS+ won't notice a problem.
Quoted 6 lines
This is obviously a really nasty failure mode. At the very least, open of new files should fail with
-EIO. Preferably the fs should force a read-only remount on IO errors. Given that the vast majority of HFS+ filesystems Linux is likely to be used with are on hotpluggable media, I think this FS should be marked EXPERIMENTAL until such integrity problems are addressed.
Currently nobody tells fs about such events, so even if I check for write errors, it can still take a while until the error is detected. I acknowledge that this is a problem, but I don't think it's a HFS+ specific problem, there is currently no standard procedure for fs what to do in such situations, so I didn't implement anything.
It would be nice if the fs would be tould about plug/unplug events, e.g. HFS+ could check the mount count to see if it was connected to a different host in the meantime and react appropriately.
Quoted 2 lines
Having the whole directory tree seemingly pinned in memory is probably something that wants addressing as well.
This actually might be a real HFS+ problem :), I have to look into it.
bye, Roman
Quoted 28 lines · Roman Zippel
Hi,
Quoted 10 lines · Matt Mackall
I've noticed a few problems with HFS+ support in recent kernels on another user's machine running Ubuntu (Warty) running 2.6.8.1-3-powerpc. I'm not in a position to extensively test or fix either of these problem because of the fs tools situation so I'm just passing this on.
First, it reports inappropriate blocks to stat(2). It uses 4096 byte blocks rather than 512 byte blocks which stat callers are expecting. This seriously confuses du(1) (and me, for a bit). Looks like it may be forgetting to set
s_blocksize_bits.This should be fixed since 2.6.10.
Quoted 10 lines
Second, if an HFS+ filesystem mounted via Firewire or USB becomes detached, the filesystem appears to continue working just fine. I can find on the entire tree, despite memory pressure. I can even create new files that continue to appear in directory listings! Writes to such files succeed (they're async, of course) and the typical app is none the wiser. It's only when apps attempt to read later that they encounter problems. It turns out that various apps including scp ignore IO errors on read and silently copy zero-filled files to the destination. So I got this report as "why aren't the pictures I took off my camera visible on my website?"
Quoted 2 lines
HFS+ metadata is also in the page cache, so as long as everything is cached, HFS+ won't notice a problem.
It's failing to notice errors at sync time or when such pages get flushed due to memory pressure.
Quoted 9 lines
Quoted 6 lines
This is obviously a really nasty failure mode. At the very least, open of new files should fail with
-EIO. Preferably the fs should force a read-only remount on IO errors. Given that the vast majority of HFS+ filesystems Linux is likely to be used with are on hotpluggable media, I think this FS should be marked EXPERIMENTAL until such integrity problems are addressed.Currently nobody tells fs about such events, so even if I check for write errors, it can still take a while until the error is detected.
It should catch up within the flush interval or at the next sync, at least. And then fail all further writes. Consider the scenario of a user sitting at their laptop when a power glitch offlines their external drive. They can copy files onto it for the next hour, delete the originals and be completely unaware that anything is wrong unless they happen to check dmesg.
Quoted 3 lines
It would be nice if the fs would be tould about plug/unplug events, e.g. HFS+ could check the mount count to see if it was connected to a different host in the meantime and react appropriately.
An FS-level callback when the underlying block device went for a walk would be a nice hook just about everywhere. And at least a start on the problem.
Signature · 1 line
Mathematics is the supreme nostalgia of our time.