New: hfsplus caused data loss

Bugme-new Andrew Morton 4 participants 5 messages Aug 20, 2009 – Aug 21, 2009 1 message not in the archive VFS

lore

Share & tools

lore

A message that is not in the archive (sent off-list or not yet received): [email protected]/

Andrew Morton

author

2009-08-20 22:03 UTC #1 lore raw

(switched to email. Please respond via emailed reply-to-all, not via the bugzilla web interface).

Quoted 70 lines · On Thu, 20 Aug 2009 02:17:21 GMT [email protected] wrote:

http://bugzilla.kernel.org/show_bug.cgi?id=14021
Summary: hfsplus caused data loss
Product: File System
Version: 2.5
Kernel Version: 2.6.31-rc5
Platform: All
OS/Version: Linux
Tree: Mainline
Status: NEW
Severity: normal
Priority: P1
Component: HFS/HFSPLUS
AssignedTo: [email protected]
ReportedBy: [email protected]
Regression: Yes
Hi.
I am trying to package a new version of Apple's own fsck/mkfs for HFS+ filesystems so that they can be used in Linux as a way to transfer data among computers, but I had quite a surprise when I was using the hfsplus module.
I just created a 100MB file with nulls (dd if=/dev/null ...) and I created an HFS+ filesystem on it.
Then, I loop mounted it and tried to use it a little bit (in particular, applying patches with quilt on a source tree). The commands spit some errors about not being able to create links (I never had that problem before) and the directory where I was became empty!
Furthermore, here is a quite, quite strange directory listing:
,----
| rbrito@chagas:/media/usb7$ ls -lAF
| ls: hfsprogs_332.14-7.diff.gz: No such file or directory
| ls: hfsprogs_332.14-7.dsc: No such file or directory
| ls: hfsprogs_332.14.orig.tar.gz: No such file or directory
| ls: hfsprogs_332.18-1.diff.gz: No such file or directory
| ls: hfsprogs_332.18-1.dsc: No such file or directory
| ls: hfsprogs_332.18-1_amd64.changes: No such file or directory
| ls: hfsprogs_332.18-1_amd64.deb: No such file or directory
| ls: hfsprogs_332.18.orig.tar.gz: No such file or directory
| total 1636
| drwxr-xr-x 1 rbrito rbrito     39 Aug 17 08:13 hfsprogs-332.14/
| drwxr-xr-x 1 rbrito rbrito     39 Aug 17 08:13 hfsprogs-332.14/
| drwxr-xr-x 1 rbrito rbrito     45 Aug 17 15:30 hfsprogs-332.18/
| -rw-r--r-- 1 rbrito rbrito  35609 Aug 17 08:13 hfsprogs_332.14-7.diff.gz
| -rw-r--r-- 1 rbrito rbrito   1193 Aug 17 08:13 hfsprogs_332.14-7.dsc
| -rw-r--r-- 1 rbrito rbrito 714035 Aug 17 08:13 hfsprogs_332.14.orig.tar.gz
| -rw-r--r-- 1 rbrito rbrito  35342 Aug 17 15:26 hfsprogs_332.18-1.diff.gz
| -rw-r--r-- 1 rbrito rbrito    954 Aug 17 15:26 hfsprogs_332.18-1.dsc
| -rw-r--r-- 1 rbrito rbrito   2148 Aug 17 15:26
hfsprogs_332.18-1_amd64.changes
| -rw-r--r-- 1 rbrito rbrito 135398 Aug 17 15:26 hfsprogs_332.18-1_amd64.deb
| -rw-r--r-- 1 rbrito rbrito 732449 Aug 17 08:35 hfsprogs_332.18.orig.tar.gz
| rbrito@chagas:/media/usb7$
`----
I am using kernel 2.6.31-rc5-1rb.pre6 (that is, rc5 with git updates up to one or two days before rc6).
There are no messages in the dmesg, besides these:
,----
| [30991.501804] loop: module loaded
| [30991.513337] hfs: create hidden dir...
| [39897.867830] hfs: create hidden dir...
| [39960.061622] hfs: splitting index node...

Quoted 7 lines

`----
No stack traces, no nothing. Oh, I still have the disk image that I created, if it is of any interest.
Please let me know if I can provide any further information.

Gee. Nobody really does much maintenance work on hfs/hfsplus any more. I cc'ed the ppc guys as I expect that most HFS users are over there.

It seems like a pretty gross failure - others should be hitting it.

I wonder if it's a weird interaction with the loop driver.

Rogério Brito

2009-08-20 22:17 UTC #2 lore raw

Hi, Andrew.

Quoted 9 lines · Andrew Morton

Quoted 7 lines · On Thu, 20 Aug 2009 02:17:21 GMT [email protected] wrote:
,----
| [30991.501804] loop: module loaded
| [30991.513337] hfs: create hidden dir...
| [39897.867830] hfs: create hidden dir...
| [39960.061622] hfs: splitting index node...
`----

Quoted 4 lines

Quoted 2 lines
No stack traces, no nothing. Oh, I still have the disk image that I created, if it is of any interest.
Gee. Nobody really does much maintenance work on hfs/hfsplus any more.

Right.

I think that I could run some stress tests (say, like the exhaustive regression tests that the ntfs-3g developers use), just to get some basic functionality on HFS+ working.

Quoted 1 line

I cc'ed the ppc guys as I expect that most HFS users are over there.

Thanks once again, Andrew.

Quoted 3 lines

It seems like a pretty gross failure - others should be hitting it.
I wonder if it's a weird interaction with the loop driver.

Just to make it clear: the reason why I loop-mounted the filesystem was just to work with a low-risk situation and submit the image to the recently "ported" fsck, just to get a little bit more confidence that I didn't mess with the userspace side of things.

But, then, this surprise.

I had some minor inconsistencies with the HFS+ driver in the past, but they were fixed by running the fsck.hfsplus that I packaged and I didn't bother reporting it before. My mistake. :-(

Thanks, Rogério Brito.

Signature · 7 lines

Rogério Brito : rbrito@{mackenzie,ime.usp}.br : GPG key 1024D/7C2CAEB8
http://www.ime.usp.br/~rbrito : http://meusite.mackenzie.com.br/rbrito
Projects: algorithms.berlios.de : lame.sf.net : vrms.alioth.debian.org
_______________________________________________
Linuxppc-dev mailing list
[email protected]
https://lists.ozlabs.org/listinfo/linuxppc-dev

Brad Boyer

2009-08-21 06:19 UTC #3 lore raw

In reply to Andrew Morton

Quoted 85 lines · Andrew Morton

(switched to email. Please respond via emailed reply-to-all, not via the bugzilla web interface).
Quoted 70 lines · On Thu, 20 Aug 2009 02:17:21 GMT [email protected] wrote:
http://bugzilla.kernel.org/show_bug.cgi?id=14021
Summary: hfsplus caused data loss
Product: File System
Version: 2.5
Kernel Version: 2.6.31-rc5
Platform: All
OS/Version: Linux
Tree: Mainline
Status: NEW
Severity: normal
Priority: P1
Component: HFS/HFSPLUS
AssignedTo: [email protected]
ReportedBy: [email protected]
Regression: Yes
Hi.
I am trying to package a new version of Apple's own fsck/mkfs for HFS+ filesystems so that they can be used in Linux as a way to transfer data among computers, but I had quite a surprise when I was using the hfsplus module.
I just created a 100MB file with nulls (dd if=/dev/null ...) and I created an HFS+ filesystem on it.
Then, I loop mounted it and tried to use it a little bit (in particular, applying patches with quilt on a source tree). The commands spit some errors about not being able to create links (I never had that problem before) and the directory where I was became empty!
Furthermore, here is a quite, quite strange directory listing:
,----
| rbrito@chagas:/media/usb7$ ls -lAF
| ls: hfsprogs_332.14-7.diff.gz: No such file or directory
| ls: hfsprogs_332.14-7.dsc: No such file or directory
| ls: hfsprogs_332.14.orig.tar.gz: No such file or directory
| ls: hfsprogs_332.18-1.diff.gz: No such file or directory
| ls: hfsprogs_332.18-1.dsc: No such file or directory
| ls: hfsprogs_332.18-1_amd64.changes: No such file or directory
| ls: hfsprogs_332.18-1_amd64.deb: No such file or directory
| ls: hfsprogs_332.18.orig.tar.gz: No such file or directory
| total 1636
| drwxr-xr-x 1 rbrito rbrito     39 Aug 17 08:13 hfsprogs-332.14/
| drwxr-xr-x 1 rbrito rbrito     39 Aug 17 08:13 hfsprogs-332.14/
| drwxr-xr-x 1 rbrito rbrito     45 Aug 17 15:30 hfsprogs-332.18/
| -rw-r--r-- 1 rbrito rbrito  35609 Aug 17 08:13 hfsprogs_332.14-7.diff.gz
| -rw-r--r-- 1 rbrito rbrito   1193 Aug 17 08:13 hfsprogs_332.14-7.dsc
| -rw-r--r-- 1 rbrito rbrito 714035 Aug 17 08:13 hfsprogs_332.14.orig.tar.gz
| -rw-r--r-- 1 rbrito rbrito  35342 Aug 17 15:26 hfsprogs_332.18-1.diff.gz
| -rw-r--r-- 1 rbrito rbrito    954 Aug 17 15:26 hfsprogs_332.18-1.dsc
| -rw-r--r-- 1 rbrito rbrito   2148 Aug 17 15:26
hfsprogs_332.18-1_amd64.changes
| -rw-r--r-- 1 rbrito rbrito 135398 Aug 17 15:26 hfsprogs_332.18-1_amd64.deb
| -rw-r--r-- 1 rbrito rbrito 732449 Aug 17 08:35 hfsprogs_332.18.orig.tar.gz
| rbrito@chagas:/media/usb7$
`----
I am using kernel 2.6.31-rc5-1rb.pre6 (that is, rc5 with git updates up to one or two days before rc6).
There are no messages in the dmesg, besides these:
,----
| [30991.501804] loop: module loaded
| [30991.513337] hfs: create hidden dir...
| [39897.867830] hfs: create hidden dir...
| [39960.061622] hfs: splitting index node...
Quoted 7 lines
`----
No stack traces, no nothing. Oh, I still have the disk image that I created, if it is of any interest.
Please let me know if I can provide any further information.

Quoted 6 lines

Gee. Nobody really does much maintenance work on hfs/hfsplus any more. I cc'ed the ppc guys as I expect that most HFS users are over there.
It seems like a pretty gross failure - others should be hitting it.
I wonder if it's a weird interaction with the loop driver.

I can explain the ls output in a very generic sense. This is what you get if the lookup fails on a name returned by a readdir. I suspect that creating hard links is failing in some way. In particular, the "hidden dir" mentioned in the log is used to save the real files for hard links.

As a quick explanation, HFS+ doesn't have a real concept of hard links. Apple hacked it in after the fact. The way it works is that there is a special hidden directory, and the real catalog entries that match the real names are just references to the hidden file. Both HFS and HFS+ have no separate notion of directory entries and inode data. The metadata is stored in records keyed by file name.

Hard links were added after I stopped working on the HFS+ code, so I don't know it in detail. I know it works for reading hard links, but I never actually tried creating more links.

Brad Boyer
[email protected]

Benjamin Herrenschmidt

2009-08-21 07:30 UTC #4 lore raw

In reply to Andrew Morton

Quoted 2 lines · Andrew Morton

(switched to email. Please respond via emailed reply-to-all, not via the bugzilla web interface).

Is Roman Zippel still around ? (I added him to the CC list). AFAIK, He maintains HFS and HFS+ (and was the last one to do any major work on them).

Cheers,
Ben.

Quoted 93 lines

Quoted 70 lines · On Thu, 20 Aug 2009 02:17:21 GMT [email protected] wrote:
http://bugzilla.kernel.org/show_bug.cgi?id=14021
Summary: hfsplus caused data loss
Product: File System
Version: 2.5
Kernel Version: 2.6.31-rc5
Platform: All
OS/Version: Linux
Tree: Mainline
Status: NEW
Severity: normal
Priority: P1
Component: HFS/HFSPLUS
AssignedTo: [email protected]
ReportedBy: [email protected]
Regression: Yes
Hi.
I am trying to package a new version of Apple's own fsck/mkfs for HFS+ filesystems so that they can be used in Linux as a way to transfer data among computers, but I had quite a surprise when I was using the hfsplus module.
I just created a 100MB file with nulls (dd if=/dev/null ...) and I created an HFS+ filesystem on it.
Then, I loop mounted it and tried to use it a little bit (in particular, applying patches with quilt on a source tree). The commands spit some errors about not being able to create links (I never had that problem before) and the directory where I was became empty!
Furthermore, here is a quite, quite strange directory listing:
,----
| rbrito@chagas:/media/usb7$ ls -lAF
| ls: hfsprogs_332.14-7.diff.gz: No such file or directory
| ls: hfsprogs_332.14-7.dsc: No such file or directory
| ls: hfsprogs_332.14.orig.tar.gz: No such file or directory
| ls: hfsprogs_332.18-1.diff.gz: No such file or directory
| ls: hfsprogs_332.18-1.dsc: No such file or directory
| ls: hfsprogs_332.18-1_amd64.changes: No such file or directory
| ls: hfsprogs_332.18-1_amd64.deb: No such file or directory
| ls: hfsprogs_332.18.orig.tar.gz: No such file or directory
| total 1636
| drwxr-xr-x 1 rbrito rbrito     39 Aug 17 08:13 hfsprogs-332.14/
| drwxr-xr-x 1 rbrito rbrito     39 Aug 17 08:13 hfsprogs-332.14/
| drwxr-xr-x 1 rbrito rbrito     45 Aug 17 15:30 hfsprogs-332.18/
| -rw-r--r-- 1 rbrito rbrito  35609 Aug 17 08:13 hfsprogs_332.14-7.diff.gz
| -rw-r--r-- 1 rbrito rbrito   1193 Aug 17 08:13 hfsprogs_332.14-7.dsc
| -rw-r--r-- 1 rbrito rbrito 714035 Aug 17 08:13 hfsprogs_332.14.orig.tar.gz
| -rw-r--r-- 1 rbrito rbrito  35342 Aug 17 15:26 hfsprogs_332.18-1.diff.gz
| -rw-r--r-- 1 rbrito rbrito    954 Aug 17 15:26 hfsprogs_332.18-1.dsc
| -rw-r--r-- 1 rbrito rbrito   2148 Aug 17 15:26
hfsprogs_332.18-1_amd64.changes
| -rw-r--r-- 1 rbrito rbrito 135398 Aug 17 15:26 hfsprogs_332.18-1_amd64.deb
| -rw-r--r-- 1 rbrito rbrito 732449 Aug 17 08:35 hfsprogs_332.18.orig.tar.gz
| rbrito@chagas:/media/usb7$
`----
I am using kernel 2.6.31-rc5-1rb.pre6 (that is, rc5 with git updates up to one or two days before rc6).
There are no messages in the dmesg, besides these:
,----
| [30991.501804] loop: module loaded
| [30991.513337] hfs: create hidden dir...
| [39897.867830] hfs: create hidden dir...
| [39960.061622] hfs: splitting index node...
Quoted 7 lines
`----
No stack traces, no nothing. Oh, I still have the disk image that I created, if it is of any interest.
Please let me know if I can provide any further information.
Gee. Nobody really does much maintenance work on hfs/hfsplus any more. I cc'ed the ppc guys as I expect that most HFS users are over there.
It seems like a pretty gross failure - others should be hitting it.
I wonder if it's a weird interaction with the loop driver.
_______________________________________________
Linuxppc-dev mailing list
[email protected]
https://lists.ozlabs.org/listinfo/linuxppc-dev

Andrew Morton

author

2009-08-21 07:37 UTC #5 lore raw

Quoted 7 lines · On Fri, 21 Aug 2009 17:29:01 +1000 Benjamin Herrenschmidt…

Quoted 2 lines · Andrew Morton
(switched to email. Please respond via emailed reply-to-all, not via the bugzilla web interface).
Is Roman Zippel still around ? (I added him to the CC list). AFAIK, He maintains HFS and HFS+ (and was the last one to do any major work on them).

I don't think I've heard from Roman since January of this year.