PROBLEM: hfsplus filesystem allows opendir() on plain files
1. hfsplus filesystem allows opendir() on plain files.
2. When opendir() is called on a plain file stored in an hfsplus filesystem, opendir() fails to return 0 and set errno to ENOTDIR.
3. filesystem, hfsplus
4. Linux version 2.6.27-7-generic (buildd@palmer) (gcc version 4.3.2 (Ubuntu 4.3.2-1ubuntu11) ) #1 SMP Tue Nov 4 19:33:20 UTC 2008
5. unknown
6. n/a
7. Try the following program on files and directories first under a mountpoint for an ext3 partition then under a mountpoint for an hfsplus partition.
///////////////////////////////////
#include <dirent.h>
#include <stdio.h>
#include <errno.h>
// usage: opendir_test <file>|<directory>
int main( int argc, char** argv )
{
DIR* result = opendir( argv[1] );
printf("ENOTDIR is 0x%x\n", ENOTDIR );
printf("opendir '%s' returned 0x%x, errno 0x%x\n", argv[1], result,
errno );
return 0;
}//////////////////////////////////
I get the following output:
damian@damian-laptop:~/code/opendir$ mkdir test_directory damian@damian-laptop:~/code/opendir$ echo hello > test_file.txt damian@damian-laptop:~/code/opendir$ ./opendir_test test_directory ENOTDIR is 0x14 opendir 'test_directory' returned 0x88f6008, errno 0x0 damian@damian-laptop:~/code/opendir$ ./opendir_test test_file.txt ENOTDIR is 0x14 opendir 'test_file.txt' returned 0x0, errno 0x14 # /osx is an HFSPLUS mountpoint damian@damian-laptop:~/code/opendir$ cd /media/osx # mkdir test_directory && echo hello > test_file.txt performed in OSX damian@damian-laptop:/media/osx$ ./opendir_test test_directory ENOTDIR is 0x14 opendir 'test_directory' returned 0x919d008, errno 0x0 # erroneous result follows damian@damian-laptop:/media/osx$ ./opendir_test test_file.txt ENOTDIR is 0x14 opendir 'test_file.txt' returned 0x9a23008, errno 0x0 damian@damian-laptop:/media/osx$
8. If some fields are empty or look unusual you may have an old version. Compare to the current minimal requirements in Documentation/Changes.
Linux damian-laptop 2.6.27-7-generic #1 SMP Tue Nov 4 19:33:20 UTC 2008 i686 GNU/Linux
Gnu C 4.3.2-1ubuntu11) Gnu make 3.81 binutils Binutils util-linux 2.14 mount support module-init-tools 3.3-pre11 e2fsprogs 1.41.3 reiserfsprogs 3.6.19 pcmciautils 014 Linux C Library 2.8.90 Dynamic linker (ldd) 2.8.90 Procps 3.2.7 Net-tools 1.60 Kbd 1.14.1 Sh-utils 6.10 udev 124 wireless-tools 29 Modules Loaded ipv6 aes_i586 aes_generic nls_utf8 hfsplus i915 drm af_packet binfmt_misc btusb sco bridge stp bnep rfcomm l2cap bluetooth ppdev acpi_cpufreq cpufreq_stats cpufreq_powersave cpufreq_conservative cpufreq_userspace cpufreq_ondemand freq_table container wmi pci_slot sbs sbshc iptable_filter ip_tables x_tables sbp2 parport_pc lp parport joydev snd_hda_intel arc4 snd_pcm_oss snd_mixer_oss iTCO_wdt ecb crypto_blkcipher iTCO_vendor_support snd_pcm ath9k evdev appleir pcspkr appletouch snd_seq_dummy mac80211 snd_seq_oss cfg80211 snd_seq_midi snd_rawmidi snd_seq_midi_event snd_seq video snd_timer snd_seq_device output battery snd ac intel_agp button soundcore agpgart shpchp pci_hotplug snd_page_alloc ext3 jbd mbcache sd_mod crc_t10dif sr_mod cdrom sg ata_generic ata_piix usbhid hid pata_acpi ohci1394 libata ieee1394 scsi_mod ehci_hcd uhci_hcd dock usbcore sky2 thermal processor fan fbcon tileblit font bitblit softcursor fuse
thanks
d
Signature · 2 lines
damian stewart | skype: damiansnz | [email protected] frey | live art with machines | http://www.frey.co.nz
Quoted 4 lines · Damian Stewart
1. hfsplus filesystem allows
opendir()on plain files.2. When
opendir()is called on a plain file stored in an hfsplus filesystem,opendir()fails to return 0 and set errno to ENOTDIR.
Yes, this is a side effect of an intentional behavior. In osx, you can open "file/rsrc" to get the resource fork of "file". This behavior is emulated inside hfsplus on Linux, which means that to some degree every file acts like a directory. There isn't a readdir operation so it's not possible to list these, but it is possible to do a lookup on this specific name. The code path taken by opendir inside the kernel checks inode->i_op->lookup and returns ENOTDIR if this function pointer is not set.
We can't fix this without either eliminating this way to open the resource fork or making real changes to the generic Linux code. I'm not sure what would be the appropriate thing to do in this case. Someone on the mailing list is likely to have a strong opinion, so there are likely to be suggestions of one sort or another.
Maybe the check in the generic code to decide what is a "directory" should be changed? We could possibly check inode->i_fop->readdir.
Brad Boyer
[email protected]
Quoted 2 lines · Brad Boyer
Maybe the check in the generic code to decide what is a "directory" should be changed? We could possibly check
inode->i_fop->readdir.
Hi Brad,
Thanks for your reply.
This issue came up because at least one major project (the VLC video player) is using 'opendir()' to determine whether or not a path is a file or directory. (They probably ought to be using 'stat()', but they're not, so this issue shows up.)
I've reported it on the VLC forums but only got very dismissive replies. The few developers who posted bluntly said it was a 'kernel bug' so they weren't willing to do anything about it; which is a shame, because it leaves those VLC users with videos on HFS+ drives unable to play their videos with the latest version of VLC.
Anyways, perhaps your suggestion is the right one, in this case.
Cheers
d
Signature · 2 lines
damian stewart | skype: damiansnz | [email protected] frey | live art with machines | http://www.frey.co.nz
Quoted 4 lines · Damian Stewart
This issue came up because at least one major project (the VLC video player) is using '
opendir()' to determine whether or not a path is a file or directory. (They probably ought to be using 'stat()', but they're not, so this issue shows up.)
Unless they're already needing to open any directories they check, doing an opendir call to check a file type seems less than ideal. Hopefully they had a reason, but I know how that sort of thing can happen.
Quoted 5 lines
I've reported it on the VLC forums but only got very dismissive replies. The few developers who posted bluntly said it was a 'kernel bug' so they weren't willing to do anything about it; which is a shame, because it leaves those VLC users with videos on HFS+ drives unable to play their videos with the latest version of VLC.
Well, I actually tend to agree that it is less than ideal behavior on the part of the kernel in spite of thinking that they are doing something stupid. It really ought to work.
Quoted 1 line
Anyways, perhaps your suggestion is the right one, in this case.
We'll see what comes out of this.
Brad Boyer
[email protected]
Quoted 2 lines · Brad Boyer
We can't fix this without either eliminating this way to open the resource fork
Which sould have happened long ago. See the reiser4 discussion. You're breaking userspace semantic without a good reason.
Quoted 6 lines · Christoph Hellwig
Quoted 2 lines · Brad Boyer
We can't fix this without either eliminating this way to open the resource fork
Which sould have happened long ago. See the reiser4 discussion. You're breaking userspace semantic without a good reason.
I personally don't care about the feature myself. I'm not sure what good it does anyway, and I don't know of anything on Linux that uses it. However, I also wasn't the one that added it. My original version of hfsplus did not have any way to access them. Maybe it's best to just remove it and see who complains about it. I do want to point out that I don't maintain that code anymore and don't have time to do that work myself. As far as I know, hfsplus is officially unmaintained these days.
Just as a side note, hfs has the same feature. The two modules are basically the same code these days.
Brad Boyer
[email protected]
Al Viro
maintainerQuoted 11 lines · Brad Boyer
I personally don't care about the feature myself. I'm not sure what good it does anyway, and I don't know of anything on Linux that uses it. However, I also wasn't the one that added it. My original version of hfsplus did not have any way to access them. Maybe it's best to just remove it and see who complains about it. I do want to point out that I don't maintain that code anymore and don't have time to do that work myself. As far as I know, hfsplus is officially unmaintained these days.
Just as a side note, hfs has the same feature. The two modules are basically the same code these days.
You know what... I thought that HFS+ did not support links; looking at the source, it appears that hardlinks *are* supported and that brings all the deadlocks into the game, same as we had in reiser4 case.
Which is to say, this stuff *must* go. In case of HFS it's obnoxious, but not immediately letal. In case of HFS+ it's an instant DoS.
Quoted 3 lines · Al Viro
You know what... I thought that HFS+ did not support links; looking at the source, it appears that hardlinks *are* supported and that brings all the deadlocks into the game, same as we had in reiser4 case.
Apple added hard links with osx. Back when I wrote hfsplus originally, there wasn't any such thing on HFS+ either. The support for the hack that added the appearance of hard links was added after I stopped maintaining the code. It's really just a matter of hiding the real file in a special place and putting stubs with special settings where all the links appear in the namespace. In all truth both HFS and HFS+ are inherently incompatible with the very concept of a hard link or having a file still exist after removal. It's just a horrible hack that makes it look like it works.
Quoted 2 lines
Which is to say, this stuff *must* go. In case of HFS it's obnoxious, but not immediately letal. In case of HFS+ it's an instant DoS.
Feel free. I disavowed any responsibility for this code years ago. And before anyone thinks of complaining, I'll point out that I wasn't the one that submitted hfsplus for inclusion in the kernel either.
Brad Boyer
[email protected]
Al Viro
maintainerQuoted 9 lines · Brad Boyer
Apple added hard links with osx. Back when I wrote hfsplus originally, there wasn't any such thing on HFS+ either. The support for the hack that added the appearance of hard links was added after I stopped maintaining the code. It's really just a matter of hiding the real file in a special place and putting stubs with special settings where all the links appear in the namespace. In all truth both HFS and HFS+ are inherently incompatible with the very concept of a hard link or having a file still exist after removal. It's just a horrible hack that makes it look like it works.
Well... That's not too unreasonably way to do them, actually - just treat your hidden directory as real inode table. The tricky part is keeping i_ino stable through all that, but if it's done...
FWIW, fs/qnx4 uses a somewhat similar scheme, IIRC.
To make things workable you need
* hidden directory that would never be reachable by syscalls
* recognizable files that would contain references to names
in that directory
* lookup that would normally lead to such file getting inode
of object in hidden directory
* link(2) with target being *not* one of those would start with
moving target to hidden directory and creating such "link" file in its place. Then it would create a link file in source. That, BTW, would have interesting implications wrt locking - we'd need to modify target's parent, which can get sucky.
* unlink(2) with victim dentry not being a "link" file *and* being
busy would move the victim to hidden. Otherwise it'd kill the victim outright.
* final iput() of file that'd been taken to hidden does deletion
of directory entry in hidden if i_nlink is 0.
The interesting part is locking - you need to deal with lookup vs. eviction of link(2) target to hidden and with rename vs. the same thing.
Quoted 6 lines
Quoted 2 lines
Which is to say, this stuff *must* go. In case of HFS it's obnoxious, but not immediately letal. In case of HFS+ it's an instant DoS.
Feel free. I disavowed any responsibility for this code years ago. And before anyone thinks of complaining, I'll point out that I wasn't the one that submitted hfsplus for inclusion in the kernel either.
Not blaming you...
Al Viro
maintainerQuoted 35 lines · Al Viro
Quoted 9 lines · Brad Boyer
Apple added hard links with osx. Back when I wrote hfsplus originally, there wasn't any such thing on HFS+ either. The support for the hack that added the appearance of hard links was added after I stopped maintaining the code. It's really just a matter of hiding the real file in a special place and putting stubs with special settings where all the links appear in the namespace. In all truth both HFS and HFS+ are inherently incompatible with the very concept of a hard link or having a file still exist after removal. It's just a horrible hack that makes it look like it works.
Well... That's not too unreasonably way to do them, actually - just treat your hidden directory as real inode table. The tricky part is keeping
i_inostable through all that, but if it's done...FWIW, fs/qnx4 uses a somewhat similar scheme, IIRC.
To make things workable you need
* hidden directory that would never be reachable by syscalls
* recognizable files that would contain references to names
in that directory
* lookup that would normally lead to such file getting inode
of object in hidden directory
* link(2) with target being *not* one of those would start with
moving target to hidden directory and creating such "link" file in its place. Then it would create a link file in source. That, BTW, would have interesting implications wrt locking - we'd need to modify target's parent, which can get sucky.
* unlink(2) with victim dentry not being a "link" file *and* being
busy would move the victim to hidden. Otherwise it'd kill the victim outright.
* finaliput()of file that'd been taken to hidden does deletion
of directory entry in hidden ifi_nlinkis 0.
Quoted 2 lines
The interesting part is locking - you need to deal with lookup vs. eviction of link(2) target to hidden and with rename vs. the same thing.
PS: that's for handling link(2) on non-directories with that kind of layout, *NOT* for file-cross-directory hybrids or links to directories.
Hi,
Quoted 6 lines · Al Viro
You know what... I thought that HFS+ did not support links; looking at the source, it appears that hardlinks *are* supported and that brings all the deadlocks into the game, same as we had in reiser4 case.
Which is to say, this stuff *must* go. In case of HFS it's obnoxious, but not immediately letal. In case of HFS+ it's an instant DoS.
At least on OS X 10.5 (Leopard) HFS+ not only supports hard links but also supports hard links on directories (though they are read-only)!
btw. That is how time machine works (the built in backup tool that backs up to a HFS+ file system and then each hour it creates a new directory tree by doing hard links to the previous one including the directories but the backups are read-only).
Supporting that on Linux as it is today is likely to be impossible without major work which no-one is prepared to do / accept...
Best regards,
Anton
Signature · 3 lines
Anton Altaparmakov <aia21 at cam.ac.uk> (replace at with @) Unix Support, Computing Service, University of Cambridge, CB2 3QH, UK Linux NTFS maintainer, http://www.linux-ntfs.org/