get/set FAT filesystem attribute bits

This patch adds a set of ioctls to get and set the FAT filesystem native attribute bits, including the unused bits (6 and 7.)

It also includes some very minor code cleanups; mostly by macroizing a few idioms.

-hpa

Index: linux-2.5/fs/fat/dir.c
===================================================================
RCS file: /home/hpa/kernel/bkcvs/linux-2.5/fs/fat/dir.c,v retrieving revision 1.28

diff -u -r1.28 dir.c
@@ -663,7 +663,7 @@
 		both = 1;
 		break;
 	default:
-		return -EINVAL;
+		return fat_generic_ioctl(inode, filp, cmd, arg);
 	}
 
 	d1 = (struct dirent __user *)arg;

Index: linux-2.5/fs/fat/file.c
===================================================================
RCS file: /home/hpa/kernel/bkcvs/linux-2.5/fs/fat/file.c,v retrieving revision 1.26

diff -u -r1.26 file.c
@@ -19,6 +19,7 @@
 	.read		= generic_file_read,
 	.write		= fat_file_write,
 	.mmap		= generic_file_mmap,
+	.ioctl		= fat_generic_ioctl,
 	.fsync		= file_fsync,
 	.readv		= generic_file_readv,
 	.writev		= generic_file_writev,

Index: linux-2.5/fs/fat/inode.c
===================================================================
RCS file: /home/hpa/kernel/bkcvs/linux-2.5/fs/fat/inode.c,v retrieving revision 1.108

Index: linux-2.5/include/linux/compat_ioctl.h
===================================================================
RCS file: /home/hpa/kernel/bkcvs/linux-2.5/include/linux/compat_ioctl.h,v retrieving revision 1.35

diff -u -r1.35 compat_ioctl.h
@@ -571,6 +571,9 @@
 COMPATIBLE_IOCTL(DEVFSDIOC_SET_EVENT_MASK)
 COMPATIBLE_IOCTL(DEVFSDIOC_RELEASE_EVENT_QUEUE)
 COMPATIBLE_IOCTL(DEVFSDIOC_SET_DEBUG_MASK)
+/* FAT */
+COMPATIBLE_IOCTL(FAT_IOCTL_GET_ATTRIBUTES)
+COMPATIBLE_IOCTL(FAT_IOCTL_SET_ATTRIBUTES)
 /* Raw devices */
 COMPATIBLE_IOCTL(RAW_SETBIND)
 COMPATIBLE_IOCTL(RAW_GETBIND)

Index: linux-2.5/include/linux/msdos_fs.h
===================================================================
RCS file: /home/hpa/kernel/bkcvs/linux-2.5/include/linux/msdos_fs.h,v retrieving revision 1.41

diff -u -r1.41 msdos_fs.h
@@ -95,6 +95,9 @@
  */
 #define	VFAT_IOCTL_READDIR_BOTH		_IOR('r', 1, struct dirent [2])
 #define	VFAT_IOCTL_READDIR_SHORT	_IOR('r', 2, struct dirent [2])
+/* Badness: 1..6 conflict with linux/videotext.h, despite ioctl-numbers.txt */
+#define FAT_IOCTL_GET_ATTRIBUTES	_IOR('r', 0x10, __u8)
+#define FAT_IOCTL_SET_ATTRIBUTES	_IOW('r', 0x11, __u8)
 
 /* 
  * vfat shortname flags

@@ -202,6 +205,13 @@
 	return container_of(inode, struct msdos_inode_info, vfs_inode);
 }
 
+static inline u8 msdos_attr(struct inode *inode)
+{
+	return MSDOS_MKATTR(inode->i_mode) |
+		MSDOS_I(inode)->i_attrs |
+		(S_ISDIR(inode->i_mode) ? ATTR_DIR : 0);
+}
+
 static inline void fat16_towchar(wchar_t *dst, const __u8 *src, size_t len)
 {
 #ifdef __BIG_ENDIAN

@@ -267,6 +277,8 @@
 int fat_fill_super(struct super_block *sb, void *data, int silent,
 		   struct inode_operations *fs_dir_inode_ops, int isvfat);
 extern int fat_notify_change(struct dentry * dentry, struct iattr * attr);
+extern int fat_generic_ioctl(struct inode * inode, struct file * filp,
+			     unsigned int cmd, unsigned long arg);
 
 /* fat/misc.c */
 extern void fat_fs_panic(struct super_block *s, const char *fmt, ...);

Index: linux-2.5/include/linux/msdos_fs_i.h
===================================================================
RCS file: /home/hpa/kernel/bkcvs/linux-2.5/include/linux/msdos_fs_i.h,v retrieving revision 1.11

diff -u -r1.11 msdos_fs_i.h
@@ -20,7 +20,6 @@
 	int i_start;	/* first cluster or 0 */
 	int i_logstart;	/* logical first cluster */
 	int i_attrs;	/* unused attribute bits */
-	int i_ctime_ms;	/* unused change time in milliseconds */
 	loff_t i_pos;	/* on-disk position of directory entry or 0 */
 	struct hlist_node i_fat_hash;	/* hash by i_location */
 	struct inode vfs_inode;

Instead of adding another ioctl, wouldn't an xattr be more appropriate? For instance, system.fatattrs containing a text representation of the attribute bits.

i.e.
a = archive
d = directory
h = hidden
r = read only
s = system
v = volume
6 = unused bit 6
7 = unused bit 7

and

bash-3.00$ getfattr -n system.fatattrs dosfile.txt # file: dosfile.txt
system.fatattrs="ar"

bash-3.00$

Nicholas Miell <[email protected]>

This really worries me, because it's not clear to me that Microsoft isn't going to add NTFS-style xattrs to FAT in the future. There is a very specific reason why they might want to do that: since they want to keep NTFS secret and proprietary, FAT is the published interchange format that other devices can use to exchange data with MS operating systems. If we then have overloaded the xattr mechanism, that would be very ugly.

-hpa

That's why I put fatattrs in the system namespace, which is wholly owned by the Linux kernel. Any theoretical FAT-with-xattrs variant would put those xattrs in the user namespace.

On another note, NTFS-style xattrs (aka named streams) are unrelated to Linux xattrs. A named stream is a separate file with a funny name, while a Linux xattr is a named extension to struct stat.

Nicholas Miell <[email protected]>

OK, that does make it more sensible. I do note, however, that ext2/ext3 do not seem to export their attributes (chattr/lsattr) in this way; I do also note that the xattr code wherever it has been implemented is just painfully complex.

I'll see if I can weed it down to some kind of sane size.

-hpa

Yeah, I contemplated adding system.fattattrs, system.ntfsattrs, and system.linuxattrs (for the ext2 attrs that have popped up in several other filesystems) a while ago, but xattrs seem to be the red-headed left-handed stepchild of the Linux VFS and I lost interest in the project.

Nice to see someone else interested in it, though.

Nicholas Miell <[email protected]>

I'm honestly not sure that using an ASCII string in an xattr is the sane way of doing this. Even a binary byte in an xattr would make more sense in some ways.

I think the xattr mechanism is ignored largely because it's painfully complex.

A plus with using xattr is that in theory (but of course not in practice!) it would let one store a copy of a DOS filesystem on an ext3 (or xfs, or...) filesystem and have it restored, all using standard (but by necessity, xattr-aware) tools. However, the splitting of xattr into namespaces may very well make that impossible, since what's a "system" attribute to one filesystem is a "user" attribute to another. Classic design flaw, by the way.

Anyway, I'm going to send out something to the various maintainers of DOS-based filesystems (FAT, CIFS, NTFS) and see what they think.

-hpa

ASCII strings require no special tools to manipulate from shell scripts.

The design does allow users to store whatever they want as an xattr without having to worry about how the kernel chooses to interpret it, though. (i.e. the user namespace is just a byte array that the kernel stores for you, while the system/security namespaces are probably generated and interpreted on demand.)

Nicholas Miell <[email protected]>

[Pruning the Cc: list.]

You need some kind of special tool anyway, i.e. getfattr/setfattr. What tool you use isn't really important.

The fact that getfattr/setfattr can't deal with attributes that aren't ASCII strings seem like flaws in these tools.

Exactly, and that's a total screwup. It makes something that would otherwise be possible -- for some filesystems to have an attribute (call it "system.dosattrib") which is used, and for others which is stored. The problem is that with the current design, that won't happen.

Encoding this in the namespace, therefore preventing this kind of compatiblity, is daft. From the looks of it, the CIFS people were planning to do the "put everything in user.*" workaround for this design error.

-hpa

I was talking about getdosattr and setdosattr (and the corresponding pair for every other filesystem with it's own set of special attributes).

getfattr and setfattr are standard tools already provided with the distro.

They can. Non-ASCII xattrs are either base64 encoded or octal escaped.

Try

getfattr -n system.posix_acl_access some_file_with_an_acl getfattr -e text -n system.posix_acl_access some_file_with_an_acl

for a quick example.

I responded to this in the other thread already.

Nicholas Miell <[email protected]>

This is incorrect. NTFS has two different beasts:

- Extended Attributes (EAs) which are just like Linux xattrs and in Windows a very similar API exists as in Linux for accessing them (on NTFS volumes only - VFAT does not support EAs in Windows).

- Named streams for which you are pretty much correct that they are like a file with a funny name but note that named streams share the same inode as their parent (unnamed) stream. And this is not just the inode number, it is the same on-disk inode. This means that the ACLs, dos attributes, etc, are all valid for both the unnamed and all named streams. So you cannot for example allow user A to access the unnamed stream but not some named stream and vice versa you cannot allow user B to access some named stream but not the unnamed stream. In Windows these beasts are accessed as filename::namedstreamname and in Linux there is no existing API to access named streams at all that I am aware off. Again, as EAs, in Windows named streams only are supported on NTFS volumes - VFAT does not support them.

The major difference is that access to named streams is much faster on a per-stream basis as they are separate entities, i.e. stream A and stream B share the same inode but are otherwise accessed independently and are stored independently on disk (and as such can be read/written simultaneously). EAs on the other hand are all linked together and are in fact just one large blob of data. Resizing one EA modifies potentially all of them since all EAs are stored in a single byte stream, not very dissimilar to a named stream in fact. So EAs are more suited for storage of constant length structures while named streams are better suited for variable length data (e.g. icons for executables, name of document author, etc).

One interesting bit of trivia is that Windows uses named streams very extensively while it _never_ uses EAs. In fact I have never seen a Windows OS or application that uses EAs. They were added to be compatible with OS/2 EAs when it came out but since OS/2 died they now just seem like old baggage/backwards compatibility in Windows that is no longer used. (If anyone knows of a Windows application that uses EAs please let me know. I would be most interested in knowing about it!)

Hope this clears things up a bit as far as NTFS is concerned...

I don't know what API would be best for accessing named streams on NTFS but an xattrs like interface is not suitable IMO. You really want to be able to open them and access them like normal files. An interface similar to the Solaris openat() system call (see
http://docs.sun.com/app/docs/doc/816-0212/6m6nd4nc7?a=view) that has been discussed on LKML before seems like a good way to deal with this but I am more interested in getting normal write support into NTFS at present than I am in fancy features like EAs and named streams...

Best regards,

Anton

Anton Altaparmakov <aia21 at cam.ac.uk> (replace at with @)
Unix Support, Computing Service, University of Cambridge, CB2 3QH, UK
Linux NTFS maintainer / IRC: #ntfs on irc.freenode.net
WWW: http://linux-ntfs.sf.net/ & http://www-stu.christs.cam.ac.uk/~aia21/

[ why this is incorrect omitted for brevity ]

This would probably explain why I've never heard of them. In my brief perusal of the MSDN Library in search of more information about these beasts, the only hint I could find related to their existence is in parameters to ZwCreateFile/NtCreateFile (which are themselves mostly undocumented). The high level file API certainly doesn't support their use, AFAICT.

I think it's reasonably safe to assume that hpa's worry that FAT might get EAs in the future is unfounded. (Named streams is still a possibility, but I don't think Microsoft is all that interested in making improvements to a filesystem that people use so that they don't have to license NTFS.)

Good luck with that.

Nicholas Miell <[email protected]>

D'oh! They probably should be renamed _cs (centiseconds ;)

Yes, I wanted to use them for the syslinux installer.

-hpa