Case-preserving & case-insensitive filesystems, kernel 2.4.19-pre4+2.5.8

Hi,

# Problem:
/$ mount -t vfat /dev/hdc2 /hdc2
/$ cd /hdc2
/hdc2$ cat LONGFILENAME_ABCD
cat: LONGFILENAME_ABCD: No such file or directory
/hdc2$ touch LongFileName_aBcD
/hdc2$ ls -l
-rwxr-xr-x 1 root root 0 Jan 9 00:05 LONGFILENAME_ABCD
/hdc2$
# Samba (and other?) software expects "LongFileName_aBcD" here

My patches are provided on:
http://short.vellum.cz/priv/vfat/

http://short.vellum.cz/priv/vfat/linux-2.4.19-pre4-vfat5.diff
Linus kernel 2.4.19-pre4, tested for almost a month [included]
http://short.vellum.cz/priv/vfat/linux-2.5.8-vfat5.diff
Linus kernel 2.5.8, only simple port and tested once
http://short.vellum.cz/priv/vfat/vfat_test_suite_tcl-2.4-fix1.diff
update of:
http://bmrc.berkeley.edu/people/chaffee/vfat_test_suite_tcl
to get it working for testing of recent kernels

Already solved in the past by Alexander Viro (1999-05-26) ("Vogon filesystems"):

	http://marc.theaimsgroup.com/?l=linux-fsdevel&m=92792807215416&w=2
obsoletes his previous patches (more or less, I hope):
	http://marc.theaimsgroup.com/?l=linux-fsdevel&m=92478709702476&w=2
	http://marc.theaimsgroup.com/?l=linux-fsdevel&m=91894002829117&w=2
	http://marc.theaimsgroup.com/?l=linux-kernel&m=91636997404967&w=2

Bug is present at least in:

	Alan Cox:    2.4.18-pre7-ac1, 2.4.19-pre4
	Linus:       2.5.3, 2.5.8
	David Jones: patch-2.5.2-dj6 (only read the diff, no filename-specifics)
	RedHat:      kernel-2.4.9-21
	Mandrake:    kernel-2.4.8-34.1mdk

Thanks to:
developed in cooperation with Jan Kara <[email protected]> bugreported and bravely tested by Jan Kolar <[email protected]>

Problem introduced in 2.1.x (dcache implementation) and it hits Samba servers trying to provide its share from vfat partition:

When W32 client wants to create "long filename", it will first check for its existence (in upper-cased form) - it will fail but vfat (dentries) will cache the long filename. The problem is that the forthcoming file create (in different case) will create the file in its original upper-cased form!

This results in all filenames always uppercased when created from W9x clients on (at least letter-connected) disk from Samba share running on Linux vfat. This is a problem while pushing Linux to business where managers want to keep their data (disks) w/o dependence on the new technology Linux (=to use vfat).

The key point of the patch is the condition added to top of vfs_rename():
+ if (old_dentry->d_inode == new_dentry->d_inode
+ && (!old_dentry->d_op->d_compare_rename || (old_dir != new_dir
+ || old_dentry->d_op->d_compare_rename(old_dentry, &old_dentry->d_name, &new_dentry->d_name))))
+ return 0;

Corresponds to the following sequence of conditions:
inode !=
DO
old-style filesystem
return 0 (hard-linked)
dir !=
return 0 (hard-linked across directories)
anycase name compare NOT MATCH
return 0 (hard-linked in the same directory)
DO RENAME

I solved it by 'virtual hardlinking' the file to all its name case-aliases. This should be safe for files but it may be a problem for directories - kernel currently doesn't expect directories to have hardlinks but for now I don't know about any specific problems of it.

Currently it is protected by mandatory revalidation (vfat_revalidate()) of all dentries:

+/* We have to always do the revalidate as after unlink (etc.) there still may
+ * exist other case-different dentries for the same inode. It would be also
+ * possible to discard such aliases by going through d_alias links during the
+ * unlink. "strictcase" doesn't have case-different dentries but "longna~1"
+ * style aliases still exist there.
+ */

It was patched with target to vfat (to be able to provide Samba shares from vfat volumes) but I also implemented it to affs as proof-of-concept for case-preserving & case-insensitive filesystems which are capable of hardlinks (as vfat cannot have valid hardlinks, of course).

It is not optimal solution but a bunch of problems ceases to be. Anyway even the original vfat implementation uses dentries revalidation a lot, my patch just increases their amount. This revalidations can cost a lot during entries modified in big directories - it would be nice to do another optimizing-only patch to remove any revalidation by d_alias links utilization during dentry delete. I have such in-progress patch ready, it hooks on d_delete() and d_move() but I dropped it now to minimize the affected code and introduced bugs.

All the sane (or case-preserving & case-insensitive but not yet patched) filesystems don't have dentry_operations->d_compare_rename() implemented and thus the patch doesn't affect them in any way. Patched filesystems MUST have d_hash() and d_compare() case sensitive but case-insensitive comparison must be implemented by d_compare_rename(). The filesystems usually have to be fixed for proper renaming between two case-insensitively matching entries in the same directory.

I am not sure how to hehave during rename("LongFileName","longfi~1"). I consider these two names as hardlinks and thus "0" is returned and no action is done.

Jan "Lace" Kratochvil,
patch developed in cooperation with Jan Kara

Inlined patch linux-2.4.19-pre4-vfat5.diff: (for other patches see URLs above)
-------------------------------------------

diff -u -ru linux-2.4.19-pre4-clean/Documentation/filesystems/vfs.txt linux-2.4.19-pre4-vfat/Documentation/filesystems/vfs.txt
@@ -405,6 +405,7 @@
 	int (*d_revalidate)(struct dentry *);
 	int (*d_hash) (struct dentry *, struct qstr *);
 	int (*d_compare) (struct dentry *, struct qstr *, struct qstr *);
+	int (*d_compare_rename) (struct dentry *, struct qstr *, struct qstr *);
 	void (*d_delete)(struct dentry *);
 	void (*d_release)(struct dentry *);
 	void (*d_iput)(struct dentry *, struct inode *);

@@ -417,7 +418,13 @@
 
   d_hash: called when the VFS adds a dentry to the hash table
 
-  d_compare: called when a dentry should be compared with another
+  d_compare: called when a dentry should be compared with another.
+	Case-sensitive for all case-preserving filesystems, no matter if the
+	filesystem structure is case-sensitive itself
+
+  d_compare_rename: compare equivalency of two dentries, no matter their case.
+	Optional - implement only for case-insensitive but case-preserving
+	filesystems, dentries always compared by d_compare when not provided
 
   d_delete: called when the last reference to a dentry is
 	deleted. This means no-one is using the dentry, however it is

diff -u -ru linux-2.4.19-pre4-clean/fs/namei.c linux-2.4.19-pre4-vfat/fs/namei.c
@@ -1714,9 +1714,6 @@
 	int error;
 	struct inode *target;
 
-	if (old_dentry->d_inode == new_dentry->d_inode)
-		return 0;
-
 	error = may_delete(old_dir, old_dentry, 1);
 	if (error)
 		return error;

@@ -1794,9 +1791,6 @@
 {
 	int error;
 
-	if (old_dentry->d_inode == new_dentry->d_inode)
-		return 0;
-
 	error = may_delete(old_dir, old_dentry, 0);
 	if (error)
 		return error;

@@ -1835,6 +1829,12 @@
 	       struct inode *new_dir, struct dentry *new_dentry)
 {
 	int error;
+
+	if (old_dentry->d_inode == new_dentry->d_inode
+	    && (!old_dentry->d_op->d_compare_rename || (old_dir != new_dir
+	                || old_dentry->d_op->d_compare_rename(old_dentry, &old_dentry->d_name, &new_dentry->d_name))))
+		return 0;
+
 	if (S_ISDIR(old_dentry->d_inode->i_mode))
 		error = vfs_rename_dir(old_dir,old_dentry,new_dir,new_dentry);
 	else

diff -u -ru linux-2.4.19-pre4-clean/include/linux/dcache.h linux-2.4.19-pre4-vfat/include/linux/dcache.h
@@ -87,6 +87,7 @@
 	int (*d_revalidate)(struct dentry *, int);
 	int (*d_hash) (struct dentry *, struct qstr *);
 	int (*d_compare) (struct dentry *, struct qstr *, struct qstr *);
+	int (*d_compare_rename) (struct dentry *, struct qstr *, struct qstr *);
 	int (*d_delete)(struct dentry *);
 	void (*d_release)(struct dentry *);
 	void (*d_iput)(struct dentry *, struct inode *);