cifs: fix case-insensitive filename matching and hashing
Subsystems (4)
cover cifs: fix case-insensitive filename matching and hashing
This patchset embodies a new scheme for handling case-insensitive filename matching. The problem currently is that we rely in the NLS subsystem to handle this, and it's quite frankly not up to the task if the local charset is UTF8. There are two major problems:
1) The NLS subsystem assumes that characters are always a single byte which is clearly wrong in the case of UTF8.
...and even if it did handle that better:
2) nls_utf8.ko has no concept of upper or lowercase characters. All bytes passed in are identity-mapped, and there are no tables to do this mapping.
This scheme instead converts the characters to wchar_t's using the normal NLS routines, and then uses a converted version of the mapping table that Windows uses to convert those to uppercase. With that scheme, case-insensitive handling now works, even if UTF8 is in use.
This patchset is fairly new, but I've given it some smoke testing and it seems to be ok. It's possibly a candidate for 3.12, given the brokenness of the existing CI code. Getting it into linux-next soonish would be good.
FWIW, it might be reasonable to move the winucase.c file into generic NLS code if other filesystems have the desire to use it. I'm not opposed to doing that, but didn't want to do so unless there was interest in using this in other filesystems.
Jeff Layton (3):
cifs: add new case-insensitive conversion routines that are based on
wchar_t's
cifs: convert case-insensitive dentry ops to use new case conversion
routines
cifs: add winucase_convert.pl to Documentation/ directory
Documentation/filesystems/cifs/winucase_convert.pl | 62 ++ fs/cifs/Makefile | 2 +- fs/cifs/cifs_unicode.h | 2 + fs/cifs/dir.c | 58 +- fs/cifs/winucase.c | 661 +++++++++++++++++++++ 5 files changed, 776 insertions(+), 9 deletions(-) create mode 100755 Documentation/filesystems/cifs/winucase_convert.pl create mode 100644 fs/cifs/winucase.c
Signature · 1 line
1.8.3.1
PATCH 1/3 cifs: add new case-insensitive conversion routines that are based on wchar_t's
The existing NLS case conversion routines do not appropriately handle the (now common) case where the local host is using UTF8. This is because nls_utf8 has no support at all for converting a utf8 string between cases and the NLS infrastructure in general cannot handle a multibyte input character.
In any case, what we really need for cifs is to emulate how we expect the server to convert the character to upper or lowercase. Thus, even if we had routines that could handle utf8 case conversion, we likely would end up with the wrong result if the name ends up being in the upper planes.
This patch adds a new scheme for doing unicode case conversion. The case conversion tables that Microsoft has published for Windows 8 have been converted to a set of lookup tables, and a routine is added to convert a wchar_t from lower to uppercase using those tables.
--- fs/cifs/Makefile | 2 +- fs/cifs/cifs_unicode.h | 2 + fs/cifs/winucase.c | 661 +++++++++++++++++++++++++++++++++++++++++++++++++ 3 files changed, 664 insertions(+), 1 deletion(-) create mode 100644 fs/cifs/winucase.c
fs/cifs/Makefile +1 -1
@@ -6,7 +6,7 @@ obj-$(CONFIG_CIFS) += cifs.o cifs-y := cifsfs.o cifssmb.o cifs_debug.o connect.o dir.o file.o inode.o \ link.o misc.o netmisc.o smbencrypt.o transport.o asn1.o \ cifs_unicode.o nterr.o xattr.o cifsencrypt.o \ - readdir.o ioctl.o sess.o export.o smb1ops.o + readdir.o ioctl.o sess.o export.o smb1ops.o winucase.o cifs-$(CONFIG_CIFS_ACL) += cifsacl.o
fs/cifs/cifs_unicode.h +2 -0
@@ -91,6 +91,8 @@ extern __le16 *cifs_strndup_to_utf16(const char *src, const int maxlen,#endif /* CONFIG_CIFS_SMB2 */ #endif +wchar_t cifs_toupper (wchar_t in); +/* * UniStrcat: Concatenate the second string to the first *
fs/cifs/winucase.c new file +661 -0
661 lines not on this page · raw message
Signature · 1 line
1.8.3.1
A message that is not in the archive (sent off-list or not yet received): 1377028211-17624-1-git-send-email-jlayton-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org
PATCH 2/3 cifs: convert case-insensitive dentry ops to use new case conversion routines
Have the case-insensitive d_compare and d_hash routines convert each character in the filenames to wchar_t's and then use the new cifs_toupper routine to convert those into uppercase.
With this scheme we should more closely emulate the case conversion that the servers will do.
--- fs/cifs/dir.c | 58 ++++++++++++++++++++++++++++++++++++++++++++++++++-------- 1 file changed, 50 insertions(+), 8 deletions(-)
fs/cifs/dir.c +50 -8
@@ -32,6 +32,7 @@#include "cifsproto.h" #include "cifs_debug.h" #include "cifs_fs_sb.h" +#include "cifs_unicode.h" static void renew_parental_timestamps (struct dentry *direntry)
@@ -834,12 +835,17 @@ static int cifs_ci_hash(const struct dentry *dentry, struct qstr *q) {struct nls_table *codepage =CIFS_SB (dentry->d_sb)->local_nls;unsigned long hash; -int i; +wchar_t c; +int i, charlen; hash =init_name_hash (); -for (i =0 ; i < q->len; i++) - hash =partial_name_hash (nls_tolower (codepage, q->name[i]), - hash); +for (i =0 ; i < q->len; i += charlen) { + charlen = codepage->char2uni (&q->name[i], q->len - i, &c); +/* error out if we can't convert the character */ +if (unlikely (charlen <0 )) +return charlen; + hash =partial_name_hash (cifs_toupper (c), hash); + } q->hash =end_name_hash (hash);return 0 ;
@@ -849,11 +855,47 @@ static int cifs_ci_compare(const struct dentry *parent, const struct dentry *den unsigned int len, const char *str, const struct qstr *name) { struct nls_table *codepage = CIFS_SB(parent->d_sb)->local_nls; +wchar_t c1, c2; +int i, l1, l2; -if ((name->len == len) && - (nls_strnicmp (codepage, name->name, str, len) ==0 )) -return 0 ; -return 1 ; +/* +* We make the assumption here that uppercase characters in the local +* codepage are always the same length as their lowercase counterparts. +* +* If that's ever not the case, then this will fail to match it. +*/ +if (name->len != len) +return 1 ; + +for (i =0 ; i < len; i += l1) { +/* Convert characters in both strings to UTF-16. */ + l1 = codepage->char2uni (&str[i], len - i, &c1); + l2 = codepage->char2uni (&name->name[i], name->len - i, &c2); + +/* +* If we can't convert either character, just declare it to +* be 1 byte long and compare the original byte. +*/ +if (unlikely (l1 <0 && l2 <0 )) { +if (str[i] != name->name[i]) +return 1 ; + l1 =1 ; +continue ; + } + +/* +* Here, we again ass|u|me that upper/lowercase versions of +* a character are the same length in the local NLS. +*/ +if (l1 != l2) +return 1 ; + +/* Now compare uppercase versions of these characters */ +if (cifs_toupper (c1) !=cifs_toupper (c2)) +return 1 ; + } + +return 0 ; } const struct dentry_operations cifs_ci_dentry_ops = {
Signature · 1 line
1.8.3.1
PATCH 3/3 cifs: add winucase_convert.pl to Documentation/ directory
In reply to Jeff LaytonAdd the script used to generate the case-conversion tables to the Documentation/ directory, in case we ever need to update or regenerate these tables in the future.
--- Documentation/filesystems/cifs/winucase_convert.pl | 62 ++++++++++++++++++++++ 1 file changed, 62 insertions(+) create mode 100755 Documentation/filesystems/cifs/winucase_convert.pl
Documentation/filesystems/cifs/winucase_convert.pl new file +62 -0
@@ -0,0 +1,62 @@ +#!/usr/bin/perl -w +# +# winucase_convert.pl -- convert "Windows 8 Upper Case Mapping Table.txt" to +# a two-level set of C arrays. +# +# Copyright 2013: Jeff Layton <[email protected]> +# +# This program is free software: you can redistribute it and/or modify +# it under the terms of the GNU General Public License as published by +# the Free Software Foundation, either version 3 of the License, or +# (at your option) any later version. +# +# This program is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. +# +# You should have received a copy of the GNU General Public License +# along with this program. If not, see <http://www.gnu.org/licenses/>. +# + +while(<>) { + next if (!/^0x(..)(..)\t0x(....)\t/); + $firstchar = hex($1); + $secondchar = hex($2); + $uppercase = hex($3); + + $top[$firstchar][$secondchar] = $uppercase; +} + +for ($i = 0; $i < 256; $i++) { + next if (!$top[$i]); + + printf("static const wchar_t t2_%2.2x[256] = {", $i); + for ($j = 0; $j < 256; $j++) { + if (($j % 8) == 0) { + print "\n\t"; + } else { + print " "; + } + printf("0x%4.4x,", $top[$i][$j] ? $top[$i][$j] : 0); + } + print "\n};\n\n"; +} + +printf("static const wchar_t *const toplevel[256] = {", $i); +for ($i = 0; $i < 256; $i++) { + if (($i % 8) == 0) { + print "\n\t"; + } elsif ($top[$i]) { + print " "; + } else { + print " "; + } + + if ($top[$i]) { + printf("t2_%2.2x,", $i); + } else { + print "NULL,"; + } +} +print "\n};\n\n";
Signature · 1 line
1.8.3.1