Skip to contentSkip to conversation

hfsplus: fixes worst-case unicode to char conversion of file names and attributes

PATCH Hin-Tak Leung 2 participants 2 messages Apr 23, 2014

Subsystems (2)
lore
Share & tools
lore
#0 lore raw

------------------------------

Quoted 54 lines · BST Andrew Morton
Quoted 28 lines · On Thu, 17 Apr 2014 20:30:20 +0100 Hin-Tak Leung <[email protected]> wrote:

From: Hin-Tak Leung <[email protected]>

The HFS Plus Volume Format specification (TN1150) states that file names are stored internally as a maximum of 255 unicode characters, as defined by The Unicode Standard, Version 2.0 [Unicode, Inc. ISBN 0-201-48345-9]. File names are converted by the NLS system on Linux before presented to the user.

255 CJK characters converts to UTF-8 with 1 unicode character to up to 3 bytes, and to GB18030 with 1 unicode character to up to 4 bytes. Thus, trying in a UTF-8 locale to list files with names of more than 85 CJK characters results in:

     $ ls /mnt
     ls: reading directory /mnt: File name too long

The receiving buffer to hfsplus_uni2asc() needs to be 255 x NLS_MAX_CHARSET_SIZE bytes, not 255 bytes as the code has always been.

Similar consideration applies to attributes, which are stored internally as a maximum of 127 UTF-16BE units. See XNU source for an up-to-date reference on attributes.

Strictly speaking, the maximum value of NLS_MAX_CHARSET_SIZE = 6 is not attainable in the case of conversion to UTF-8, as going beyond 3 bytes requires the use of surrogate pairs, i.e. consuming two input units.

Quoted 9 lines

Thanks Anton Altaparmakov for reviewing an earlier version of this change.

This patch fixes all callers of hfsplus_uni2asc(), and also enables the use of long non-English file names in HFS+. The getting and setting, and general usage of long non-English attributes requires further forthcoming work, in the following patches of this series.

fs/hfsplus/xattr.c: In function 'hfsplus_listxattr':
fs/hfsplus/xattr.c:673: error: label 'out' used but not defined

fs/hfsplus/xattr.c renamed from fs/hfsplus/xattr.c~hfsplus-fixes-worst-case-unicode-to-char-conversion-of-file-names-and-attributes-fix +0 -0
@@ -726,6 +726,7 @@ ssize_t hfsplus_listxattr(struct dentry
 

end_listxattr:
    kfree(strbuf);
+out:
    hfs_find_exit(&fd);
    return res;
}

Quoted 3 lines

But obviously the patches you sent were not the patches you tested. What happened here?

Hi Andrew,

Am terribly sorry - my fault of using/working/testing under one kernel (3.13.9),
then cherry-picking onto Linus' HEAD and collapsing there. A couple of small
changes got dropped/messed up. There is the missing headers
(#include <linux/nls.h>) in xattr*.c you spotted in the other mail; the above
is meant to be corrected by re-using the label nearby (as kfree(null) is okay...), rather than adding a new label.

fs/hfsplus/xattr.c +1 -1
index 3ddf50c..2a38647 100644
@@ -670,7 +670,7 @@ ssize_t hfsplus_listxattr(struct dentry *dentry, char *buffer, size_t size)
                        XATTR_MAC_OSX_PREFIX_LEN + 1, GFP_KERNEL);
        if (!strbuf) {
                res = -ENOMEM;
-               goto out;
+               goto end_listxattr;
        }
 
        err = hfsplus_find_attr(inode->i_sb, inode->i_ino, NULL, &fd);

Do you want me to re-submit? I checked the diff of the two branches, it is just these two issues.

Really sorry about that.

Hin-Tak

--
To unsubscribe from this list: send the line "unsubscribe linux-fsdevel" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html

#1 lore raw
Quoted 37 lines · On Wed, 23 Apr 2014 02:43:00 +0100 (BST) Hin-Tak Leung…
Quoted 10 lines

+++ a/fs/hfsplus/xattr.c

(quoted diff) +0 -0
@@ -726,6 +726,7 @@ ssize_t hfsplus_listxattr(struct dentry
 

end_listxattr:
______ kfree(strbuf);
+out:
______ hfs_find_exit(&fd);
______ return res;
}

Quoted 3 lines

But obviously the patches you sent were not the patches you tested. What happened here?

Hi Andrew,

Am terribly sorry - my fault of using/working/testing under one kernel (3.13.9),
then cherry-picking onto Linus' HEAD and collapsing there. A couple of small
changes got dropped/messed up. There is the missing headers
(#include <linux/nls.h>) in xattr*.c you spotted in the other mail; the above
is meant to be corrected by re-using the label nearby (as kfree(null) is okay...), rather than adding a new label.

fs/hfsplus/xattr.c +1 -1
index 3ddf50c..2a38647 100644
@@ -670,7 +670,7 @@ ssize_t hfsplus_listxattr(struct dentry *dentry, char *buffer, size_t size)
                        XATTR_MAC_OSX_PREFIX_LEN + 1, GFP_KERNEL);
        if (!strbuf) {
                res = -ENOMEM;
-               goto out;
+               goto end_listxattr;
        }
 
        err = hfsplus_find_attr(inode->i_sb, inode->i_ino, NULL, &fd);
Quoted 3 lines

Do you want me to re-submit? I checked the diff of the two branches, it is just these two issues.

Is OK - what I have now saves 30 cycles on a path which never happens ;)

It would help if you could grab tomorrow's linux-next and retest, to check that everything landed OK.

How were you testing this anyway? It's a pretty intricate patchset and presumably some effort went into developing the testcases. Perhaps you have something which we can immortalize in tools/testing/selftests/?