hfsplus: fixes worst-case unicode to char conversion of file names and attributes
Subsystems (2)
------------------------------
Quoted 54 lines · BST Andrew Morton
Quoted 28 lines · On Thu, 17 Apr 2014 20:30:20 +0100 Hin-Tak Leung <[email protected]> wrote:
From: Hin-Tak Leung <[email protected]>
The HFS Plus Volume Format specification (TN1150) states that file names are stored internally as a maximum of 255 unicode characters, as defined by The Unicode Standard, Version 2.0 [Unicode, Inc. ISBN 0-201-48345-9]. File names are converted by the NLS system on Linux before presented to the user.
255 CJK characters converts to UTF-8 with 1 unicode character to up to 3 bytes, and to GB18030 with 1 unicode character to up to 4 bytes. Thus, trying in a UTF-8 locale to list files with names of more than 85 CJK characters results in:
$ ls /mnt ls: reading directory /mnt: File name too longThe receiving buffer to
hfsplus_uni2asc()needs to be 255 xNLS_MAX_CHARSET_SIZEbytes, not 255 bytes as the code has always been.Similar consideration applies to attributes, which are stored internally as a maximum of 127 UTF-16BE units. See XNU source for an up-to-date reference on attributes.
Strictly speaking, the maximum value of
NLS_MAX_CHARSET_SIZE= 6 is not attainable in the case of conversion to UTF-8, as going beyond 3 bytes requires the use of surrogate pairs, i.e. consuming two input units.Quoted 9 lines
Thanks Anton Altaparmakov for reviewing an earlier version of this change.
This patch fixes all callers of
hfsplus_uni2asc(), and also enables the use of long non-English file names in HFS+. The getting and setting, and general usage of long non-English attributes requires further forthcoming work, in the following patches of this series.
fs/hfsplus/xattr.c: In function 'hfsplus_listxattr':fs/hfsplus/xattr.c:673: error: label 'out' used but not definedfs/hfsplus/xattr.c renamed from fs/hfsplus/xattr.c~hfsplus-fixes-worst-case-unicode-to-char-conversion-of-file-names-and-attributes-fix +0
-0@@ -726,6 +726,7 @@ ssize_t hfsplus_listxattr(struct dentry
end_listxattr:
kfree(strbuf);
+out:
hfs_find_exit(&fd);
return res;
}
Quoted 3 lines
But obviously the patches you sent were not the patches you tested. What happened here?
Hi Andrew,
Am terribly sorry - my fault of using/working/testing under one kernel (3.13.9),
then cherry-picking onto Linus' HEAD and collapsing there. A couple of small
changes got dropped/messed up. There is the missing headers
(#include <linux/nls.h>) in xattr*.c you spotted in the other mail; the above
is meant to be corrected by re-using the label nearby (as kfree(null) is okay...), rather than adding a new label.
fs/hfsplus/xattr.c +1 -1
@@ -670,7 +670,7 @@ ssize_t hfsplus_listxattr(struct dentry *dentry, char *buffer, size_t size) XATTR_MAC_OSX_PREFIX_LEN + 1, GFP_KERNEL);if (!strbuf) { res = -ENOMEM ; - goto out; + goto end_listxattr; } err =hfsplus_find_attr (inode->i_sb, inode->i_ino,NULL , &fd);
Do you want me to re-submit? I checked the diff of the two branches, it is just these two issues.
Really sorry about that.
Hin-Tak
--
To unsubscribe from this list: send the line "unsubscribe linux-fsdevel" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html
Quoted 37 lines · On Wed, 23 Apr 2014 02:43:00 +0100 (BST) Hin-Tak Leung…
Quoted 10 lines
+++
a/fs/hfsplus/xattr.c(quoted diff) +0
-0@@ -726,6 +726,7 @@ ssize_t hfsplus_listxattr(struct dentry
end_listxattr:
______ kfree(strbuf);
+out:
______hfs_find_exit(&fd);
______ return res;
}Quoted 3 lines
But obviously the patches you sent were not the patches you tested. What happened here?
Hi Andrew,
Am terribly sorry - my fault of using/working/testing under one kernel (3.13.9),
then cherry-picking onto Linus' HEAD and collapsing there. A couple of small
changes got dropped/messed up. There is the missing headers
(#include <linux/nls.h>) in xattr*.c you spotted in the other mail; the above
is meant to be corrected by re-using the label nearby (as kfree(null) is okay...), rather than adding a new label.fs/hfsplus/xattr.c +1
-1@@ -670,7 +670,7 @@ ssize_t hfsplus_listxattr(struct dentry *dentry, char *buffer, size_t size) XATTR_MAC_OSX_PREFIX_LEN + 1, GFP_KERNEL);if (!strbuf) { res = -ENOMEM ; - goto out; + goto end_listxattr; } err =hfsplus_find_attr (inode->i_sb, inode->i_ino,NULL , &fd);
Quoted 3 lines
Do you want me to re-submit? I checked the diff of the two branches, it is just these two issues.
Is OK - what I have now saves 30 cycles on a path which never happens ;)
It would help if you could grab tomorrow's linux-next and retest, to check that everything landed OK.
How were you testing this anyway? It's a pretty intricate patchset and presumably some effort went into developing the testcases. Perhaps you have something which we can immortalize in tools/testing/selftests/?