From: Hin-Tak Leung Subject: Re: [PATCH V2] hfsplus: fixes worst-case unicode to char conversion of file names and attributes Date: Tue, 8 Apr 2014 17:07:33 +0100 (BST) Message-ID: <1396973253.12333.YahooMailBasic@web172303.mail.ir2.yahoo.com> Reply-To: htl10@users.sourceforge.net Mime-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Transfer-Encoding: QUOTED-PRINTABLE Cc: linux-fsdevel@vger.kernel.org, slava@dubeyko.com, viro@zeniv.linux.org.uk, hch@infradead.org To: anton@tuxera.com Return-path: Received: from nm29-vm7.bullet.mail.ir2.yahoo.com ([212.82.97.76]:23383 "EHLO nm29-vm7.bullet.mail.ir2.yahoo.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1756473AbaDHQHf convert rfc822-to-8bit (ORCPT ); Tue, 8 Apr 2014 12:07:35 -0400 Sender: linux-fsdevel-owner@vger.kernel.org List-ID: Hi Anton, ------------------------------ On Tue, Apr 8, 2014 4:06 PM BST Anton Altaparmakov wrote: >Hi Hin-Tak, > >That now looks good. > >I agree with you that doing a major cleanup of how attribute names are= handled is a separate patch.=A0 As you say that is much more work and = it is not as trivial as this.=A0 I think doing the attribute handling p= roperly requires reworking how it all fits together because it is impos= ing the name length checks at the wrong level - they need to be done af= ter conversion to UTF-16 not whilst they are still in the current NLS (= except of course you want to check that they are not too big to fit in = the buffer though it is not at all clear to me why the argument is bein= g copied into a buffer in the first place at least in one of the places= I looked at - seems a waste of CPU time to me). > The bigger attribute change part has a mixture of prefixes (in bytes an= d nls local strings), and also moving/concatenating within the buffer, = it needs a lot more thinking and work. But I think just to address what= comes out of hfsplus_uni2asc() (from on disk to in-memory), the patch = is all that is needed. =46or the copying of argument, if you are referring to the len being pa= ssed into uni2asc - it is being used as both a limit and also a return = value for the actual number of byte used. >Also, I don't think it is needed to set up a kmem cache for this - kma= lloc is perfect - as at any one time there will be a vanishingly small = number of these buffers allocated at any one time.=A0 After all, how ma= ny concurrent readdir() calls are there going to be on a volume at any = one time?=A0 It is not like the struct inode or other long lived memory= structures where you end up with thousands of them in memory at any on= e time... > I haven't been able to find any reference to which to use (grep'ing und= er Documentation/* ) - there are a lot of *alloc in the kernel . Even f= inding kzalloc does what calloc in userland (and kcalloc is a totally d= ifferent beast) took some effort. The attribute ones seem to need kzalloc (or, just '\0' at the beginning= to null terminate if empty) - they are being used in concatenating, et= c. As i said, cleaning that up will be a lot more work. >Just one thing I wanted to mention in case you did not know it: kfree(= NULL) is allowed - in fact kfree() function starts by doing "if (NULL a= rgument) return;" so in your patch you could instead of adding an addit= ional exit label, just use the generic one that also does a kfree() whi= ch is safe as the pointer is NULL.=A0 As this is only error handling co= de path it does not matter that you incur an additional function call o= verhead. > Thanks. I did have the impression and google'd to confirm that a change= to make it work that way happened in 2005/2006. I have played with var= ious ways of re-arranging the code, specifically whether strbuf should = go before or after find_init, and how they could fail (independently), = but in the end, the problem is simply that only one of them can do "if = fail then return err". The other will have to clean up the opposite par= ty before the returning. There might be future addition where the order= is important. >Whether you want to change the exit label or not, you can change the C= C: to Reviewed-By: for me if you like and I would suggest you send it t= o Andrew Morton for inclusion through his patch series. Thanks. I'll possibly change a few words in the commit message. Let's s= ee if others have any more to say. Hin-Tak >On 8 Apr 2014, at 03:19, Hin-Tak Leung wrote: > >> From: Hin-Tak Leung >>=20 >> The HFS Plus Volume Format specification (TN1150) states that >> file names are stored internally as a maximum of 255 unicode >> characters, as defined by The Unicode Standard, Version 2.0 >> [Unicode, Inc. ISBN 0-201-48345-9]. File names are converted by >> the NLS system on Linux before presented to the user. >>=20 >> 255 CJK characters converts to UTF-8 with 1 unicode character >> to up to 3 bytes, and to GB18030 with 1 unicode character to >> up to 4 bytes. Thus, trying in a UTF-8 locale to list files >> with names of more than 85 CJK characters results in: >>=20 >>=A0 =A0 $ ls /mnt >>=A0 =A0 ls: reading directory /mnt: File name too long >>=20 >> The receiving buffer needs to be 255 x NLS_MAX_CHARSET_SIZE bytes, >> not 255 bytes as the code has always been. >>=20 >> Similar consideration applies to attributes, which are stored >> internally as a maximum of 127 UTF-16BE units in HFS+. >>=20 >> Strictly speaking, the maximum value of NLS_MAX_CHARSET_SIZE =3D 6 >> is not attainable in the case of conversion to UTF-8, as that >> requires the use of surrogate pairs, i.e. consuming two storage unit= s. >>=20 >> Thanks Anton Altaparmakov for reviewing an earlier version of >> this change. >>=20 >> Signed-off-by: Hin-Tak Leung >> CC: Anton Altaparmakov >> CC: Vyacheslav Dubeyko >> CC: Al Viro >> CC: Christoph Hellwig >> --- >> fs/hfsplus/dir.c=A0=A0=A0| 12 ++++++++++-- >> fs/hfsplus/xattr.c | 15 ++++++++++++--- >> 2 files changed, 22 insertions(+), 5 deletions(-) >>=20 >> diff --git a/fs/hfsplus/dir.c b/fs/hfsplus/dir.c >> index bdec665..f95abef 100644 >> --- a/fs/hfsplus/dir.c >> +++ b/fs/hfsplus/dir.c >> @@ -12,6 +12,7 @@ >> #include >> #include >> #include >> +#include >>=20 >> #include "hfsplus_fs.h" >> #include "hfsplus_raw.h" >> @@ -127,7 +128,7 @@ static int hfsplus_readdir(struct file *file, st= ruct dir_context *ctx) >> =A0=A0=A0 struct inode *inode =3D file_inode(file); >> =A0=A0=A0 struct super_block *sb =3D inode->i_sb; >> =A0=A0=A0 int len, err; >> -=A0=A0=A0 char strbuf[HFSPLUS_MAX_STRLEN + 1]; >> +=A0=A0=A0 char *strbuf; >> =A0=A0=A0 hfsplus_cat_entry entry; >> =A0=A0=A0 struct hfs_find_data fd; >> =A0=A0=A0 struct hfsplus_readdir_data *rd; >> @@ -139,6 +140,11 @@ static int hfsplus_readdir(struct file *file, s= truct dir_context *ctx) >> =A0=A0=A0 err =3D hfs_find_init(HFSPLUS_SB(sb)->cat_tree, &fd); >> =A0=A0=A0 if (err) >> =A0=A0=A0 =A0=A0=A0 return err; >> +=A0=A0=A0 strbuf =3D kmalloc(NLS_MAX_CHARSET_SIZE * HFSPLUS_MAX_STR= LEN + 1, GFP_KERNEL); >> +=A0=A0=A0 if (!strbuf) { >> +=A0=A0=A0 =A0=A0=A0 err =3D -ENOMEM; >> +=A0=A0=A0 =A0=A0=A0 goto failed_strbuf_alloc; >> +=A0=A0=A0 } >> =A0=A0=A0 hfsplus_cat_build_key(sb, fd.search_key, inode->i_ino, NUL= L); >> =A0=A0=A0 err =3D hfs_brec_find(&fd, hfs_find_rec_by_key); >> =A0=A0=A0 if (err) >> @@ -193,7 +199,7 @@ static int hfsplus_readdir(struct file *file, st= ruct dir_context *ctx) >> =A0=A0=A0 =A0=A0=A0 hfs_bnode_read(fd.bnode, &entry, fd.entryoffset, >> =A0=A0=A0 =A0=A0=A0 =A0=A0=A0 fd.entrylength); >> =A0=A0=A0 =A0=A0=A0 type =3D be16_to_cpu(entry.type); >> -=A0=A0=A0 =A0=A0=A0 len =3D HFSPLUS_MAX_STRLEN; >> +=A0=A0=A0 =A0=A0=A0 len =3D NLS_MAX_CHARSET_SIZE * HFSPLUS_MAX_STRL= EN; >> =A0=A0=A0 =A0=A0=A0 err =3D hfsplus_uni2asc(sb, &fd.key->cat.name, s= trbuf, &len); >> =A0=A0=A0 =A0=A0=A0 if (err) >> =A0=A0=A0 =A0=A0=A0 =A0=A0=A0 goto out; >> @@ -246,6 +252,8 @@ next: >> =A0=A0=A0 } >> =A0=A0=A0 memcpy(&rd->key, fd.key, sizeof(struct hfsplus_cat_key)); >> out: >> +=A0=A0=A0 kfree(strbuf); >> +failed_strbuf_alloc: >> =A0=A0=A0 hfs_find_exit(&fd); >> =A0=A0=A0 return err; >> } >> diff --git a/fs/hfsplus/xattr.c b/fs/hfsplus/xattr.c >> index 4e27edc..cfbb23d 100644 >> --- a/fs/hfsplus/xattr.c >> +++ b/fs/hfsplus/xattr.c >> @@ -8,6 +8,7 @@ >>=20 >> #include "hfsplus_fs.h" >> #include >> +#include >> #include "xattr.h" >> #include "acl.h" >>=20 >> @@ -645,8 +646,7 @@ ssize_t hfsplus_listxattr(struct dentry *dentry,= char *buffer, size_t size) >> =A0=A0=A0 struct hfs_find_data fd; >> =A0=A0=A0 u16 key_len =3D 0; >> =A0=A0=A0 struct hfsplus_attr_key attr_key; >> -=A0=A0=A0 char strbuf[HFSPLUS_ATTR_MAX_STRLEN + >> -=A0=A0=A0 =A0=A0=A0 =A0=A0=A0 XATTR_MAC_OSX_PREFIX_LEN + 1] =3D {0}= ; >> +=A0=A0=A0 char *strbuf; >> =A0=A0=A0 int xattr_name_len; >>=20 >> =A0=A0=A0 if ((!S_ISREG(inode->i_mode) && >> @@ -666,6 +666,13 @@ ssize_t hfsplus_listxattr(struct dentry *dentry= , char *buffer, size_t size) >> =A0=A0=A0 =A0=A0=A0 return err; >> =A0=A0=A0 } >>=20 >> +=A0=A0=A0 strbuf =3D kzalloc(NLS_MAX_CHARSET_SIZE * HFSPLUS_ATTR_MA= X_STRLEN + >> +=A0=A0=A0 =A0=A0=A0 =A0=A0=A0 XATTR_MAC_OSX_PREFIX_LEN + 1, GFP_KER= NEL); >> +=A0=A0=A0 if (!strbuf) { >> +=A0=A0=A0 =A0=A0=A0 res =3D -ENOMEM; >> +=A0=A0=A0 =A0=A0=A0 goto failed_strbuf_alloc; >> +=A0=A0=A0 } >> + >> =A0=A0=A0 err =3D hfsplus_find_attr(inode->i_sb, inode->i_ino, NULL,= &fd); >> =A0=A0=A0 if (err) { >> =A0=A0=A0 =A0=A0=A0 if (err =3D=3D -ENOENT) { >> @@ -692,7 +699,7 @@ ssize_t hfsplus_listxattr(struct dentry *dentry,= char *buffer, size_t size) >> =A0=A0=A0 =A0=A0=A0 if (be32_to_cpu(attr_key.cnid) !=3D inode->i_ino= ) >> =A0=A0=A0 =A0=A0=A0 =A0=A0=A0 goto end_listxattr; >>=20 >> -=A0=A0=A0 =A0=A0=A0 xattr_name_len =3D HFSPLUS_ATTR_MAX_STRLEN; >> +=A0=A0=A0 =A0=A0=A0 xattr_name_len =3D NLS_MAX_CHARSET_SIZE * HFSPL= US_ATTR_MAX_STRLEN; >> =A0=A0=A0 =A0=A0=A0 if (hfsplus_uni2asc(inode->i_sb, >> =A0=A0=A0 =A0=A0=A0 =A0=A0=A0 (const struct hfsplus_unistr *)&fd.key= ->attr.key_name, >> =A0=A0=A0 =A0=A0=A0 =A0=A0=A0 =A0=A0=A0 =A0=A0=A0 strbuf, &xattr_nam= e_len)) { >> @@ -718,6 +725,8 @@ ssize_t hfsplus_listxattr(struct dentry *dentry,= char *buffer, size_t size) >> =A0=A0=A0 } >>=20 >> end_listxattr: >> +=A0=A0=A0 kfree(strbuf); >> +failed_strbuf_alloc: >> =A0=A0=A0 hfs_find_exit(&fd); >> =A0=A0=A0 return res; >> } >> --=20 >> 1.9.0 > >--=20 >Anton Altaparmakov (replace at with @) >Senior Kernel Engineer, Tuxera Inc., http://www.tuxera.com/ >Linux NTFS maintainer > -- To unsubscribe from this list: send the line "unsubscribe linux-fsdevel= " in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html