Skip to contentSkip to conversation

Bug#333776: linux-2.6: vfat driver in 2.6.12 is not properly case-insensitive

DISCUSSION Horms 5 participants 8 messages Oct 28, 2005 – Oct 29, 2005 3 messages not in the archive

lore
Share & tools
lore

A message that is not in the archive (sent off-list or not yet received): [email protected]

A message that is not in the archive (sent off-list or not yet received): [email protected]

A message that is not in the archive (sent off-list or not yet received): [email protected]

Horms

author
#3 lore raw

Ogawa-san,

I'm bringing this to you attention because a) I'm not sure who to ask and b) I'm not sure what the correct behaviour is.

When a vfat filesystem is mounted isocharset=iso8859-1, then the following works:

touch a.txt
ls A.txt

But when it is mounted isocharset=utf8, then ls complains, file not found:

touch a.txt
ls A.txt

That is, in utf8, a =! A on vfat, and thus its not case insensitive as one might expect.

I took a quick look in fs/nls/nls_utf8.c and I see that this is intentional.

static struct nls_table table = {

        .charset        = "utf8",
        .uni2char       = uni2char,
        .char2uni       = char2uni,
        .charset2lower  = identity,     /* no conversion */
        .charset2upper  = identity,
        .owner          = THIS_MODULE,

};

I guess it is charset2lower or charset2upper that vfat is calling, which make no conversion, thus leading to the problem I outlined above.

My question is: Is this behaviour correct, or is it a bug?

Signature · 1 line
Horms
#4 lore raw
Quoted 7 lines · Horms

static struct nls_table table = {

        .charset        = "utf8",
        .uni2char       = uni2char,
        .char2uni       = char2uni,
        .charset2lower  = identity,     /* no conversion */
        .charset2upper  = identity,
        .owner          = THIS_MODULE,
Quoted 6 lines

};

I guess it is charset2lower or charset2upper that vfat is calling, which make no conversion, thus leading to the problem I outlined above.

My question is: Is this behaviour correct, or is it a bug?

This is known bug. For fixing this bug cleanly, we will need to much change the both of nls and filesystems.

Thanks.

Signature · 1 line
OGAWA Hirofumi <[email protected]>
#5 lore raw
Quoted 11 lines · OGAWA Hirofumi
Quoted 9 lines · Horms

static struct nls_table table = {

        .charset        = "utf8",
        .uni2char       = uni2char,
        .char2uni       = char2uni,
        .charset2lower  = identity,     /* no conversion */
        .charset2upper  = identity,
        .owner          = THIS_MODULE,

};

Quoted 7 lines
Quoted 4 lines

I guess it is charset2lower or charset2upper that vfat is calling, which make no conversion, thus leading to the problem I outlined above.

My question is: Is this behaviour correct, or is it a bug?

This is known bug. For fixing this bug cleanly, we will need to much change the both of nls and filesystems.

And fatfs has "utf8" option, probably the behavior is preferable than "iocharset=utf8". However, unfortunately "utf8" has problem too.

Signature · 1 line
OGAWA Hirofumi <[email protected]>
#6 lore raw
Quoted 16 lines · OGAWA Hirofumi
Quoted 14 lines · OGAWA Hirofumi
Quoted 12 lines · Horms

static struct nls_table table = {

        .charset        = "utf8",
        .uni2char       = uni2char,
        .char2uni       = char2uni,
        .charset2lower  = identity,     /* no conversion */
        .charset2upper  = identity,
        .owner          = THIS_MODULE,

};

I guess it is charset2lower or charset2upper that vfat is calling, which make no conversion, thus leading to the problem I outlined above.

Quoted 7 lines
Quoted 4 lines
Quoted 1 line

My question is: Is this behaviour correct, or is it a bug?

This is known bug. For fixing this bug cleanly, we will need to much change the both of nls and filesystems.

And fatfs has "utf8" option, probably the behavior is preferable than "iocharset=utf8". However, unfortunately "utf8" has problem too.

Thanks

#7 lore raw
In reply to OGAWA Hirofumi

Hi,

Quoted 8 lines · OGAWA Hirofumi
Quoted 4 lines · Horms

I guess it is charset2lower or charset2upper that vfat is calling, which make no conversion, thus leading to the problem I outlined above.

My question is: Is this behaviour correct, or is it a bug?

This is known bug. For fixing this bug cleanly, we will need to much change the both of nls and filesystems.

Using per locale collation sequences? :-)

Do you know, how Windows handles the problem of differing collation sequences on the file system? Or is the file system always dependend on the locale of the Windows version, which created the file system?

I'm so happy, that Unix is not case insensitive :-)

Regards

Ingo Oeser

#8 lore raw
Quoted 7 lines · Ingo Oeser
Quoted 2 lines

This is known bug. For fixing this bug cleanly, we will need to much change the both of nls and filesystems.

Using per locale collation sequences? :-)

Do you know, how Windows handles the problem of differing collation sequences on the file system?

I don't know. Why do we need to care the collation sequences here?

Quoted 2 lines

Or is the file system always dependend on the locale of the Windows version, which created the file system?

Probably, yes. I think we need to know on-disk filename's code set.

Signature · 1 line
OGAWA Hirofumi <[email protected]>
#9 lore raw
Quoted 9 lines · OGAWA Hirofumi
Quoted 7 lines · Ingo Oeser
Quoted 2 lines

This is known bug. For fixing this bug cleanly, we will need to much change the both of nls and filesystems.

Using per locale collation sequences? :-)

Do you know, how Windows handles the problem of differing collation sequences on the file system?

Quoted 6 lines

I don't know. Why do we need to care the collation sequences here?

Quoted 2 lines

Or is the file system always dependend on the locale of the Windows version, which created the file system?

Probably, yes. I think we need to know on-disk filename's code set.

If FAT stores the filenames in 8 bits (non-UTF) then yes, it will be in the current locale/code page of the Windows system writing them (e.g. that happens with the names of EAs in NTFS).

If the names are stored in 16-bit Unicode like on NTFS then obviously they are completely locale/code page independent. (Makes my life in NTFS a _lot_ easier. Especially since the NTFS volume contains an upcase table for the full 16-bit Unicode which we load and use to do upcasing for the case insensitive comparisons...)

Best regards,

Anton

Signature · 4 lines
Anton Altaparmakov <aia21 at cam.ac.uk> (replace at with @)
Unix Support, Computing Service, University of Cambridge, CB2 3QH, UK
Linux NTFS maintainer / IRC: #ntfs on irc.freenode.net
WWW: http://linux-ntfs.sf.net/ & http://www-stu.christs.cam.ac.uk/~aia21/
#10 lore raw
Quoted 11 lines · Anton Altaparmakov
Quoted 1 line

Probably, yes. I think we need to know on-disk filename's code set.

If FAT stores the filenames in 8 bits (non-UTF) then yes, it will be in the current locale/code page of the Windows system writing them (e.g. that happens with the names of EAs in NTFS).

If the names are stored in 16-bit Unicode like on NTFS then obviously they are completely locale/code page independent. (Makes my life in NTFS a _lot_ easier. Especially since the NTFS volume contains an upcase table for the full 16-bit Unicode which we load and use to do upcasing for the case insensitive comparisons...)

Yes, I got to know it from fs/ntfs/*. :) Unfortunately, FAT stores 8/16bits codeset filename always. (Unicode (UCS2?) is stored in only case of longname.)

Thanks.

Signature · 1 line
OGAWA Hirofumi <[email protected]>