Skip to contentSkip to conversation

jfs, special characters

DISCUSSION Frederik Eaton 3 participants 6 messages Feb 26, 2005 – Feb 28, 2005 VFS

lore
Share & tools
lore
#0 lore raw

Does JFS have problems with files with special characters in them?

$ ls
ls: 04-The_Dark_Of_The_Matin?e.ogg: No such file or directory
[1]$ rm 04-The_Dark_Of_The_Matin\?e.ogg
rm: cannot lstat `04-The_Dark_Of_The_Matin?e.ogg': No such file or directory

Frederik

#1 lore raw
Quoted 7 lines · Frederik Eaton

Does JFS have problems with files with special characters in them?

$ ls
ls: 04-The_Dark_Of_The_Matin?e.ogg: No such file or directory
[1]$ rm 04-The_Dark_Of_The_Matin\?e.ogg
rm: cannot lstat `04-The_Dark_Of_The_Matin?e.ogg': No such file or directory

Probably, you should ask this question on the jfs-discussion mailing list.

Sonny

#2 lore raw
In reply to Frederik Eaton
Quoted 1 line · Frederik Eaton

Does JFS have problems with files with special characters in them?

Sometimes. jfs attempts to store pathnames in 16-bit unicode, and uses the mount option iocharset to determine what to convert to/from.

In the 2.4 kernel, the default value of iocharset is determined by the kernel config option, CONFIG_NLS_DEFAULT. If an existing pathname exists that doesn't map to the iocharset, jfs has the problem you describe. Mounting with -o iocharset=utf8 should let you access any existing files.

In the 2.6 kernel, the default was changed to store each byte of the filename as a 16-bit value in the directory without using any conversion. There can still be a problem if an existing pathname has a value with a non-zero high-order byte. Again mounting with -o iocharset=utf8 will let you access all files.

Quoted 6 lines
$ ls
ls: 04-The_Dark_Of_The_Matin?e.ogg: No such file or directory
[1]$ rm 04-The_Dark_Of_The_Matin\?e.ogg
rm: cannot lstat `04-The_Dark_Of_The_Matin?e.ogg': No such file or directory

Frederik

Signature · 2 lines
David Kleikamp
IBM Linux Technology Center
#3 lore raw
Quoted 12 lines · Dave Kleikamp
Quoted 1 line · Frederik Eaton

Does JFS have problems with files with special characters in them?

Sometimes. jfs attempts to store pathnames in 16-bit unicode, and uses the mount option iocharset to determine what to convert to/from.

In the 2.4 kernel, the default value of iocharset is determined by the kernel config option, CONFIG_NLS_DEFAULT. If an existing pathname exists that doesn't map to the iocharset, jfs has the problem you describe. Mounting with -o iocharset=utf8 should let you access any existing files.

Quoted 10 lines

In the 2.6 kernel, the default was changed to store each byte of the filename as a 16-bit value in the directory without using any conversion. There can still be a problem if an existing pathname has a value with a non-zero high-order byte. Again mounting with -o iocharset=utf8 will let you access all files.

Quoted 4 lines
$ ls
ls: 04-The_Dark_Of_The_Matin?e.ogg: No such file or directory
[1]$ rm 04-The_Dark_Of_The_Matin\?e.ogg
rm: cannot lstat `04-The_Dark_Of_The_Matin?e.ogg': No such file or directory

Thanks, you were right that I recently upgraded to 2.6 (actually I transferred the hard drive between machines), and with a "mount /home -o remount,iocharset=utf8" at least I can access the files now. But the names aren't what they should be, for instance I see:

04-The_Dark_Of_The_MatinΘe.ogg

(that's a theta). Were they supposed to be correct?

As an aside, now, I just tried "mount /home -o remount,iocharset=iso8859-1", and got same problem

ls: '/home/shared/frederik/Franz_Ferdinand/Franz_Ferdinand/04-The_Dark_Of_The_Matin?e.ogg': No such file or directory

But when I repeated the "mount /home -o remount,iocharset=utf8" command, the listing didn't show the name with a theta character, but instead with two question marks:

04-The_Dark_Of_The_Matin??e.ogg

Strange behavior. I.e., upon changing the mount options and then changing them back I get a different listing.

Frederik
-
To unsubscribe from this list: send the line "unsubscribe linux-fsdevel" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html

#4 lore raw
Quoted 8 lines · Frederik Eaton

Thanks, you were right that I recently upgraded to 2.6 (actually I transferred the hard drive between machines), and with a "mount /home -o remount,iocharset=utf8" at least I can access the files now. But the names aren't what they should be, for instance I see:

04-The_Dark_Of_The_MatinΘe.ogg

(that's a theta). Were they supposed to be correct?

Not necessarily. Even if the character returned is correct utf8, the terminal (xterm?) displaying the name may be assuming a different character set. It's also possible that on the older system, CONFIG_NLS_DEFAULT was not set to the same locale that the system used, so the character stored in the name was not the right unicode, but it was accessed consistently on the old system so it worked as long as nothing changed.

Quoted 2 lines

As an aside, now, I just tried "mount /home -o remount,iocharset=iso8859-1", and got same problem

This is no surprise. iso8859-1 has the same mapping as iocharset=none (the 2.6 default). (The "none" keyword is first supported in 2.6.11-rc4-mm1 and 2.4.30-pre2, so you probably won't be able to use it.)

Quoted 10 lines

ls: '/home/shared/frederik/Franz_Ferdinand/Franz_Ferdinand/04-The_Dark_Of_The_Matin?e.ogg': No such file or directory

But when I repeated the "mount /home -o remount,iocharset=utf8" command, the listing didn't show the name with a theta character, but instead with two question marks:

04-The_Dark_Of_The_Matin??e.ogg

Strange behavior. I.e., upon changing the mount options and then changing them back I get a different listing.

That I wouldn't expect. I really have no clue why it isn't consistent.

The easiest way to "fix" these files (assuming there aren't too many) is to mount with iocharset=utf8, rename the files using only ascii characters, remount again with iocharset=iso8859-1, and rename the files to their proper names. After that you should be able to mount without any iocharset flag and get sane behavior.

Quoted 1 line

Frederik

Signature · 7 lines
David Kleikamp
IBM Linux Technology Center

-
To unsubscribe from this list: send the line "unsubscribe linux-fsdevel" in
the body of a message to [email protected]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
#5 lore raw
Quoted 5 lines

The easiest way to "fix" these files (assuming there aren't too many) is to mount with iocharset=utf8, rename the files using only ascii characters, remount again with iocharset=iso8859-1, and rename the files to their proper names. After that you should be able to mount without any iocharset flag and get sane behavior.

OK, thanks.

Frederik