bug? VFAT copy problem
Hi all
I've recently found a problem with the VFAT module.
I'm working on a gentoo box with kernel 2.6.17 gentoo patched (recently upgraded from 2.6.15)
I've got a samba share on a reiser partition and a daily cron will copy its content to an external usb disk with vfat file system. The copy is done via bash scripting and some checks are done before mirroring the files. btw _some_ random files are copied lowering all chars in the file name.
Here is an example:
# pwd /home/b/dir # ls -l -rwxr-xr-x 1 b users 676389 Aug 10 2004 DSCN5967(1).JPG -rwxr-xr-x 1 b users 710090 Aug 10 2004 DSCN5968(1).JPG -rwxr-xr-x 1 b users 732903 Aug 10 2004 DSCN5981.JPG.rem.2006-10-27-1543 -rwxr-xr-x 1 b users 622595 Aug 10 2004 DSCN5982.JPG.rem.2006-10-27-1543 # cp -v * /mnt/iomega/dir/ `DSCN5967(1).JPG' -> `/mnt/iomega/dir/DSCN5967(1).JPG' `DSCN5968(1).JPG' -> `/mnt/iomega/dir/DSCN5968(1).JPG' `DSCN5981.JPG.rem.2006-10-27-1543' -> `/mnt/iomega/dir/DSCN5981.JPG.rem.2006-10-27-1543' `DSCN5982.JPG.rem.2006-10-27-1543' -> `/mnt/iomega/dir/DSCN5982.JPG.rem.2006-10-27-1543' # ls -l /mnt/iomega/dir/ total 2784 -rwxr-xr-x 1 b users 676389 Oct 27 16:55 DSCN5967(1).JPG -rwxr-xr-x 1 b users 710090 Oct 27 16:55 DSCN5968(1).JPG -rwxr-xr-x 1 b users 732903 Oct 27 16:55 dscn5981.jpg.rem.2006-10-27-1543 -rwxr-xr-x 1 b users 622595 Oct 27 16:55 dscn5982.jpg.rem.2006-10-27-1543
even if 'cp' gives correct destination names, an 'ls' shows that this is not always true.
Here is a second quick experiment I've made:
# pwd /home/b/dir # cp -v DSCN5981.JPG.rem.2006-10-27-1543 DSCN5981.JPG.rem.9999-99-99-9999 `DSCN5981.JPG.rem.2006-10-27-1543' -> `DSCN5981.JPG.rem.9999-99-99-9999' # cp -v DSCN5981.JPG.rem.9999-99-99-9999 /mnt/iomega/dir/ `DSCN5981.JPG.rem.9999-99-99-9999' -> `/mnt/iomega/dir/DSCN5981.JPG.rem.9999-99-99-9999' # ls -l /mnt/iomega/dir/ total 2784 -rwxr-xr-x 1 b users 676389 Oct 27 16:55 DSCN5967(1).JPG -rwxr-xr-x 1 b users 710090 Oct 27 16:55 DSCN5968(1).JPG -rwxr-xr-x 1 b users 732903 Oct 27 17:01 DSCN5981.JPG.rem.9999-99-99-9999 -rwxr-xr-x 1 b users 732903 Oct 27 16:55 dscn5981.jpg.rem.2006-10-27-1543 -rwxr-xr-x 1 b users 622595 Oct 27 16:55 dscn5982.jpg.rem.2006-10-27-1543
if the code will not show up correctly you can find the full code listings at this url:
http://forums.gentoo.org/viewtopic-t-511205-highlight-.html
(thread in italian, the code is quite self explaining)
Thanks in advance, I can do any further test if you need, and provide any other info on the system.
Signature · 6 lines
Matteo 'Peach' Pescarin ICQ UIN = 71110111 Jabber ID = [email protected] Web Site = http://www.smartart.it GeCHI = http://www.gechi.it
Quoted 5 lines · The Peach
I've recently found a problem with the VFAT module.
I'm working on a gentoo box with kernel 2.6.17 gentoo patched (recently upgraded from 2.6.15)I've got a samba share on a reiser partition and a daily cron will copy its content to an external usb disk with vfat file system. The copy is done via bash scripting and some checks are done before mirroring the files. btw _some_ random files are copied lowering all chars in the file name.
I couldn't reproduce this for now. Could you tell mount options which you used? and after mount, "cat /proc/mounts", please.
Quoted 20 lines
Here is an example:
# pwd /home/b/dir # ls -l(quoted diff) +0
-4-rwxr-xr-x 1 b users 676389 Aug 10 2004 DSCN5967(1).JPG -rwxr-xr-x 1 b users 710090 Aug 10 2004 DSCN5968(1).JPG -rwxr-xr-x 1 b users 732903 Aug 10 2004 DSCN5981.JPG.rem.2006-10-27-1543 -rwxr-xr-x 1 b users 622595 Aug 10 2004 DSCN5982.JPG.rem.2006-10-27-1543# cp -v * /mnt/iomega/dir/ `DSCN5967(1).JPG' -> `/mnt/iomega/dir/DSCN5967(1).JPG' `DSCN5968(1).JPG' -> `/mnt/iomega/dir/DSCN5968(1).JPG' `DSCN5981.JPG.rem.2006-10-27-1543' -> `/mnt/iomega/dir/DSCN5981.JPG.rem.2006-10-27-1543' `DSCN5982.JPG.rem.2006-10-27-1543' -> `/mnt/iomega/dir/DSCN5982.JPG.rem.2006-10-27-1543' # ls -l /mnt/iomega/dir/ total 2784(quoted diff) +0
-4-rwxr-xr-x 1 b users 676389 Oct 27 16:55 DSCN5967(1).JPG -rwxr-xr-x 1 b users 710090 Oct 27 16:55 DSCN5968(1).JPG -rwxr-xr-x 1 b users 732903 Oct 27 16:55 dscn5981.jpg.rem.2006-10-27-1543 -rwxr-xr-x 1 b users 622595 Oct 27 16:55 dscn5982.jpg.rem.2006-10-27-1543
Signature · 1 line
OGAWA Hirofumi <[email protected]>
Quoted 2 lines · On Tue, 21 Nov 2006 02:32:43 +0900 OGAWA Hirofumi…
I couldn't reproduce this for now. Could you tell mount options which you used? and after mount, "cat
/proc/mounts", please.
# mount | grep vfat /dev/sdb1 on /mnt/iomega type vfat (rw,uid=1000,gid=100,codepage=850,iocharset=iso8859-15)
it seems only related to those kind of files, but I don't know how to inspect the "file properties" and why these files behave like this.
As you can see and with a strace made on cp, the files _seems_ to be copied with the correct case, whilst it isn't, as seen with "ls". This and other things let me think is a vfat problem.
Signature · 6 lines
Matteo 'Peach' Pescarin ICQ UIN = 71110111 Jabber ID = [email protected] Web Site = http://www.smartart.it GeCHI = http://www.gechi.it
Quoted 11 lines · The Peach
Quoted 2 lines · On Tue, 21 Nov 2006 02:32:43 +0900 OGAWA Hirofumi…
I couldn't reproduce this for now. Could you tell mount options which you used? and after mount, "cat
/proc/mounts", please.# mount | grep vfat /dev/sdb1 on /mnt/iomega type vfat (rw,uid=1000,gid=100,codepage=850,iocharset=iso8859-15)it seems only related to those kind of files, but I don't know how to inspect the "file properties" and why these files behave like this.
As you can see and with a strace made on cp, the files _seems_ to be copied with the correct case, whilst it isn't, as seen with "ls". This and other things let me think is a vfat problem.
Hmm... This may be the dentry cache handling problem of fat.
Can you try the attached debug patch? And if you comment-in the following parts, does this problem fix?
@@ -787,6 +830,9 @@ static int vfat_rmdir(struct inode *dir,
clear_nlink(inode);
inode->i_mtime = inode->i_atime = CURRENT_TIME_SEC; fat_detach(inode);
+ /* need to revalidate for next create */
+ table = (sbi->options.name_check == 's') ? 3 : 1;
+// dentry->d_op = &vfat_dentry_ops[table];
@@ -811,6 +858,9 @@ static int vfat_unlink(struct inode *dir
clear_nlink(inode);
inode->i_mtime = inode->i_atime = CURRENT_TIME_SEC; fat_detach(inode);
+ /* need to revalidate for next create */
+ table = (sbi->options.name_check == 's') ? 3 : 1;
+// dentry->d_op = &vfat_dentry_ops[table];
Signature · 1 line
OGAWA Hirofumi <[email protected]>
---
fs/vfat/namei.c | 56 ++++++++++++++++++++++++++++++++++++++++++++++++++++++-- 1 file changed, 54 insertions(+), 2 deletions(-)
fs/vfat/namei.c renamed from fs/vfat/namei.c~vfat-debug +54 -2
@@ -29,6 +29,8 @@ static int vfat_revalidate(struct dentry {int ret =1 ; +printk ("%s: name %s, nd %p, flags %08x\n" , __FUNCTION__, + dentry->d_name.name, nd, nd ? nd->flags :0 );if (!dentry->d_inode && nd && !(nd->flags &LOOKUP_CONTINUE ) && (nd->flags &LOOKUP_CREATE ))/*
@@ -83,6 +85,10 @@ static int vfat_hashi(struct dentry *den name = qstr->name; len =vfat_striptail_len (qstr); +printk ("%s: parent %p, parent->d_op %p\n" , + __FUNCTION__, dentry, dentry->d_op); +printk ("%s: parent %s, name %s\n" , __FUNCTION__, + dentry->d_name.name, name); hash =init_name_hash ();while (len--)
@@ -100,6 +106,9 @@ static int vfat_cmpi(struct dentry *dentstruct nls_table *t =MSDOS_SB (dentry->d_inode->i_sb)->nls_io;unsigned int alen, blen; +printk ("%s: parent %p, parent->d_op %p\n" , + __FUNCTION__, dentry, dentry->d_op); +printk ("%s: a %s, b %s\n" , __FUNCTION__, a->name, b->name);/* A filename cannot end in '.' or we treat it like it has none */ alen =vfat_striptail_len (a); blen =vfat_striptail_len (b);
@@ -659,6 +668,34 @@ static int vfat_add_entry(struct inode *if (err) goto cleanup; +{ +int i; +for (i = nr_slots -1 ; i >=0 ; i--) { +if (i == (nr_slots -1 )) { +struct msdos_dir_entry *de = + (struct msdos_dir_entry *)&slots[i]; +printk ("%s: %d: %c%c%c%c%c%c%c%c, %c%c%c\n" , + __FUNCTION__, + nr_slots -1 - i, + de->name[0 ], de->name[1 ], de->name[2 ], + de->name[3 ], de->name[4 ], de->name[5 ], + de->name[6 ], de->name[7 ], + de->ext[0 ], de->ext[1 ], de->ext[2 ]); +continue ; + } +printk ("%s: %d: %c%c%c%c%c, %c%c%c%c%c%c, %c%c" , + __FUNCTION__, + nr_slots -1 - i, + slots[i].name0_4[0 ], slots[i].name0_4[2 ], + slots[i].name0_4[4 ], slots[i].name0_4[6 ], + slots[i].name0_4[8 ], + slots[i].name5_10[0 ], slots[i].name5_10[2 ], + slots[i].name5_10[4 ], slots[i].name5_10[6 ], + slots[i].name5_10[8 ], slots[i].name5_10[10 ], + slots[i].name11_12[0 ], slots[i].name11_12[2 ]); +printk ("\n" ); + } +} err =fat_add_entries (dir, slots, nr_slots, sinfo);if (err) goto cleanup;
@@ -692,6 +729,7 @@ static struct dentry *vfat_lookup(structstruct dentry *alias;int err, table; +printk ("%s: name %s\n" , __FUNCTION__, dentry->d_name.name);lock_kernel (); table = (MSDOS_SB (sb)->options.name_check =='s' ) ?2 :0 ; dentry->d_op = &vfat_dentry_ops[table];
@@ -714,6 +752,7 @@ static struct dentry *vfat_lookup(structelse {iput (inode);unlock_kernel (); +printk ("%s: d_op %p\n" , __FUNCTION__, alias->d_op);return alias; }
@@ -726,6 +765,7 @@ error:if (dentry) { dentry->d_op = &vfat_dentry_ops[table]; dentry->d_time = dentry->d_parent->d_inode->i_version; +printk ("%s: d_op %p\n" , __FUNCTION__, dentry->d_op); }return dentry; }
@@ -742,6 +782,7 @@ static int vfat_create(struct inode *dirlock_kernel (); ts =CURRENT_TIME_SEC ; +printk ("%s: name %s\n" , __FUNCTION__, dentry->d_name.name); err =vfat_add_entry (dir, &dentry->d_name,0 ,0 , &ts, &sinfo);if (err) goto out;
@@ -761,14 +802,16 @@ static int vfat_create(struct inode *dird_instantiate (dentry, inode); out:unlock_kernel (); +printk ("%s: err %d\n" , __FUNCTION__, err);return err; }static int vfat_rmdir (struct inode *dir,struct dentry *dentry) {struct inode *inode = dentry->d_inode; +struct msdos_sb_info *sbi =MSDOS_SB (inode->i_sb);struct fat_slot_info sinfo; -int err; +int err, table;lock_kernel ();
@@ -787,6 +830,9 @@ static int vfat_rmdir(struct inode *dir,clear_nlink (inode); inode->i_mtime = inode->i_atime =CURRENT_TIME_SEC ;fat_detach (inode); +/* need to revalidate for next create */ + table = (sbi->options.name_check =='s' ) ?3 :1 ; +// dentry->d_op = &vfat_dentry_ops[table]; out:unlock_kernel ();
@@ -796,8 +842,9 @@ out:static int vfat_unlink (struct inode *dir,struct dentry *dentry) {struct inode *inode = dentry->d_inode; +struct msdos_sb_info *sbi =MSDOS_SB (inode->i_sb);struct fat_slot_info sinfo; -int err; +int err, table;lock_kernel ();
@@ -811,6 +858,9 @@ static int vfat_unlink(struct inode *dirclear_nlink (inode); inode->i_mtime = inode->i_atime =CURRENT_TIME_SEC ;fat_detach (inode); +/* need to revalidate for next create */ + table = (sbi->options.name_check =='s' ) ?3 :1 ; +// dentry->d_op = &vfat_dentry_ops[table]; out:unlock_kernel ();
@@ -1019,6 +1069,8 @@ static int vfat_fill_super(struct super_ sb->s_root->d_op = &vfat_dentry_ops[0 ]; else sb->s_root->d_op = &vfat_dentry_ops[2]; +printk ("%s: s_root %p, d_op %p\n" , __FUNCTION__, + sb->s_root, sb->s_root->d_op);return 0 ; }
_
Hi,
Have vfat a limit of a file size when copy ? I tried copy 4.4 Megas to a vfat partition but I got a limit size exceed
Thanks,
--
Sérgio M. B.
Quoted 33 lines · OGAWA Hirofumi
Quoted 11 lines · The Peach
Quoted 2 lines · On Tue, 21 Nov 2006 02:32:43 +0900 OGAWA Hirofumi…
I couldn't reproduce this for now. Could you tell mount options which you used? and after mount, "cat
/proc/mounts", please.# mount | grep vfat /dev/sdb1 on /mnt/iomega type vfat (rw,uid=1000,gid=100,codepage=850,iocharset=iso8859-15)it seems only related to those kind of files, but I don't know how to inspect the "file properties" and why these files behave like this.
As you can see and with a strace made on cp, the files _seems_ to be copied with the correct case, whilst it isn't, as seen with "ls". This and other things let me think is a vfat problem.Hmm... This may be the dentry cache handling problem of fat.
Can you try the attached debug patch? And if you comment-in the following parts, does this problem fix?
fs/vfat/namei.c renamed from fs/vfat/namei.c~vfat-debug +6
-0@@ -787,6 +830,9 @@ static int vfat_rmdir(struct inode *dir,clear_nlink (inode); inode->i_mtime = inode->i_atime =CURRENT_TIME_SEC ;fat_detach (inode); +/* need to revalidate for next create */ + table = (sbi->options.name_check =='s' ) ?3 :1 ; +// dentry->d_op = &vfat_dentry_ops[table]; @@ -811,6 +858,9 @@ static int vfat_unlink(struct inode *dirclear_nlink (inode); inode->i_mtime = inode->i_atime =CURRENT_TIME_SEC ;fat_detach (inode); +/* need to revalidate for next create */ + table = (sbi->options.name_check =='s' ) ?3 :1 ; +// dentry->d_op = &vfat_dentry_ops[table];
Hi Sérgio :)
* Sergio Monteiro Basto <[email protected]> dixit:
Quoted 1 line
Have vfat a limit of a file size when copy ?
2GB, if I recall correctly. FAT32 itself has a limit of 4GB-1 for
file size, but Linux restricts it even more (don't ask me why).
Raúl Núñez de Arenas Coronado
Signature · 2 lines
Linux Registered User 88736 | http://www.dervishd.net It's my PC and I'll cry if I want to... RAmen!
Quoted 1 line · On Wed, 22 Nov 2006 00:46:43 +0900 OGAWA Hirofumi…
Can you try the attached debug patch?
Here is the debug for the copy of a normal file over vfat:
# cp -v DSCN5970\(1\).JPG /mnt/loop/ `DSCN5970(1).JPG' -> `/mnt/loop/DSCN5970(1).JPG'
and the output on dmesg:vfat_hashi: parent d528a514, parent->d_op e0fbf5c0vfat_hashi: parent /, name DSCN5970(1).JPGvfat_lookup: name DSCN5970(1).JPGvfat_hashi: parent d528a514, parent->d_op e0fbf5c0vfat_hashi: parent /, name DSCN5970(1).JPGvfat_cmpi: parent d528a514, parent->d_op e0fbf5c0vfat_cmpi: a DSCN5970(1).JPG, b DSCN5970(1).JPGvfat_revalidate: name DSCN5970(1).JPG, nd c9f1af30, flags 00000300vfat_lookup: name DSCN5970(1).JPGvfat_create: name DSCN5970(1).JPGvfat_add_entry: 0: DSCN59~1, JPGvfat_add_entry: 1: DSCN5, 970(1), .Jvfat_add_entry: 2: PGvfat_create: err 0
and the copy of an "abnormal" file:
# cp -v DSCN5980.JPG /mnt/loop/ `DSCN5980.JPG' -> `/mnt/loop/DSCN5980.JPG'
and its dmesg output:vfat_hashi: parent d528a514, parent->d_op e0fbf5c0vfat_hashi: parent /, name DSCN5980.JPGvfat_cmpi: parent d528a514, parent->d_op e0fbf5c0vfat_cmpi: a dscn5980.jpg, b DSCN5980.JPGvfat_hashi: parent d528a514, parent->d_op e0fbf5c0vfat_hashi: parent /, name DSCN5980.JPGvfat_cmpi: parent d528a514, parent->d_op e0fbf5c0vfat_cmpi: a dscn5980.jpg, b DSCN5980.JPGvfat_create: name dscn5980.jpgvfat_add_entry: 0: DSCN5980, JPGvfat_add_entry: 1: dscn5, 980.jp, gvfat_create: err 0
then:
# ls -l /mnt/loop/ totale 1363 -rwxr-xr-x 1 root root 695514 22 nov 16:21 DSCN5970(1).JPG -rwxr-xr-x 1 root root 699770 22 nov 16:25 dscn5980.jpg
and its output:vfat_hashi: parent d528a514, parent->d_op e0fbf5c0vfat_hashi: parent /, name DSCN5970(1).JPGvfat_cmpi: parent d528a514, parent->d_op e0fbf5c0vfat_cmpi: a DSCN5970(1).JPG, b DSCN5970(1).JPGvfat_revalidate: name DSCN5970(1).JPG, nd c9f18ef8, flags 00000000vfat_lookup: name DSCN5970(1).JPGvfat_hashi: parent d528a514, parent->d_op e0fbf5c0vfat_hashi: parent /, name dscn5980.jpgvfat_cmpi: parent d528a514, parent->d_op e0fbf5c0vfat_cmpi: a dscn5980.jpg, b dscn5980.jpg
Quoted 2 lines
And if you comment-in the
following parts, does this problem fix?
I will repatch asap and post the result.
Signature · 6 lines
Matteo 'Peach' Pescarin ICQ UIN = 71110111 Jabber ID = [email protected] Web Site = http://www.smartart.it GeCHI = http://www.gechi.it
Quoted 5 lines · DervishD
* Sergio Monteiro Basto <[email protected]> dixit:
Quoted 1 line
Have vfat a limit of a file size when copy ?
2GB, if I recall correctly. FAT32 itself has a limit of 4GB-1 for
file size, but Linux restricts it even more (don't ask me why).
It was fixed already, 2.6.x can handle 4GB-1. 2.4.x has limit of 2GB-1 (there is patch).
Signature · 1 line
OGAWA Hirofumi <[email protected]>
Quoted 5 lines · The Peach
and the copy of an "abnormal" file:
# cp -v DSCN5980.JPG /mnt/loop/ `DSCN5980.JPG' -> `/mnt/loop/DSCN5980.JPG'
Quoted 13 lines
and its dmesg output:
vfat_hashi: parentd528a514,parent->d_ope0fbf5c0vfat_hashi: parent /, name DSCN5980.JPGvfat_cmpi: parentd528a514,parent->d_ope0fbf5c0vfat_cmpi: a dscn5980.jpg, b DSCN5980.JPGvfat_hashi: parentd528a514,parent->d_ope0fbf5c0vfat_hashi: parent /, name DSCN5980.JPGvfat_cmpi: parentd528a514,parent->d_ope0fbf5c0vfat_cmpi: a dscn5980.jpg, b DSCN5980.JPGvfat_create: name dscn5980.jpgvfat_add_entry: 0: DSCN5980, JPGvfat_add_entry: 1: dscn5, 980.jp, gvfat_create: err 0
This seems to be nasty problem of dentry cache handling.
Quoted 4 lines
Quoted 2 lines
And if you comment-in the
following parts, does this problem fix?I will repatch asap and post the result.
Thanks. Probably instead of some overheads, this patch will fix a problem.
Signature · 1 line
OGAWA Hirofumi <[email protected]>
Hi OGAWA :)
* OGAWA Hirofumi <[email protected]> dixit:
Quoted 9 lines
Quoted 5 lines · DervishD
* Sergio Monteiro Basto <[email protected]> dixit:
Quoted 1 line
Have vfat a limit of a file size when copy ?
2GB, if I recall correctly. FAT32 itself has a limit of 4GB-1 for
file size, but Linux restricts it even more (don't ask me why).It was fixed already, 2.6.x can handle 4GB-1. 2.4.x has limit of 2GB-1 (there is patch).
Cool! Thanks for the information :)
Raúl Núñez de Arenas Coronado
Signature · 2 lines
Linux Registered User 88736 | http://www.dervishd.net It's my PC and I'll cry if I want to... RAmen!
Quoted 1 line · On Thu, 23 Nov 2006 01:15:46 +0900 OGAWA Hirofumi…
Thanks. Probably instead of some overheads, this patch will fix a problem.
here it is, did the very same experiments than before. Normal file:
# cp -v DSCN5970\(1\).JPG /mnt/loop/ `DSCN5970(1).JPG' -> `/mnt/loop/DSCN5970(1).JPG'
dmesg:vfat_hashi: parent d1801e94, parent->d_op e0fbc620vfat_hashi: parent /, name DSCN5970(1).JPGvfat_cmpi: parent d1801e94, parent->d_op e0fbc620vfat_cmpi: a DSCN5970(1).JPG, b DSCN5970(1).JPGvfat_revalidate: name DSCN5970(1).JPG, nd d65eaeec, flags 00000001vfat_lookup: name DSCN5970(1).JPGvfat_hashi: parent d1801e94, parent->d_op e0fbc620vfat_hashi: parent /, name DSCN5970(1).JPGvfat_cmpi: parent d1801e94, parent->d_op e0fbc620vfat_cmpi: a DSCN5970(1).JPG, b DSCN5970(1).JPGvfat_revalidate: name DSCN5970(1).JPG, nd d65eaf24, flags 00000300vfat_lookup: name DSCN5970(1).JPGvfat_create: name DSCN5970(1).JPGvfat_add_entry: 0: DSCN59~1, JPGvfat_add_entry: 1: DSCN5, 970(1), .Jvfat_add_entry: 2: PGvfat_create: err 0
Abnormal file:
# cp -v DSCN5980.JPG /mnt/loop/ `DSCN5980.JPG' -> `/mnt/loop/DSCN5980.JPG'
dmesg:vfat_hashi: parent d1801e94, parent->d_op e0fbc620vfat_hashi: parent /, name DSCN5980.JPGvfat_cmpi: parent d1801e94, parent->d_op e0fbc620vfat_cmpi: a dscn5980.jpg, b DSCN5980.JPGvfat_revalidate: name dscn5980.jpg, nd d65eaeec, flags 00000001vfat_lookup: name DSCN5980.JPGvfat_hashi: parent d1801e94, parent->d_op e0fbc620vfat_hashi: parent /, name DSCN5980.JPGvfat_cmpi: parent d1801e94, parent->d_op e0fbc620vfat_cmpi: a DSCN5980.JPG, b DSCN5980.JPGvfat_revalidate: name DSCN5980.JPG, nd d65eaf24, flags 00000300vfat_lookup: name DSCN5980.JPGvfat_create: name DSCN5980.JPGvfat_add_entry: 0: DSCN5980, JPGvfat_create: err 0
and:
# ls -l /mnt/loop/ totale 1363 -rwxr-xr-x 1 root root 695514 22 nov 20:04 DSCN5970(1).JPG -rwxr-xr-x 1 root root 699770 22 nov 20:07 dscn5980.jpg
dmesg:vfat_hashi: parent d1801e94, parent->d_op e0fbc620vfat_hashi: parent /, name DSCN5970(1).JPGvfat_cmpi: parent d1801e94, parent->d_op e0fbc620vfat_cmpi: a DSCN5970(1).JPG, b DSCN5970(1).JPGvfat_revalidate: name DSCN5970(1).JPG, nd d2512eec, flags 00000000vfat_lookup: name DSCN5970(1).JPGvfat_hashi: parent d1801e94, parent->d_op e0fbc620vfat_hashi: parent /, name dscn5980.jpgvfat_cmpi: parent d1801e94, parent->d_op e0fbc620vfat_cmpi: a DSCN5980.JPG, b dscn5980.jpgvfat_revalidate: name DSCN5980.JPG, nd d2512eec, flags 00000000
:(
Signature · 6 lines
Matteo 'Peach' Pescarin ICQ UIN = 71110111 Jabber ID = [email protected] Web Site = http://www.smartart.it GeCHI = http://www.gechi.it
Quoted 56 lines · The Peach
Quoted 1 line · On Thu, 23 Nov 2006 01:15:46 +0900 OGAWA Hirofumi…
Thanks. Probably instead of some overheads, this patch will fix a problem.
here it is, did the very same experiments than before. Normal file:
# cp -v DSCN5970\(1\).JPG /mnt/loop/ `DSCN5970(1).JPG' -> `/mnt/loop/DSCN5970(1).JPG'dmesg:
vfat_hashi: parentd1801e94,parent->d_ope0fbc620vfat_hashi: parent /, name DSCN5970(1).JPGvfat_cmpi: parentd1801e94,parent->d_ope0fbc620vfat_cmpi: a DSCN5970(1).JPG, b DSCN5970(1).JPGvfat_revalidate: name DSCN5970(1).JPG, ndd65eaeec, flags 00000001vfat_lookup: name DSCN5970(1).JPGvfat_hashi: parentd1801e94,parent->d_ope0fbc620vfat_hashi: parent /, name DSCN5970(1).JPGvfat_cmpi: parentd1801e94,parent->d_ope0fbc620vfat_cmpi: a DSCN5970(1).JPG, b DSCN5970(1).JPGvfat_revalidate: name DSCN5970(1).JPG, ndd65eaf24, flags 00000300vfat_lookup: name DSCN5970(1).JPGvfat_create: name DSCN5970(1).JPGvfat_add_entry: 0: DSCN59~1, JPGvfat_add_entry: 1: DSCN5, 970(1), .Jvfat_add_entry: 2: PGvfat_create: err 0Abnormal file:
# cp -v DSCN5980.JPG /mnt/loop/ `DSCN5980.JPG' -> `/mnt/loop/DSCN5980.JPG'dmesg:
vfat_hashi: parentd1801e94,parent->d_ope0fbc620vfat_hashi: parent /, name DSCN5980.JPGvfat_cmpi: parentd1801e94,parent->d_ope0fbc620vfat_cmpi: a dscn5980.jpg, b DSCN5980.JPGvfat_revalidate: name dscn5980.jpg, ndd65eaeec, flags 00000001vfat_lookup: name DSCN5980.JPGvfat_hashi: parentd1801e94,parent->d_ope0fbc620vfat_hashi: parent /, name DSCN5980.JPGvfat_cmpi: parentd1801e94,parent->d_ope0fbc620vfat_cmpi: a DSCN5980.JPG, b DSCN5980.JPGvfat_revalidate: name DSCN5980.JPG, ndd65eaf24, flags 00000300vfat_lookup: name DSCN5980.JPGvfat_create: name DSCN5980.JPGvfat_add_entry: 0: DSCN5980, JPGvfat_create: err 0and:
# ls -l /mnt/loop/ totale 1363(quoted diff) +0
-2-rwxr-xr-x 1 root root 695514 22 nov 20:04 DSCN5970(1).JPG -rwxr-xr-x 1 root root 699770 22 nov 20:07 dscn5980.jpg
Quoted 12 lines
dmesg:
vfat_hashi: parentd1801e94,parent->d_ope0fbc620vfat_hashi: parent /, name DSCN5970(1).JPGvfat_cmpi: parentd1801e94,parent->d_ope0fbc620vfat_cmpi: a DSCN5970(1).JPG, b DSCN5970(1).JPGvfat_revalidate: name DSCN5970(1).JPG, ndd2512eec, flags 00000000vfat_lookup: name DSCN5970(1).JPGvfat_hashi: parentd1801e94,parent->d_ope0fbc620vfat_hashi: parent /, name dscn5980.jpgvfat_cmpi: parentd1801e94,parent->d_ope0fbc620vfat_cmpi: a DSCN5980.JPG, b dscn5980.jpgvfat_revalidate: name DSCN5980.JPG, ndd2512eec, flags 00000000
This is different thing. Please try "shortname=winnt" or "shortname=mixed" mount option for shortname (default is shortname=lower).
Signature · 1 line
OGAWA Hirofumi <[email protected]>
Quoted 2 lines · On Thu, 23 Nov 2006 04:29:15 +0900 OGAWA Hirofumi…
This is different thing. Please try "shortname=winnt" or "shortname=mixed" mount option for shortname (default is shortname=lower).
finally it works (mounted with shortname=winnt).
Did this patch really solve the issue or was just a "shortname" option problem? I didn't even know that option would made the difference. now I just should get rid of verbose output.
Signature · 6 lines
Matteo 'Peach' Pescarin ICQ UIN = 71110111 Jabber ID = [email protected] Web Site = http://www.smartart.it GeCHI = http://www.gechi.it
Quoted 9 lines · The Peach
Quoted 2 lines · On Thu, 23 Nov 2006 04:29:15 +0900 OGAWA Hirofumi…
This is different thing. Please try "shortname=winnt" or "shortname=mixed" mount option for shortname (default is shortname=lower).
finally it works (mounted with shortname=winnt).
Did this patch really solve the issue or was just a "shortname" option problem? I didn't even know that option would made the difference. now I just should get rid of verbose output.
The both of patch and option should be needed. Because the following filename is not shortname, shortname option doesn't affect.
If you test shortname=winnt without patch, it should still show the problem of following filename, but it should be rare case though.
Can you test it?
Quoted 2 lines
(quoted diff) +0
-2-rwxr-xr-x 1 b users 732903 Oct 27 16:55 dscn5981.jpg.rem.2006-10-27-1543 -rwxr-xr-x 1 b users 622595 Oct 27 16:55 dscn5982.jpg.rem.2006-10-27-1543
Signature · 1 line
OGAWA Hirofumi <[email protected]>
Quoted 7 lines · On Thu, 23 Nov 2006 04:51:56 +0900 OGAWA Hirofumi…
The both of patch and option should be needed. Because the following filename is not shortname, shortname option doesn't affect.
If you test shortname=winnt without patch, it should still show the problem of following filename, but it should be rare case though.
Can you test it?
unfortunately not, the files have been renamed and renaming them back I can't reproduce the magical effect they did before.
Now I will try to rescan for any kind of files with long names and see if with the option on and without the patch it will re-generate that kind of errors they gave me.
I must thank you a lot, maybe a bottle of wine will do it ;)
anyway I didn't get why some files will copy with the right case and other don't. Was it a problem with the dentry table randomly failing in saving the filename? the shortname option in the Docs isn't well accurate in explaining the meaning of "Windows X rule for display/create", I will investigate after having checked the 32 GB of files. thanks again.
Signature · 6 lines
Matteo 'Peach' Pescarin ICQ UIN = 71110111 Jabber ID = [email protected] Web Site = http://www.smartart.it GeCHI = http://www.gechi.it
Quoted 7 lines · DervishD
Hi Sérgio :)
* Sergio Monteiro Basto <[email protected]> dixit:
Quoted 1 line
Have vfat a limit of a file size when copy ?
2GB, if I recall correctly. FAT32 itself has a limit of 4GB-1 for
file size, but Linux restricts it even more (don't ask me why).
May I say that FAT32 have a bigger limit (at least on last Windows).
The file in question have 4491771904 bytes, 4288 mega bytes , 4.2G or 4.5G if use powers of 1000.
I copy the file from one FAT32 to my computer (ext3), now I would like to copy to other FAT32. But I just copy 4096 mega bytes (and terminate with an error of limit file exceed) .
Have you a solution for the case ? Now I have the file in ext3 and I couldn't copy to any vfat :)
I have a solution with cifs or smbmount, but in same computer ?
Thanks,
Quoted 3 lines
Raúl Núñez de Arenas Coronado
Signature · 1 line
Sérgio M.B.
Hi,
We encountered this problem before,it did being caused by dcache. When creating files, VFS would check the dcache first and pass the dentry it found there to vfat_create(). Instead of dentry , using the path info passed by user solved the problem.
namei.c.new renamed from namei.c +1 -1
@@ -742,7 +742,7 @@ lock_kernel(); ts = CURRENT_TIME_SEC; - err = vfat_add_entry(dir, &dentry->d_name, 0, 0, &ts, &sinfo); + err = vfat_add_entry(dir, &nd->last, 0, 0, &ts, &sinfo); if (err) goto out; dir->i_version++;
From: OGAWA Hirofumi <[email protected]>
Subject: Re: bug? VFAT copy problem
Date: Wed, 22 Nov 2006 00:46:43 +0900
Quoted 36 lines
Quoted 11 lines · The Peach
Quoted 2 lines · On Tue, 21 Nov 2006 02:32:43 +0900 OGAWA Hirofumi…
I couldn't reproduce this for now. Could you tell mount options which you used? and after mount, "cat
/proc/mounts", please.# mount | grep vfat /dev/sdb1 on /mnt/iomega type vfat (rw,uid=1000,gid=100,codepage=850,iocharset=iso8859-15)it seems only related to those kind of files, but I don't know how to inspect the "file properties" and why these files behave like this.
As you can see and with a strace made on cp, the files _seems_ to be copied with the correct case, whilst it isn't, as seen with "ls". This and other things let me think is a vfat problem.Hmm... This may be the dentry cache handling problem of fat.
Can you try the attached debug patch? And if you comment-in the following parts, does this problem fix?
(quoted diff) +6
-1@@ -787,6 +830,9 @@ static int vfat_rmdir(struct inode *dir, clear_nlink(inode); inode->i_mtime = inode->i_atime = CURRENT_TIME_SEC; fat_detach(inode); + /* need to revalidate for next create */ + table = (sbi->options.name_check == 's') ? 3 : 1; +// dentry->d_op = &vfat_dentry_ops[table];@@ -811,6 +858,9 @@ static int vfat_unlink(struct inode *dir clear_nlink(inode); inode->i_mtime = inode->i_atime = CURRENT_TIME_SEC; fat_detach(inode); + /* need to revalidate for next create */ + table = (sbi->options.name_check == 's') ? 3 : 1; +// dentry->d_op = &vfat_dentry_ops[table]; --OGAWA Hirofumi <[email protected]>
Hi Sérgio :)
* Sergio Monteiro Basto <[email protected]> dixit:
Quoted 8 lines · DervishD
Quoted 5 lines
* Sergio Monteiro Basto <[email protected]> dixit:
Quoted 1 line
Have vfat a limit of a file size when copy ?
2GB, if I recall correctly. FAT32 itself has a limit of 4GB-1 for
file size, but Linux restricts it even more (don't ask me why).May I say that FAT32 have a bigger limit (at least on last Windows).
I really don't know, but from microsoft technical information
(the first or second hit when googling for "FAT32 size limit"), the limit they specify in FAT32 is 2^32-1.
I may be wrong, but with 32 bits you cannot address more than
2^32 bytes, I don't know how can you create a bigger-than-4Gb file in a FAT32 filesystem without resorting to tricks like this:
forum.doom9.org/archive/index.php/t-20689.html
Looks like FAT-32 (don't ask me how because I don't know the
internals) can store a file bigger than 4GB, but you cannot *save* it. So you won't be able to put the file you have back to any FAT32 filesystem, I'm afraid.
Quoted 2 lines
Have you a solution for the case ? Now I have the file in ext3 and I couldn't copy to any vfat :)
No, I don't have any idea about how to do it, sorry :(
Quoted 1 line
I have a solution with cifs or smbmount, but in same computer ?
I doubt. That limit is a hard limit as far as I know. Google for
"FAT32 limit" and you'll get a lot of answers telling it. I don't really know how did you get a bigger than 4GB file saved in a FAT32 filesystem in the first place ;))
Raúl Núñez de Arenas Coronado
Signature · 2 lines
Linux Registered User 88736 | http://www.dervishd.net It's my PC and I'll cry if I want to... RAmen!
Quoted 9 lines · [email protected] writes:
We encountered this problem before,it did being caused by dcache. When creating files, VFS would check the dcache first and pass the dentry it found there to
vfat_create(). Instead of dentry , using the path info passed by user solved the problem.
namei.c.new line 748
Quoted 8 lines
namei.c.new renamed from namei.c +1
-1@@ -742,7 +742,7 @@ lock_kernel(); ts = CURRENT_TIME_SEC; - err = vfat_add_entry(dir, &dentry->d_name, 0, 0, &ts, &sinfo); + err = vfat_add_entry(dir, &nd->last, 0, 0, &ts, &sinfo); if (err) goto out; dir->i_version++;
Interesting hack. However, mkdir() and rename() have same problem.
Signature · 1 line
OGAWA Hirofumi <[email protected]>
Quoted 5 lines · On Thu, 23 Nov 2006 10:13:01 +0100 DervishD <[email protected]> wrote:
Quoted 2 lines
Have you a solution for the case ? Now I have the file in ext3 and I couldn't copy to any vfat :)
No, I don't have any idea about how to do it, sorry :(
maybe using ntfs-3g driver with fuse or use the extX windows driver (if the need was read from Windows). I'm feeling quite confortable with the first solution, whilst the second is suggested by the official linux ntfs support page
Signature · 6 lines
Matteo 'Peach' Pescarin ICQ UIN = 71110111 Jabber ID = [email protected] Web Site = http://www.smartart.it GeCHI = http://www.gechi.it
Quoted 3 lines · The Peach
anyway I didn't get why some files will copy with the right case and other don't. Was it a problem with the dentry table randomly failing in saving the filename?
If it's shortname (the detail of shortname is following), the "shortname=" option solves a problem. If it's longname (not shortname), the patch solves a problem. The your case seems to need both.
* 1) Valid characters for the 8.3 format alias are any combination of
* letters, uppercase alphabets, digits, any of the
* following special characters:
* $ % ' ` - @ { } ~ ! # ( ) & _ ^
* In this case Longfilename is not stored in disk.
*
* WinNT's Extension:
* File name and extension name is contain uppercase/lowercase
* only. And it is expressed by CASE_LOWER_BASE and CASE_LOWER_EXT. *
* 2) File name is 8.3 format, but it contain the uppercase and
* lowercase char, muliti bytes char, etc. In this case numtail is not
* added, but Longfilename is stored.
*
* 3) When the one except for the above, or the following special
* character are contained:
* . [ ] ; , + =
* numtail is added, and Longfilename must be stored in disk .
Signature · 1 line
OGAWA Hirofumi <[email protected]>
Quoted 18 lines · DervishD
* Sergio Monteiro Basto <[email protected]> dixit:
Quoted 8 lines · DervishD
Quoted 5 lines
* Sergio Monteiro Basto <[email protected]> dixit:
Quoted 1 line
Have vfat a limit of a file size when copy ?
2GB, if I recall correctly. FAT32 itself has a limit of 4GB-1 for
file size, but Linux restricts it even more (don't ask me why).May I say that FAT32 have a bigger limit (at least on last Windows).
I really don't know, but from microsoft technical information
(the first or second hit when googling for "FAT32 size limit"), the limit they specify in FAT32 is 2^32-1.I may be wrong, but with 32 bits you cannot address more than
2^32 bytes, I don't know how can you create a bigger-than-4Gb file in a FAT32 filesystem without resorting to tricks like this:
Quoted 6 lines
forum.doom9.org/archive/index.php/t-20689.html
Looks like FAT-32 (don't ask me how because I don't know the
internals) can store a file bigger than 4GB, but you cannot *save* it. So you won't be able to put the file you have back to any FAT32 filesystem, I'm afraid.
Right. FAT's size field is 32bit, so *file* of FAT has limit of 4GB-1. (Since directory doesn't have size, in theoretically it can exceed 4GB-1)
Hm.. Maybe MS added a new hack to FAT..?
Signature · 1 line
OGAWA Hirofumi <[email protected]>
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
OGAWA Hirofumi escreveu:
Quoted 4 lines
Right. FAT's size field is 32bit, so *file* of FAT has limit of 4GB-1. (Since directory doesn't have size, in theoretically it can exceed 4GB-1)
Hm.. Maybe MS added a new hack to FAT..?
Ogawa, MS don't added a new hack to FAT32... This file system don't support file size with more than 4Gb:
<http://msdn2.microsoft.com/en-us/library/aa365678.aspx>
Best Regards,
- --
Renato S. Yamane
Fingerprint: 68AE A381 938A F4B9 8A23 D11A E351 5030 D420 515A PGP Server: http://pgp.mit.edu/ --> KeyID: 0xD420515A
<http://www.renatoyamane.com>
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.2 (GNU/Linux)
Comment: Using GnuPG with SUSE - http://enigmail.mozdev.org
iD8DBQFFZYsZ41FQMNQgUVoRAresAJ0cyYTTOMr9Zrik2ojqquUC8wwEwgCfTe1g 93wBDWCheZSSfMyiBHIfmNc=
=kmsy
-----END PGP SIGNATURE-----
Quoted 9 lines · Renato S. Yamane
OGAWA Hirofumi escreveu:
Quoted 4 lines
Right. FAT's size field is 32bit, so *file* of FAT has limit of 4GB-1. (Since directory doesn't have size, in theoretically it can exceed 4GB-1)
Hm.. Maybe MS added a new hack to FAT..?
Ogawa, MS don't added a new hack to FAT32... This file system don't support file size with more than 4Gb:
<http://msdn2.microsoft.com/en-us/library/aa365678.aspx>
Thanks. Yes. However, Sergio's windows seems to handle file size more than 4GB-1...
Signature · 1 line
OGAWA Hirofumi <[email protected]>
Hi Matteo :)
* The Peach <[email protected]> dixit:
Quoted 5 lines
Quoted 3 lines · DervishD
Quoted 2 lines
Have you a solution for the case ? Now I have the file in ext3 and I couldn't copy to any vfat :)
Quoted 7 lines
Quoted 2 lines
No, I don't have any idea about how to do it, sorry :(maybe using ntfs-3g driver with fuse or use the extX windows driver (if the need was read from Windows). I'm feeling quite confortable with the first solution, whilst the second is suggested by the official linux ntfs support page
Of course, such options exist, but Sergio was asking to write to
a FAT-32, so... Myself, I use ext2 for my external disks if I can, and I install the ext2-ifs driver for windows to read them. It works like a charm.
Raúl Núñez de Arenas Coronado
Signature · 2 lines
Linux Registered User 88736 | http://www.dervishd.net It's my PC and I'll cry if I want to... RAmen!
Quoted 2 lines · OGAWA Hirofumi
. Yes. However, Sergio's windows seems to handle file size more than 4GB-1..
I not finish my tests yet, but so far I was wrong. Could happened that transfer never had pass by one FAT32.
Thanks,
--
Sérgio M. B.