FAT driver enhancement
Hi,
A while ago I initiated a thread about mounting a NTFS partition as FAT partition. The problem is that FAT partitions do not have a real fingerprint, so the FAT driver mounts almost anything.
The current 2.5 driver only tests if some values in the bootsector are non-zero. IMHO, this is not strict enough. For example, the number of FATs is always 1 or 2 (anyone ever seen more ?). Besides, when there are two FATs, all entries in those FATs should be equal. If they are not, we deal with a non-FAT or broken FAT partition, and we should not mount.
It's not a real fingerprint, but what are the chances all sectors of what we think is the FAT are equal on non-FAT filesystems ? Yes, when you just did a
dd if=/dev/zero of=/dev/partition; mkfs.somefs /dev/partition
there is a chance, but that's an empty filesystem. Data corruption isn't that bad on an empty disk. We know that a FAT is at the beginning of a partition and I assume that any other filesystem will fill up those first sectors very soon.
Questions:
1) How do you think about the checking of the FAT tables ? It definitely will slow down the mount.
2) If I implement it, where shoud it go ? At the moment, I hacked fat_read_super, for there the FAT fs is validated, but I got the feeling this is not the place to be.
3) Anyone seen more than two FATs on a filesystem ? Can I assume there is a limit ?
4) Comments, anyone ?
Jos
Quoted 23 lines · Jos Hulzink
Hi,
A while ago I initiated a thread about mounting a NTFS partition as FAT partition. The problem is that FAT partitions do not have a real fingerprint, so the FAT driver mounts almost anything.
The current 2.5 driver only tests if some values in the bootsector are non-zero. IMHO, this is not strict enough. For example, the number of FATs is always 1 or 2 (anyone ever seen more ?). Besides, when there are two FATs, all entries in those FATs should be equal. If they are not, we deal with a non-FAT or broken FAT partition, and we should not mount.
It's not a real fingerprint, but what are the chances all sectors of what we think is the FAT are equal on non-FAT filesystems ? Yes, when you just did a
dd if=/dev/zero of=/dev/partition; mkfs.somefs
/dev/partitionthere is a chance, but that's an empty filesystem. Data corruption isn't that bad on an empty disk. We know that a FAT is at the beginning of a partition and I assume that any other filesystem will fill up those first sectors very soon.
Quoted 4 lines
Questions:
1) How do you think about the checking of the FAT tables ? It definitely will slow down the mount.
Unfortunately if FAT table has bad sector, FAT tables may not be the same.
Quoted 6 lines
2) If I implement it, where shoud it go ? At the moment, I hacked
fat_read_super, for there the FAT fs is validated, but I got the feeling this is not the place to be.
3) Anyone seen more than two FATs on a filesystem ? Can I assume there is a limit ?
4) Comments, anyone ?
How about writing the mount.xxx/fsck.xxx? The mount.xxx/fsck.xxx can check many of the ordinary FAT status. If the something occurs, output message to user. And user can handle it by option etc.
Regards.
Signature · 1 line
OGAWA Hirofumi <[email protected]>
Hi!
Quoted 21 lines
A while ago I initiated a thread about mounting a NTFS partition as FAT partition. The problem is that FAT partitions do not have a real fingerprint, so the FAT driver mounts almost anything.
The current 2.5 driver only tests if some values in the bootsector are non-zero. IMHO, this is not strict enough. For example, the number of FATs is always 1 or 2 (anyone ever seen more ?). Besides, when there are two FATs, all entries in those FATs should be equal. If they are not, we deal with a non-FAT or broken FAT partition, and we should not mount.
It's not a real fingerprint, but what are the chances all sectors of what we think is the FAT are equal on non-FAT filesystems ? Yes, when you just did a
dd if=/dev/zero of=/dev/partition; mkfs.somefs
/dev/partitionthere is a chance, but that's an empty filesystem. Data corruption isn't that bad on an empty disk. We know that a FAT is at the beginning of a partition and I assume that any other filesystem will fill up those first sectors very soon.
Quoted 4 lines
Questions:
1) How do you think about the checking of the FAT tables ? It definitely will slow down the mount.
Reading FATs is very fast, and they are probably going to be needed, anyway. I guess its okay.
Quoted 5 lines
2) If I implement it, where shoud it go ? At the moment, I hacked
fat_read_super, for there the FAT fs is validated, but I got the feeling this is not the place to be.
3) Anyone seen more than two FATs on a filesystem ? Can I assume there is a limit ?
No. I think you can only have two.
Pavel
Signature · 2 lines
(about SSSCA) "I don't say this lightly. However, I really think that the U.S. no longer is classifiable as a democracy, but rather as a plutocracy." --hpa
Quoted 26 lines · OGAWA Hirofumi
Quoted 23 lines · Jos Hulzink
Hi,
A while ago I initiated a thread about mounting a NTFS partition as FAT partition. The problem is that FAT partitions do not have a real fingerprint, so the FAT driver mounts almost anything.
The current 2.5 driver only tests if some values in the bootsector are non-zero. IMHO, this is not strict enough. For example, the number of FATs is always 1 or 2 (anyone ever seen more ?). Besides, when there are two FATs, all entries in those FATs should be equal. If they are not, we deal with a non-FAT or broken FAT partition, and we should not mount.
It's not a real fingerprint, but what are the chances all sectors of what we think is the FAT are equal on non-FAT filesystems ? Yes, when you just did a
dd if=/dev/zero of=/dev/partition; mkfs.somefs
/dev/partitionthere is a chance, but that's an empty filesystem. Data corruption isn't that bad on an empty disk. We know that a FAT is at the beginning of a partition and I assume that any other filesystem will fill up those first sectors very soon.
Quoted 7 lines
Quoted 4 lines
Questions:
1) How do you think about the checking of the FAT tables ? It definitely will slow down the mount.
Unfortunately if FAT table has bad sector, FAT tables may not be the same.
And then you don't want to mount unless you know what you are doing. And those knowing what they are doing can be bothered to use some kind of "force" option in this case. Or perhaps an option that selects which FAT to trust.
Helge Hafting
Quoted 32 lines · Helge Hafting
Quoted 31 lines · OGAWA Hirofumi
Quoted 27 lines · Jos Hulzink
Hi,
A while ago I initiated a thread about mounting a NTFS partition as FAT partition. The problem is that FAT partitions do not have a real fingerprint, so the FAT driver mounts almost anything.
The current 2.5 driver only tests if some values in the bootsector are non-zero. IMHO, this is not strict enough. For example, the number of FATs is always 1 or 2 (anyone ever seen more ?). Besides, when there are two FATs, all entries in those FATs should be equal. If they are not, we deal with a non-FAT or broken FAT partition, and we should not mount.
It's not a real fingerprint, but what are the chances all sectors of what we think is the FAT are equal on non-FAT filesystems ? Yes, when you just did a
dd if=/dev/zero of=/dev/partition; mkfs.somefs
/dev/partitionthere is a chance, but that's an empty filesystem. Data corruption isn't that bad on an empty disk. We know that a FAT is at the beginning of a partition and I assume that any other filesystem will fill up those first sectors very soon.
Questions:
1) How do you think about the checking of the FAT tables ? It definitely will slow down the mount.
Quoted 7 lines
Quoted 2 lines
Unfortunately if FAT table has bad sector, FAT tables may not be the same.
And then you don't want to mount unless you know what you are doing. And those knowing what they are doing can be bothered to use some kind of "force" option in this case. Or perhaps an option that selects which FAT to trust.
I mean I/O error, not data damage.
Signature · 1 line
OGAWA Hirofumi <[email protected]>
Quoted 13 lines · OGAWA Hirofumi
Quoted 11 lines · Helge Hafting
Quoted 9 lines · OGAWA Hirofumi
Quoted 4 lines · Jos Hulzink
Questions:
1) How do you think about the checking of the FAT tables ? It definitely will slow down the mount.
Unfortunately if FAT table has bad sector, FAT tables may not be the same.
Quoted 6 lines
Quoted 4 lines
And then you don't want to mount unless you know what you are doing. And those knowing what they are doing can be bothered to use some kind of "force" option in this case. Or perhaps an option that selects which FAT to trust.
I mean I/O error, not data damage.
It is the block layer's responsibility to retry such soft errors and recover.
Probably the best you can do, is retry the read a few times if there is an error reported, and then fail if it persists.
Quoted 14 lines · Mike Fedyk
Quoted 13 lines · OGAWA Hirofumi
Quoted 11 lines · Helge Hafting
Quoted 9 lines · OGAWA Hirofumi
Quoted 4 lines · Jos Hulzink
Questions:
1) How do you think about the checking of the FAT tables ? It definitely will slow down the mount.
Unfortunately if FAT table has bad sector, FAT tables may not be the same.
Quoted 8 lines
Quoted 6 lines
Quoted 4 lines
And then you don't want to mount unless you know what you are doing. And those knowing what they are doing can be bothered to use some kind of "force" option in this case. Or perhaps an option that selects which FAT to trust.
I mean I/O error, not data damage.
It is the block layer's responsibility to retry such soft errors and recover.
No it's the low level driver
Quoted 2 lines
Probably the best you can do, is retry the read a few times if there is an error reported, and then fail if it persists.
I/O error retrying from the fs is a bad idea, the low level driver should already have exhausted the possibilities to recover the data. If the fs receives an I/O error, that's the final result and retrying would serve no sane purpose.
Signature · 1 line
Jens Axboe
Quoted 14 lines · Mike Fedyk
Quoted 13 lines · OGAWA Hirofumi
Quoted 11 lines · Helge Hafting
Quoted 9 lines · OGAWA Hirofumi
Quoted 4 lines · Jos Hulzink
Questions:
1) How do you think about the checking of the FAT tables ? It definitely will slow down the mount.
Unfortunately if FAT table has bad sector, FAT tables may not be the same.
Quoted 8 lines
Quoted 6 lines
Quoted 4 lines
And then you don't want to mount unless you know what you are doing. And those knowing what they are doing can be bothered to use some kind of "force" option in this case. Or perhaps an option that selects which FAT to trust.
I mean I/O error, not data damage.
It is the block layer's responsibility to retry such soft errors and recover.
Yes.
Quoted 2 lines
Probably the best you can do, is retry the read a few times if there is an error reported, and then fail if it persists.
Umm, there is a `copy of FAT table' for this kind of error. If the I/O error occurs, the FAT driver should use the other FAT table.
Signature · 1 line
OGAWA Hirofumi <[email protected]>
Quoted 7 lines · OGAWA Hirofumi
Quoted 3 lines · Mike Fedyk
Quoted 1 line
I mean I/O error, not data damage.
It is the block layer's responsibility to retry such soft errors and recover.
Yes.
But what about the data damage errors ?
Quoted 5 lines
Quoted 2 lines
Probably the best you can do, is retry the read a few times if there is an error reported, and then fail if it persists.
Umm, there is a `copy of FAT table' for this kind of error. If the I/O error occurs, the FAT driver should use the other FAT table.
How should the FAT driver know that the first FAT is bad if it doesn't scan the FAT ? You don't want the second FAT to be used, you want the mount to fail, and fsck.xxx to fix the mess. Who tells you that the second copy of the FAT is the correct one, and not the first ?
Jos
Quoted 15 lines · Jos Hulzink
Quoted 7 lines · OGAWA Hirofumi
Quoted 3 lines · Mike Fedyk
Quoted 1 line
I mean I/O error, not data damage.
It is the block layer's responsibility to retry such soft errors and recover.
Yes.
But what about the data damage errors ?
Quoted 3 lines
Quoted 2 lines
Probably the best you can do, is retry the read a few times if there is an error reported, and then fail if it persists.
Quoted 7 lines
Quoted 2 lines
Umm, there is a `copy of FAT table' for this kind of error. If the I/O error occurs, the FAT driver should use the other FAT table.
How should the FAT driver know that the first FAT is bad if it doesn't scan the FAT ? You don't want the second FAT to be used, you want the mount to fail, and fsck.xxx to fix the mess. Who tells you that the second copy of the FAT is the correct one, and not the first ?
FAT16/FAT32 use the second entry of FAT table for data damage. The 1 bit of second entry is a clean/dirty unmount flag.
But, it's not perfect. Furthermore, currently not implemented.
Signature · 1 line
OGAWA Hirofumi <[email protected]>
Quoted 4 lines · Jos Hulzink
How should the FAT driver know that the first FAT is bad if it doesn't scan the FAT ? You don't want the second FAT to be used, you want the mount to fail, and fsck.xxx to fix the mess. Who tells you that the second copy of the FAT is the correct one, and not the first ?
Seems to me you would want a mount-time option to the FAT fs code to say "use FAT#<n>", defaulting to the first if no parm given. If that copy of the FAT has any problems, fail the mount.
Then you'd want the fsck.fat to have a similar option, saying "use FAT#<n> for the check" - that way if the FATs are out of sync, you could do a dry run check on each FAT, and go with the one that seemed to be better. Perhaps even having the tool allow you to pick and choose if needed (although this would probably be better as a seperate tool, that allowed you to view a file given a selected FAT and copy it to a clean file system.)
Hi,
Quoted 11 lines · David D. Hagood
Quoted 4 lines · Jos Hulzink
How should the FAT driver know that the first FAT is bad if it doesn't scan the FAT ? You don't want the second FAT to be used, you want the mount to fail, and fsck.xxx to fix the mess. Who tells you that the second copy of the FAT is the correct one, and not the first ?
Seems to me you would want a mount-time option to the FAT fs code to say "use FAT#<n>", defaulting to the first if no parm given. If that copy of the FAT has any problems, fail the mount.
Then you'd want the fsck.fat to have a similar option, saying "use FAT#<n> for the check" - that way if the FATs are out of sync, you could do a dry run check on each FAT, and go with the one that seemed to be better. Perhaps even having the tool allow you to pick and choose if needed (although this would probably be better as a seperate tool, that allowed you to view a file given a selected FAT and copy it to a clean file system.)
If I apply the "Think big" patch for a second, I say, check individually for each entry with fsck.fat and build a fat of the entries that are still ok. This could be a special --rebuild option or whatever.
Regards,
Thunder
Signature · 2 lines
Thunder from the hill. Citizen of our universe.