Blockbusting news, results end
The drive finally reallocated the block and there are no longer any visible bad blocks.
I will not be able to perform the following planned test:
Well, in a future weekend, I will try to see if ext2fs really takes action on permanently bad blocks that are detected during normal operations on a mounted partition.
But I think the underlying defects remain in need of correction. Toshiba knows about theirs but will probably never say if they make any fixes. Mr. Reiser and friends have plans to add important features, and I am unable to detect if ext2fs needs it. (As mentioned before, I understand that ext2fs can do it during formatting and fsck, but no one seems to be saying what happens if a permanently bad block is detected during normal operation on a mounted partition.)
Hi!
Quoted 2 lines
The drive finally reallocated the block and there are no longer any visible bad blocks.
And what was the operation that made it realocate?
Pavel
Signature · 2 lines
When do you have a heart between your knees? [Johanka's followup: and *two* hearts?]
Pavel Machek replied to me:
Quoted 4 lines
Quoted 2 lines
The drive finally reallocated the block and there are no longer any visible bad blocks.
And what was the operation that made it realocate?
At first I wasn't sure. I noticed that the drive was behaving differently when I told dd to use bs=4096 instead of 512. Until seeing Oleg Drokin's message about ReiserFS, I thought that the drive itself was doing something differently. That didn't make much sense to me because the physical sectors are much longer than 4096 and the pseudo-sectors are the conventional 512, so why did 4096 cause different behaviour? From Oleg Drokin's message, I guess that the use of 4096 might make a difference in the sequence of read-modify-write cycles involved in the logical write operation.
This doesn't seem like a complete answer, but I don't think I'll ever know a complete answer.
Quoted 7 lines · Norman Diamond
Pavel Machek replied to me:
Quoted 4 lines
Quoted 2 lines
The drive finally reallocated the block and there are no longer any visible bad blocks.
And what was the operation that made it realocate?
Quoted 8 lines
At first I wasn't sure. I noticed that the drive was behaving differently when I told dd to use bs=4096 instead of 512. Until seeing Oleg Drokin's message about ReiserFS, I thought that the drive itself was doing something differently. That didn't make much sense to me because the physical sectors are much longer than 4096 and the pseudo-sectors are the conventional 512, so why did 4096 cause different behaviour? From Oleg Drokin's message, I guess that the use of 4096 might make a difference in the sequence of read-modify-write cycles involved in the logical write operation.
You bring up an interesting point. If the physical sector is larger than the data being written how can the drive reallocate the sector without silently losing data?
To put it in the concrete, if the physical sector were 16K and we only do a 4K write and there is a unrecoverable read error on the physical sector as part of the read-modify-write sequence what is the drive to do? The other 12K for which the drive has no data could be other files not related to the 4K being written or even filesystem meta-data. Reallocation in that case would cause silent corruption.
Perhaps what finally allowed the reallocation was that the entire physical sector finally accumulated writes to all the logical sectors needed to be a complete physical sector write.
Signature · 5 lines
________________________________________________________________ J.W. Schultz Pegasystems Technologies email address: [email protected] Remember Cernan and Schmitt
To date IDEMA has not released a formal spec for drive makers to switch to 4Kb sectors. If they have, then when running in compatibility mode, a heavy read-modify-write happens to goto the pseudo sector of 512b.
Linux can not handled new IDEMA calls currently.
Cheers,
Andre Hedrick
LAD Storage Consulting Group
Quoted 47 lines · jw schultz
Quoted 7 lines · Norman Diamond
Pavel Machek replied to me:
Quoted 4 lines
Quoted 2 lines
The drive finally reallocated the block and there are no longer any visible bad blocks.
And what was the operation that made it realocate?
Quoted 8 lines
At first I wasn't sure. I noticed that the drive was behaving differently when I told dd to use bs=4096 instead of 512. Until seeing Oleg Drokin's message about ReiserFS, I thought that the drive itself was doing something differently. That didn't make much sense to me because the physical sectors are much longer than 4096 and the pseudo-sectors are the conventional 512, so why did 4096 cause different behaviour? From Oleg Drokin's message, I guess that the use of 4096 might make a difference in the sequence of read-modify-write cycles involved in the logical write operation.
You bring up an interesting point. If the physical sector is larger than the data being written how can the drive reallocate the sector without silently losing data?
To put it in the concrete, if the physical sector were 16K and we only do a 4K write and there is a unrecoverable read error on the physical sector as part of the read-modify-write sequence what is the drive to do? The other 12K for which the drive has no data could be other files not related to the 4K being written or even filesystem meta-data. Reallocation in that case would cause silent corruption.
Perhaps what finally allowed the reallocation was that the entire physical sector finally accumulated writes to all the logical sectors needed to be a complete physical sector write.
Signature · 10 lines
________________________________________________________________ J.W. Schultz Pegasystems Technologies email address: [email protected] Remember Cernan and Schmitt - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Quoted 6 lines · Andre Hedrick
To date IDEMA has not released a formal spec for drive makers to switch to 4Kb sectors. If they have, then when running in compatibility mode, a heavy read-modify-write happens to goto the pseudo sector of 512b.
Linux can not handled new IDEMA calls currently.
Irrelevant!
I am assuming that these numbers are applicable (one is unknown):
logical sector size == 512B physical sector size == ???B page size/filesystem block size == 4KB
Whether the write of a 4KB block is a single 4KB write or 8 512B writes i expect the drive will gather it so that 4KB is being written. If that 4KB (or whatever size write) is less than the size of a physical sector a read-modify-write cycle must occur inside the drive. If the read portion of the read-modify-write fails the requested write(s) must either fail or data in the logical sectors adjacent to the write will become corrupted. This is whether or not the physical sector is relocated.
Judging by logic and earlier discussions on the matter of write errors if that write fails because of a read error in the read-modify-write to the application it would appear to have succeeded unless it were synchronous all the way to the media.
This sounds very much like the described behaviour of the Toshiba drive that started this whole thread.
To the best of my knowledge we have no way to know what the physical sector size of a disk is. For that matter it might not even be a constant. Perhaps the someone could inform us what the drive will actually do if the read portion of a read-modify-write fails.
Quoted 62 lines
Cheers,
Andre Hedrick
LAD Storage Consulting GroupQuoted 30 lines · jw schultz
Quoted 7 lines · Norman Diamond
Pavel Machek replied to me:
Quoted 4 lines
Quoted 2 lines
The drive finally reallocated the block and there are no longer any visible bad blocks.
And what was the operation that made it realocate?
Quoted 8 lines
At first I wasn't sure. I noticed that the drive was behaving differently when I told dd to use bs=4096 instead of 512. Until seeing Oleg Drokin's message about ReiserFS, I thought that the drive itself was doing something differently. That didn't make much sense to me because the physical sectors are much longer than 4096 and the pseudo-sectors are the conventional 512, so why did 4096 cause different behaviour? From Oleg Drokin's message, I guess that the use of 4096 might make a difference in the sequence of read-modify-write cycles involved in the logical write operation.
You bring up an interesting point. If the physical sector is larger than the data being written how can the drive reallocate the sector without silently losing data?
To put it in the concrete, if the physical sector were 16K and we only do a 4K write and there is a unrecoverable read error on the physical sector as part of the read-modify-write sequence what is the drive to do? The other 12K for which the drive has no data could be other files not related to the 4K being written or even filesystem meta-data. Reallocation in that case would cause silent corruption.
Quoted 17 lines
Perhaps what finally allowed the reallocation was that the entire physical sector finally accumulated writes to all the logical sectors needed to be a complete physical sector write.
Signature · 10 lines
________________________________________________________________ J.W. Schultz Pegasystems Technologies email address: [email protected] Remember Cernan and Schmitt - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
Signature · 5 lines
________________________________________________________________ J.W. Schultz Pegasystems Technologies email address: [email protected] Remember Cernan and Schmitt
Quoted 9 lines · jw schultz
Quoted 6 lines · Andre Hedrick
To date IDEMA has not released a formal spec for drive makers to switch to 4Kb sectors. If they have, then when running in compatibility mode, a heavy read-modify-write happens to goto the pseudo sector of 512b.
Linux can not handled new IDEMA calls currently.
Irrelevant!
Andre, if you were trying to say what i am about to, i apologise.
Quoted 5 lines
I am assuming that these numbers are applicable (one is unknown):
logical sector size == 512B physical sector size == ???B page size/filesystem block size == 4KB
I have dialoged with Eric Mudama. He is 99% sure that no manufacturer of is making ATA drives with physical sectors larger than 512B. I'll let that statement trump Norman Diamond's until i hear otherwise.
The drive manufacturers would like to be able to go to a larger physical sector but the read-modify-write is just too scary. If they could be sure of market acceptance of drives that required all I/O to be in larger units they would build them because it would allow greater capacity (and i'm guessing speed as well) on the same physical hardware. I look forward to the day with Linux has enough market-share to influence the hardware manufacturers.
In any case, unless someone can authoritatively contradict Eric we can ignore the firmware read-modify-write implications of unreadable sectors.
End-of-subthread!
or in the words of Emily Latella "Never-mind".
Quoted 8 lines · Norman Diamond
The drive finally reallocated the block and there are no longer any visible bad blocks.
I will not be able to perform the following planned test:
Well, in a future weekend, I will try to see if ext2fs really takes action on permanently bad blocks that are detected during normal operations on a mounted partition.
Quoted 12 lines
But I think the underlying defects remain in need of correction. Toshiba knows about theirs but will probably never say if they make any fixes. Mr. Reiser and friends have plans to add important features, and I am unable to detect if ext2fs needs it. (As mentioned before, I understand that ext2fs can do it during formatting and fsck, but no one seems to be saying what happens if a permanently bad block is detected during normal operation on a mounted partition.)
Badblocks support is in reiser4, and anyone is welcome to update the patch for V3, or sponsor us to do it. We are very low on cash, so we only work on V4, as that is the only way it will get to ship.
Signature · 1 line
Hans
Hans Reiser <[email protected]> wrote:
HR> Badblocks support is in reiser4, and anyone is welcome to update the HR> patch for V3, or sponsor us to do it. We are very low on cash, so we
Actually that v3 patch does not do bad blocks remapping in case of write failure, it only does remapping when you manually ask it. And biggest part of badblocks support in reiser3 is in reiserfsck and tools (and in not that bad shape, last time I looked).
As for remapping bad blocks on write failure, the only PC OS that was doing this that comes to my mind is Novell Netware (I think they called it a "hotfix" or something like that).
Bye,
Oleg
Quoted 12 lines · Oleg Drokin
Hans Reiser <[email protected]> wrote:
HR> Badblocks support is in reiser4, and anyone is welcome to update the HR> patch for V3, or sponsor us to do it. We are very low on cash, so we
Actually that v3 patch does not do bad blocks remapping in case of write failure, it only does remapping when you manually ask it. And biggest part of badblocks support in reiser3 is in reiserfsck and tools (and in not that bad shape, last time I looked).
As for remapping bad blocks on write failure, the only PC OS that was doing this that comes to my mind is Novell Netware (I think they called it a "hotfix" or something like that).
Quoted 6 lines
Bye,
Oleg
I wasn't saying we would do what is in Netware, and as far as being in bad shape is concerned, from the user's point of view it either works or it doesn't.
Signature · 1 line
Hans
Hello!
Quoted 11 lines · Hans Reiser
Quoted 10 lines
HR> Badblocks support is in reiser4, and anyone is welcome to update the HR> patch for V3, or sponsor us to do it. We are very low on cash, so we Actually that v3 patch does not do bad blocks remapping in case of write failure, it only does remapping when you manually ask it. And biggest part of badblocks support in reiser3 is in reiserfsck and tools (and in not that bad shape, last time I looked).
As for remapping bad blocks on write failure, the only PC OS that was doing this that comes to my mind is Novell Netware (I think they called it a "hotfix"
or something like that).I wasn't saying we would do what is in Netware, and as far as being in
Sure, I was not claiming this either, but original man seems to be implying such a feature as something common.
Quoted 2 lines
bad shape is concerned, from the user's point of view it either works or it doesn't.
Sure. And in modern desktop world it does not work (if SMART does not want to do it).
Bye,
Oleg
Quoted 7 lines · Oleg Drokin
Quoted 6 lines
bad shape is concerned, from the user's point of view it either works or it doesn't.
Quoted 9 lines
Sure. And in modern desktop world it does not work (if SMART does not want to do it).
Bye,
Oleg
I don't understand what SMART has to do with it. I was talking about a patch is either ready for use, or it isn't, and "almost ready" is worthless.
Signature · 1 line
Hans
Hello!
Quoted 7 lines · Hans Reiser
Quoted 5 lines
Quoted 2 lines
bad shape is concerned, from the user's point of view it either works or it doesn't.
Sure. And in modern desktop world it does not work (if SMART does not want to
do it).I don't understand what SMART has to do with it. I was talking about a patch is either ready for use, or it isn't, and "almost ready" is worthless.
Ah. Last time I checked, patch was ready to use.
But it is not strictly necessary, the reason of the patch was to mark bad blocks in mounted filesystems "on the fly". Everything else should be handled by reiserfsprogs anyway. Without reiserfsprogs support (which was ready too), it is worthless.
Bye,
Oleg
Quoted 5 lines
Everything else should be handled
by reiserfsprogs anyway. Without reiserfsprogs support (which was ready too),
If it isn't included in what we release, the job isn't done, and the work is useless. I don't know why people think that if they have written a patch that they and only they know where to get and how to apply, something useful has been done.
Signature · 1 line
Hans
Hello!
Quoted 7 lines · Hans Reiser
Quoted 3 lines
Everything else should be handled
by reiserfsprogs anyway. Without reiserfsprogs support (which was ready too),If it isn't included in what we release, the job isn't done, and the work is useless. I don't know why people think that if they have written a patch that they and only they know where to get and how to apply, something useful has been done.
The kernel patch was handed to Vitaly with instructions on how to apply and how to use. (And Vitaly added necessary support to reiserfsprogs, without which this patch would be useless)
Now the decision of whenever to release this piece of code (the patch and the reiserfsprogs parts) is not mine.
Bye,
Oleg
Quoted 12 lines · Oleg Drokin
Hello!
Quoted 8 lines · Hans Reiser
Quoted 3 lines
Everything else should be handled
by reiserfsprogs anyway. Without reiserfsprogs support (which was ready too),If it isn't included in what we release, the job isn't done, and the work is useless. I don't know why people think that if they have written a patch that they and only they know where to get and how to apply, something useful has been done.
Quoted 5 lines
The kernel patch was handed to Vitaly with instructions on how to apply and how to use. (And Vitaly added necessary support to reiserfsprogs, without which this patch would be useless)
Now the decision of whenever to release this piece of code (the patch and the reiserfsprogs parts) is not mine.
The code needed to be tested first before releasing, it is done by now and I am about to put on our ftp site 3.6.12-pre1 release with this bad block support.
Signature · 2 lines
Thanks, Vitaly Fertman