Skip to contentSkip to conversation

Dying disk and filesystem choice.

DISCUSSION monkeyiq 12 participants 23 messages May 24, 2001 – May 25, 2001

lore
Share & tools
lore

monkeyiq

author
#0 lore raw

Hi,
Could I please be CC'd replies.

To keep it short and sweet, I have a 45Gb IBM drive that
is slowly dying by getting more bad sectors. I have already returned my first one to get the current disk, so would like to use the current one for a while before returning it for another disk that will prolly just start dying again.

I am using reiserfs at the moment, which doesn't really like to work on a dying drive. for example, doing a make fails to work even though it is *creating* files on the disk, it fails to do so because it hits new bad sectors and doesn't seem to remap them.

I am wondering what advise on filesystem choice the list as and any other options I can use to get the kernel to remap bad blocks.

Thanks.

Signature · 4 lines
---------------------------------------------------
It's the question, http://witme.sourceforge.net
If you think education is expensive, try ignorance.
		-- Derek Bok, president of Harvard
#1 lore raw
Quoted 28 lines · monkeyiq

Hi,
Could I please be CC'd replies.

To keep it short and sweet, I have a 45Gb IBM drive that
is slowly dying by getting more bad sectors. I have already returned my first one to get the current disk, so would like to use the current one for a while before returning it for another disk that will prolly just start dying again.

I am using reiserfs at the moment, which doesn't really like to work on a dying drive. for example, doing a make fails to work even though it is *creating* files on the disk, it fails to do so because it hits new bad sectors and doesn't seem to remap them.

I am wondering what advise on filesystem choice the list as and any other options I can use to get the kernel to remap bad blocks.

Thanks.

--
---------------------------------------------------
It's the question, http://witme.sourceforge.net
If you think education is expensive, try ignorance.
-- Derek Bok, president of Harvard

Quoted 5 lines

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/

You can get the badblocks patch from Nikita and continue using reiserfs if you want. ext2 will also work.

We haven't sent the badblocks patch to Linus solely because it is a feature not a bugfix, and code-freeze is on.

Hans

#2 lore raw
In reply to monkeyiq

It was written:

Quoted 12 lines

To keep it short and sweet, I have a 45Gb IBM drive that
is slowly dying by getting more bad sectors. I have already returned my first one to get the current disk, so would like to use the current one for a while before returning it for another disk that will prolly just start dying again.

I am using reiserfs at the moment, which doesn't really like to work on a dying drive. for example, doing a make fails to work even though it is *creating* files on the disk, it fails to do so because it hits new bad sectors and doesn't seem to remap them.

Quoted 3 lines

I am wondering what advise on filesystem choice the list as and any other options I can use to get the kernel to remap bad blocks.

Well reiserfs is probably a very bad choice at this point. It does not have any bad blocks support (yet), so as soon as you have a bad block you are stuck.

You should probably use ext2, along with the badblocks program to check which blocks in the disk are bad. It will do a read-write test, and either your disk will remap the bad blocks, or they will be detected.

Even so, keep your backups current...

Cheers, Andreas

Signature · 3 lines
Andreas Dilger  \ "If a man ate a pound of pasta and a pound of antipasto,
                 \  would they cancel out, leaving him still hungry?"
http://www-mddsp.enel.ucalgary.ca/People/adilger/               -- Dogbert
#3 lore raw
In reply to monkeyiq
Quoted 3 lines · Andreas Dilger

Well reiserfs is probably a very bad choice at this point. It does not have any bad blocks support (yet), so as soon as you have a bad block you are stuck.

reiserfs doesn't, but the HD usually has transparently in its firmware. So it hits a bad block; you see an IO error and the next time you hit the block the firmware has mapped in a fresh one from its internal reserves.

-Andi

#4 lore raw
Quoted 4 lines · Andi Kleen

reiserfs doesn't, but the HD usually has transparently in its firmware. So it hits a bad block; you see an IO error and the next time you hit the block the firmware has mapped in a fresh one from its internal reserves.

Drives have remapping capability, but it's the first I've heard of HD firmware doing it automatically. I'd be very interested in reading the relevant documentation, if you could provide a pointer. Seems to me if a drive *could* do this, you'd certainly want to turn it (automatic remapping) off. There's way too much chance that a system will read the remapped sector and assume that it contains the original data. That would be hopelessly corrupting.

Signature · 1 line
/Jonathan Lundell.
#5 lore raw
In reply to monkeyiq
Quoted 9 lines · Andi Kleen
Quoted 3 lines · Andreas Dilger

Well reiserfs is probably a very bad choice at this point. It does not have any bad blocks support (yet), so as soon as you have a bad block you are stuck.

reiserfs doesn't, but the HD usually has transparently in its firmware. So it hits a bad block; you see an IO error and the next time you hit the block the firmware has mapped in a fresh one from its internal reserves.

In fact you will typically only see an I/O error if the drive _can't_ remap the sector anymore, because it has run out. No point in reporting a condition that was recovered.

I'd still say, that if you get bad block errors reported from your disk it's long overdue for replacement.

Signature · 1 line
Jens Axboe
#6 lore raw
Quoted 6 lines · Jens Axboe

In fact you will typically only see an I/O error if the drive _can't_ remap the sector anymore, because it has run out. No point in reporting a condition that was recovered.

I'd still say, that if you get bad block errors reported from your disk it's long overdue for replacement.

This can't be right. It implies that the drive is returning bogus data with no error indication. Remapping a bad sector is not the same as recovering it.

Signature · 1 line
/Jonathan Lundell.
author
#7 lore raw
In reply to monkeyiq
Quoted 11 lines · Jens Axboe
Quoted 9 lines · Andi Kleen
Quoted 3 lines · Andreas Dilger

Well reiserfs is probably a very bad choice at this point. It does not have any bad blocks support (yet), so as soon as you have a bad block you are stuck.

reiserfs doesn't, but the HD usually has transparently in its firmware. So it hits a bad block; you see an IO error and the next time you hit the block the firmware has mapped in a fresh one from its internal reserves.

Quoted 9 lines

In fact you will typically only see an I/O error if the drive _can't_ remap the sector anymore, because it has run out. No point in reporting a condition that was recovered.

I'd still say, that if you get bad block errors reported from your disk it's long overdue for replacement.

Signature · 1 line
Jens Axboe

Well, I have been fighting this for a while now, its just that today it finally hit /home in a big way :( basically its giving a DoS trying to compile stuff (which needs to write 25Mb+ library files)

The drive gives the patented IBM "replace me" audio and the kernel graces me with these little chums:

May 24 14:50:22 kloof kernel: hda: dma_intr: status=0x51 { DriveReady SeekComplete Error }
May 24 14:50:22 kloof kernel: hda: dma_intr: error=0x40 { UncorrectableError }, LBAsect=78294495, sector=957496
May 24 14:50:22 kloof kernel: end_request: I/O error, dev 03:08 (hda), sector 957496
May 24 14:50:22 kloof kernel: hda: dma_intr: status=0x51 { DriveReady SeekComplete Error }
May 24 14:50:23 kloof kernel: hda: dma_intr: error=0x40 { UncorrectableError }, LBAsect=78294495, sector=957504
May 24 14:50:23 kloof kernel: end_request: I/O error, dev 03:08 (hda), sector 957504
May 24 14:50:23 kloof kernel: hda: dma_intr: status=0x51 { DriveReady SeekComplete Error }
May 24 14:50:23 kloof kernel: hda: dma_intr: error=0x40 { UncorrectableError }, LBAsect=78294495, sector=957512

I already knew that the drive was rooted by using the IBM DFT tool. I am backing up as I type. Then, roll the dice with yet another IBM drive :-/

Signature · 4 lines
---------------------------------------------------
It's the question, http://witme.sourceforge.net
If you think education is expensive, try ignorance.
		-- Derek Bok, president of Harvard
#8 lore raw
In reply to monkeyiq
Quoted 6 lines · Jonathan Lundell
Quoted 4 lines · Andi Kleen

reiserfs doesn't, but the HD usually has transparently in its firmware. So it hits a bad block; you see an IO error and the next time you hit the block the firmware has mapped in a fresh one from its internal reserves.

Quoted 7 lines

Drives have remapping capability, but it's the first I've heard of HD firmware doing it automatically. I'd be very interested in reading the relevant documentation, if you could provide a pointer. Seems to me if a drive *could* do this, you'd certainly want to turn it (automatic remapping) off. There's way too much chance that a system will read the remapped sector and assume that it contains the original data. That would be hopelessly corrupting.

There are two scenarios: read and write. For write doing remapping transparent is all fine, as the data is destroyed anyways.
For read it returns an IO error once and the next time you read from that block it contains fresh (or partly recovered) data.

-Andi

#9 lore raw
Quoted 15 lines · Andi Kleen
Quoted 6 lines · Jonathan Lundell
Quoted 4 lines · Andi Kleen

reiserfs doesn't, but the HD usually has transparently in its firmware. So it hits a bad block; you see an IO error and the next time you hit the block the firmware has mapped in a fresh one from its internal reserves.

Quoted 7 lines

Drives have remapping capability, but it's the first I've heard of HD firmware doing it automatically. I'd be very interested in reading the relevant documentation, if you could provide a pointer. Seems to me if a drive *could* do this, you'd certainly want to turn it (automatic remapping) off. There's way too much chance that a system will read the remapped sector and assume that it contains the original data. That would be hopelessly corrupting.

Quoted 4 lines

There are two scenarios: read and write. For write doing remapping transparent is all fine, as the data is destroyed anyways.
For read it returns an IO error once and the next time you read from that block it contains fresh (or partly recovered) data.

What HDs are we talking about, specifically?

WRT writes, how does the drive detect the error?

WRT reads, there are too many filesystems that would accept the second (no-IO-error) read as being the original good data.

IBM's UltraStar drives have an option (a bit in a vendor-unique mode page) that enables automatic reassignment, but it's done safely. If an unrecoverable read error is reported, the block is entered in a list of reassignment candidates. If that block is subsequently written, it's written back to the original location, and then verified. If the verify fails, the block is reassigned and rewritten; if it succeeds, it's left in the original location, and the block is removed from the reassignment candidate list.

Notice that invalid data is never returned without an error indication. That's critical.

Signature · 1 line
/Jonathan Lundell.
#10 lore raw
In reply to monkeyiq
Quoted 13 lines · Andi Kleen
Quoted 3 lines · Andreas Dilger

Well reiserfs is probably a very bad choice at this point. It does not have any bad blocks support (yet), so as soon as you have a bad block you are stuck.

reiserfs doesn't, but the HD usually has transparently in its firmware. So it hits a bad block; you see an IO error and the next time you hit the block the firmware has mapped in a fresh one from its internal reserves.

(quoted diff) +0 -2
-Andi
-
Quoted 4 lines

To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/

No, reiserfs does have badblock support!!!!

You just have to get it as a separate patch from us because it was written after code freeze.

Hans

#11 lore raw
In reply to monkeyiq
Quoted 5 lines · Hans Reiser

No, reiserfs does have badblock support!!!!

You just have to get it as a separate patch from us because it was written after code freeze.

Any chance that you'll be putting them on www.namesys.com for easy download?

-Dave

#12 lore raw
In reply to monkeyiq

Excellent!

Will this be in resierfs 4.0 then?

cu

jjs

Quoted 4 lines · Hans Reiser

No, reiserfs does have badblock support!!!!

You just have to get it as a separate patch from us because it was written after code freeze.

#13 lore raw
In reply to monkeyiq
Quoted 4 lines · Hans Reiser

No, reiserfs does have badblock support!!!!

You just have to get it as a separate patch from us because it was written after code freeze.

IMHO we are not that deep into code freeze anymore. Freevxfs got added in linux-2.4.5-pre*, so I think that a patch that adds a useful feature like badblock support would be OK.

Erik

Signature · 5 lines
J.A.K. (Erik) Mouw, Information and Communication Theory Group, Department
of Electrical Engineering, Faculty of Information Technology and Systems,
Delft University of Technology, PO BOX 5031,  2600 GA Delft, The Netherlands
Phone: +31-15-2783635  Fax: +31-15-2781843  Email: [email protected]
WWW: http://www-ict.its.tudelft.nl/~erik/
#14 lore raw
Quoted 3 lines

IMHO we are not that deep into code freeze anymore. Freevxfs got added in linux-2.4.5-pre*, so I think that a patch that adds a useful feature like badblock support would be OK.

FreeVxFS changes precisely nothing in the behaviour of any other fs - its like adding a new driver.

Updating Reiserfs requires a lot more care because it has the potential to harm existing stable setups

#15 lore raw
Quoted 9 lines · On Thursday, May 24, 2001 11:16:58 PM +0100 Alan Cox…
Quoted 3 lines

IMHO we are not that deep into code freeze anymore. Freevxfs got added in linux-2.4.5-pre*, so I think that a patch that adds a useful feature like badblock support would be OK.

FreeVxFS changes precisely nothing in the behaviour of any other fs - its like adding a new driver.

Updating Reiserfs requires a lot more care because it has the potential to harm existing stable setups

This has been mostly covered, but just in case. There are two different freezes, the kernel, and in reiserfs. The reiserfs part isn't something Alan or Linus have imposed on us, we just wanted to limit the reiserfs changes as much as possible during the early kernel releases.

The end result is that some larger scale issues are unfixed (memory pressure from VM, lost files after a crash), but we have been able to focus on the critical hoses-my-files/crashes-my-box kinds of bugs.

-chris

#16 lore raw
Quoted 19 lines · Chris Mason
Quoted 9 lines · On Thursday, May 24, 2001 11:16:58 PM +0100 Alan Cox…
Quoted 3 lines

IMHO we are not that deep into code freeze anymore. Freevxfs got added in linux-2.4.5-pre*, so I think that a patch that adds a useful feature like badblock support would be OK.

FreeVxFS changes precisely nothing in the behaviour of any other fs - its like adding a new driver.

Updating Reiserfs requires a lot more care because it has the potential to harm existing stable setups

This has been mostly covered, but just in case. There are two different freezes, the kernel, and in reiserfs. The reiserfs part isn't something Alan or Linus have imposed on us, we just wanted to limit the reiserfs changes as much as possible during the early kernel releases.

Quoted 5 lines

The end result is that some larger scale issues are unfixed (memory pressure from VM, lost files after a crash), but we have been able to focus on the critical hoses-my-files/crashes-my-box kinds of bugs.

-chris

No, our policy is strictly in sync with and reflective of that of the rest of the linux-kernel. Since the ac series has a different policy, we can be different in regards to the ac series.

And I don't begin to comprehend your not sending in the lost disk space after crash bug fix (I assume it is what you mean when you refer to lost files after a crash, because I know of no lost files after a crash bug, please phrase things more carefully), and it really annoys me and the users, frankly. Why you consider that a feature is beyond me.

monstr, could you fix it please and send the fix in? We can't wait for Chris to send it in any longer.

Thx,

Hans

#17 lore raw
Quoted 3 lines · On Friday, May 25, 2001 09:21:42 AM -0700 Hans Reiser <[email protected]>…

No, our policy is strictly in sync with and reflective of that of the rest of the linux-kernel. Since the ac series has a different policy, we can be different in regards to the ac series.

Not really, our policy has been much more restrictive than the rest of the kernel. Look at the patches we didn't send in.

Quoted 6 lines

And I don't begin to comprehend your not sending in the lost disk space after crash bug fix (I assume it is what you mean when you refer to lost files after a crash, because I know of no lost files after a crash bug, please phrase things more carefully), and it really annoys me and the users, frankly. Why you consider that a feature is beyond me.

The patch is a _huge_ change to the way files are deleted and truncated, to what happens during mount, and to the way transactions work. It is effectively a format extension, and must be verified against both 2.2.x kernels and 2.4.x kernels, in both disk formats.

Before I even consider introducing a change of this size, I want to be as sure as I can the rest of the code is stable. It is the only way we can debug it and stay sane. Even after I release the code, I won't want it in an ac series for a while. It does much more harm than good if it somehow ruins compatibility with an older kernel, especially in 2.4.x.

Yes, it is a bug fix. But, it is a very different kind of bug fix than something that corrupts files at random, or something that doesn't get buffers to disk at the right time.

I won't pretend the fix isn't important, but I won't allow larger changes to ruin the progress we've made so far.

-chris

#19 lore raw
In reply to monkeyiq
Quoted 15 lines · J Sloan

Excellent!

Will this be in resierfs 4.0 then?

cu

jjs

Quoted 4 lines · Hans Reiser

No, reiserfs does have badblock support!!!!

You just have to get it as a separate patch from us because it was written after code freeze.

No, version 4 won't ship until september 2002, these features are all v3.7, which will be sent to Linus as soon as 2.5.1 opens up.

Hans

#20 lore raw
In reply to monkeyiq
Quoted 13 lines · Erik Mouw
Quoted 4 lines · Hans Reiser

No, reiserfs does have badblock support!!!!

You just have to get it as a separate patch from us because it was written after code freeze.

IMHO we are not that deep into code freeze anymore. Freevxfs got added in linux-2.4.5-pre*, so I think that a patch that adds a useful feature like badblock support would be OK.

Erik

Quoted 11 lines

--
J.A.K. (Erik) Mouw, Information and Communication Theory Group, Department of Electrical Engineering, Faculty of Information Technology and Systems, Delft University of Technology, PO BOX 5031, 2600 GA Delft, The Netherlands
Phone: +31-15-2783635 Fax: +31-15-2781843 Email: [email protected]
WWW: http://www-ict.its.tudelft.nl/~erik/
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/

vxfs is probably a completely separate fs that won't destabilize other filesystems, or at least I hope so. ReiserFS is stable code now, we aren't going to change that by adding features unless they go into the ac series for six weeks or so first. ReiserFS is the SuSE recommended FS, and we can't go destabilizing it. I am told by sistina.com (maintainers of LVM) that in their surveys of the users of LVM, 90% use ReiserFS, and the users of LVM tend very much to be persons with mission critical servers. We sent Linus a patch to mark us as stable not experimental. When we say stable, it means something. Right now it means zero (yes, zero) new bug reports that are not user error or old versions or fsck, since 2.4.4 came out.

fsck is improving dramatically in stability every week. I used it myself last week, and got my data back minus the root directory after trashing the front of my partition accidentally. (Which gave me a chance to review its end user usability, which is also improving.) We aren't yet ready to pass the zero a random block and see it recover always excepting what was zero'd test, but we will be before long. One of the things we realized recently is that if the user knows what got trashed, and he can tell this to the FS, it can be very useful for bitmap blocks. Sorry, I wander here.

Hans

author
#21 lore raw
In reply to monkeyiq
Quoted 9 lines · Hans Reiser

No, reiserfs does have badblock support!!!!

You just have to get it as a separate patch from us because it was written after code freeze.

Hans

It might be nice to have a link to that patch from the "download" page. I didn't see that patch the first time from ftp://ftp.namesys.com/pub/misc-patches/

because I assumed that the interesting patches were linked from the downloads page. Atleast a hint that there is a badblocks patch available on the ftp site on the downloads page, so that the reader can go grab this patch.

Hopefully this will extend my mileage until drive replacement :) Though I think I'll back a few more things up before attempting a kernel compile.

Signature · 4 lines
---------------------------------------------------
It's the question, http://witme.sourceforge.net
If you think education is expensive, try ignorance.
		-- Derek Bok, president of Harvard
#22 lore raw
In reply to monkeyiq
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

"erik" == Erik Mouw <[email protected]> writes:

Quoted 4 lines · Hans Reiser
Quoted 4 lines

No, reiserfs does have badblock support!!!!

You just have to get it as a separate patch from us because it was written after code freeze.

erik> IMHO we are not that deep into code freeze anymore. Freevxfs got added erik> in linux-2.4.5-pre*, so I think that a patch that adds a useful feature erik> like badblock support would be OK.

A new filesystem don't touch anything if you don't use it. Some reasoning for inclusion of new drivers. Big changes in a working driver could make it not to work (very bad in the stable series).

Later, Juan.

Signature · 2 lines
In theory, practice and theory are the same, but in practice they 
are different -- Larry McVoy