Skip to contentSkip to conversation

reiserFS?

DISCUSSION Xavier Roche 11 participants 16 messages Jul 16, 2006 – Jul 20, 2006

lore
Share & tools
lore
#0 lore raw
Quoted 1 line

It simply the best filesystem for many kinds of usage patterns.

The most frightening too. Reiserfs might be suitable for very specific appliactions, but to use it in production machine, you need to have some guts.

My last reiserfs partition was blown up two days ago, because of a bad sector, plus a fatal oops, looping endlessly. This was the second time, and the last one, as none of my ext3 filesystems *ever* had similar problems, despite numerous other bad sector issues. Not mentionning the funny "recovery" tool, which generally finishes to trash your data.

Jul 14 23:35:29 linux kernel: hdh: dma_intr: status=0x51 { DriveReady SeekComplete Error }
Jul 14 23:35:29 linux kernel: hdh: dma_intr: error=0x40 { UncorrectableError }, LBAsect=12458384, sector=12458383
Jul 14 23:35:29 linux kernel: ide: failed opcode was: unknown
Jul 14 23:35:29 linux kernel: end_request: I/O error, dev hdh, sector 12458383
Jul 14 23:35:29 linux kernel: ------------[ cut here ]------------
Jul 14 23:35:29 linux kernel: kernel BUG at fs/reiserfs/file.c:620!
..
Jul 14 23:35:29 linux kernel:  <0>Fatal exception: panic in 5 seconds

The funny part is that 14 july is the french's fireworks day, generally launched around midnight.

#1 lore raw
Quoted 6 lines · Xavier Roche
Quoted 1 line

It simply the best filesystem for many kinds of usage patterns.

The most frightening too. Reiserfs might be suitable for very specific appliactions, but to use it in production machine, you need to have some guts.

Quoted 6 lines

My last reiserfs partition was blown up two days ago, because of a bad sector, plus a fatal oops, looping endlessly. This was the second time, and the last one, as none of my ext3 filesystems *ever* had similar problems, despite numerous other bad sector issues. Not mentionning the funny "recovery" tool, which generally finishes to trash your data.

I don't quite understand. You are supposed to dd_rescue the whole block device to a working drive and use fsck on the copy. Whatever is lost in the process must of course be restored from a recent backup. But, as a friend of mine put it recently, people don't need backup, they only need restore ; )

fsck on a faulty drive might cause even more damage - how do you know that other areas of the device are OK?

I also oppose the ReiserFS-v3.x design philosophy regarding faulty storage layer, but in any case where your _live_ data is valuable and uptime counts, you _really_should_ use a RAID of some sort.

Kind regards,
uziel

PS: Your mail was line-wrapped really bad, you might want to look into that.

#2 lore raw
Quoted 5 lines · Christian Trefzer

I don't quite understand. You are supposed to dd_rescue the whole block device to a working drive and use fsck on the copy. Whatever is lost in the process must of course be restored from a recent backup. But, as a friend of mine put it recently, people don't need backup, they only need restore ; )

If the disk is known to be bad, yes, and the number of bad blocks is growing. On the other hand, disks can and will have a few bad blocks, or bad writes that don't mean the disk is going bad, and a modern filesystem should be robust enough that a single failed sector doesn't cause the filesystem to go completely kaput.

In fact, one of the scary trends with hard drives is that size is continuing to grow expoentially, access times linearly (more or less), and error rates (errors per kilobytes per unit time) are remaining more or less constant.

The fact that reiserfs uses a single B-tree to store all of its data means that very entertaining things can happen if you lose a sector containing a high-level node in the tree. It's even more entertaining if you have image files (like initrd files) in reiserfs format stored in reiserfs, and you run the recovery program on the filesystem.....

Yes, I know that reiserfs4 is alleged to fix this problem, but as far as I know it is still using a single unitary tree, with all of the pitfalls that this entails.

Now, that being said, that by itself is not a reason not to decide not to include reseirfs4 into the mainline sources. (I might privately get amused when system administrators use reiserfs and then report massive data loss, but that's my own failure of chairty; I'm working on it.) For the technical reasons why resierfs4 hasn't been integrated, please see the mailing list archives.

- Ted

#3 lore raw
Quoted 5 lines · Theodore Tso

If the disk is known to be bad, yes, and the number of bad blocks is growing. On the other hand, disks can and will have a few bad blocks, or bad writes that don't mean the disk is going bad, and a modern filesystem should be robust enough that a single failed sector doesn't cause the filesystem to go completely kaput.

I never trust a single hard drive with data that cannot be instantly re-generated anyway (eg squid caches). The fact that some people have hardware errors should not require every single fs to accommodate random bad-sectors. Feel free to use ext3 or other fs which handles this situation (and other situations) better, but reiserfs works fine on good hardware. It has been my root filesystem on many systems with no problems whatsoever, even with cheap SATA drives.

Quoted 15 lines

In fact, one of the scary trends with hard drives is that size is continuing to grow expoentially, access times linearly (more or less), and error rates (errors per kilobytes per unit time) are remaining more or less constant.

The fact that reiserfs uses a single B-tree to store all of its data means that very entertaining things can happen if you lose a sector containing a high-level node in the tree. It's even more entertaining if you have image files (like initrd files) in reiserfs format stored in reiserfs, and you run the recovery program on the filesystem.....

Yes, I know that reiserfs4 is alleged to fix this problem, but as far as I know it is still using a single unitary tree, with all of the pitfalls that this entails.

Quoted 7 lines

Now, that being said, that by itself is not a reason not to decide not to include reseirfs4 into the mainline sources. (I might privately get amused when system administrators use reiserfs and then report massive data loss, but that's my own failure of chairty; I'm working on it.) For the technical reasons why resierfs4 hasn't been integrated, please see the mailing list archives.

I read the archives, and most of the problems pointed out during the review were fixed relatively quickly, followed by a flame war due to some suggesting that reiser4 should not be able to affect VFS semantics, and other such matters (which IMO should be outside of the scope of a code review). There has been no follow-up review as far as I can tell. The discussion quickly degenerated into a personality argument against Mr Reiser, with several developers taking a strong position against reiser4 (the exact reasons for which are not specified).

I don't quite know where reiser4 stands at the moment, given that it is in -mm and has been for a very long time. I also looked at the patch again, and it is indeed quite well isolated from the rest of the kernel so I don't understand why it is not being merged as an EXPERIMENTAL option.

Regardless, it is available in -mm for anyone to use, and last I checked, works incredibly well leaving other filesystems miles behind in terms of speed. I haven't tested it enough to comment on the reliability, but if it is as reliable as reiserfs, it is sufficient for me and many others who use RAID and a UPS.

Regards,
LL

#4 lore raw
Quoted 5 lines · Lexington Luthor

I read the archives, and most of the problems pointed out during the review were fixed relatively quickly, followed by a flame war due to some suggesting that reiser4 should not be able to affect VFS semantics, and other such matters (which IMO should be outside of the scope of a code review). There has been no follow-up review as far as I can tell.

As far as I know not all of the problems were fixed. And it has been observed that given the abuse and accusations that were directed at the people who did decide to review it, that it would not at all surprising if some (all?) of reviewers may have decided they had better things to do. Getting things merged into mainline is not a right, and the reviewers are volunteers.....

Speaking for myself, since I don't enjoy being accused of partisanship and being ascribed of having a desire to backstab reiserfs, I have a personal policy to avoid reiserfs review, and recuse myself from any votes within program committee discussions regarding Hans Reiser. Being accused of taking unfair advantage of my volunteer activities is something I allow myself to get into once.

- Ted

#5 lore raw

"Why Reuser 4 still is not in" doc (was: 'reiserFS?')

Quoted 6 lines · Theodore Tso

As far as I know not all of the problems were fixed. And it has been observed that given the abuse and accusations that were directed at the people who did decide to review it, that it would not at all surprising if some (all?) of reviewers may have decided they had better things to do. Getting things merged into mainline is not a right, and the reviewers are volunteers.....

Maybe it's too late and reiser 4 will get in in the next release, but I've written this doc into the kernelnewbies' wiki:
http://wiki.kernelnewbies.org/WhyReiser4IsNotIn . If you disagree with something in that doc, edit it or just answer to this mail what you want to see in it and I'll add it myself.

#6 lore raw

Re: "Why Reuser 4 still is not in" doc

Quoted 5 lines · Diego Calleja

Maybe it's too late and reiser 4 will get in in the next release, but I've written this doc into the kernelnewbies' wiki:
http://wiki.kernelnewbies.org/WhyReiser4IsNotIn . If you disagree with something in that doc, edit it or just answer to this mail what you want to see in it and I'll add it myself.

Hi,

Thanks for the doc and for answering the FAQs.

Do you know of a list of current issues or problems with the reiser4 code that need to be resolved prior to a merge or another review? I have looked around and the mailing list archives only contain one review and the issues pointed out in there (aside from the VFS integration debate) have been resolved as far as I can tell.

I would like to jump in and perhaps help reiser4 along by trying to fix any simple issues, but I am unable to find any specifics as to why it is not being merged in its current state. Is there any message or page I am missing?

Thanks,
LL

#7 lore raw

Re: "Why Reuser 4 still is not in" doc

Quoted 1 line

(aside from the VFS integration debate)

Anybody know what's in Reiser4 that VFS doesn't like (link please)?

#8 lore raw

Re: "Why Reuser 4 still is not in" doc

Quoted 2 lines · Joshua Hudson
Quoted 1 line

(aside from the VFS integration debate)

Anybody know what's in Reiser4 that VFS doesn't like (link please)?

Reiser4 plug-ins have (had?) the ability to alter the semantics of things, like making files into directories inside which you could see meta-files like file/uid and file/size which contained meta-data and such accessible as normal files to all the unix tools (which is a very good idea IMO). You could get things like chmod by just 'echo root >file/owner' or something, very nice.

This was frowned upon by kernel developers who felt that it belonged in the kernel VFS (if at all), rather than in reiser4 directly.

Regards,
LL

#9 lore raw

Re: "Why Reuser 4 still is not in" doc

Quoted 5 lines
Quoted 2 lines
Quoted 1 line

(aside from the VFS integration debate)

Anybody know what's in Reiser4 that VFS doesn't like (link please)?

Reiser4 plug-ins have (had?) the ability to alter the semantics of things, like making files into directories inside

Yes, it changes the semantics. Suddenly you can "cd linux-2.6.17.tar.bz2". But what will stat() return? S_IFDIR? S_IFREG? S_IFANY? A .tar parser in kernelspace is almost never the right thing. And then a cpio parser, because that's what initramfs'es are made of. Not to forget .zip, because that's omnipresent. Oh of course we'd also need bzip2 and gzip decoder. BASE64 and UU anyone?

Quoted 5 lines

which you could see meta-files like
file/uid and file/size which contained meta-data and such accessible as normal files to all the unix tools (which is a very good idea IMO). You could get things like chmod by just 'echo root

Quoted 1 line

file/owner' or something, very nice.

I wish you a lot of fun with users in LDAP or other exotic storage methods. By making Everything possible through echo, you are violating the unix philosophy that one tool should do one thing (though echo does just that). And in this case, echo would be chown, chmod, tar, bzip2 all at once. This sounds familiar, I think I have seen this with explorer.exe (and its uncountable DLLs), which lets you change everything within the same window.

What I think is promising are the compression/encryption plugins. ext2 and 3 had an attribute (lsattr) for compression but it does not seem like ever implemented.

Quoted 2 lines

This was frowned upon by kernel developers who felt that it belonged in the kernel VFS (if at all), rather than in reiser4 directly.

Jan Engelhardt

#10 lore raw

Re: "Why Reuser 4 still is not in" doc

Quoted 1 line · Jan Engelhardt

Yes, it changes the semantics. Suddenly you can "cd linux-2.6.17.tar.bz2". But what will stat() return? S_IFDIR? S_IFREG? S_IFANY? A .tar parser in kernelspace is almost never the right thing. And then a cpio parser, because that's what initramfs'es are made of. Not to forget .zip, because that's omnipresent. Oh of course we'd also need bzip2 and gzip decoder. BASE64 and UU anyone?

Is there any particular reason that the parsers need to be in kernel-space. The reiser4 plugins seem like an ideal counterpart to FUSE. Imagine being able to automatically FUSE-mount a tar file as a filesystem when you cd into it. stat() need not return S_IFDIR since everything is a directory anyway (only normal directories need S_IFDIR, just like currently). When you cd into a tar file, a FUSE-fs kicks in and provides access to the tar file as a normal filesystem inside it - from userspace.

Quoted 2 lines

I wish you a lot of fun with users in LDAP or other exotic storage methods.
By making Everything possible through echo, you are violating the unix philosophy that one tool should do one thing (though echo does just that). And in this case, echo would be chown, chmod, tar, bzip2 all at once. This sounds familiar, I think I have seen this with explorer.exe (and its uncountable DLLs), which lets you change everything within the same window.

And why can meta-data not be accessed as files? To me, a lowly userspace developer, it seems even more inline with the UNIX way of things. bzip2 can be in userspace while still providing data to kernel space via a FUSE-like interface.

Regards,
LL

#11 lore raw

Re: "Why Reuser 4 still is not in" doc

In reply to Diego Calleja
Quoted 5 lines · Diego Calleja

Maybe it's too late and reiser 4 will get in in the next release, but I've written this doc into the kernelnewbies' wiki:
http://wiki.kernelnewbies.org/WhyReiser4IsNotIn . If you disagree with something in that doc, edit it or just answer to this mail what you want to see in it and I'll add it myself.

It could be improved in two ways in order to better convince people:

Firstly, mention some (perhaps even all) of the actual technical issues preventing the inclusion of Reiser4. Right now, you are giving much space to the political issues (or their non-existence). This answers a different question ("Why we are fed up with the Reiser4 discussion") than the one in the title, and in the worst case might leave a reader with the impression that political reasons are more important after all.

Secondly, the questions in the FAQ part should probably be put in a less loaded form. For example, Q1 could just read objectively:
"Why can't Reiser4 be included as an experimental feature?" As it stands now, I fear too many people's reaction upon reading it will be: "That's not my question." At least leave off the plaintive initial "but" on Q1 and Q2. It puts off the readers you want to convince.

HTH
Tilman

#12 lore raw
In reply to Theodore Tso

It is a sad reflection that we have these regular 'fs wars'; and that most of them are driven by peoples bad experiences with particular filesystems.

That leads me to ask what level of testing is performed on each filesystem - are there filesystem torture tests that are getting run by someone (who?) on various filesystems (preferably on large TB sized ones, preferably with simulated failures and resets)? The discussions on Ext4 a few weeks ago made me think that the thing I'd value more than anything else would be a damn good test regime.

It would be much nicer if the fs wars came down to peoples particularly good experiences with filesystems rather than people selecting file systems based on which one has lost them data most rarely.

Dave

P.S. For the record I use Reiser for large (>500GB) fs since I couldn't get Ext3 stable on one a year or so ago and xfs failed
the 'recover from hitting reset' test. A couple of years
ago I wouldn't touch Reiser because of NFS issues, but it seems to have grown out of that.

Signature · 4 lines
 -----Open up your eyes, open up your mind, open up your code -------   
/ Dr. David Alan Gilbert    | Running GNU/Linux on Alpha,68K| Happy  \ 
\ gro.gilbert @ treblig.org | MIPS,x86,ARM,SPARC,PPC & HPPA | In Hex /
 \ _________________________|_____ http://www.treblig.org   |_______/
#13 lore raw
Quoted 7 lines · Dr. David Alan Gilbert

That leads me to ask what level of testing is performed on each filesystem - are there filesystem torture tests that are getting run by someone (who?) on various filesystems (preferably on large TB sized ones, preferably with simulated failures and resets)? The discussions on Ext4 a few weeks ago made me think that the thing I'd value more than anything else would be a damn good test regime.

As far as I know ext2/3 is the only filesystem with a fsck tool that has a regression test suite. It's always amazed me that filesystem consistency checkers and repair tools get so little attenion by most filesystem developers.

As far as random torture testing, Pavel has written a random test tool that punches random errors into random blocks of a filesystem, and that was used to uncover a couple of cases that e2fsprogs didn't handle cleanly. Those were reported to me, and I fixed them, and the edge cases were encorprated into the regression test suite.

IIRC it came up in discussion a few weeks ago one LKML or linux-fsdevel (I can't remember which), and I believe either someone from XFS or Reiser team was going to take Pavel's torture tester and adapt it do some robustifying of their filesystem's repair capabilities.

Finally, the good folks at the Stanford Metacompilation group did some very interesting work to find bugs in three common Linux filesystems:

http://keeda.stanford.edu/~junfeng/papers/osdi04/osdi04.html

Regards,

- Ted

#14 lore raw
In reply to Christian Trefzer

Christian Trefzer schrieb am 2006-07-16:

Quoted 2 lines

fsck on a faulty drive might cause even more damage - how do you know that other areas of the device are OK?

All drives I bought in the recent years shipped with ARRE and ARWE set. Any halfway decent fsck tool can just rewrite the b0rked sector and the drive transparently remaps a spare.

If reiser4 crashes the kernel instead, it's not failure resilient and is certainly not ready for production.

Quoted 3 lines

I also oppose the ReiserFS-v3.x design philosophy regarding faulty storage layer, but in any case where your _live_ data is valuable and uptime counts, you _really_should_ use a RAID of some sort.

...and certainly a backup system that gives you FAST access to the data. RAID doesn't protect against accidents with rm(1) or, worse, wipe(1).

Signature · 1 line
Matthias Andree
#15 lore raw
In reply to Christian Trefzer
Quoted 14 lines · Christian Trefzer
Quoted 6 lines · Xavier Roche
Quoted 1 line

It simply the best filesystem for many kinds of usage patterns.

The most frightening too. Reiserfs might be suitable for very specific appliactions, but to use it in production machine, you need to have some guts.

Quoted 6 lines

My last reiserfs partition was blown up two days ago, because of a bad sector, plus a fatal oops, looping endlessly. This was the second time, and the last one, as none of my ext3 filesystems *ever* had similar problems, despite numerous other bad sector issues. Not mentionning the funny "recovery" tool, which generally finishes to trash your data.

Quoted 6 lines

I don't quite understand. You are supposed to dd_rescue the whole block device to a working drive and use fsck on the copy. Whatever is lost in the process must of course be restored from a recent backup. But, as a friend of mine put it recently, people don't need backup, they only need restore ; )

Well, many a home user simply doesn't have a a spare block device of the same size. The hassle of reinstalling instead of just waiting out a fsck is something still.

The ext filesystems are nice in that they have spare superblocks, if the main one dies from a bad sector, the spares still work so you don't loose the entire fs to only a few damaged sectors.

Quoted 3 lines

fsck on a faulty drive might cause even more damage - how do you know that other areas of the device are OK?

Somehow, that has saved my day quite a few times with ext2. I only lost a few files, then went shopping for a new disk. :-)

Helge Hafting