Skip to contentSkip to conversation

reiser4 plugins

DISCUSSION Daniel Arnold 3 participants 6 messages Jun 24, 2005 – Jun 26, 2005

lore
Share & tools
lore
#0 lore raw

Hello,

I looked at the whole thread regarding "reiser4 plugins" via Kerneltrap and felt myself committed to say thank you to those people like Hans that have a vision and are working hard to let it come true (and not only talk about what might could be done ;-) ).

See the following comments as a pure outsiders user perspective view (Note: I have virtually no knowledge of the technical kernel internals) of the possibilities that Reiser4 and related technologies provide for our beloved Linux operating system as I fear that you missed this view somehow in your heated debate:

Although I'm currently using ReiserFS v3 at my (Suse) Linux box (and am happy that it uses my small hard disk space better than other file systems and that I could always repair the data on the file system in some minutes at least in large parts in case my _hardware_ caused a system crash; I had a evil graphics card...), I'm very excited of the possibilities v4 provides and wonder why there are people at LKML that don't see those possibilities although they have a lot of more knowledge than I have on files system issues (as I'm mainly a user). Perhapes some are beside technical concerns trapped into to a "traditional Unix religion" (for me this is the real reason why e.g. the free BSD's are constantly forked, as everyone thinks he has found the only true way in the "Unix religion").

Here are only some possibilities that come into my mind regarding Reiser4:

One big thing are all the nice applications that want databases like MySQL. As they need that extra database software with its own tools and structures they are often too complicated for a normal end user. Having the same software using an intelligent file system without need for a special database would be a big step in end user usability especially but not only at the desktop. And of course you can easily manipulate your database with easy standard tools like a file manager or from command line like cp, mv, rm which would be again a big step back to simplicity.

Another idea is a CVS-like revision file system. Especially in /etc/ this would help a lot... Imagine a revision-plugin for Reiser4: This system would be self documenting as you don't need to keep in mind where, when, what you changed (and for system scripts it would be easier to track the manual manipulations; especially a problem at system upgrades). You could easily make a diff of two revisions of a file e.g. for /etc/squid/squid.conf (a huge single config file) with standard command line tools like diff (like "diff /etc/squid/squid.conf/rev-0 /etc/squid/squid.conf/rev-current").

Another possibility would be e.g. replacing the RPM-database by a structure in the Reiser4 file system. Imagine the structure of the rpm database which software is installed sitting below /etc/packages/, e.g.
in /etc/packages/apache/. With removing the /etc/packages/apache/ directory a clever Reiser4-plugin would instantly remove the whole package (don't know how to handle the problem of executing necessary system scripts during that procedure though). That this idea is highly demanded can be seen on Apple computers and on a wired approach at a new FreeBSD distribution called PC-BSD, where everything of a application is installed in a own separate directory as on Windows...

Of course there are competiting approaches to this problem via hiding the database structure to the user and reinventing again the file system level as e.g. FUSE (which is/was also controversial at LKML but also highly demanded by end users). You could also mount a rpm database with a proper FUSE-plugin and let it look like a file system but this approach is not that elegant, needs a lot more software than the "filesystem as a database vision" and has of course poor performance by design compared to a direct solution like Reiser4.

And even FUSE is higly demanded as you can see e.g. with the KIO-Slaves of KDE where the filesystem view is implemented even in a higher level (to my knowledge KIO only exists as KDE needed a solution that works right now and not after many flames and discussions about a system like FUSE).

So I personally see FUSE and Reiser4 somewhat related and beeing objected by some people at LKML beside implementational problems by the same reasons of "traditional" thinking.

My personal vision would be using FUSE as a transition tool for changing existing structures back to the file system style and for mounting non-traditional remote database structures (as SSH/SFTP or IMAP like it is already done by the KIO-system in KDE) and Reiser4-plugins for sophisticated structures that enable direct fast local database access without the need of additional software.

Both ideas would be the killer feature of Linux desktops if used in applications on larger scale.

So I wish you very much luck with getting this finally into the Linux 2.6 main kernel as this software works right now and is needed right now. There is always room for later tuning in which kernel layer a specific function has to be...

Have fun,
Daniel Arnold

#1 lore raw

Daniel Arnold <[email protected]> wrote:

[...]

Quoted 4 lines

Although I'm currently using ReiserFS v3 at my (Suse) Linux box (and am happy that it uses my small hard disk space better than other file systems and that I could always repair the data on the file system in some minutes at least in large parts

Not enough in my book.

Quoted 9 lines

in case my _hardware_ caused a
system crash; I had a evil graphics card...), I'm very excited of the possibilities v4 provides and wonder why there are people at LKML that don't see those possibilities although they have a lot of more knowledge than I have on files system issues (as I'm mainly a user). Perhapes some are beside technical concerns trapped into to a "traditional Unix religion" (for me this is the real reason why e.g. the free BSD's are constantly forked, as everyone thinks he has found the only true way in the "Unix religion").

OK, let's look.

Quoted 4 lines

Here are only some possibilities that come into my mind regarding Reiser4:

One big thing are all the nice applications that want databases like MySQL.

Shoving the RDBMS into the kernel doesn't solve this, quite to the contrary. It makes the whole system less flexible (if you don't want MySQL but something else, you can't get it easily). The whole is /more/ complex (kernel space programming is /a huge lot harder/ than userspace programming). It will be /much more/ unstable, as any idiotic bug in the RDBMS will bring the /whole system/ down, or screw up completely unrelated innocent bystanders.

Quoted 2 lines

As they need that extra database software with its own tools and
structures they are often too complicated for a normal end user.

Having stuff inside the kernel doesn't make it magically simple(r) to use. Quite the opposite.

Quoted 6 lines

Having
the same software using an intelligent file system without need for a special database would be a big step in end user usability especially but not only at the desktop. And of course you can easily manipulate your database with easy standard tools like a file manager or from command line like cp, mv, rm which would be again a big step back to simplicity.

If what you want to do can be expressed in terms of ln(1), mv(1), rm(1) et al, do use them and be happy: No need for any fancy database stuff. If it can't, I'm sorry, but it /will/ be complex anyway.

Quoted 9 lines

Another idea is a CVS-like revision file system. Especially in /etc/ this would help a lot... Imagine a revision-plugin for Reiser4: This system would be self documenting as you don't need to keep in mind where, when, what you changed (and for system scripts it would be easier to track the manual manipulations; especially a problem at system upgrades). You could easily make a diff of two revisions of a file e.g. for /etc/squid/squid.conf (a huge single config file) with standard command line tools like diff (like "diff /etc/squid/squid.conf/rev-0 /etc/squid/squid.conf/rev-current").

And why don't you use a version management system for this? You could use RCS (simple, easy to use; but doesn't handle sideways relations at all), or something more complex. Userland solutions /do/ exist, and work fine. They even handle saving stuff over the network, etc.

Quoted 10 lines

Another possibility would be e.g. replacing the RPM-database by a structure in the Reiser4 file system. Imagine the structure of the rpm database which software is installed sitting below /etc/packages/, e.g. in /etc/packages/apache/. With removing the /etc/packages/apache/ directory a clever Reiser4-plugin would instantly remove the whole package (don't know how to handle the problem of executing necessary system scripts during that procedure though). That this idea is highly demanded can be seen on Apple computers and on a wired approach at a new FreeBSD distribution called PC-BSD, where everything of a application is installed in a own separate directory as on Windows...

And shove RPM into the kernel...

All this means that you have /zilch/ possibility to savage your data if the system with the custom plugins isn't available. Thanks, had more than enough fun with corrupted "databases" elsewhere. No need to make /all/ data dependent on something like that.

Quoted 8 lines

Of course there are competiting approaches to this problem via hiding the database structure to the user and reinventing again the file system level as e.g. FUSE (which is/was also controversial at LKML but also highly demanded by end users). You could also mount a rpm database with a proper FUSE-plugin and let it look like a file system but this approach is not that elegant, needs a lot more software than the "filesystem as a database vision" and has of course poor performance by design compared to a direct solution like Reiser4.

You have to consider not only the userland code, but also the kernel code. That it comes "for free" with ReiserFS doesn't make it go away. Plus the fact that whatever it provides will be ideal for a few applications, a bad fit (but livable) for many, and completely useless (or even plainly counterproductive) for some. Yes, there are operating systems around that don't have "files", everything is in a database. PHBs love them, the people who develop for them or have to manage them hate them.

Again, operating systems with funny "just as applications require" structured files were common until Unix came along with the simple "a file is a sequence of bytes, do as you wish with it but in userland" and the lot went away like a bad dream. Unix is about simple, generic infrastructure on which you can build what /you/ want, not just what /the system/ allows. Don't kill that.

Quoted 9 lines

And even FUSE is higly demanded as you can see e.g. with the KIO-Slaves of KDE where the filesystem view is implemented even in a higher level (to my knowledge KIO only exists as KDE needed a solution that works right now and not after many flames and discussions about a system like FUSE).

So I personally see FUSE and Reiser4 somewhat related and beeing objected by some people at LKML beside implementational problems by the same reasons of "traditional" thinking.

Funny. The "traditional" thinking was once radical, and swept away much of what you advocate...

[...]

Quoted 2 lines

Both ideas would be the killer feature of Linux desktops if used in applications on larger scale.

Chicken and egg: Applications using strange filesystem semantics won't show up until said semantics are commonplace, which won't be until there is extensive use...

Quoted 2 lines

So I wish you very much luck with getting this finally into the Linux 2.6 main kernel as this software works right now

It hasn't been integrated exactly because it /doesn't/ work right.

Quoted 2 lines

and is needed right
now.

Very few people have expressed any need for it.

Quoted 2 lines

There is always room for later tuning in which kernel layer a
specific function has to be...

No. It has to be done right (or nearly right) from the start. Also, screwing around with filesystems /is/ risky, as it could affect innocent bystanding filesystems.

Signature · 4 lines
Dr. Horst H. von Brand                   User #22616 counter.li.org
Departamento de Informatica                     Fono: +56 32 654431
Universidad Tecnica Federico Santa Maria              +56 32 654239
Casilla 110-V, Valparaiso, Chile                Fax:  +56 32 797513
#2 lore raw
Quoted 11 lines · Horst von Brand
Quoted 9 lines · Daniel Arnold

Another idea is a CVS-like revision file system. Especially in /etc/ this would help a lot... Imagine a revision-plugin for Reiser4: This system would be self documenting as you don't need to keep in mind where, when, what you changed (and for system scripts it would be easier to track the manual manipulations; especially a problem at system upgrades). You could easily make a diff of two revisions of a file e.g. for /etc/squid/squid.conf (a huge single config file) with standard command line tools like diff (like "diff /etc/squid/squid.conf/rev-0 /etc/squid/squid.conf/rev-current").

Quoted 4 lines

And why don't you use a version management system for this? You could use RCS (simple, easy to use; but doesn't handle sideways relations at all), or something more complex. Userland solutions /do/ exist, and work fine. They even handle saving stuff over the network, etc.

And this can be solved much nicer by using inotify/dnotify and a callout to whatever version management system the user prefers. Why lock oneself into just one version managed filesystem?

The same goes for a local search engine such as spotlight, it doesn't have to be integrated into the kernel, all that has to be in the kernel is the infrastructure that can inform the engine about file changes. And look, we have just that in dnotify and inotify, generic infrastructure that can be made to work on all filesystems, and is not just limited to one.

Quoted 11 lines
Quoted 10 lines

Another possibility would be e.g. replacing the RPM-database by a structure in the Reiser4 file system. Imagine the structure of the rpm database which software is installed sitting below /etc/packages/, e.g. in /etc/packages/apache/. With removing the /etc/packages/apache/ directory a clever Reiser4-plugin would instantly remove the whole package (don't know how to handle the problem of executing necessary system scripts during that procedure though). That this idea is highly demanded can be seen on Apple computers and on a wired approach at a new FreeBSD distribution called PC-BSD, where everything of a application is installed in a own separate directory as on Windows...

Quoted 6 lines

And shove RPM into the kernel...

All this means that you have /zilch/ possibility to savage your data if the system with the custom plugins isn't available. Thanks, had more than enough fun with corrupted "databases" elsewhere. No need to make /all/ data dependent on something like that.

Well, generic filesystem transactions would be nice, but it is a hard problem to solve. There would have to be limits on how large a transaction can be, there must be some way to handle deadlocks where two different processes try to modify the same file concurrently. Then there's security, what happens if one process starts a transaction and then asks a suid process to write to a file in that transaction, can the suid application handle that? What if the unprivileged process has started a different transaction already, can the suid process handle getting a -EDEADLOCK error properly?

I'd love to have filesystem transactions in Linux, but I don't belive its easy to do and I don't know if the complexity is worth it. And besides, if someone shows that it is possible, and desirable, I'd like to have them on ext3. So I would prefer to see a generic transaction API in the VFS that can be implemented by multiple filsystemes. This does not belong in just one filesystem IMHO.

/Christer

Signature · 4 lines
"Just how much can I get away with and still go to heaven?"

Freelance consultant specializing in device driver programming for Linux 
Christer Weinigel <[email protected]>  http://www.weinigel.se
#3 lore raw
In reply to Horst von Brand

Am Samstag, 25. Juni 2005 02:40 schrieb Horst von Brand:

Quoted 10 lines

Daniel Arnold <[email protected]> wrote:

[...]

Quoted 4 lines

Although I'm currently using ReiserFS v3 at my (Suse) Linux box (and am happy that it uses my small hard disk space better than other file systems and that I could always repair the data on the file system in some minutes at least in large parts

Not enough in my book.

Well I'm no expert in data rescue after a severe hardware crash and with ext2 or ext3 it wouldn't have been any better (even worse with ext2). So I'm just wondering why you have such high expectations towards any Reiser-FS-version but not at the same time to other file systems.

And beside that the good usage of my harddisk space (20 Gigabytes) is for me a _very_ strong point in using ReiserFS as the computer I'm talking about is not able to use hard disks larger than 32GB and I don't want to buy a new computer because of that.

And have you ever heared of the problem of Wikipedia storage? Their old SCSI storage system had no additional slot left, so there was need to buy an entire new storage array and this wasn't/isn't cheap. And of course the database grows exponentially. So there was need for a solution that is orders of magnitudes larger. The first solution was reducing the data storage problem at the root: compression (gzip compressed chunks of 20 following versions of an article; reduced the database by 85%). The second step is the new storage hardware. So a filesystem with intelligent disk usage is very much appreciated there too, regardless how big the storage array is...

Quoted 10 lines
Quoted 2 lines

One big thing are all the nice applications that want databases like MySQL.

Shoving the RDBMS into the kernel doesn't solve this, quite to the contrary. It makes the whole system less flexible (if you don't want MySQL but something else, you can't get it easily). The whole is /more/ complex (kernel space programming is /a huge lot harder/ than userspace programming). It will be /much more/ unstable, as any idiotic bug in the RDBMS will bring the /whole system/ down, or screw up completely unrelated innocent bystanders.

Ok. You're afraid that this would be again some kind of kernel-httpd. But it won't be for the following reasons:

The storage part of a database only one part of a database application. Many programming lines of a database package are about creating nice tools and interfaces for easy access and manipulation of the stored data, like a SQL-interpreter and such.

Filesystems _are_ storage backends, so existing database applications are duplicating this storage part today as their programmers and users are anoyed by the poor and less flexible possibilities the filesystems provide.

Reiser4 won't provide anything but the storage part. All the tools accessing this thing are userland applications.

You can access this "reiser4-database" with standard tools like cp, rm, mv, ls and such but you could also create an userland SQL-interface for this - and this will be done for sure as this would be a very easy possibility to enhance existing applications based on SQL-databases using the new features of Reiser4 beside existing databases too.

What userland tools you use to acess this filesystem is upon your decission (of course also which tools do exist). Why not acess a filesystem additionally with SQL?

E.g. Wikipedia has a constant problem with CPU usage of their servers, so people are flaming that PostgreSQL is faster than MySQL and Wikipedia should not use MySQL. But what about a solution based on Reiser4 (revision storage and on the fly compression needed) with a userland SQL-interface? Reiser4 _is_ fast and is data loss safe.

Quoted 9 lines
Quoted 5 lines

the same software using an intelligent file system without need for a special database would be a big step in end user usability especially but not only at the desktop. And of course you can easily manipulate your database with easy standard tools like a file manager or from command line like cp, mv, rm which would be again a big step back to simplicity.

If what you want to do can be expressed in terms of ln(1), mv(1), rm(1) et al, do use them and be happy: No need for any fancy database stuff. If it can't, I'm sorry, but it /will/ be complex anyway.

Well of course ln, mv, rm are indeed simple and you won't base complex actions on these tools as they are simply not the right tools for complex jobs but in principle you can do it and you will do it if the action you want to do is simple enough. In case you want a more powerfull solution you choose a different tool to acess the same data, see above.

Quoted 4 lines

And why don't you use a version management system for this? You could use RCS (simple, easy to use; but doesn't handle sideways relations at all), or something more complex. Userland solutions /do/ exist, and work fine. They even handle saving stuff over the network, etc.

Well yes I could do it for myself. But the majority of users (Im talking of "innocent stupid" computer users here) will never get those benefits as it is simply to complex and so distributions as Suse, RedHat and others will not base a key feature of their systems on an additional complexity caused by a daemon like CVS and of course those revision systems aren't really nice to look at via a file manager (ok there exists Cervisia-KIO-slave in KDE - again a filesystem plugin in a high level application).

And here again: The storage part of revision systems is not that big it only needs to be flexible enough. Most important are the userland tools that enable you manipulating the data. And why did Subversion not choose plain file storage as CVS? They simply disliked the limitations caused by todays standard file systems and so they went away from the file system to a database.

Quote of their FAQ:

"The repository is built on a database (currently Berkeley DB) and exports a C API that simulates a filesystem -- a versioned filesystem."

So with Reiser4 Subversion wouldn't need to base the storage on a database and do unnecessary double work and could base their storage on a prooven stable file system as Reiser4.

Quoted 3 lines
Quoted 2 lines

Another possibility would be e.g. replacing the RPM-database by a structure in the Reiser4 file system. [...]

And shove RPM into the kernel...

No I never said that the RPM-tool should get into the kernel as this wouldn't make any sense. The userland RPM-tool will be still the right tool in complex cases but you can also manipulate the database with other userland tools if you like.

Quoted 3 lines

Yes, there are operating systems around that
don't have "files", everything is in a database. PHBs love them, the people who develop for them or have to manage them hate them.

I would also dislike an operating system where I can't acess my data in standard ways with a hierarchical filesystem but how could Reiser4 be a problem regarding that? Reiser4 does not touch the file system view it only enhances it and makes it even more transparent as internal structures (that are until now only accessible via special tools) are now also a file.

Quoted 6 lines

Again, operating systems with funny "just as applications require" structured files were common until Unix came along with the simple "a file is a sequence of bytes, do as you wish with it but in userland" and the lot went away like a bad dream. Unix is about simple, generic infrastructure on which you can build what /you/ want, not just what /the system/ allows. Don't kill that.

This won't kill simplicity. It would bring back simplicity where it has sadly gone. And of course Unix was the OS with the radical "everything is a file vision". What are devices? What is below /proc? These are all systems that enable you access to completly different things via the well known file system.

So Reiser4 is a consequent way in the "everything is a file roadmap".

Quoted 3 lines

Chicken and egg: Applications using strange filesystem semantics won't show up until said semantics are commonplace, which won't be until there is extensive use...

Well one could simply create a userland SQL-interface for Reiser4 and many server applications could use its benefits in no time (for sure in the beginnings "real SQL databases" would have more features).

Quoted 3 lines

No. It has to be done right (or nearly right) from the start. Also, screwing around with filesystems /is/ risky, as it could affect innocent bystanding filesystems.

How can Reiser4 affect other parts of the kernel or harm other file systems if it dos not touch them (And Reiser4 in itself seems to be very stable according to all reports I could find)? Honestly I don't know how this can be possible but maybe I can't see it as I'm to much a user with to less knowledge...

Daniel Arnold

#4 lore raw
Quoted 10 lines · Daniel Arnold

Am Samstag, 25. Juni 2005 02:40 schrieb Horst von Brand:

Quoted 9 lines

Daniel Arnold <[email protected]> wrote:

[...]

Quoted 4 lines

Although I'm currently using ReiserFS v3 at my (Suse) Linux box (and am happy that it uses my small hard disk space better than other file systems and that I could always repair the data on the file system in some minutes at least in large parts

Quoted 6 lines
Quoted 1 line

Not enough in my book.

Well I'm no expert in data rescue after a severe hardware crash and with ext2 or ext3 it wouldn't have been any better (even worse with ext2). So I'm just wondering why you have such high expectations towards any Reiser-FS-version but not at the same time to other file systems.

Experimental data point is that on loss of single disk sectors the /whole filesystem/ got irretrievably lost. Haven't seen that on ext2...

Quoted 4 lines

And beside that the good usage of my harddisk space (20 Gigabytes) is for me a _very_ strong point in using ReiserFS as the computer I'm talking about is not able to use hard disks larger than 32GB and I don't want to buy a new computer because of that.

And because /you/ can't afford a decent machine /we all/ have to endure ReiserFS?!

Quoted 10 lines

And have you ever heared of the problem of Wikipedia storage? Their old SCSI storage system had no additional slot left, so there was need to buy an entire new storage array and this wasn't/isn't cheap. And of course the database grows exponentially. So there was need for a solution that is orders of magnitudes larger. The first solution was reducing the data storage problem at the root: compression (gzip compressed chunks of 20 following versions of an article; reduced the database by 85%). The second step is the new storage hardware. So a filesystem with intelligent disk usage is very much appreciated there too, regardless how big the storage array is...

And on the extreme other end people worry about the cost of the storage they require, and that should somehow also be a burden for /all/ Linux developers...

Don't you see that this argument makes no sense? Sure, I too would like 25GHz CPUs, and 10GigEth to the Internet at home, and filesystems that store data in no space at all. Doesn't make it happen just like that.

Quoted 11 lines
Quoted 10 lines
Quoted 2 lines

One big thing are all the nice applications that want databases like MySQL.

Shoving the RDBMS into the kernel doesn't solve this, quite to the contrary. It makes the whole system less flexible (if you don't want MySQL but something else, you can't get it easily). The whole is /more/ complex (kernel space programming is /a huge lot harder/ than userspace programming). It will be /much more/ unstable, as any idiotic bug in the RDBMS will bring the /whole system/ down, or screw up completely unrelated innocent bystanders.

Quoted 7 lines

Ok. You're afraid that this would be again some kind of kernel-httpd. But it won't be for the following reasons:

The storage part of a database only one part of a database application. Many programming lines of a database package are about creating nice tools and interfaces for easy access and manipulation of the stored data, like a SQL-interpreter and such.

So? Placing the rest into the kernel won't magically make it take up no RAM (quite the contrary), or be blindingly fast, or maximally convenient to use, or deadlock-free, or whatever.

Quoted 4 lines

Filesystems _are_ storage backends, so existing database applications are duplicating this storage part today as their programmers and users are anoyed by the poor and less flexible possibilities the filesystems provide.

Maybe it is because their storage use patterns just /don't fit/ the regular needs of normal applications, for which filesystems have been carefully designed and tuned?

[...]

Quoted 5 lines

E.g. Wikipedia has a constant problem with CPU usage of their servers, so people are flaming that PostgreSQL is faster than MySQL and Wikipedia should not use MySQL. But what about a solution based on Reiser4 (revision storage and on the fly compression needed) with a userland SQL-interface? Reiser4 _is_ fast and is data loss safe.

It very much remains to be seen if some generic filesystem is even a reasonable fit to their needs, forget about their requirements on data security (Yes, backups are a must; but if you have to pull them down each single time something goes wrong the setup is useless. And you won't know without much more tests and experience in real settings.). Fast compared to what? It is of no use if user time goes down while system time goes up by the same (or larger) ammount.

[...]

Quoted 5 lines
Quoted 4 lines

And why don't you use a version management system for this? You could use RCS (simple, easy to use; but doesn't handle sideways relations at all), or something more complex. Userland solutions /do/ exist, and work fine. They even handle saving stuff over the network, etc.

Quoted 8 lines

Well yes I could do it for myself. But the majority of users (Im talking of "innocent stupid" computer users here) will never get those benefits as it is simply to complex and so distributions as Suse, RedHat and others will not base a key feature of their systems on an additional complexity caused by a daemon like CVS and of course those revision systems aren't really nice to look at via a file manager (ok there exists Cervisia-KIO-slave in KDE - again a filesystem plugin in a high level application).

Right. So you compensate for dumb users, who wouldn't distinguish an version control system from a camel, by making the filesystem into a version control system, which they won't understand anyway, lets alone make productive use of it.

Quoted 2 lines

And here again: The storage part of revision systems is not that big it only needs to be flexible enough.

Yep. Regular files have been plenty enough for all that is out there. Doesn't that tell you something?

Quoted 2 lines

Most important are the userland tools that
enable you manipulating the data.

Exactly.

Quoted 4 lines

And why did Subversion not choose plain
file storage as CVS? They simply disliked the limitations caused by todays standard file systems and so they went away from the file system to a database.

And used a database backend that (surprise!) works on files itself. That way they can run wherever the filesystem has what is needed for DB, and they don't have to duplicate the specialized word Sleepycat has done. Smart move.

Quoted 2 lines

So with Reiser4 Subversion wouldn't need to base the storage on a database

... but I'd suggest keeping wide away from it, as very few people have ReiserFS...

Quoted 1 line

and do unnecessary double work

What double work? There is /one/ database involved when running on, e.g., ext3. Any "unnecessary double work" would squarely be within ReiserFS... just one more reason /not/ to use it, wouldn't you say?

Quoted 2 lines

and could base their storage on a
prooven

Not yet, sorry.

Quoted 1 line

stable

Very much remains to be seen. Jury deliberations up to this point indicate it isn't for version 4. Judging from the history of ReiserFS versions, it is dismal.

Quoted 11 lines

file system as Reiser4.

Quoted 4 lines
Quoted 2 lines

Another possibility would be e.g. replacing the RPM-database by a structure in the Reiser4 file system. [...]

And shove RPM into the kernel...

No I never said that the RPM-tool should get into the kernel as this wouldn't make any sense. The userland RPM-tool will be still the right tool in complex cases but you can also manipulate the database with other userland tools if you like.

Look at the various RPM tools: They are mostly a thin shell around functionality in a library. /That/ is what you'd need to push into the kernel (or at least a large part of the library). Just like mv(1) is not much more than a shell around rename(2).

[...]

Quoted 5 lines
Quoted 3 lines

No. It has to be done right (or nearly right) from the start. Also, screwing around with filesystems /is/ risky, as it could affect innocent bystanding filesystems.

How can Reiser4 affect other parts of the kernel

It needs to be integrated with the rest...

Quoted 2 lines

or harm other file
systems if it dos not touch them

Not touching them because the whole design is feet in the air, head firmly planted on the ground doesn't count.

Quoted 2 lines

(And Reiser4 in itself seems to be very
stable according to all reports I could find)?

ReiserFS 3 has a very bad record, so...

Quoted 3 lines

Honestly I don't know how
this can be possible but maybe I can't see it as I'm to much a user with to less knowledge...

Signature · 4 lines
Dr. Horst H. von Brand                   User #22616 counter.li.org
Departamento de Informatica                     Fono: +56 32 654431
Universidad Tecnica Federico Santa Maria              +56 32 654239
Casilla 110-V, Valparaiso, Chile                Fax:  +56 32 797513
#5 lore raw

Am Sonntag, 26. Juni 2005 22:15 schrieb Horst von Brand:

Quoted 7 lines
Quoted 4 lines

And beside that the good usage of my harddisk space (20 Gigabytes) is for me a _very_ strong point in using ReiserFS as the computer I'm talking about is not able to use hard disks larger than 32GB and I don't want to buy a new computer because of that.

And because /you/ can't afford a decent machine /we all/ have to endure ReiserFS?!

No you don't need to endure anything because of me. If you don't want Reiser-FS v3 oder v4 you don't need to use it. There are several filesystems to choose from integrated in the kernel...

I only wanted to make a point that there is a significant group of people and circumstances where the "harddisks are cheap" argument doesn't count and where a filesystem as ReiserFS is needed (that's why I also made the Wikipedia example).

Other people have other priorities and thus they might choose ext3. It's perfectly fine with me but why should I endure it not using Reiser4 with the vanilla kernel (okay okay I can make my own custom patched kernel, so we all end up with incompatible individual kernels in the end if you everytime get the answer: "Make your own kernel if you want feature $foobar.")?

Quoted 3 lines

So? Placing the rest into the kernel won't magically make it take up no RAM (quite the contrary), or be blindingly fast, or maximally convenient to use, or deadlock-free, or whatever.

I was talking about non-existant Reiser4 plugins. I only wanted to illustrate what is possible with this flexible system. At the moment I don't see that the current Reiser4-code with all the existing plugins makes the kernel fat (and I assume it will also not do this in future).

Quoted 8 lines
Quoted 4 lines

Filesystems _are_ storage backends, so existing database applications are duplicating this storage part today as their programmers and users are anoyed by the poor and less flexible possibilities the filesystems provide.

Maybe it is because their storage use patterns just /don't fit/ the regular needs of normal applications, for which filesystems have been carefully designed and tuned?

Well I guess someone would say the same to a /proc and a /dev device system if it wouldn't exist...

Let's look how far we can go with the file system and not say in the beginnings of an attempt: "This is not the thing it was intended to be." Let's look if people are convinced by Reiser4 and switch back away from databases. But this will only work if Reiser4 is in the vanilla kernel. It's a pure psychological thing that it will only then be used at large.

Quoted 4 lines

Right. So you compensate for dumb users, who wouldn't distinguish an version control system from a camel, by making the filesystem into a version control system, which they won't understand anyway, lets alone make productive use of it.

No sometimes the dumb user suddenly understands what it is all about if you make it easy enough. You can hide the complicated things of current version systems with a nice GUI (like KDE does with Cervisia) but the user will always have the feeling that he somehow lost touch with the system and feels uncomfortable (and will have lots of problems if someday there is a failure... as he can't use the standard tools like a file manager to manipulate individual files).

Quoted 6 lines
Quoted 2 lines

So with Reiser4 Subversion wouldn't need to base the storage on a database

[...]
What double work? There is /one/ database involved when running on, e.g., ext3. Any "unnecessary double work" would squarely be within ReiserFS... just one more reason /not/ to use it, wouldn't you say?

Well they invented again the filesystem layer, as quoted by me out of their FAQ, due to limitations in current file systems that Reiser4 tries to solve. I don't expect Subversion throwing this layer away but at least there could be a backend for Reiser4 that can be used by those people that like the idea and maybe people are suddenly using such a backend simply because they can touch their repository with a filemanager too and feel now much more comfortable in case something went wrong with it?

And those reinvented filesystem layers can be found in many places:
*KDE-KIO
*GNOME-VFS
*FUSE

And more and more applications are unsing databases as their backends (most times only because it looks cool) and the system gets more and more instransparent from the file system perspective.

All the possibilities I was illustrating are currently _non-existant_. But why are they only possible ontop of the the current _existing_ Reiser4?

*good disk usage for very small files (in databases most fields are very small)
*Atomic operations (very much needed for transactions) *exensible flexible system (custom extensions in case you need them)

But I should stop arguing now (as I can't anyways say much to the deeper technical side) and play with Reiser4 as it seems impossible to move you any inch but at least I made the point that Reiser4 maybe is not wanted by everyone but at least by a significant group of users and thus it would be very sad if it does not make it into the vanilla kernel soon.

Have fun,
Daniel Arnold