silent semantic changes with reiser4
Quoted 13 lines
Quoted 3 lines · Spam
Quoted 2 lines · Horst von Brand
Quoted 1 line · Hans Reiser
Andrew Morton wrote:
[...]
Quoted 3 lines
Quoted 2 lines
Quoted 2 lines
Andrew, we need to compete with WinFS and Dominic Giampaolo's filesystem for Apple,
Quoted 8 lines
Quoted 5 lines
Quoted 1 line
Says who?
At least I as a user see the benefits largely. The workstation / desktop user has much more need for such features, than perhaps the normal Linux sysadmin.
Show benefits, show there are no downsides. Show that the same can't possibly be done in userland. Then I'll consider buying it.
I am not saying it couldn't be done in userland. Most things can.
The problem with userland is that it doesn't provide generic access across most applications. For example GnomeVFS is limited to Gnome applications only. The same for KDE. to be able to coordinate application developer on anything is just almost impossible.
Still. Why do you oppose plugins, streams and meta files? The could
be valuable and easy to use tools for many purposes. One could
be would be advanced ACLs defined as a meta-file using XML format.
Quoted 11 lines
AFAICS: Smallish benefits for some narrow application niche. Mayor problems (no decent multiuser way of using it, extremely complex stuff placed in the kernel). Userland solutions are all around us right now, and work just fine. No, placing the same gunk in the kernel won't make it magically use no time or memory, more like exactly the opposite. Besides, all the visions waved around here have very little in common (except the idea that they make all restrictions magically dissapear).
Get a list of required features, show concrete ways of using it, outline possible implementations, discuss resource usage. I.e., do "engineering". Then we have something to discuss.
I am not sure what you refer to as "required features". Do you mean example of usage and contents for streams, plugins and meta files?
Quoted 11 lines
Quoted 10 lines
Quoted 10 lines
Quoted 2 lines
and that means we need to put search engine and database
functionality into the filesystem.Please don't. Unix works and is extremely popular because it _doesn't_ try to be smart (policy vs mechanism distinction, simple abstractions that can be combined endlessly). If you add this to the filesystems, you either redo _all_ userland to understand _one_ particular way of doing "smart things" (which will turn out to be almost exactly the dumbest possible for some uses, and then you are stuck), or get lots of shards from broken apps (and users, and sysadmins).
Quoted 5 lines
Quoted 2 lines
Indeed. But you forget one thing. Combining lots of small userland tools can actually make things more complex - at least for the user.
So? Placing lots of little things in kernel does the same, plus screws over sysadmins, and makes kernel hacker's job harder. I prefer the former alone.
Difference being that it could be transparent in a way otherwise not possible.
Quoted 45 lines
Quoted 2 lines
That said, I do not see that anyone has suggested to remove this way of working with applications.
Excuse me, if files suddenly become directories, and you can't have some "reserved" file/directory names, and... you are breaking lots of stuff.
Quoted 1 line
Are the new and old features mutually exclusive?
Yep.
Quoted 7 lines
Quoted 5 lines
Quoted 2 lines
It takes 11 years of serious
research to build a clean storage layer able to handle doing that.Great! Build an experimental OS showing how to use it, and through that see if the ideas hold any water
_inreal, day-to-day, down-to-earth,practice_.There are many uses already.
Where?
Quoted 3 lines
It could be encryption, compression,
meta-data, file streams, sorting, searching and a multitude of uses and ideas that haven't been discovered yet."Could", "multitude ... haven't been discovered yet" is _not_ "many current uses".
Quoted 9 lines
Quoted 5 lines
Quoted 2 lines
Reiser4 has done that, finally. None of the other Linux filesystems have.
How come nobody before you in the almost 30 years of Unix has ever seen the need for this indispensable feature?
I, as a user, have for a long time. Do not drag everyone over the same edge. Users have different needs. We should try to be as forthcoming as possible to provide the tools and options for them.
Why? Breaking the basic paradigms for some user whim is not good OS design in my book, sorry. Many "bright ideas" have been thrown out over the years for just the same reasons.
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
The next major release of ReiserFS is going to be bursting with
semantic enhancements, because the prerequisites for them are in place now. None of the other Linux filesystems have those prerequisites.
Quoted 8 lines
Quoted 4 lines
Quoted 1 line
To me that would mean that ReiserFS has no place in Linux.
Why do you say this? I thought Linux embraced advancements and new technologies.
If they are right. I think breaking the base of the OS is not a good idea. Perhaps ReiserFS has a place elsewhere (in MacOS, or Windows, where this is supposedly so so badly needed?), but not here.
You mean in Windows and MacOS where much of this functionality already exists?
Quoted 12 lines
Quoted 11 lines
Quoted 11 lines
Quoted 2 lines
They won't be able to keep up with the semantic enhancements. This metafiles and file-directories stuff is actually fairly trivial stuff.
Maybe. But the question of it being of any use aren't trivially answered "yes". My gut feeling is that the answer is a resounding "no", but that's just me. Direcories are directories, if you want to ship directory-like stuff around, use directories (or tarfiles, or whatever). Just look what happened to the much, much easier stuff of splitting SUID/SGID into
capabilities: Nothing at all whatsoever, because they have no decent place to stay in the hallowed Unix APIs. Sure, Linux is now large enough to be able to _define_ APIs to follow, but even so it isn't at all easy to do so.
Quoted 3 lines
Quoted 3 lines
Exactly. The problem I see in this discussion is the lack of just this - APIs. Because the lack of good APIs we cannot make any larger changes or bring new features because old applications will break.
I'll chop this up...
Quoted 3 lines
No. There is a complete lack of answers to why it should be done ("I think it would be nice", "MSFT is trying to do it", "Look at MacOS" is far from enough)
No perhaps not by itself. But it certainly should be considered.
These two OS'es have most of the market share. They have great resources trying to make things that work for their users.
Users are more important than you think. Linux and any OS can do what it wants, but if it wants to keep users then it has to listen to them.
Quoted 3 lines
, on what the alternatives are ("Sorry, it must be in the kernel because it is impossible to get application writers to agree on how to do it"
This is a perfectly valid reason. If it is necessary to move something into the core of Linux because it couldn't otherwise be done then it should seriously be considered.
Quoted 2 lines
will just add another incompatible way of "doing almost the same", this
time in the kernel),
People on this list are actually trying to make it to be fully compatible with old legacy applications.
If you hadn't noticed, Reiser4 does work. The meta files do work. The discussion is wether parts of Reiser4 should be made generic and fit into the VFS.
Quoted 4 lines
and last but not least what the impact is (performance
wise, application breakage, user/sysadmin expectation breakage, incompatibility with other systems, kernel development impact, to list a few). Hash this out, and we can talk.
You paint up a doomsday picture here. Nothing of this magnitude has happened and won't. besides, what is wrong if sysadmins would have to learn something new?
Quoted 9 lines
Quoted 4 lines
I think that Linux should make new API that will completely separate applications from the storage media. This way we will not as easily break applications when new features are developed. This should be the main priority.
The "files + directories" stuff is a nice abstraction, and works today on all random-access storage media. You don't care much if it is on a CD, a floppy, a disk, a RAID array, or a flash card, do you? It won't work on tapes, or printers; but that is to be expected.
you didn't actually comment on what I said. I mentioned API's nothing about directories and files. API being the keyword here.
If there are good API's that applications can use, for example to access filesystems, then it will be easier and much more possible to introduce new things into the filesystem without breaking existing applications. One example of preventing application breakage could
be to have a versioning system. Applications could query which
version is supported, or vice versa.
From what I understand it is much harder to do anything today because of the fixed ways. This is why we have this discussion.
Quoted 8 lines
[...]
Quoted 2 lines
Application breakage has happened before between virtually every kernel change. It isn't new. Just take devfs and sysfs.
Oops, now emacs stopped working because devfs was added? gcc needed a mayor overhaul because of /sys vs /proc? Gimme a break. That was
_basicsystemtool_breakage, not all-around application breakage.
Emacs seem to work fine with the file-as-dir on my Reiser4 partition. And so does actually any other application I have used on it. Be it in Gnome, KDE or in plain shell's.
Quoted 4 lines
Quoted 4 lines
Quoted 3 lines
Quoted 3 lines
Look guys, in 1993 I anticipated the battle would be here, and I build the foundation for a defensive tower right at the spot MS and Apple are now maneuvering towards.
Quoted 8 lines
Quoted 5 lines
Quoted 2 lines
Why place a protective tower around the pit into which they are trying desperately to throw themselves? ;-)
It is easy to say this. But the fact is that users can actually use these new features for good things.
And can abuse them for bad things, and the features aren't free. Balance benefits (very minor) against costs (quite large).
Abuse depends on permissions and security policies which is your job as admin to look after.
It would be possible to limit, for example), the types of plugins that a user can either use or load.
Do you give your users access to modprobe or tools like that?
Quoted 14 lines
Quoted 13 lines
Quoted 13 lines
Quoted 6 lines
Help me get the next level on the tower before
they get here. It is one hell of a foundation, they won't be able to shake it, their trees are not as powerful. Don't move reiser4 into vfs, use reiser4 as the vfs. Don't write filesystems, write file plugins and disk format plugins and all the other kinds of plugins, and you won't be missing any expressive power that you really want....Better write libraries handling whatever weird format you have in mind for each use. Even applications. I don't expect to be able to use emacs to edit a JPEG or PostgreSQL database. Neither do I expect gimp to be able to edit a poem. The current trend (which I heartily agree with) is to take stuff _out_ of the kernel (for flexibility, access, development ease, even performance).
Quoted 6 lines
Quoted 3 lines
If I want to be able to quickly find files by a certain author I
could search the meta-data fields (if they are used) with normal
tools such as grep.Iff the metadata is there. It won't (everybody would have to agree on placing that on the files for it to work), so it is useless.
No it is not useless. It just means the file has no meta information. Just as it is today. No change.
Quoted 11 lines
Quoted 4 lines
Compression or encryption is another example.
Would you rather patch every application there is for Linux to enable support for reading these files and directories?
No. But an encrypted or compressed filessytem doesn't need weird files. Easiest on applications is just to serve the decrypted/uncompressed files for use via the traditional APIs. Next easiest is provide programs that handle cryptography and compression and can be combined with whatever unaware program you care to use. Funny, that is exactly the way things work now...
So you can setup a directory that has some files compressed and some other encrypted and yet they will be available to use with tar, cp, ls, mc, Nautilus, Emacs etc etc?
Quoted 2 lines · Horst von Brand
Build an experimental OS showing how to use it, and through that see if the ideas hold any water
_inreal, day-to-day, down-to-earth,practice_.
Hey, if you can think of a way to build an experimental OS and get it used in real, day-to-day, down-to-earth, practice - i.e. used by a diverse audience of thousands of vocal groups doing different things with it, please let me know.
Until you do we have to lobby for some things to be tried in Linux before we know if they really work.
-- Jamie
Quoted 1 line
Don't move reiser4 into vfs, use reiser4 as the vfs.
Ah ... that seems to be a good insight into what you're driving for. That would seem to be good for you ... and bad for other filesystems. So whilst I can see why you'd want it - but I don't think it's a good idea. Competition is good.
Quoted 2 lines · Andrew
And what are the licensing implications of plugins? Are they derived works? Must they be GPL'ed?
You replied:
Quoted 4 lines
Quoted 2 lines
And what are the licensing implications of plugins? Are they derived works?
Yes.
I find it odd that you cut out the last part of his question. So must they be GPL'ed or not?
M.
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
"Martin" == Martin J Bligh <[email protected]> writes:
Quoted 2 lines · Martin> Andrew wrote:
And what are the licensing implications of plugins? Are they derived
Quoted 1 line
works? Must they be GPL'ed?
Martin> [Hans] replied:
Quoted 4 lines
Quoted 4 lines
Quoted 2 lines
And what are the licensing implications of plugins? Are they derived works?
Yes.
Martin> I find it odd that you cut out the last part of his question. So Martin> must they be GPL'ed or not?
If they are derived works of GPL'ed software, then they obviously must be GPL'ed (or else they are illegal). But if the authors obtain an alternate license from Hans/Namesys, then they are bound by the terms of that license. (Of course, this case would most likely apply only to Reiser4 running under a different operating system.)
Signature · 4 lines
Hubert Chan <[email protected]> - http://www.uhoreg.ca/ PGP/GnuPG key: 1024D/124B61FA Fingerprint: 96C5 012F 5F74 A5F7 1FF7 5291 AF29 C719 124B 61FA Key available at wwwkeys.pgp.net. Encrypted e-mail preferred.
vfs2 (was Re: silent semantic changes with reiser4)
In reply to Hans ReiserHi!
Quoted 4 lines
Quoted 3 lines
Yes, but I didn't say "flame Christoph and ignore the issues" ;)
Oh....;-)
...
Quoted 2 lines
enhancements. This metafiles and file-directories stuff is actually fairly trivial stuff.
Now I'm scared.
not as powerful. Don't
Quoted 4 lines
move reiser4 into vfs, use reiser4 as the vfs. Don't write filesystems, write file plugins and disk format plugins and all the other kinds of plugins, and you won't be missing any expressive power that you really want....
So you want to replace vfs. Fine, but you should have told us before.
Also vfs replacement probably should not be hidden in fs/reiserfs4...
Signature · 1 line
64 bytes from 195.113.31.123: icmp_seq=28 ttl=51 time=448769.1 ms
file-as-dir vs. dir? (was Re: silent semantic changes with reiser4)
In reply to Christoph Hellwig(Apologies for beating on a dead horse here...)
I seems like one confusion here is that everyone has a different idea of what the semantics we're talking about are. I see two main ones:
Hans's: A file, a directory, and an attribute are functionally equivalent (except for S_ISxxx and hardlinks). That is, /usr/bin/metas makes sense, and it's not talking about a program called metas. This also means that /foo/metas/metas might exist and needs dealing with.
Linus's (I think): A directory is just a directory (no attributes and no read()able data). A file can contain attributes, where attributes can be "file" attributes or "directory" attributes. That is, a file is also a subtree with posix-like semantics (except for hardlink stuff). So doing "touch /tmp/foo; cat /tmp/foo/metas" fails, rather than doing something that's probably useless. "touch /tmp/foo; touch /tmp/foo/bar; touch /tmp/foo/bar/baz" fails on the last touch because bar is a file attribute and recursive magic is disallowed.
Which one are we talking about?
FWIW, I like the latter version a lot better, as it removes a lot of ambiguity. If I see a path like /tmp/foo/metas/uid, it is either a uid attribute (i.e. writing it has security implications) or it is a standard file (i.e. writing it just writes it). But _I can tell which one_ by fstat()ing /tmp/foo! If it's a directory, than I have either a named stream called uid or a genuine file called uid (and I can tell which by fstat()ing /tmp/foo/metas), but if it's a file then I have the magic uid. And I'm guaranteed that there's no other funny business, because /tmp/foo is a "file with a subtree," which means that it's not an attribute. This way I know exactly what I'm dealing with.
As an added bonus, we could have O_NOMETAS which means that "files" may not be traversed. Then someone who wants to make sure they get a real file can do it. If recursive files-as-dirs were allowed, that might not do quite what the caller expected. If you really wanted a directory with some data attached, but it in dir/.mystuff, because that's probably what you meant, and any existing tool will do exactly the right thing with it.
In the former version, AFAIK, I have a mess. In UNIX, a path is a path.
It's trivial to understand and there's no funny business parsing it.
But now a path has a slippery meaning where the token "metas" could be a plain old token or it could be magic or it could be magic some other way. The security implications and the possbility for something to break horribly are scary.
--Andy