fs: Restore files_lock and set_fs_root exports

Hello, Andrew,

Some export-removal work causes breakage for an out-of-tree filesystem. Could you please apply the attached patch to restore the exports for files_lock and set_fs_root?

Thanx, Paul

----- End forwarded message -----

diff -urpN -X ../dontdiff linux-2.5/fs/file_table.c linux-2.5-MVFS/fs/file_table.c
@@ -26,6 +26,7 @@ EXPORT_SYMBOL(files_stat); /* Needed by 
 
 /* public. Not pretty! */
 spinlock_t __cacheline_aligned_in_smp files_lock = SPIN_LOCK_UNLOCKED;
+EXPORT_SYMBOL(files_lock);
 
 static spinlock_t filp_count_lock = SPIN_LOCK_UNLOCKED;
 

diff -urpN -X ../dontdiff linux-2.5/fs/namespace.c linux-2.5-MVFS/fs/namespace.c
@@ -1207,6 +1207,7 @@ void set_fs_root(struct fs_struct *fs, s
 		mntput(old_rootmnt);
 	}
 }
+EXPORT_SYMBOL(set_fs_root);
 
 /*
  * Replace the fs->{pwdmnt,pwd} with {mnt,dentry}. Put the old values.

----- End forwarded message -----

What out of tree filesystem, and what the heck is it doing?

Without proper explanation it's vetoed.

btw, any reason you put half the world in the Cc list? Al and Andrew I see, but do the other people on the Cc list have to do with it? And you forgot the person that killed the export.

MVFS, as was correctly guessed from my diff. It is providing a view into a source-code control system, so that a given process can specify the version it wishes to see. Yes, different processes then see a different filesystem tree at the same mount point.

What additional explanation are you looking for?

I figured that there was no way that you would miss it, so there was no point in adding to an already overly long CC line. ;-)

Thanx, Paul

We have that in the VFS as namespace and it has no business in a filesystem driver. And we have been telling this for more than a year.

The explanation is so good that we can veto that patch with a reason, as it should really be obvious to you an anyone involved.

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Well, autofsng patches (new set forthcoming) use set_fs_root/set_fs_pwd to pivot a call_usermodehelper process into the triggering process's namespace.

It has no need however for files_lock.

- --
Mike Waychison
Sun Microsystems, Inc.
1 (650) 352-5299 voice
1 (416) 202-8336 voice

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
NOTICE: The opinions expressed in this email are held by me, and may not represent the views of Sun Microsystems, Inc.
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.2.5 (GNU/Linux)
Comment: Using GnuPG with Thunderbird - http://enigmail.mozdev.org

iD8DBQFB3aEPdQs4kOxk3/MRAlvmAJ4sXcg0Cx8+00WrNEkXW4T7Ji3tKwCfVYPO 43IFyNGeDo85sqDJCprLR8I=
=Vbec
-----END PGP SIGNATURE-----

Once we get anywhere where this is needed we'll find a better interface for that. Like call_usermodehelper_in_namespace() or something even better.

Ah, a function like that would be nice to help with some issues that I imagine the virtual-linux (or whatever that patch is called) people will be having with udev and hotplug. But as they haven't complained yet, I haven't been very moved to look into it...

thanks,

greg k-h

What, in name of everything unholy, is *filesystem* doing with set_fs_root()?

It is using it to set the process's view of the source-code control system to the desired version. So process A can see version 2.1 while process B sees version 1.5, and so that either process can change the version that it sees on the fly.

Thanx, Paul

Yep, you win the prize, it is MVFS.

This is the usual port of an existing body of code to the Linux kernel. It is not asking for a new export, only restoration of a previously existing export.

Thanx, Paul

Sorry, but "our code is badly misdesigned" does not make a valid excuse when you have been told, repeatedly, by many people, for at least a year that you needed to sanitize your design.

Request denied. And no, it never had been intended to be exported or, indeed, used by filesystems.

The obvious searches did not find this for me. Any pointers so that I can bring to the MVFS guys' attention any alternatives that might have been recommended?

Thanx, Paul

"Use recursive bindings instead of trying to take over the entire mount tree and mirroring it within your fs code. And do that explicitly from userland".

Guys, the technical arguments are all of course correct. But the fact remains that we broke things without any notice. (Yes, perhaps someone did say something at sometime, but the affected party didn't know, and it's our job to make sure that they knew). (These exports were removed in October - the IBM guys need to work out why it took so long to detect the breakage).

I think the exports should be restored. So does Linus ("Not that I like it all that much, but I don't think we should break existing modules unless we have a very specific reason to break just those modules.").

Which begs the question "how do we ever get rid of these things when we have no projected date for Linux-2.8"?

I'd propose:

a) Create Documentation/feature-removal-schedule.txt which describes things which are going away, when, why, who is involved, etc.

b) Mark things deprecated where appropriate.

c) For module exports it would be nice if we could get the module loader to print a message "module foo.o uses deprecated symbol files_lock". Dunno how hard that would be.

Alternatively perhaps we could arrange for the symbol to be marked deprecated if it's used from a module:

   #ifdef MODULE
   #define deprecated_if_module __deprecated
   #else
   #define deprecated_if_module
   #endif

Just something to really draw people's attention to the fact that their code will break at a well-defined date.

No. I specificly asked the question how they're using it and they're use is

a) completely buggy
b) poking so deep in the kernel that the user falls under the GPL derived works clause. As a copyright holder of quite a bit of fs/*.c I certainly wouldn't give IBM a special exception to use it even if it was exported.

These exports were only added for intermezzo during 2.4.x and with the removal of intermezzo they go. They never were a public API, and that they were needed at all was a managment mistake in how that code was merged.

We've been removing stuff that has

(1) no intree users
(2) doesn't make sense as API

there's simply no guarantee with out of tree users. I've gotten quite a few mails for the recent removals, and I could tell everyone a better way to solve it. Just as we did to IBM for this problem (very highlevel as they don't show sources), but they insist on their completely broken way of doing it.

Fine. Completely agree. Sometimes people do need to be forced to make such changes - I don't think anyone would disagree with that.

What's under discussion here is "how to do it". Do we just remove things when we notice them, or do we give (say) 12 months notice?

Remove when we notice with a short (measured in weeks) period where that removal happens only in -mm. It's a price people have to pay for not submitting their code upstream.

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Not everyone has cycles to follow to -mm.

I'd much rather see deprecation warnings in mainline releases for at least one if not two releases.

- --
Mike Waychison
Sun Microsystems, Inc.
1 (650) 352-5299 voice
1 (416) 202-8336 voice

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
NOTICE: The opinions expressed in this email are held by me, and may not represent the views of Sun Microsystems, Inc.
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.2.5 (GNU/Linux)
Comment: Using GnuPG with Thunderbird - http://enigmail.mozdev.org

iD8DBQFB3gJTdQs4kOxk3/MRAq9uAKCRHF7aF/vviLfIQl3fvv4eZSYpCACgh82/ 5aTd4a6BqeGISYPZDDUvhSg=
=mhET
-----END PGP SIGNATURE-----

let me chime in as the author/maintainer of Tux, which is an out-of-tree patch that relies on VFS internals. VFS changes constantly break Tux in one way or another, but i've not complained once because:

_I simply have no right to complain_

either i submit the code and then i can expect it to be taken into account, or i dont. It's _the_ basic quid pro quo that keeps technology moving towards mainline Linux. So if i have to fix up Tux (both for exports and for other internal details), then that's the price. Often i have to change Tux to do things differently - and in most cases it's Tux that was doing things incorrectly. And note that Tux is not a binary-only module, it's a fully GPL patch.

does this cause more work for me? Sure. Would i prefer to have less work? Yes, but not in such a shortsighted way. Tux staying external is a choice i made and i also care about Linux and i very much like the way the VFS is evolving.

so my strong position is that even asking for any 'warning period' for changes in VFS internals (including exports/unexports) would be extremely rude. It would be rude not only towards the authors and maintainers of mainline VFS code, but also towards other external trees/drivers who do _not_ ask for any special status and accept the
deal: "follow internals, notice kernel people if they do bad stuff (extremely rare in my case) and fix/redesign stuff if the external tree is broken (much more common)".

Ingo

<sarcasm>
<osdl-salespitch>
Unfortunately you don't have the financial and political powers IBM has, so your opinion doesn't matter as much. Maybe you should become OSDL member to influence the direction of Linux development. </osdl-salespitch>
</sarcasm>

If a filesystem is binary-only, then it should get the same treatment as other binary-only things like graphic drivers and machine-virtualization drivers: either stay on a designated kernel version or else fix your driver so it work with mainline.

Greetz, Antonio Vargas aka winden of network

http://wind.codepixel.com/

Las cosas no son lo que parecen, excepto cuando parecen lo que si son.

No, I'd say that unexports are different. They can clearly break existing code and so should only be undertaken with caution, and with lengthy notice if possible.

And it _is_ possible here, because there are no plans to change the exported functions, and it's only two lines of code, dammit.

The cost to us of maintaining those two lines of code for a year is basically zero.

The cost to others of us removing those two lines of code without warning is appreciable.

Obvious solution: don't remove the two lines of code without warning.

The only reason I can see for peremptorily removing those two lines of code is that there is some benefit to doing so which outweighs the downstream cost of doing so. Nobody has demonstrated such a benefit.

Christoph, it would be better to constraint yourself to commenting on other people's actions rather than entering into premature speculation regarding their motivations, especially when that speculation involves accusations of corruption.

I have outlined the reasons for delaying the removal of these exports. If you can demonstrate that those reasons are invalid, and that I had good reason for also believing that those reasons are invalid then that would be an appropriate stage at which to start to speculate about ulterior motivations, thanks very much.

As mentioned before we only unexported symbols were we were pretty clear that all users of it are indeep utterly broken. I got about a dozend replies for this patches, and for more than half of it where the reporter was either the author or the module was opensource I could help them to actually fix their code. In this case the code is far more broken than the others, but we've even been trying to help them fix their code for more than a year, but IBM folks have been constanly refusing.

But as long as IBM people don't fix their %$^%! they'll continue annoying us and the Distibutors about adding more and more hacks for it, and other people will write similarly crappy code using it again and making the removal even more difficult.

Hey, you got messages in this thread from half a dozen kernel contributors here and still stand against all of them and for IBM who's violating our copyrights that don't allow non-propritary derived works from it which any user of these deeply-internal modules clearly is.

I don't know nor care nor want to speculate over anyones motives, but the situation is at least strange enough to deserve a little sarcastic anekdote.

They didn't fix their code because it worked - no reason to do so.

Telling people "this is going away in 12 months" gives them reason to fix the code. And a reasonable amount of time to do so and to distribute the new version.

Maybe, maybe not. But is it appropriate for us to be trying to control someone else's internal activities via alterations to the kernel?

I doubt if people would be silly enough to use a deprecated export in new code after the export has been scheduled for removal. If they do then yes, sorry, we have to draw the line somewhere.

You still have not demonstrated any benefit to any party from not delaying the removal of these two exports.

Trivial one: it saves memory 8)

I did a bit research using grep, sort, uniq and wc:

between 2.6.9 and 2.6.10:
414 EXPORT_SYMBOL's were removed

since 2.6.10:
90 EXPORT_SYMBOL's were removed in Linus' tree

in 2.6.10-mm2 excluding linus.patch:
71 EXPORT_SYMBOL's are removed

Notes:
- EXPORT_SYMBOL_GPL's weren't counted in any way
- if an EXPORT_SYMBOL was moved, it wasn't counted (using uniq)
- small mistakes in the numbers might be possible since my method to measure them was't at a scientific level, but after a quick look it seems the numbers are roughly correct

Resurrecting and documenting all of these recently removed EXPORT_SYMBOL's because some company might have found some way to (ab)use one or more of them costs us:
- some extra work
- wasts space for all users of Linux (e.g. some of the recent removals are "remove EXPORT_SYMBOL'ed but completely unused function" patches I sent)

cu
Adrian

       "Is there not promise of rain?" Ling Tan asked suddenly out
        of the darkness. There had been need of rain for many days.
       "Only a promise," Lao Er said.
                                       Pearl S. Buck - Dragon Seed

Note that this alone is *NOT* enough for removal - there are groups of functions that basically are libraries and that are supposed to be exported, whether we have all of them used in the tree at given time or not.

That was done

People have been trying to get this stuff fixed for a long time. There are no sane users of it, there are no apparent legal users of it and the one remaining problem has shown no sign of wishing to change and will no doubt try the same again in twelve months.

Obviously not very well, because it didn't work. Was a removal date given?

I doubt if that is correct. Paul?

If MVFS tries the same again in twelve months, they will be going through some representative other than myself. In fact, if MVFS does not have a credible plan for accommodating a twelve-months-from-now loss of these exports within 30 days, their management will be hearing from me.

Thanx, Paul

Why should we keep junk around for 12 months that nobody has a legal reason to be using ? We broke every out of serial tty driver in 2.6.9 and in 2.6.10 for example and we will break them all again in ways we can't keep the old stuff around. (and Linus broke them all in 2.6.10 8))

There is a difference between a public API like inter_module_get() and an internal deep reference to something that is fairly private. Zapping those kind of functions is very different and I agree we should deprecate them properly.

A lot of this is coming up because old exports for private use were never really properly marked as _GPL or in other ways so we have a legacy of assumptions to tidy.

Alan

To give people a reasonable amount of time to stop using these things, of course.

The symbols were exported to non-gpl modules. People used them. Maybe they shouldn't have. Maybe they were asked not to do so, and maybe or maybe not they noticed. Certainly we shouldn't have exported these things in the first place.

We should find a way of repairing things while minimising the amount of screwing people around.

just because it was exported it doesn't mean it ever was legal.

Ok, I'll bite, here's a patch that does just that. Look good?

thanks,

greg k-h

-----------

Add Documentation/feature-removal-schedule.txt as a way to notify everyone when and what is going to be removed.

diff -Nru a/Documentation/feature-removal-schedule.txt b/Documentation/feature-removal-schedule.txt
@@ -0,0 +1,17 @@
+The following is a list of files and features that are going to be
+removed in the kernel source tree.  Every entry should contain what
+exactly is going away, why it is happening, and who is going to be doing
+the work.  When the feature is removed from the kernel, it should also
+be removed from this file.
+
+---------------------------
+
+What:	devfs
+When:	July 2005
+Files:	fs/devfs/*, include/linux/devfs_fs*.h and assorted devfs
+	function calls throughout the kernel tree
+Why:	It has been unmaintained for a number of years, has unfixable
+	races, contains a naming policy within the kernel that is
+	against the LSB, and can be replaced by using udev.
+Who:	Greg Kroah-Hartman <[email protected]>
+

another item:

What: unused exports that don't make sense as general APIs
When: as soon as noticed
Files: all
Why: Unused functions bloat the kernel and wrong exported functions
will make external driver authors write code that's buggy and unmaintainable.
Who: Christoph Hellwig <[email protected]> & others.

Heh, yeah right, might as well just add the following also if we are in the CYA mode:

What: Broken function parameters and entire functions
When: As soon as noticed
Files: All
Why: See Documentation/stable_api_nonesense.txt for why this is
necessary.
Who: Greg Kroah-Hartman <[email protected]> and others.

But I think this file is for specific things, the above, and your entry, shouldn't be needed. Well, I would hope so, but odds are it might be necessary just to keep from answering the same questions all the time...

thanks,

greg k-h

Brodo, can you add a little more info to this, please?

---
Add 2.4.x cpufreq /proc and sysctl interface removal to the feature-removal-schedule.

diffstat:=

 Documentation/feature-removal-schedule.txt |    9 +++++++++
 1 files changed, 9 insertions(+)

---

diff -Naurp ./Documentation/feature-removal-schedule.txt~cpufreq ./Documentation/feature-removal-schedule.txt
@@ -15,3 +15,12 @@ Why:	It has been unmaintained for a numb
 	against the LSB, and can be replaced by using udev.
 Who:	Greg Kroah-Hartman <[email protected]>
 
+---------------------------
+
+What:	/proc/sys/cpu and the sysctl interface to cpufreq (2.4.x interfaces)
+When:	January 2005
+Files:	drivers/cpufreq/: cpufreq_userspace.c, proc_intf.c
+	function calls throughout the kernel tree
+Why:	Deprecated, has been replaced/superseded by (what?)....
+Who:	Dominik Brodowski <[email protected]>
+

I think for something like this to be really useful, you should not just say "can be replaced with Xxxx", but have some docs (or pointers to such) for both users and developers (depending on who is affected) on _how_ to replace it, or fix it.

Also, I'm not convinced about the single-file format. If we want to do this (I don't know how much it buys, but hey, I certainly also don't have any objections), I think it would be much nicer to have a separate "deprecated" subdirectory, with one file per issue.

(Not that I think it necessarily needs to be just about deprecated or removed features - again, if we do this, I don't see why it shouldn't contain the same information about semantic changes, so that when the locking for an interface changes, you could have a

Documentation/changes/vfs-ioctl-locking.txt

that tells what the new rules are).

Would most of the files be small? Sure hope so. But so what?

Linus

Ok, that's fine with me, makes it easier for patches. I'll take my writeup and Randy's and split them out.

Hm, but things "change" all the time. "new" rules become "old" rules over time too. What should probably happen is the proper rules are documented and kept up to date, like they are today in Documentation/filesystems/Locking and Documentation/pci.txt as two examples. Showing what has changed over time in those two files is what diffs are for :)

Otherwise, the kernel changelogs are good places to dig for changes in apis, but do we really want to duplicate this in other places too?

thanks,

greg k-h

Even though these 2.4. interfaces are already gone in Dave Jones' cpufreq bitkeeper tree, here's a patch which properly announces it in Documentation/feature-removal-schedule.txt:

Add meaningful content concerning the removal of deprecated interfaces to the cpufreq core.

---

 Documentation/feature-removal-schedule.txt |   14 +++++++++++---
 1 files changed, 11 insertions(+), 3 deletions(-)

Index: 2.6.10/Documentation/feature-removal-schedule.txt
===================================================================
@@ -17,10 +17,18 @@
 
 ---------------------------
 
-What:	/proc/sys/cpu and the sysctl interface to cpufreq (2.4.x interfaces)
+What:	/proc/sys/cpu/*, sysctl and /proc/cpufreq interfaces to cpufreq (2.4.x interfaces)
 When:	January 2005
 Files:	drivers/cpufreq/: cpufreq_userspace.c, proc_intf.c
-	function calls throughout the kernel tree
-Why:	Deprecated, has been replaced/superseded by (what?)....
+Why:	/proc/sys/cpu/* has been deprecated since inclusion of cpufreq into
+	the main kernel tree. It bloats /proc/ unnecessarily and doesn't work
+	well with the "governor"-based design of cpufreq.
+	/proc/cpufreq/* has also been deprecated for a long time and was only
+	meant for usage during 2.5. until the new sysfs-based interface became
+	ready. It has an inconsistent interface which doesn't work well with
+	userspace setting the frequency. The output from /proc/cpufreq/* can
+	be emulated using "cpufreq-info --proc" (cpufrequtils).
+	Both interfaces are superseded by the cpufreq interface in
+	/sys/devices/system/cpu/cpu%n/cpufreq/.
 Who:	Dominik Brodowski <[email protected]>
 

Applied, thanks.

greg k-h

As it is already removed in Davej's cpufreq bitkeeper tree, I don't know if it's still desired or not, but:

Applied, thanks.

greg k-h

IBM might want to also note that anyone wanting to do so needs an IBM patent license for non GPL use ..

given this, I actually think it might be better to make it a _GPL export right away, anything else is setting people up in an entrapment kind of way.

There could be no entrapment unless IBM chose to go after people who used RCU from a binary module.

Thanx, Paul

Last time I checked with IBM's lawyers, they were not interested in doing so. Nonetheless, you are correct, non-GPL use of RCU would require a conversation with IBM. And in fact the five relevant patents (one lapsed) are called out in Documentation/RCU/RTFP.txt. The four non-lapsed patents are called out as "contributed under GPL", so I think that we are covered.

Thanx, Paul

What happens when the feature is just not (ab)usable in the way proposed ?

At that point there is a specific _reason_ to break it, aka "that function simply doesn't exist any more".

I'm known for happily breaking binary modules, but I think we should do it only if we have a reason _other_ than "let's break a module".

Linus

Note that the function that started this thread weren't removed with nwfs in mind - in fact I doubt anyone here has seen the module or it's requirements.

Arjan and I have produced lists of unused exported symbols and went through the list what made sense and what not, and these exports were up high on the doesn't make sense and any use we could imagine would be harmfull list.

The explained use only seconds that opinion - it has caused constant trouble for years where IBM has been pushing the big distros to relax VFS sanity checks and add additioal exports for their totally broken kernel driver.

That no beeing enough I know of at least two propritary Unix variants where it cause exactly the same trouble.

how about "it can be static, and static functions without external references can be better optimized by gcc (and are, as of version 3.4)" (where function pointers are seen as external references, and EXPORT_SYMBOL does just that)

Indeed!

We are getting lists of symbols used by the various products, as well as working to get them tested more frequently against kernel.org trees instead of only against distro releases. IBM being what it is, -finding- all such products is a challenge, but so it goes.

Thanx, Paul

Thank you for the pointer! By this, you mean do mount operations in conjunction with namespaces, right?

I will follow up with more detail as I learn more. The current issue seems to be with removeable devices. Their users want to be accessing a particular version, but still see a memory stick that was subsequently mounted outside of the view. Straightforward use of mounts and namespaces would prevent the memory stick from being visible to users that were already in view.

Thanx, Paul

Namespaces or chroots, not much difference... The point is that you want to duplicate the mount tree with your stuff added at some point. Instead of doing duplication on directory tree level (i.e. having your fs code try and mirror the stuff from other filesystems), you can do that completely outside of fs code; either by cloning the namespace first or by building a chroot jail with mount --rbind / <location of jail> and chrooting in there. Then just mount the stuff that really comes from your data at whatever place you want.

There is a way to deal with that and since 2.7 is not going to materialize, we'd better go and resurrect that project in 2.6... Basically, having shared and asymmetrically shared subtrees between namespaces/locations in the same namespace.

I'll try to get the detailed description of that stuff (partial sharing) written down in a couple of days and post an RFC on l-k and fsdevel.

It's doable, it's not particulary scary, but it sure as hell *far* easier to do in generic code; there we have access to vfsmount trees and there lives all code that modifies them, so we don't have to screw with mirroring directory trees of other filesystems.

Note that even now you can simply go ahead and mount that stick inside
of view explicitly. That will work; the question is how to make it automatic and do that in a sane way.

OK, after much thrashing, here is what the ClearCase users need. Sorry for the length -- the first few paragraphs gives the flavor of it and the rest goes into more detail with examples.

Thoughts?

Thanx, Paul

------------------------------------------------------------------------

ClearCase provides a filesystem view of revision-control repositories. ClearCase users can specify the desired revision directly in the pathname ("view extended naming", for example "/views/v1.1/vob/foo/bar.c") or they can associate a particular revision with a process and its descendants ("setview context naming", for example "/vob/foo/bar.c"). Specifying the revision in the pathname is useful for diffing and such, and associating a revision with a process is useful for builds and testing using standard tools, sort of like chroot jails.

ClearCase users need the root filesystem to be overlaid on each view, so that "/etc/passwd" and "/views/v1.1/etc/passwd" reference the same file.

The current hope is that adding (a) shared and asymmetrically shared subtrees between namespaces/locations in the same namespace, (b) stackable LSM modules, and (c) dynamic recursive union mount would enable Linux to provide this in a technically sound manner. [But this is not clear to me yet.]

More details on what ClearCase users want to see follow.

1. "View extended naming", where the revision is specified by
the pathname.

a. Users explicitly specify that they want to see a specific
revision (e.g., v1.1) using the ClearCase "startview" command. This revision is then visible to all processes.

b. The pathname fully specifies the revision and file, e.g.,
"/views/v1.1/vob/foo/bar.c", where the "startview" command has initialized an MVFS view of the v1.1 version on "/views/v1.1/vob/foo". ("vob" stands for "versioned object base".)

c. Note that the entire filesystem is also visible within
the view, so that /etc/passwd" is an synonym for "/views/v1.1/etc/passwd". This can be thought of as (currently mythical) "dynamic union mount rbind". See #4 below for more detail.

2. "Setview context naming", where revision is -not- specified
by the pathname.

a. A particular process designates a particular revision
(e.g., "v1.1") using a ClearCase "setview" command. This designation applies to both the process and its future descendants (any descendants already created are unaffected. The "setview" command does an implicit "startview" if needed.

b. That process and any of its descendants will then see the
specified revision via "/vob/foo/bar.c", where an MVFS filesystem has been mounted on "/vob/foo".

c. Changes made via the "setview context naming" paths
are visible in "view extended naming" paths, with the same restrictions as called out in #1c above.

d. A given process can mix and match "view extended naming"
and "setview context naming" references, so that both "/views/v1.1/vob/foo/bar.c" and "/vob/foo/bar.c" may be used to refer to the same file, but in that case "/views/v1.2/vob/foo/bar.c" might reference a different file. These synonyms for the same file need to have the properties of the mythical "dynamic union mount rbind" described in #4 below.

3. ClearCase users are -not- permitted to mount over a file or
directory within an MVFS filesystem. Therefore, if a user mounts a memory stick over (say) "/vob/foo/mnt", the results are undefined.

4. ClearCase users need to be able to access multiple repositories
simultaneously (e.g., for related projects). So there might be a "/vob/foo" and "/vob/oof" project visible simultaneously.

There are some specialized but important testing situations that require separate top-level directories, e.g., "/vob" and "/tmp/vobtest".

5. The ClearCase users need the mythical "dynamic union mount rbind"
to set things up so that any access to a non-MVFS file made from within a view gives the same results as a direct access to that file would give if no MVFS filesystems were mounted. For example, any access to "/views/v1.1/etc/passwd" must give exactly the same results as the same access to "/etc/passwd" would if there were no MVFS filesystems mounted.

The "dynamic union mount rbind" therefore needs to have the following properties:

a. The underlying inodes retain the same link count
no matter how many ClearCase views they are accessible from. So, if the link count of /etc/passwd is initially 1, it remains 1 even though it is accessible via "/etc/passwd", "/views/v1.1/etc/passwd", "/views/v1.2/etc/passwd", and so on. Similarly, if the link count of foo/bar.c in the v1.1 revision is initially 1, it remains 1 even when it is accessible both via "/vob/foo/bar.c" and "/views/v1.1/vob/foo/bar.c".

Hard-link tricks break user scripts and programs.

b. The types of the underlying files remain the same
regardless of how many views they are visible in and regardless of how they are accessed. So, if /etc/passwd is a normal file, it will appear to be a normal file when accessed via "/etc/passwd", "/views/v1.1/etc/passwd", "/views/v1.2/etc/passwd", and so on. Similarly, if foo/bar.c is a normal file, is will appear to be a normal file when accessed via "/vob/foo/bar.c" and "/views/v1.1/vob/foo/bar.c".

Symbolic-link tricks break user scripts and Makefiles.

c. The types of the underlying filesystem remain the
same regardless of which view is used. This is required by user scripts that use things like "df -k".

d. Any changes to any non-MVFS filesystem are immediately
visible in the view. Some examples:

i. A mount in the root filesystem, e.g.,

mount /dev/cdrom /mnt/cdrom

must result in the CDROM being visible in both "/mnt/cdrom" and in "/views/v1.1/mnt/cdrom". Ditto for NFS mounts and automounting.

In this case "immediately" means "before the mount command's process exits". This need for immediacy applies both to explicit mounts and to autofs-induced mounts. Ditto for unmounts.

ii. Dynamic filesystems such as /proc must
exhibit their dynamic nature whether accessed via "/proc", "/views/v1.1/proc", or "/views/v1.2/proc".

Again, changes must be immediately visible in all views, where "immediately" means that there should be no "time warps". For example, any monotonic counters must be seen as monotonic, despite successive reads happening from different views.

e. Any changes made via "view extended naming" or via
"setview context naming" are visible in the other view. For example, the file created by

touch /vob/foo/bar.c.new

would also be immediately visible as "/views/v1.1/vob/foo/bar.c.new" and vice versa.

f. Symbolic links from inside an MVFS filesystem act as
expected, with the expected advice that views use relative rather than absolute symlinks for targets within the view.

However, absolute symbolic links from inside an MVFS filesystem to files that are outside an MVFS filesystem work as expected.

Interesting; so clearcase mvfs currently extends the linux VFS to do all this.... It would be very interesting to get these extensions to linux posted on this mailing list in patch form, since it could well be a nice generic enhancement of linux. (Assuming Al doesn't go WTF over the code of course :-)

Just given the interfaces it makes use of, I would guess that Al would more than go WTF over the current code. Most would probably say that the current code does not extend Linux VFS, but rather subverts it. Plus it was written by people without any history of Linux-kernel development, so one would anticipate much heartburn over style and usage issues. ;-)

All that aside, if you really really want to see it, I will see what I can do about getting it to you.

Thanx, Paul

I don't see how (b) has anything to do with this. Anyone care to explain that?

thanks,

greg k-h

It would allow tracking the processes that are using a given view, so that state associated with that view could be cleaned up when the last process exits. One case that motivates this approach:

1. one process creates a view (e.g,. "setview" so that

	"/vob/foo/bar.c" references version 1.2, just as
	"/views/v1.2/vob/foo/bar.c" would),

2. this process forks off several descendants, then exits, and

3. the descendant processes eventually exit.

The underlying filesystem could use stackable LSM modules to track fork()s and exit()s, allowing it to work out when all processes using a given view had terminated.

It is quite possible that there is a better way to do this. Thoughts?

Thanx, Paul

But why the "stackable" requirement? Why not just use the LSM interface and be done with it?

Or do you want to combine this LSM with SELinux or any other LSM (root-plug, seclvl, etc.)? If so, that's asking for trouble (see the stackable LSM problems that have been posted multiple times here, and on the linux-security-modules mailing list.)

thanks,

greg k-h

I believe that combining with SELinux is indeed one of the motivations. And, yes, stackable LSM modules are a bit controversial.

But any way of tracking processes retaining a reference to a view would meet the need.

Thanx, Paul

eh maybe a weird question, but why are you and not the MVFS guys asking for this export then? They can probably better explain why they need it ....

Because an person known to the communicity can justify IBM's illegal and copyright-infringe than someone totally unkown? Paul did the same already for GPFS and the unmapping_range stuff, and Greg told me has was asked similar things repeatedly but refused.

It really seems like IBM wants to abuse the positition the it's employees archived contribution to opesource projects.

As near as I can tell, they believe that they already did the best they could to explain their needs and failed to do so.

Thanx, Paul

can you try to find even ONE request for these exports from them in the mailing list archives? Since I find that hard to believe.

I did not find much when I searched yesterday, which is why I had to ask Al what advice he had given them.

Thanx, Paul

as far as I know that was mostly unrelated to the MVFS people asking for these exports, which you seem to think they did. I think differently now ;)