`rmdir .` doesn't work in 2.4

Hello Al,

why rmdir . is been deprecated in 2.4.x? I wrote software that depends on rmdir . to work (it's local software only for myself so I don't care that it may not work on unix) and I'm getting flooded by failing cronjobs since I put 2.4.0 on such machine. rmdir . makes perfect sense, the cwd dentry remains pinned by me until I cd .., when it gets finally deleted from disk. I'd like if we could resurrect such fine feature (adapting userspace is just a few liner but that isn't the point). Comments?

Andrea
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [email protected]
Please read the FAQ at http://www.tux.org/lkml/

I fail to see why this is useful. you can't do anything in the directory afterwards.

bash# mkdir foobar
bash# cd foobar/
bash# ls
bash# rmdir .
bash# touch foooooo
touch: foooooo: Operation not permitted
bash# ls

Whats the point of it?

=====================================================================
Mohammad A. Haque                              http://www.haque.net/
                                               [email protected]

  "Alcohol and calculus don't mix.             Project Lead
   Don't drink and derive." --Unknown          http://wm.themes.org/
                                               [email protected]
=====================================================================

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [email protected]
Please read the FAQ at http://www.tux.org/lkml/

To type less characters. I'll show you the code that is failing:

def binutils():

	print 'Checking out binutils'
	sys.stdout.flush()
	os.chdir(binutils_dir)
	if os.system('%s update -A -d -P' % CVS):
		print 'failed binutils checkout'
		sys.exit(1)

	print 'Compiling and installing binutils'
	sys.stdout.flush()
	os.chdir('..')
	binutils_build = 'binutils-build-%s' % BUILD
	os.mkdir(binutils_build)
	os.chdir(binutils_build)
	if os.system('../binutils/configure --target=x86_64-unknown-linux --prefix=%s && make && make install' % install_dir):
		print 'failed binutils compile'
		sys.exit(1)
	shutil.rmtree(".")

Why should I write:

	os.chdir("..")
	shutil.rmtree(binutils_build)

when I can simply write:

	shutil.rmtree(".")

I know in the above case I really could avoid the os.chdir(binutils_build) and to use cd %s; ... % (binutils_dir, install_dir) and probably
it wouldn't make much difference, but the above looks simpler and I may have other things to run while inside the binutils_dir too so it could make lots of sense to os.chdir there as I just did.

To workaround this misfeature I can simply implement a derivative class of shutil in my robot and to wrap the rmtree method to emulate the 2.2.x behaviour in userspace, but I think the old behaviour was more flexible (it was also showing how much our dcache is powerful) and I still don't see why it's been removed. Maybe it was to remove a branch from a fast path? (if so I don't think it was a good idea, there are many more overhead things that matters more and that aren't even visible to userspace)

Andrea
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [email protected]
Please read the FAQ at http://www.tux.org/lkml/

Racy. Nonportable. Has portable and simple equivalent. Again, don't bother with chdir at all - if you know the name of directory even ../name will work. It's not about the current directory. It's about the invalid last component of the name.

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [email protected]
Please read the FAQ at http://www.tux.org/lkml/

The last component of the name isn't invalid, it's a plain valid directory. If according to you rmdir ../name and rmdir pwd makes sense then according to me rmdir . makes perfect sense too.

Andrea
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [email protected]
Please read the FAQ at http://www.tux.org/lkml/

rmdir() works on _links_, not directories. Just as every other syscall of that sort. It's _not_ "find directory and kill what you've found", it's "find a link and destroy it; preserve the fs consistency".

Code from another posting has a cute little property: it may kill a
directory that has absolutely nothing to your pwd. And that's OK - you really can't get anything better than "kill whatever my pwd was called" without a lot of PITA. And yes, it can happen in 2.2, along with many other fun things.

Andrea, fix your code. Linux-only stuff is OK when there is no
portable way to achieve the same result. In your situation such way indeed exists and is prefectly doable in userland. If you want a cute DoWhatIMean wrapper around rmdir(2) - fine, you've just written one of the variants. Any reasons to keep that in the kernel? None? Thank you, case closed.

"My code stopped working on 2.4" should actually be pronouced
"My code wouldn't work on anything but 2.2 (let's ignore the related races that _are_ in 2.2), so could you please put a fix for my code into the 2.4? No, I don't want to put that wrapper into my code, put it in the kernel". Sorry, not impressed.

If this wrapper is OK for your purposes (for 99% of situations
it indeed is) - fine, use it. If you really need to destroy the directory that happens to be your pwd - sorry, no reliable way to do that without interesting locking. On _any_ UNIX out there. 2.2 included. It will happily give you -ENOENT and refuse to perform the action above in case if some other process renames your pwd. Yes, for rmdir("."); It will also return that if your directory had been already rmdir()'d. Have fun. Other Unices will return -EINVAL or -EBUSY - depends on the flavour.

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [email protected]
Please read the FAQ at http://www.tux.org/lkml/

Well, this bites.

Locking guess: use a global read-write lock, with the "write" case being deletion of "." and the "read" case being everything else. You could have one lock per CPU, with the writer needing to grab all of them in order. So removal of "." pays the cost.

If the standards gripe, well, rmdot() is a nice name.
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [email protected]
Please read the FAQ at http://www.tux.org/lkml/

It's _so_ far from the SMP cache issues that it's not even funny. So reference to brw-locks is completely bogus. What you are proposing is to serialize rmdir() and rename() (including lookups) wrt rmdir and rename. Globally. Fun, fun...

If anything, frmdir() might be a better name. However, it's really inconsistent with the whole namespace-modifying stuff. You don't have flink(fd, newname). frename() and funlink() are not even funny - _which_ link would you want to be renamed/removed?

Filesystem consists of two types of objects - files (and that includes directories, etc.) and links. Pathname can be evaluated to link and to file. Namespace syscalls (creat()/mkdir()/mknod()/symlink()/link()/ unlink()/rmdir()/rename()) operate on links. open(), truncate(), stat(), lstat(), etc. operate on files - completely different can of worms.

2.2 tried (without success) to make rmdir() and some cases of rename() act on files. Notice that if you have /foo as pwd, "." and "/foo" will evaluate to the same file, but to different links. That's what it's really about.

We could add new syscalls. However, I'm yet to see the real-world situation where they would be needed enough to warrant their inclusion. And I mean real-world, not an exercise asking for that functionality. Occam's Razor...

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [email protected]
Please read the FAQ at http://www.tux.org/lkml/

As long as nobody tried to remove ".", nothing is serialized. You can do your lookups in parallel since they can all grab the read lock at once.

If you prefer some pentuple_down thing though, cool.

Linux can tell where you got the "f" from in the first place. Deleted files could cause an error, or turn frename() into flink() and return success for funlink().

The whole "." and ".." mess is 100% special case to begin with, so this argument doesn't make sense. The mess already gives
us a link that gets replaced when some other link is renamed, and two links that get unlinked when something else is unlinked. -
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [email protected]
Please read the FAQ at http://www.tux.org/lkml/

Bzzzert. At which point do you take that lock for rmdir("foo/bar/barf/.")?

For crying out loud, rename() already got a Bastard Semantics From Hell, choke-full of special cases. Write the complete description of f* behaviour and let's see what you'll get. Then post it here, OK?

Side effects. You've got to preserve fs consistency. Operation requested by caller requires to change the links in (re)moved directory. Which is done.

Again, the main argument is Occam's Razor. I agree that in theory f*() give things that are not provided by current syscalls. No arguments here. However, unless you can show the real need to have them in the API they have no business being there. As an aside, for some filesystems they are simply impossible - try to implement frmdir() on NFS, for one thing.

The only possible benefit is in atomicity of these guys. _If_ filesystems can provide it (local ones can do that) _and_ there is a real-world stuff that needs such atomicity warranties - fine, you've got a case for inclusion of that stuff. Otherwise presumption of uselessness[1] applies.

[1] Useless until proven useful, aka. Thou Shalt Not Bloat Thy API.

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [email protected]
Please read the FAQ at http://www.tux.org/lkml/

BTW, "rmdir pwd" is not portable either.

As just said I'm really not concerned about my code.

I think it would provide nicer semantics as 2.2.x does. The performance argument seems irrelevant to me.

Andrea
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [email protected]
Please read the FAQ at http://www.tux.org/lkml/

In article <[email protected]>,

Indeed. Avoid it if you can.

But at least "rmdir pwd" is not _required_ to fail, like rmdir("."/"..").

Linus
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [email protected]
Please read the FAQ at http://www.tux.org/lkml/

"rmdir pwd" is required to fail (at least under csh, bash, ksh) if the path component contains a white space and thereof it can't be a valid replacement for Andreas "rmdir ." which was what Al initially suggested.

Yes, I'm very pickey about that; but hey, I don't want to force anyone to write GNU/Linux like rms; just valid shell code. :)

I hope you now agree that "rmdir pwd" makes not much sense - except in bug reports for bash, csh and ksh scripts or in scripts dedicated to the Win32 environment :)

  ciao - 
    Stefan

" Lession 1: Quoting.                   Lession 2: Power up the machine. "
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [email protected]
Please read the FAQ at http://www.tux.org/lkml/

Of course you should do rmdir "pwd"
But this is a userspace issue.

Eric

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [email protected]
Please read the FAQ at http://www.tux.org/lkml/

In article <[email protected]>,

It makes perfect sense, and Linux used to accept it during the 2.3.x timeframe.

However, it is against all UNIX standards, and Linux-2.4 will explicitly not allow it (there's also some parent locking issues there).

Linus
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [email protected]
Please read the FAQ at http://www.tux.org/lkml/

I may be missing something but apparently SuSv2 allows it, you can check here:

	http://www.opengroup.org/onlinepubs/007908799/xsh/rmdir.html

Infact SuSv2 doesn't even allow rmdir to return -EINVAL.

Andrea
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [email protected]
Please read the FAQ at http://www.tux.org/lkml/

Hi,

SuS always allows implementations to return other errors than the ones listed:

Implementations will not generate a different error number from the ones described here for error conditions described in this specification, but may generate additional errors unless explicitly disallowed for a particular function.

See http://www.opengroup.org/onlinepubs/007908799/xsh/errors.html

--Stephen
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [email protected]
Please read the FAQ at http://www.tux.org/lkml/

It's a hell of a pain wrt locking. You need to lock the parent, but it can cease to be your parent while you were locking it. rmdir() (and especially rename()) are major PITA as is. Adding lock/check/retry everything if we had been moved was way over the top. Again, rename() was the main offender, but rmdir() also wasn't nice.

rmdir() and rename() act on links. Yes, for directory there is a special link - one from the parent and we could try to find it and assume that caller wanted to do the thing on said link. However, it _is_ overloading the operation. Adds quite a bit of complexity into the place where we already have too much. For no good reason, since rmdir(pwd) does the same thing quite fine, is portable (unlike the rmdir(".") which will fail on every UNIX except Linux) and does not require special-casing.

All namespace-modifying operation (create/remove/rename) act on the links, not fs objects. Pathname you are passing identifies the directory entry that should be affected. Conceptually, actual removal of the inode is a side effect.

The bottom line: rmdir(".") is gone. Replace it with portable equivalent, namely

	p = getcwd(pwd, sizeof(pwd));
	if (!p)
		/* handle error - ERANGE or ENOENT */
	rmdir(p);

Shell equivalent is rmdir pwd. Also portable.
Same goes for rename(). Notice that _all_ pathes that end on . or .. are off-limits. Explicit POSIX requirement and in that case I'm 100% agree with them. Linux used to provide an extension that
* added complexity (and was handled with races in 2.2)
* was absolutely nonportable
* was inconsistent with the semantics of other operations (and
in case of rename() - with other parts of rename() semantics)
* had simple portable replacement.

He's dead, Jim.

PS: try your code on any other UNIX and see it misbehave.

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [email protected]
Please read the FAQ at http://www.tux.org/lkml/

This is a no-brainer and bad implementation, but shows it's obviously right wrt locking. (pseudocode, I ignored the uaccess details and all the other not relevant things)

	sys_rmdir(path)
	{
		char buf[PAGE_SIZE]

		err = sys_getcwd(buf, PAGE_SIZE)
		if (err)
			goto out
		
		if (!memcmp(path, ".", 2))
			path = buf
		err = 2_4_0_sys_rmdir(path)

	out:
		return err
	}

Optimizing things just a little more as _worse_ the overhead of supporting rmdir . can be reduced to a memcmp(name, ".") plus and out of line jump and I don't see a performance problem with that.

Could you enlight me on where's the locking pain?

Andrea
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [email protected]
Please read the FAQ at http://www.tux.org/lkml/

Very portable - not.

rmdir "pwd" !!!

  ciao - 
    Stefan

"     ( cd /lib ; ln -s libBrokenLocale-2.2.so libNiedersachsen.so )     "
    
Stefan Traby                Linux/ia32               fax:  +43-3133-6107-9
Mitterlasznitzstr. 13       Linux/alpha            phone:  +43-3133-6107-2
8302 Nestelbach             Linux/sparc       http://www.hello-penguin.com
Austria                                    mailto://[email protected]
Europe                                   mailto://[email protected]
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [email protected]
Please read the FAQ at http://www.tux.org/lkml/

Not very portable at all...

hpux = HP/UX 10.2

hpux:~$ mkdir foo
hpux:~$ cd foo
hpux:~/foo$ rmdir "`pwd`"
rmdir: /home/blc/foo: Cannot remove mountable directory
hpux:~/foo$ rmdir .
rmdir: cannot remove .. or .
hpux:~/foo$ rmdir /home/blc/foo
rmdir: /home/blc/foo: Cannot remove mountable directory
hpux:~/foo$ rmdir ./
rmdir: ./: Cannot remove mountable directory
hpux:~/foo$

Maybe HP/UX is messed up as well.

But anyway I'll try to step out of the building that i'm trying to blow up, regardless of whether or not it will work on a particular OS will hold onto a floor of it till it's done with it...

-bc

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [email protected]
Please read the FAQ at http://www.tux.org/lkml/

Same on FreeBSD, by the way

bash-2.04# uname -a
FreeBSD freebsd.redhat.de 5.0-20001112-CURRENT FreeBSD 5.0-20001112-CURRENT #0: Sun Nov 12 14:04:55 GMT 2000 [email protected]:/usr/src/sys/compile/BERO i386 bash-2.04# mkdir test
bash-2.04# cd test
bash-2.04# rmdir .
rmdir: .: Invalid argument

LLaP
bero

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [email protected]
Please read the FAQ at http://www.tux.org/lkml/

It seems not to return -EBUSY. As also mentioned by Andries:

If the directory is the root directory or the current working directory of any process, it is unspecified whether the function succeeds, or whether it fails and sets errno to [EBUSY].

The portable way is the one I mentioned as possible change for my code:

        os.chdir("..")
        shutil.rmtree(binutils_build)

Andrea
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [email protected]
Please read the FAQ at http://www.tux.org/lkml/

Nor solaris
[wakko@<removed>:/home/wakko/test] rmdir "pwd"
rmdir: /home/wakko/test: Invalid argument [wakko@<removed>:/home/wakko/test] uname -a
SunOS <removed> 5.8 Generic_108528-03 sun4d sparc SUNW,SPARCcenter-2000 [wakko@<removed>:/home/wakko/test]

 Lab tests show that use of micro$oft causes cancer in lab animals
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [email protected]
Please read the FAQ at http://www.tux.org/lkml/

Some other OS with a yet different retval? :)

Andrea
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [email protected]
Please read the FAQ at http://www.tux.org/lkml/

It can be much worse (irix-6.5.4):

bash# mkdir x; cd x; rmdir "pwd"
/x: Can't remove current directory or ..

Here the error message makes sense - but is totally wron in this case :(

And here is linux-2.2.18:

   cerebro:~# mkdir x; cd x;rmdir "`pwd`"
   cerebro:~/x# ls -la
   total 6
   drwxr-x---   0 root     root           35 Jan  9 05:54 .
   drwx------  69 root     root         5372 Jan  9 05:54 ..
   cerebro:~/x# cd
   cerebro:~# ls -la x
   ls: x: No such file or directory

So, no, linux certainly does NOT remove "." ;)

      -----==-                                             |
      ----==-- _                                           |
      ---==---(_)__  __ ____  __       Marc Lehmann      +--
      --==---/ / _ \/ // /\ \/ /       [email protected] |e|
      -=====/_/_//_/\_,_/ /_/\_\       XX11-RIPE         --+
    The choice of a GNU generation                       |
                                                         |
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [email protected]
Please read the FAQ at http://www.tux.org/lkml/

OK, got me on that. Yes, you'll need quoting here. Sorry.
Notice that there are two effects in the game:
* some Unices refuse to rmdir() busy directories. For them
removing the pwd is impossible. Period. You can chdir() away, but there is no promise that after
chdir foo
chdir ..
foo will refer to the directory that used to be your pwd in the middle. That's pretty obvious - consider the effect of

chdir foo
chdir .. mv foo bar
mv baz foo
So unless you have external information about behaviour of other processes the only way to pinpoint a directory is to keep it opened/pwd/root. Each of these will keep it busy and Unices that refuse to rmdir() busy ones will return -EBUSY on that.
On such systems there is _no_ reliable way to remove your current
pwd unless you can guarantee that it won't be renamed away by another process. No matter what you are doing.

* All Unices are required to refuse rmdir() on pathnames that end on
"." or "..". 2.2 is an exception in that respect - usually it allows such operation. However, that is _still_ unreliable. rename() called by another process in the right time will make rmdir(".") return -ENOENT, even though at any moment "." would resolve to the same directory. Including the window when rmdir() would fail. Notice that error value is indistinguishable from the other cases, so blind repeating rmdir(".") while you get -ENOENT is not a solution (as the matter of fact, it can trivially turn into infinite loop).
All examples mentioned in that thread (HP/UX, Solaris, *BSD) _will_
fail with "rmdir .". Some of them will fail with "rmdir <your_pwd>" too - see discussion of -EBUSY above.

The bottom line: without external information about behaviour of
other processes you can't reliably remove the directory that is your pwd now. "chdir away and rmdir by the name it used to have" works around the problem with -EBUSY (on the systems that refuse to remove busy ones) _BUT_ it is still vulnerable to "rename by another process" kind of races.

If you _have_ such external warranties - trivial wrapper will do
the trick on the systems that allow rmdir() of busy directories and the same wrapper combined with chdir away will solve the problem for all systems. There is no reason to put that in the kernel - it will not give you any additional warranties.

We _can_ pinpoint the link and do rmdir() on it reliably. We can't
do the same to inode. In principle kernel could do that, but NONE of the existing Unices (2.2 included) do such things and it would require more trickery than it's worth.

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [email protected]
Please read the FAQ at http://www.tux.org/lkml/

Hi,

Do the above while another process is renaming one of your parents and watch an innocent directory get shot down in flames, or prepare for an incorrect ENOENT.

--Stephen
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [email protected]
Please read the FAQ at http://www.tux.org/lkml/

Hi!

...when you are lucky and all directories up to your path are readable... Which is not always so. With old glibc, it will fail.
Pavel

I'm [email protected]. "In my country we have almost anarchy and I don't care."
Panos Katsaloulis describing me w.r.t. patents at [email protected]
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [email protected]
Please read the FAQ at http://www.tux.org/lkml/