rm-ing files with open file descriptors

test it[1]. Sure enough, you can rm a file that has opened file descriptors and no errors are generated. Interestingly, sun solaris does the same thing. Since this is the case, I thought this might be a feature instead of a bug (ms-win doesn't allow the rm). So, my question is where is this behavior defined? Is it a kernel issue?

I had a weird situation with my application where the user deleted all the database files while the app was still reading and writing to the opened file descriptor. What was weird to me was that the app didn't complain. It just went merrily about it's business as if nothing were wrong. Of course, after the app shut down all it's data was lost.

Since I didn't expect this behavior I wrote a simple little program to test it[1]. Sure enough, you can rm a file that has opened file descriptors and no errors are generated. Interestingly, sun solaris does the same thing. Since this is the case, I thought this might be a feature instead of a bug (ms-win doesn't allow the rm). So, my question is where is this behavior defined? Is it a kernel issue? Does POSIX define this behavior? Is it a libc issue?

I tried to google this, but couldn't think of the right terms to describe it. As I'm not on lkm, I would appreciate a CC: to <[email protected]>.

I had a weird situation with my application where the user deleted all the database files while the app was still reading and writing to the opened file descriptor. What was weird to me was that the app didn't complain. It just went merrily about it's business as if nothing were wrong. Of course, after the app shut down all it's data was lost.

Since I didn't expect this behavior I wrote a simple little program to test it[1]. Sure enough, you can rm a file that has opened file descriptors and no errors are generated. Interestingly, sun solaris does the same thing. Since this is the case, I thought this might be a feature instead of a bug (ms-win doesn't allow the rm). So, my question is where is this behavior defined? Is it a kernel issue? Does POSIX define this behavior? Is it a libc issue?

I tried to google this, but couldn't think of the right terms to describe it. As I'm not on lkm, I would appreciate a CC: to <[email protected]>.

This is a characteristic of a VFS (Virtual File System) on any Unix system. The file doesn't go away until it is closed by everybody that accesses it. However, you can remove or rename it even when it's open for write by other tasks. If a task has an open file-descriptor and keeps it open, it could 'fix' a possibly deleted file, by opening it again with

        new_fd = open("filename", O_CREAT|O_RDWR, 0644);

(without O_TRUNC) and it will remain in existance after all file descriptors are closed, because it was "created" again after it was deleted by another task.

There is no way to recreate a file with a nlink count of 0, well that is until someone adds flink(fd, newpath) to the kernel.

Well no. new_fd will refer to a completely new, empty file which has no relation to the old file at all.

There is no way to recreate a file with a nlink count of 0, well that is until someone adds flink(fd, newpath) to the kernel.

Followup to:  <[email protected]>
By author:    Miquel van Smoorenburg <[email protected]>
In newsgroup: linux.dev.kernel

Quoted 4 lines

Well no. new_fd will refer to a completely new, empty file which has no relation to the old file at all.

Quoted 3 lines

There is no way to recreate a file with a nlink count of 0, well that is until someone adds flink(fd, newpath) to the kernel.

This *might* work:

link("/proc/self/fd/40", newpath);

I had a weird situation with my application where the user deleted all the database files while the app was still reading and writing to the opened file descriptor. What was weird to me was that the app didn't complain. It just went merrily about it's business as if nothing were wrong. Of course, after the app shut down all it's data was lost.

[email protected] (christophe barb?) wrote on 19.01.02 in <[email protected]>:

Quoted 11 lines · Andreas Bombe

Quoted 6 lines

Whether that was an intended or accidental feature only someone with more insight into Unix history can answer. It's that feature that lets us do live upgrades of distributions without rebooting (executables and libraries can be replaced without affecting the currently running processes), at the very least much easier than it would be without this behaviour.

I remember that previous debian release come with a patched kernel to allow live upgrade. It was explained in the FAQ that the patch was required for this purpose.

Complete and utter bullshit. This was never true, and the FAQ never claimed this.

Quoted 2 lines

7.2 Debian claims to be able to update a running program;
how is this accomplished?

... under which was originally explained how running demons would be restarted, and later it was also mentioned that replacing in-use files is possible under Unix. Nothing more. (Google groups will happily find those versions, they were in use from 1996 to 2001 according to the archive.)

Quoted 1 line

What was in this patch?

The patch only exists in your fantasy.

MfG Kai
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/

Followup to:  <[email protected]>
By author:    Miquel van Smoorenburg <[email protected]>
In newsgroup: linux.dev.kernel

Quoted 3 lines

There is no way to recreate a file with a nlink count of 0, well that is until someone adds flink(fd, newpath) to the kernel.

This *might* work:

link("/proc/self/fd/40", newpath);

This could be hacked around ofcourse in fs/namei.c, so I tried it for fun. And indeed, with a minor correction it works:

% perl flink.pl
Success.

I now have a flink-test2.txt file. That is pretty cool ;)

Quoted 8 lines

This could be hacked around ofcourse in fs/namei.c, so I tried it for fun. And indeed, with a minor correction it works:

% perl flink.pl
Success.

I now have a flink-test2.txt file. That is pretty cool ;)

It's also a security hole.

Quoted 7 lines · Miquel van Smoorenburg

This could be hacked around ofcourse in fs/namei.c, so I tried it for fun. And indeed, with a minor correction it works:

% perl flink.pl
Success.

I now have a flink-test2.txt file. That is pretty cool ;)

It's also a security hole.

It results in:

link("/proc/self/fd/3", "flink-test2.txt") = -1 EXDEV (Invalid cross-device lin
k)

This is probably because link() doesn't look up the target of the symlink, it links the symlink itself. Linux allows symlinks with a nlink count of 2:

% ln -s a b
% ln b c
ln: `b': warning: making a hard link to a symbolic link is not portable % ls -l

lrwxrwxrwx    2 miquels  staff           1 Jan 19 11:34 b -> a
lrwxrwxrwx    2 miquels  staff           1 Jan 19 11:34 c -> a

This could be hacked around ofcourse in fs/namei.c, so I tried it for fun. And indeed, with a minor correction it works:

% perl flink.pl
Success.

I now have a flink-test2.txt file. That is pretty cool ;)

Miquel van Smoorenburg <[email protected]> said:
[...]

Quoted 9 lines

It results in:
link("/proc/self/fd/3", "flink-test2.txt") = -1 EXDEV (Invalid cross-device link) This is probably because link() doesn't look up the target of the symlink, it links the symlink itself. Linux allows symlinks with
a nlink count of 2:
% ln -s a b
% ln b c
ln: `b': warning: making a hard link to a symbolic link is not portable % ls -l

Quoted 7 lines

lrwxrwxrwx    2 miquels  staff           1 Jan 19 11:34 b -> a
lrwxrwxrwx    2 miquels  staff           1 Jan 19 11:34 c -> a

This could be hacked around ofcourse in fs/namei.c, so I tried it for fun. And indeed, with a minor correction it works:
% perl flink.pl
Success.
I now have a flink-test2.txt file. That is pretty cool ;)

This is a possible security risk: The unlinking program thinks the file is forever inaccessible, but it isn't...

Quoted 2 lines

This is a possible security risk: The unlinking program thinks the file is forever inaccessible, but it isn't...

Will the ability to access this linked file still be there across a reboot ? Or an fsck ? Tia , JimL

Quoted 1 line

I now have a flink-test2.txt file. That is pretty cool ;)

This is a possible security risk: The unlinking program thinks the file is forever inaccessible, but it isn't...

Miquel van Smoorenburg <[email protected]> said:

[...]

Quoted 19 lines

It results in:

link("/proc/self/fd/3", "flink-test2.txt") = -1 EXDEV (Invalid cross-device lin k)

This is probably because link() doesn't look up the target of the symlink, it links the symlink itself. Linux allows symlinks with a nlink count of 2:

% ln -s a b
% ln b c
ln: `b': warning: making a hard link to a symbolic link is not portable % ls -l

lrwxrwxrwx    2 miquels  staff           1 Jan 19 11:34 b -> a
lrwxrwxrwx    2 miquels  staff           1 Jan 19 11:34 c -> a

This could be hacked around ofcourse in fs/namei.c, so I tried it for fun. And indeed, with a minor correction it works:

Quoted 4 lines

% perl flink.pl
Success.

I now have a flink-test2.txt file. That is pretty cool ;)

This is a possible security risk: The unlinking program thinks the file is forever inaccessible, but it isn't...

Quoted 5 lines · [email protected] (Alexander Viro) wrote:

Quoted 1 line · Miquel van Smoorenburg

I now have a flink-test2.txt file. That is pretty cool ;)

It's also a security hole.

It may well be one when going via /proc. But is it one when going via a (hypothetical) proper flink(2)? If so, why?

Note that every process who has a filehandle open for reading can already get at the file contents and write them to a completely new file, and every process who has it open for writing can already change its contents to everything it likes. So I can see read|write checks on the file handle. Also all the usual link(2) checks. What else could be a hole?

If the file descriptor you have was opened O_RDONLY, but you have write permission on the file itself, then creating a new name for it would allow you to open it O_RDWR.

If the file descriptor you have was opened O_RDONLY, but you have write permission on the file itself, then creating a new name for it would allow you to open it O_RDWR.

Quoted 6 lines

Quoted 6 lines

This could be hacked around ofcourse in fs/namei.c, so I tried it for fun. And indeed, with a minor correction it works:

% perl flink.pl
Success.

Quoted 3 lines

Quoted 1 line

I now have a flink-test2.txt file. That is pretty cool ;)

It's also a security hole.

How is linking back a file into the normal namespace anymore a security hole as having it under /proc or keeping an fd to it open?

Quoted 3 lines

How is linking back a file into the normal namespace anymore a security hole as having it under /proc or keeping an fd to it open?

Imagine you want to delete my file, you are root.

Before, you could rm it, then kill all my processes.

This *might* work:

link("/proc/self/fd/40", newpath);

Quoted 6 lines

Well no. new_fd will refer to a completely new, empty file which has no relation to the old file at all.

There is no way to recreate a file with a nlink count of 0, well that is until someone adds flink(fd, newpath) to the kernel.

This *might* work:

link("/proc/self/fd/40", newpath);

Quoted 7 lines

Quoted 6 lines

Well no. new_fd will refer to a completely new, empty file which has no relation to the old file at all.

There is no way to recreate a file with a nlink count of 0, well that is until someone adds flink(fd, newpath) to the kernel.

Quoted 3 lines

This *might* work:

link("/proc/self/fd/40", newpath);

cat /proc/<id>/fd/<nr> > whatever
actually works.

Quoted 7 lines · Matthias Schniedermeyer

Quoted 7 lines

Quoted 6 lines

Well no. new_fd will refer to a completely new, empty file which has no relation to the old file at all.

There is no way to recreate a file with a nlink count of 0, well that is until someone adds flink(fd, newpath) to the kernel.

Quoted 6 lines

Quoted 3 lines

This *might* work:

link("/proc/self/fd/40", newpath);

cat /proc/<id>/fd/<nr> > whatever
actually works.

Once it's unliked ? I doubt it.

Quoted 12 lines · Xavier Bestel

Quoted 11 lines · Matthias Schniedermeyer

Quoted 10 lines

Quoted 6 lines

Well no. new_fd will refer to a completely new, empty file which has no relation to the old file at all.

There is no way to recreate a file with a nlink count of 0, well that is until someone adds flink(fd, newpath) to the kernel.

This *might* work:

link("/proc/self/fd/40", newpath);

Quoted 4 lines

Quoted 2 lines

cat /proc/<id>/fd/<nr> > whatever
actually works.

Once it's unliked ? I doubt it.

Egads... It certainly works, unlinked or not. Please learn the basics of Unix filesystem semantics.

Quoted 19 lines · Alexander Viro

Quoted 15 lines · Xavier Bestel

Quoted 13 lines · Matthias Schniedermeyer

Quoted 10 lines

Quoted 6 lines

Well no. new_fd will refer to a completely new, empty file which has no relation to the old file at all.

There is no way to recreate a file with a nlink count of 0, well that is until someone adds flink(fd, newpath) to the kernel.

This *might* work:

link("/proc/self/fd/40", newpath);

cat /proc/<id>/fd/<nr> > whatever
actually works.

Quoted 4 lines

Quoted 1 line

Once it's unliked ? I doubt it.

Egads... It certainly works, unlinked or not. Please learn the basics of Unix filesystem semantics.

Indeed, it works, but it doesn't with a true symlink. What kind of a link is that /proc/<id>/fd/<nr> entry ? It's not a symlink even if it looks like one.

Quoted 1 line · Xavier Bestel

On Sat, 2002-01-19 at 13:16, Matthias Schniedermeyer wrote:

Quoted 4 lines

Quoted 2 lines

cat /proc/<id>/fd/<nr> > whatever
actually works.

Once it's unliked ? I doubt it.

Egads... It certainly works, unlinked or not. Please learn the basics of Unix filesystem semantics.

Whether that was an intended or accidental feature only someone with more insight into Unix history can answer. It's that feature that lets us do live upgrades of distributions without rebooting (executables and libraries can be replaced without affecting the currently running processes), at the very least much easier than it would be without this behaviour.

Quoted 6 lines

Whether that was an intended or accidental feature only someone with more insight into Unix history can answer. It's that feature that lets us do live upgrades of distributions without rebooting (executables and libraries can be replaced without affecting the currently running processes), at the very least much easier than it would be without this behaviour.

I remember that previous debian release come with a patched kernel to allow live upgrade. It was explained in the FAQ that the patch was required for this purpose.

7.2 Debian claims to be able to update a running program;
how is this accomplished?

What was in this patch?

This is actually a long-standing UNIXism that is pretty heavily relied- upon -- for example, opening a temporary file then unlinking it before use "guarantees" that the file will not stick around in case the app crashes before it can properly close and unlink.

There is no way to recreate a file with a nlink count of 0, well that is until someone adds flink(fd, newpath) to the kernel.

Which I actually posted a patch for in 2.4.0-test1 time :)

% ll -i old
32619 -rw-r--r-- 1 remi users 14 Jul 31 15:44 old
% exec 5<old
% rm old
% ~/src/flink/flink 5 new
% ll -i new
32619 -rw-r--r-- 1 remi users 14 Jul 31 15:44 new

The more interesting part - open(O_ANONYMOUS) or something like that looked much harder to do. (IOW, I gave up ;) )

Just out of interest (I'm not actually suggesting this would be useful, or feasible): what about ilink(dev, inode_nr, "path") or iopen(dev, inode_nr)?

Or /proc/inodes/dev/<nr> ?

Quoted 5 lines

Just out of interest (I'm not actually suggesting this would be useful, or feasible): what about ilink(dev, inode_nr, "path") or iopen(dev, inode_nr)?

Or /proc/inodes/dev/<nr> ?

...which would successfully defeat any access control scheme based on directory permissions...

On Sun, Jan 20, 2002 at 09:44:31PM +0100, you [Andreas Ferber] claimed:

Quoted 5 lines · Ville Herva

Quoted 4 lines

Just out of interest (I'm not actually suggesting this would be useful, or feasible): what about ilink(dev, inode_nr, "path") or iopen(dev, inode_nr)?

Quoted 4 lines

Quoted 1 line

Or /proc/inodes/dev/<nr> ?

...which would successfully defeat any access control scheme based on directory permissions...

Yeah - it could be root-only.

But it's propably not useful anyway.

There are filesystems around (MSDOS, VFAT) that haven't got fixed inode numbers for files. There are networked filesystems where this would need radical changes to the server side. Some even make up inode numbers on the fly IIRC.

If in dire need, you could hack something together for <favorite filesystem> by groveling over the disk image. e2fsprogs' libraries should come handy...