rm-ing files with open file descriptors

This is actually a long-standing UNIXism that is pretty heavily relied- upon -- for example, opening a temporary file then unlinking it before use "guarantees" that the file will not stick around in case the app crashes before it can properly close and unlink.

One nasty side-effect is space allocation -- after unlinking a file and writing to it, you can fill the disk without the file showing up in 'ls' or 'du', etc. Hard to debug. Stronghold on Solaris used to do this with log files -- HUP did not discard the old FDs.

Signature · 2 lines

Ken.
[email protected]

On Fri, Jan 18, 2002 at 04:11:26PM -0500, Doug Alcorn wrote:

| 
| I had a weird situation with my application where the user deleted all
| the database files while the app was still reading and writing to the
| opened file descriptor.  What was weird to me was that the app didn't
| complain.  It just went merrily about it's business as if nothing were
| wrong.  Of course, after the app shut down all it's data was lost.
| 
| Since I didn't expect this behavior I wrote a simple little program to
| test it[1].  Sure enough, you can rm a file that has opened file
| descriptors and no errors are generated.  Interestingly, sun solaris
| does the same thing.  Since this is the case, I thought this might be
| a feature instead of a bug (ms-win doesn't allow the rm).  So, my
| question is where is this behavior defined?  Is it a kernel issue?
| Does POSIX define this behavior?  Is it a libc issue?  
| 
| I tried to google this, but couldn't think of the right terms to
| describe it.  As I'm not on lkm, I would appreciate a CC: to
| <[email protected]>.
| -- 
|  (__) Doug Alcorn (mailto:[email protected] http://www.lathi.net)
|  oo / PGP 02B3 1E26 BCF2 9AAF 93F1  61D7 450C B264 3E63 D543
|  |_/  If you're a capitalist and you have the best goods and they're
|       free, you don't have to proselytize, you just have to wait. 
| 
| -
| To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
| the body of a message to [email protected]
| More majordomo info at  http://vger.kernel.org/majordomo-info.html
| Please read the FAQ at  http://www.tux.org/lkml/

Richard B. Johnson

2002-01-18 21:48 UTC #3 lore raw

Quoted 15 lines · Doug Alcorn

I had a weird situation with my application where the user deleted all the database files while the app was still reading and writing to the opened file descriptor. What was weird to me was that the app didn't complain. It just went merrily about it's business as if nothing were wrong. Of course, after the app shut down all it's data was lost.
Since I didn't expect this behavior I wrote a simple little program to test it[1]. Sure enough, you can rm a file that has opened file descriptors and no errors are generated. Interestingly, sun solaris does the same thing. Since this is the case, I thought this might be a feature instead of a bug (ms-win doesn't allow the rm). So, my question is where is this behavior defined? Is it a kernel issue? Does POSIX define this behavior? Is it a libc issue?

Quoted 3 lines

I tried to google this, but couldn't think of the right terms to describe it. As I'm not on lkm, I would appreciate a CC: to <[email protected]>.

This is a characteristic of a VFS (Virtual File System) on any Unix system. The file doesn't go away until it is closed by everybody that accesses it. However, you can remove or rename it even when it's open for write by other tasks. If a task has an open file-descriptor and keeps it open, it could 'fix' a possibly deleted file, by opening it again with

         new_fd = open("filename", O_CREAT|O_RDWR, 0644);

(without O_TRUNC) and it will remain in existance after all file descriptors are closed, because it was "created" again after it was deleted by another task. However, if all descriptors are closed before you recreate it, data are permanently lost. In this case, it's a new file.

Cheers,
Dick Johnson

Penguin : Linux version 2.4.1 on an i686 machine (797.90 BogoMips).

I was going to compile a list of innovations that could be attributed to Microsoft. Once I realized that Ctrl-Alt-Del was handled in the BIOS, I found that there aren't any.

J Sloan

2002-01-18 22:00 UTC #4 lore raw

Normal unix behaviour -

It's always been that way....

jjs

Quoted 18 lines · Doug Alcorn

I had a weird situation with my application where the user deleted all the database files while the app was still reading and writing to the opened file descriptor. What was weird to me was that the app didn't complain. It just went merrily about it's business as if nothing were wrong. Of course, after the app shut down all it's data was lost.
Since I didn't expect this behavior I wrote a simple little program to test it[1]. Sure enough, you can rm a file that has opened file descriptors and no errors are generated. Interestingly, sun solaris does the same thing. Since this is the case, I thought this might be a feature instead of a bug (ms-win doesn't allow the rm). So, my question is where is this behavior defined? Is it a kernel issue? Does POSIX define this behavior? Is it a libc issue?
I tried to google this, but couldn't think of the right terms to describe it. As I'm not on lkm, I would appreciate a CC: to <[email protected]>.

Miquel van Smoorenburg

2002-01-19 00:50 UTC #5 lore raw

In article <[email protected]>,

Quoted 7 lines · Richard B. Johnson

This is a characteristic of a VFS (Virtual File System) on any Unix system. The file doesn't go away until it is closed by everybody that accesses it. However, you can remove or rename it even when it's open for write by other tasks. If a task has an open file-descriptor and keeps it open, it could 'fix' a possibly deleted file, by opening it again with

Quoted 5 lines

        new_fd = open("filename", O_CREAT|O_RDWR, 0644);
(without O_TRUNC) and it will remain in existance after all file descriptors are closed, because it was "created" again after it was deleted by another task.

Well no. new_fd will refer to a completely new, empty file which has no relation to the old file at all.

There is no way to recreate a file with a nlink count of 0, well that is until someone adds flink(fd, newpath) to the kernel.

You're a regular on this list, frankly I'm amazed that you don't know this ?

Mike.

H. Peter Anvin

2002-01-19 02:30 UTC #7 lore raw

Followup to:  <[email protected]>
By author:    Miquel van Smoorenburg <[email protected]>
In newsgroup: linux.dev.kernel

Quoted 7 lines

Well no. new_fd will refer to a completely new, empty file which has no relation to the old file at all.
There is no way to recreate a file with a nlink count of 0, well that is until someone adds flink(fd, newpath) to the kernel.

This *might* work:

link("/proc/self/fd/40", newpath);

-hpa

Signature · 3 lines

<[email protected]> at work, <[email protected]> in private!
"Unix gives you enough rope to shoot yourself in the foot."
http://www.zytor.com/~hpa/puzzle.txt	<[email protected]>

Andreas Bombe

2002-01-19 04:20 UTC #9 lore raw

In reply to H. Peter Anvin

Quoted 6 lines · Doug Alcorn

I had a weird situation with my application where the user deleted all the database files while the app was still reading and writing to the opened file descriptor. What was weird to me was that the app didn't complain. It just went merrily about it's business as if nothing were wrong. Of course, after the app shut down all it's data was lost.

Others already wrote that this is standard behaviour, but I'm in lecturing mood right now.

You have to understand that under Unix, the user can not delete files. Simple as that. Deleting files happens only at the kernel's discretion (or in fsck after an unclean reboot).

In Unix style filesystems, files (inodes) and directory entries (links) are separate things. Files as in inodes are just an entry in the filesystem referenced by number. Directory entries aka file names aka links are just a name associated with an inode number (try "ls -i" sometime).

One inode can have multiple links pointing to it, they can be created with ln (hardlinks, not "ln -s" symlinks). There is a syscall to deal with removal of those links, but none that deal with file removal.

If a program does open("xyz") or something, the directory entry "xyz" is consulted to find the inode, which is then opened. Now there is a connection file descriptor <=> inode, the directory entries are totally out of the loop now. It doesn't matter whether you move or remove the links now.

Inodes have a reference count and they are automatically deleted when it goes to zero. The reference count is the sum of link count and open count. After you remove all links to the opened file it is still referenced by the open and continues to exist. Only after the program having the file open terminates does the ref count go to zero and cause the file to be really deleted.

Whether that was an intended or accidental feature only someone with more insight into Unix history can answer. It's that feature that lets us do live upgrades of distributions without rebooting (executables and libraries can be replaced without affecting the currently running processes), at the very least much easier than it would be without this behaviour.

It just may not be the most intuitive thing (like file name == file is), but that is more than made up by its possibilities.

Signature · 1 line

Andreas Bombe <[email protected]>    DSA key 0x04880A44

Xavier Bestel

2002-01-19 10:57 UTC #8 lore raw

Quoted 7 lines · H. Peter Anvin

Followup to:  <[email protected]>
By author:    Miquel van Smoorenburg <[email protected]>
In newsgroup: linux.dev.kernel
Quoted 4 lines
Well no. new_fd will refer to a completely new, empty file which has no relation to the old file at all.

Quoted 8 lines

Quoted 3 lines
There is no way to recreate a file with a nlink count of 0, well that is until someone adds flink(fd, newpath) to the kernel.
This *might* work:
link("/proc/self/fd/40", newpath);

Mmh, in this case /proc/self/fd/40 would be a symlink to nothing ...

Miquel van Smoorenburg

2002-01-19 11:10 UTC #11 lore raw

In article <[email protected]>,

Quoted 10 lines · H. Peter Anvin

Followup to:  <[email protected]>
By author:    Miquel van Smoorenburg <[email protected]>
In newsgroup: linux.dev.kernel
Quoted 3 lines
There is no way to recreate a file with a nlink count of 0, well that is until someone adds flink(fd, newpath) to the kernel.
This *might* work:
link("/proc/self/fd/40", newpath);

I used the following perl script to test this, and alas this doesn't work (yet):

open(FD, ">flink-test.txt") || die;
print FD "flink-test\n";
unlink "flink-test.txt"|| die;
link("/proc/self/fd/" . fileno(FD), "flink-test2.txt") || die; print "Success.\n";

It results in:

link("/proc/self/fd/3", "flink-test2.txt") = -1 EXDEV (Invalid cross-device link)

This is probably because link() doesn't look up the target of the symlink, it links the symlink itself. Linux allows symlinks with a nlink count of 2:

% ln -s a b
% ln b c
ln: `b': warning: making a hard link to a symbolic link is not portable % ls -l

lrwxrwxrwx    2 miquels  staff           1 Jan 19 11:34 b -> a
lrwxrwxrwx    2 miquels  staff           1 Jan 19 11:34 c -> a

This could be hacked around ofcourse in fs/namei.c, so I tried it for fun. And indeed, with a minor correction it works:

% perl flink.pl
Success.

I now have a flink-test2.txt file. That is pretty cool ;)

Here's the patch:

fs/namei.c renamed from fs/namei.c.orig +11 -0

@@ -22,6 +22,7 @@
 #include <linux/dnotify.h>
 #include <linux/smp_lock.h>
 #include <linux/personality.h>
+#include <linux/proc_fs.h>
 
 #include <asm/namei.h>
 #include <asm/uaccess.h>

@@ -1640,6 +1641,16 @@
 			error = path_walk(from, &old_nd);
 		if (error)
 			goto exit;
+#if 1 /* flink()-like support */
+		if (old_nd.mnt->mnt_sb->s_magic == PROC_SUPER_MAGIC) {
+			path_release(&old_nd);
+			if (path_init(from, LOOKUP_POSITIVE|LOOKUP_FOLLOW,
+			    &old_nd))
+				error = path_walk(from, &old_nd);
+			if (error)
+				goto exit;
+		}
+#endif
 		if (path_init(to, LOOKUP_PARENT, &nd))
 			error = path_walk(to, &nd);
 		if (error)

Mike.

Ville Herva

2002-01-19 11:15 UTC #26 lore raw

Quoted 4 lines · you [H. Peter Anvin]

This *might* work:
link("/proc/self/fd/40", newpath);

I don't think so: firstly, it would create a cross device link and secondly, /proc/<pid>/fd/* are symbolic links. See:

/tmp>while true; do sleep 1; echo kukkanen; done > r &
[1] 19925
/tmp>L /proc/19925/fd
total 0

lrwx------    1 root     root           64 Jan 19 13:10 0 -> /dev/pts/7
l-wx------    1 root     root           64 Jan 19 13:10 1 -> /tmp/r
lrwx------    1 root     root           64 Jan 19 13:10 2 -> /dev/pts/7
/tmp>rm r               
/tmp>L /proc/19925/fd
total 0
lrwx------    1 root     root           64 Jan 19 13:10 0 -> /dev/pts/7
l-wx------    1 root     root           64 Jan 19 13:10 1 -> /tmp/r (deleted)
lrwx------    1 root     root           64 Jan 19 13:10 2 -> /dev/pts/7

/tmp>ln /proc/19925/fd/1 r2
ln: /proc/19925/fd/1: warning: making a hard link to a symbolic link is not portable
ln: create hard link r2' to /proc/19925/fd/1': Invalid cross-device link

I think one has to use lsof, ps and/or fuser and then kill.

-- v --

Alexander Viro

2002-01-19 11:28 UTC #12 lore raw

Quoted 7 lines · Miquel van Smoorenburg

This could be hacked around ofcourse in fs/namei.c, so I tried it for fun. And indeed, with a minor correction it works:
% perl flink.pl
Success.
I now have a flink-test2.txt file. That is pretty cool ;)

It's also a security hole.

Miquel van Smoorenburg

2002-01-19 12:02 UTC #14 lore raw

In article <[email protected]>,

Quoted 11 lines · Alexander Viro

Quoted 7 lines · Miquel van Smoorenburg
This could be hacked around ofcourse in fs/namei.c, so I tried it for fun. And indeed, with a minor correction it works:
% perl flink.pl
Success.
I now have a flink-test2.txt file. That is pretty cool ;)
It's also a security hole.

How is linking back a file into the normal namespace anymore a security hole as having it under /proc or keeping an fd to it open?

I've searched google on the subject but couldn't find anything relevant. Yes this has been proposed a few times for both BSD and Linux, often in combination with "unattached open" (O_NULL or somesuch) that opens a file with a nlink count of 0. It's supposed to be a perfect way to create a new file and link it atomically into place without creating (named) tempfiles.

Mike.

Matthias Schniedermeyer

2002-01-19 12:16 UTC #27 lore raw

Quoted 10 lines

Quoted 6 lines
Well no. new_fd will refer to a completely new, empty file which has no relation to the old file at all.
There is no way to recreate a file with a nlink count of 0, well that is until someone adds flink(fd, newpath) to the kernel.
This *might* work:
link("/proc/self/fd/40", newpath);

cat /proc/<id>/fd/<nr> > whatever
actually works.

"Even Better(tm)" would be if
ln /proc/<id>/fd/<nr> whatever
or
ln <Inode> whatever (lsof delivers the needed Inode-nr) would work.

Otherwise you have a problem to recover files when you have insufficent disc space. Imagine a file that is 1GB in size and you have 512MB left on the hdd. Currently i don't see a chance to recover such a file without problems.

Bis denn

Signature · 4 lines

Real Programmers consider "what you see is what you get" to be just as 
bad a concept in Text Editors as it is in women. No, the Real Programmer
wants a "you asked for it, you got it" text editor -- complicated, 
cryptic, powerful, unforgiving, dangerous.

Xavier Bestel

2002-01-19 12:23 UTC #28 lore raw

Quoted 7 lines · Matthias Schniedermeyer

Quoted 7 lines
Quoted 6 lines
Well no. new_fd will refer to a completely new, empty file which has no relation to the old file at all.
There is no way to recreate a file with a nlink count of 0, well that is until someone adds flink(fd, newpath) to the kernel.

Quoted 6 lines

Quoted 3 lines
This *might* work:
link("/proc/self/fd/40", newpath);
cat /proc/<id>/fd/<nr> > whatever
actually works.

Once it's unliked ? I doubt it.

Alexander Viro

2002-01-19 12:30 UTC #29 lore raw

Quoted 8 lines · Xavier Bestel

Quoted 7 lines · Matthias Schniedermeyer
Quoted 7 lines
Quoted 6 lines
Well no. new_fd will refer to a completely new, empty file which has no relation to the old file at all.
There is no way to recreate a file with a nlink count of 0, well that is until someone adds flink(fd, newpath) to the kernel.

Quoted 8 lines

Quoted 6 lines
Quoted 3 lines
This *might* work:
link("/proc/self/fd/40", newpath);
cat /proc/<id>/fd/<nr> > whatever
actually works.
Once it's unliked ? I doubt it.

Egads... It certainly works, unlinked or not. Please learn the basics of Unix filesystem semantics.

Xavier Bestel

2002-01-19 12:47 UTC #30 lore raw

Quoted 16 lines · Alexander Viro

Quoted 12 lines · Xavier Bestel
Quoted 11 lines · Matthias Schniedermeyer
Quoted 10 lines
Quoted 6 lines
Well no. new_fd will refer to a completely new, empty file which has no relation to the old file at all.
There is no way to recreate a file with a nlink count of 0, well that is until someone adds flink(fd, newpath) to the kernel.
This *might* work:
link("/proc/self/fd/40", newpath);

Quoted 7 lines

Quoted 4 lines
Quoted 2 lines
cat /proc/<id>/fd/<nr> > whatever
actually works.
Once it's unliked ? I doubt it.
Egads... It certainly works, unlinked or not. Please learn the basics of Unix filesystem semantics.

Indeed, it works, but it doesn't with a true symlink. What kind of a link is that /proc/<id>/fd/<nr> entry ? It's not a symlink even if it looks like one.

Rogier Wolff

2002-01-19 13:19 UTC #31 lore raw

Quoted 20 lines · Xavier Bestel

Quoted 19 lines · Alexander Viro
Quoted 15 lines · Xavier Bestel
Quoted 13 lines · Matthias Schniedermeyer
Quoted 10 lines
Quoted 6 lines
Well no. new_fd will refer to a completely new, empty file which has no relation to the old file at all.
There is no way to recreate a file with a nlink count of 0, well that is until someone adds flink(fd, newpath) to the kernel.
This *might* work:
link("/proc/self/fd/40", newpath);
cat /proc/<id>/fd/<nr> > whatever
actually works.

Quoted 8 lines

Quoted 4 lines
Quoted 1 line
Once it's unliked ? I doubt it.
Egads... It certainly works, unlinked or not. Please learn the basics of Unix filesystem semantics.
Indeed, it works, but it doesn't with a true symlink. What kind of a link is that /proc/<id>/fd/<nr> entry ? It's not a symlink even if it looks like one.

Highly correct!

Roger.

Signature · 4 lines

** [email protected] ** http://www.BitWizard.nl/ ** +31-15-2137555 **
*-- BitWizard writes Linux device drivers for any device you may have! --*
* There are old pilots, and there are bold pilots. 
* There are also old, bald pilots.

Horst von Brand

2002-01-19 14:51 UTC #6 lore raw

Miquel van Smoorenburg <[email protected]> said:

[...]

Quoted 2 lines

There is no way to recreate a file with a nlink count of 0, well that is until someone adds flink(fd, newpath) to the kernel.

It stays around under /proc/<pid>/fd/<xxx>, so you could grab it there. It is _announced_ as a symlink to the deleted file, but you get its contents anyway. [2.4.18pre4]

Signature · 1 line

Horst von Brand			     http://counter.li.org # 22616

christophe barbé

2002-01-19 14:51 UTC #33 lore raw

Quoted 6 lines · Andreas Bombe

Whether that was an intended or accidental feature only someone with more insight into Unix history can answer. It's that feature that lets us do live upgrades of distributions without rebooting (executables and libraries can be replaced without affecting the currently running processes), at the very least much easier than it would be without this behaviour.

I remember that previous debian release come with a patched kernel to allow live upgrade. It was explained in the FAQ that the patch was required for this purpose.

7.2 Debian claims to be able to update a running program;
how is this accomplished?

The debian FAQ has been updated because now debian uses a pristine kernel.

What was in this patch?

Christophe

Signature · 6 lines

Christophe Barbé <[email protected]>
GnuPG FingerPrint: E0F6 FADF 2A5C F072 6AF8  F67A 8F45 2F1E D72C B41E

Cats are rather delicate creatures and they are subject to a good
many ailments, but I never heard of one who suffered from insomnia.
--Joseph Wood Krutch

Horst von Brand

2002-01-19 15:21 UTC #15 lore raw

In reply to Alexander Viro

Miquel van Smoorenburg <[email protected]> said:

[...]

Quoted 16 lines

It results in:
link("/proc/self/fd/3", "flink-test2.txt") = -1 EXDEV (Invalid cross-device lin
k)
This is probably because link() doesn't look up the target of the symlink, it links the symlink itself. Linux allows symlinks with a nlink count of 2:
% ln -s a b
% ln b c
ln: `b': warning: making a hard link to a symbolic link is not portable % ls -l
lrwxrwxrwx    2 miquels  staff           1 Jan 19 11:34 b -> a
lrwxrwxrwx    2 miquels  staff           1 Jan 19 11:34 c -> a

Quoted 7 lines

This could be hacked around ofcourse in fs/namei.c, so I tried it for fun. And indeed, with a minor correction it works:
% perl flink.pl
Success.
I now have a flink-test2.txt file. That is pretty cool ;)

This is a possible security risk: The unlinking program thinks the file is forever inaccessible, but it isn't...

Signature · 1 line

Horst von Brand			     http://counter.li.org # 22616

Horst von Brand

2002-01-19 15:25 UTC #32 lore raw

Quoted 2 lines · Alexander Viro

Quoted 1 line · Xavier Bestel
On Sat, 2002-01-19 at 13:16, Matthias Schniedermeyer wrote:

[...]

Quoted 7 lines

Quoted 4 lines
Quoted 2 lines
cat /proc/<id>/fd/<nr> > whatever
actually works.
Once it's unliked ? I doubt it.
Egads... It certainly works, unlinked or not. Please learn the basics of Unix filesystem semantics.

It does show up as a broken symlink for ls(1)...

Signature · 1 line

Horst von Brand			     http://counter.li.org # 22616

Mr. James W. Laferriere

2002-01-19 15:33 UTC #16 lore raw

In reply to Horst von Brand

Quoted 19 lines · Horst von Brand

Miquel van Smoorenburg <[email protected]> said:
[...]
Quoted 9 lines
It results in:
link("/proc/self/fd/3", "flink-test2.txt") = -1 EXDEV (Invalid cross-device link) This is probably because link() doesn't look up the target of the symlink, it links the symlink itself. Linux allows symlinks with
a nlink count of 2:
% ln -s a b
% ln b c
ln: `b': warning: making a hard link to a symbolic link is not portable % ls -l
Quoted 7 lines
lrwxrwxrwx    2 miquels  staff           1 Jan 19 11:34 b -> a
lrwxrwxrwx    2 miquels  staff           1 Jan 19 11:34 c -> a
This could be hacked around ofcourse in fs/namei.c, so I tried it for fun. And indeed, with a minor correction it works:
% perl flink.pl
Success.
I now have a flink-test2.txt file. That is pretty cool ;)

Quoted 2 lines

This is a possible security risk: The unlinking program thinks the file is forever inaccessible, but it isn't...

Will the ability to access this linked file still be there across a reboot ? Or an fsck ? Tia , JimL

Miquel van Smoorenburg

2002-01-19 17:54 UTC #18 lore raw

In reply to Horst von Brand

According to Horst von Brand:

Quoted 4 lines

Quoted 1 line
I now have a flink-test2.txt file. That is pretty cool ;)
This is a possible security risk: The unlinking program thinks the file is forever inaccessible, but it isn't...

Why. If you keep an fd open to it it's accessible anyway, and if you like you can copy it to a new file. Or you could link(2) it beforehand, etc etc

Mike.

Kai Henningsen

2002-01-19 19:07 UTC #13 lore raw

In reply to Alexander Viro

[email protected] (Alexander Viro) wrote on 19.01.02 in <[email protected]>:

Quoted 11 lines · Miquel van Smoorenburg

Quoted 8 lines
This could be hacked around ofcourse in fs/namei.c, so I tried it for fun. And indeed, with a minor correction it works:
% perl flink.pl
Success.
I now have a flink-test2.txt file. That is pretty cool ;)
It's also a security hole.

It may well be one when going via /proc. But is it one when going via a (hypothetical) proper flink(2)? If so, why?

Note that every process who has a filehandle open for reading can already get at the file contents and write them to a completely new file, and every process who has it open for writing can already change its contents to everything it likes. So I can see read|write checks on the file handle. Also all the usual link(2) checks. What else could be a hole?

MfG Kai

Kai Henningsen

2002-01-19 19:07 UTC #34 lore raw

In reply to christophe barbé

[email protected] (christophe barbé) wrote on 19.01.02 in <[email protected]>:

Quoted 11 lines · Andreas Bombe

Quoted 6 lines
Whether that was an intended or accidental feature only someone with more insight into Unix history can answer. It's that feature that lets us do live upgrades of distributions without rebooting (executables and libraries can be replaced without affecting the currently running processes), at the very least much easier than it would be without this behaviour.
I remember that previous debian release come with a patched kernel to allow live upgrade. It was explained in the FAQ that the patch was required for this purpose.

Complete and utter bullshit. This was never true, and the FAQ never claimed this.

Quoted 2 lines

7.2 Debian claims to be able to update a running program;
how is this accomplished?

... under which was originally explained how running demons would be restarted, and later it was also mentioned that replacing in-use files is possible under Unix. Nothing more. (Google groups will happily find those versions, they were in use from 1996 to 2001 according to the archive.)

Quoted 1 line

What was in this patch?

The patch only exists in your fantasy.

MfG Kai

christophe barbé

2002-01-20 03:44 UTC #10 lore raw

In reply to Andreas Bombe

Quoted 20 lines · Kai Henningsen

[email protected] (christophe barb?) wrote on 19.01.02 in <[email protected]>:
Quoted 11 lines · Andreas Bombe
Quoted 6 lines
Whether that was an intended or accidental feature only someone with more insight into Unix history can answer. It's that feature that lets us do live upgrades of distributions without rebooting (executables and libraries can be replaced without affecting the currently running processes), at the very least much easier than it would be without this behaviour.
I remember that previous debian release come with a patched kernel to allow live upgrade. It was explained in the FAQ that the patch was required for this purpose.
Complete and utter bullshit. This was never true, and the FAQ never claimed this.
Quoted 2 lines
7.2 Debian claims to be able to update a running program;
how is this accomplished?

Quoted 8 lines

... under which was originally explained how running demons would be restarted, and later it was also mentioned that replacing in-use files is possible under Unix. Nothing more. (Google groups will happily find those versions, they were in use from 1996 to 2001 according to the archive.)
Quoted 1 line
What was in this patch?
The patch only exists in your fantasy.

Ok you are right. I've checked old versions of this FAQ and this patch only exists in my fantasy.

I take your 'Complete and utter bullshit' comment as a debian compliment and not as an insult.

Christophe

Quoted 7 lines

MfG Kai
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/

Signature · 6 lines

Christophe Barbé <[email protected]>
GnuPG FingerPrint: E0F6 FADF 2A5C F072 6AF8  F67A 8F45 2F1E D72C B41E

Ce que l'on conçoit bien s'énonce clairement,
Et les mots pour le dire arrivent aisément.
   Nicolas Boileau, L'Art poétique

Chris Wedgwood

2002-01-20 03:55 UTC #20 lore raw

In reply to Horst von Brand

On Sat, Jan 19, 2002 at 06:28:36AM -0500, Alexander Viro wrote:

It's also a security hole.

As such would an flink impementation also be considered a bad thing?

--cw

Rob Landley

2002-01-20 04:30 UTC #19 lore raw

Quoted 23 lines · Horst von Brand

Miquel van Smoorenburg <[email protected]> said:
[...]
Quoted 19 lines
It results in:
link("/proc/self/fd/3", "flink-test2.txt") = -1 EXDEV (Invalid cross-device lin k)
This is probably because link() doesn't look up the target of the symlink, it links the symlink itself. Linux allows symlinks with a nlink count of 2:
% ln -s a b
% ln b c
ln: `b': warning: making a hard link to a symbolic link is not portable % ls -l
lrwxrwxrwx    2 miquels  staff           1 Jan 19 11:34 b -> a
lrwxrwxrwx    2 miquels  staff           1 Jan 19 11:34 c -> a
This could be hacked around ofcourse in fs/namei.c, so I tried it for fun. And indeed, with a minor correction it works:

Quoted 7 lines

Quoted 4 lines
% perl flink.pl
Success.
I now have a flink-test2.txt file. That is pretty cool ;)
This is a possible security risk: The unlinking program thinks the file is forever inaccessible, but it isn't...

It's only accessable to the same user. (permissions 700 in /proc/blah/fs.) If you're running at the same user, you can attach a debugger to the process and look at anything it's got. So it's just an easier way to do something you could effectively already do anyway...

Rob

Rob Landley

2002-01-20 04:30 UTC #35 lore raw

In reply to Mr. James W. Laferriere

Quoted 4 lines · Ken Brownfield

This is actually a long-standing UNIXism that is pretty heavily relied- upon -- for example, opening a temporary file then unlinking it before use "guarantees" that the file will not stick around in case the app crashes before it can properly close and unlink.

It's fun with named pipes, too. Allows you to use the child's PID as the temp file name really easily...

Rob

Rob Landley

2002-01-20 04:31 UTC #17 lore raw

Quoted 5 lines · Mr. James W. Laferriere

Quoted 2 lines
This is a possible security risk: The unlinking program thinks the file is forever inaccessible, but it isn't...
Will the ability to access this linked file still be there across a reboot ? Or an fsck ? Tia , JimL

Actually deleting files with a link count of zero is one of the things fsck is for, I believe...

Rob

Remi Turk

2002-01-20 14:23 UTC #36 lore raw

Quoted 2 lines · Miquel van Smoorenburg

There is no way to recreate a file with a nlink count of 0, well that is until someone adds flink(fd, newpath) to the kernel.

Which I actually posted a patch for in 2.4.0-test1 time :)

% ll -i old
32619 -rw-r--r-- 1 remi users 14 Jul 31 15:44 old
% exec 5<old
% rm old
% ~/src/flink/flink 5 new
% ll -i new
32619 -rw-r--r-- 1 remi users 14 Jul 31 15:44 new

The more interesting part - open(O_ANONYMOUS) or something like that looked much harder to do. (IOW, I gave up ;) )

Happy hacking

Remi

Signature · 6 lines

See the light and feel my warm desire,
Run through your veins like the evening sun
It will live but no eyes will see it,
I'll bless your name before I die.

Key fingerprint = CC90 A1BA CF6D 891C 5B88  C543 6C5F C469 8F20 70F4

Richard Kettlewell

2002-01-20 15:30 UTC #21 lore raw

Quoted 10 lines · [email protected] (Kai Henningsen) writes:

Quoted 5 lines · [email protected] (Alexander Viro) wrote:
Quoted 1 line · Miquel van Smoorenburg
I now have a flink-test2.txt file. That is pretty cool ;)
It's also a security hole.
It may well be one when going via /proc. But is it one when going via a (hypothetical) proper flink(2)? If so, why?

Quoted 6 lines

Note that every process who has a filehandle open for reading can already get at the file contents and write them to a completely new file, and every process who has it open for writing can already change its contents to everything it likes. So I can see read|write checks on the file handle. Also all the usual link(2) checks. What else could be a hole?

If the file descriptor you have was opened O_RDONLY, but you have write permission on the file itself, then creating a new name for it would allow you to open it O_RDWR.

I'm not 100% convinced by this argument. If you really want a particular user not to be able to write to a file, the certain answer is to set its permissions appropriately, rather than rely on it having no name.

One could make the hypothetical flink(2) only work on O_RDWR file descriptors, or only on files owned by the euid of the calling process.

Signature · 1 line

http://www.greenend.org.uk/rjk/

Doug McNaught

2002-01-20 18:22 UTC #22 lore raw

Quoted 3 lines · Richard Kettlewell

If the file descriptor you have was opened O_RDONLY, but you have write permission on the file itself, then creating a new name for it would allow you to open it O_RDWR.

Are you sure about this? Permissions are stored in the inode, not the directory entry.

-Doug

Signature · 2 lines

Let us cross over the river, and rest under the shade of the trees.
   --T. J. Jackson, 1863

Ville Herva

2002-01-20 20:03 UTC #37 lore raw

On Sun, Jan 20, 2002 at 03:23:59PM +0100, you [Remi Turk] claimed:

Quoted 11 lines

Which I actually posted a patch for in 2.4.0-test1 time :)
% ll -i old
32619 -rw-r--r-- 1 remi users 14 Jul 31 15:44 old
% exec 5<old
% rm old
% ~/src/flink/flink 5 new
% ll -i new
32619 -rw-r--r-- 1 remi users 14 Jul 31 15:44 new

Quoted 2 lines

The more interesting part - open(O_ANONYMOUS) or something like that looked much harder to do. (IOW, I gave up ;) )

Just out of interest (I'm not actually suggesting this would be useful, or feasible): what about ilink(dev, inode_nr, "path") or iopen(dev, inode_nr)?

Or /proc/inodes/dev/<nr> ?

-- v --

Andreas Ferber

2002-01-20 20:44 UTC #38 lore raw

Quoted 5 lines · Ville Herva

Just out of interest (I'm not actually suggesting this would be useful, or feasible): what about ilink(dev, inode_nr, "path") or iopen(dev, inode_nr)?
Or /proc/inodes/dev/<nr> ?

...which would successfully defeat any access control scheme based on directory permissions...

Andreas

Signature · 3 lines

       Andreas Ferber - dev/consulting GmbH - Bielefeld, FRG
     ---------------------------------------------------------
         +49 521 1365800 - [email protected] - www.devcon.net

Ville Herva

2002-01-20 21:09 UTC #39 lore raw

On Sun, Jan 20, 2002 at 09:44:31PM +0100, you [Andreas Ferber] claimed:

Quoted 9 lines · Ville Herva

Quoted 5 lines
Just out of interest (I'm not actually suggesting this would be useful, or feasible): what about ilink(dev, inode_nr, "path") or iopen(dev, inode_nr)?
Or /proc/inodes/dev/<nr> ?
...which would successfully defeat any access control scheme based on directory permissions...

Yeah - it could be root-only.

But it's propably not useful anyway.

-- v --

Miquel van Smoorenburg

2002-01-20 23:11 UTC #23 lore raw

In article <[email protected]>,

Quoted 3 lines · Richard Kettlewell

If the file descriptor you have was opened O_RDONLY, but you have write permission on the file itself, then creating a new name for it would allow you to open it O_RDWR.

/proc allows for this anyway.

open("knuth.txt", O_RDONLY)             = 3
unlink("knuth.txt")                     = 0
open("/proc/self/fd/3", O_RDWR)         = 4

Mike.

Horst von Brand

2002-01-21 09:07 UTC #40 lore raw

In reply to Ville Herva

Quoted 6 lines · Ville Herva

On Sun, Jan 20, 2002 at 09:44:31PM +0100, you [Andreas Ferber] claimed:
Quoted 5 lines · Ville Herva
Quoted 4 lines
Just out of interest (I'm not actually suggesting this would be useful, or feasible): what about ilink(dev, inode_nr, "path") or iopen(dev, inode_nr)?

Quoted 8 lines

Quoted 4 lines
Quoted 1 line
Or /proc/inodes/dev/<nr> ?
...which would successfully defeat any access control scheme based on directory permissions...
Yeah - it could be root-only.
But it's propably not useful anyway.

There are filesystems around (MSDOS, VFAT) that haven't got fixed inode numbers for files. There are networked filesystems where this would need radical changes to the server side. Some even make up inode numbers on the fly IIRC.

If in dire need, you could hack something together for <favorite filesystem> by groveling over the disk image. e2fsprogs' libraries should come handy...

Signature · 1 line

Horst von Brand			     http://counter.li.org # 22616

Ville Herva

2002-01-21 09:21 UTC #41 lore raw

In reply to Ville Herva

On Mon, Jan 21, 2002 at 10:06:57AM +0100, you [Horst von Brand] claimed:

Quoted 5 lines

There are filesystems around (MSDOS, VFAT) that haven't got fixed inode numbers for files. There are networked filesystems where this would need radical changes to the server side. Some even make up inode numbers on the fly IIRC.

True.

Quoted 3 lines

If in dire need, you could hack something together for <favorite filesystem> by groveling over the disk image. e2fsprogs' libraries should come handy...

Well, I'll have to come up with the dire need first - or any need indeed :). Just playing around with the idea.

IIRC, Stephen Tweedie had made such a patch for ext2.

-- v --

Pavel Machek

2002-01-23 22:45 UTC #24 lore raw