ext3 and undeletion
A message that is not in the archive (sent off-list or not yet received): [email protected]
Quoted 3 lines
but I don't want a Netware filesystem running on Linux, I want a *native* Linux filesystem (i.e. ext3) that has the ability to queue deleted files should I configure it to.
Rather than implementing this in the filesystem itself, I'd first try writing a libc shim that overrides unlink(). You could copy files to safety, or do anything else you want, before they actually get deleted...
Regards,
Dan
Quoted 11 lines · Dan Maas
Quoted 3 lines
but I don't want a Netware filesystem running on Linux, I want a *native* Linux filesystem (i.e. ext3) that has the ability to queue deleted files should I configure it to.
Rather than implementing this in the filesystem itself, I'd first try writing a libc shim that overrides
unlink(). You could copy files to safety, or do anything else you want, before they actually get deleted...Regards,
Dan
Yes... unlink() becomes mv /path/filename /deleted/path/filename Simple. For idiot users, you can just make such an alias for those who insist in doing rm * instead of rm \* after they had used a wild-card as a file-name... It happens:
`ls *.* >files` ...is typoed to: `ls *.>* files`
If somebody then recreates the same file and deletes it again -- tough.
Cheers,
Dick Johnson
Penguin : Linux version 2.4.1 on an i686 machine (797.90 BogoMips).
111,111,111 * 111,111,111 = 12,345,678,987,654,321
Quoted 10 lines · Richard B. Johnson
Quoted 8 lines · Dan Maas
Quoted 3 lines
but I don't want a Netware filesystem running on Linux, I want a *native* Linux filesystem (i.e. ext3) that has the ability to queue deleted files should I configure it to.
Rather than implementing this in the filesystem itself, I'd first try writing a libc shim that overrides
unlink(). You could copy files to safety, or do anything else you want, before they actually get deleted...
Quoted 5 lines
Quoted 3 lines
Regards,
DanYes...
unlink()becomesmv /path/filename /deleted/path/filenameSimple. For idiot users, you can just make such an alias for those
It would be nice if there was a 'deleted' dir per mount point, as that would keep similar speeds as rm. Also, 'deleted' would probably have to be marked writable, but not readable and would need a suid binary to read that dir and limit the output to only list files owned by the calling uid. But that's a bit too offtopic for this list...
Mike
Quoted 5 lines · Mike Fedyk
Quoted 4 lines · Richard B. Johnson
Quoted 3 lines
Rather than implementing this in the filesystem itself, I'd first try writing a libc shim that overrides
unlink(). You could copy files to safety, or do anything else you want, before they actually get deleted...
Quoted 8 lines
Quoted 2 lines
Yes...
unlink()becomesmv /path/filename /deleted/path/filenameSimple. For idiot users, you can just make such an alias for thoseIt would be nice if there was a 'deleted' dir per mount point, as that would keep similar speeds as rm. Also, 'deleted' would probably have to be marked writable, but not readable and would need a suid binary to read that dir and limit the output to only list files owned by the calling uid. But that's a bit too offtopic for this list...
Yes, the deleted (prefer /.deleted or similar) directory would _have_ to be per mount point for a few reasons:
1) speed - copying all deleted files across mountpoints would be _slow_.
2) space - you would have to have a _huge_ root directory otherwise.
3) locality - need to handle network filesystems properly (e.g. being able to undelete a file on a network fs if it was deleted on a different host).
While I've seen this "change unlink in libc" suggestion many, many times I don't think I've ever seen it implemented. Is it just because the people who can do it don't want to, and by the time the people who want it can implement it they don't want it anymore?
Cheers, Andreas
--
Andreas Dilger
http://sourceforge.net/projects/ext2resize/
http://www-mddsp.enel.ucalgary.ca/People/adilger/
Re: ext3 and undeletion (libtrash)
In reply to Dan MaasQuoting Andreas Dilger <[email protected]> on Mon, Feb 25 12:49:
Quoted 3 lines
While I've seen this "change unlink in libc" suggestion many, many times I don't think I've ever seen it implemented.
libtrash does this: http://www.m-arriaga.net/software/libtrash/
I've never actually used it, but I've been meaning to look at it for quite a while.
Omen
Signature · 3 lines
"What is this talk of 'release'? Klingons do not make software 'releases'. Our software 'escapes,' leaving a bloody trail of designers and quality assurance people in its wake."
Quoted 10 lines · Martin Dalecki
Quoted 7 lines
Quoted 4 lines
For the educated user it was always a pain
in the you know where, to constantly run out of quota space due to file versioning.Ahh, so we'd need to chown the files to root (or a configurable user and group) to get around the quota issue.
Welcome to my kill-file. This just shows that you don't even have basic background.
Thank you.
Now, if I'm talking out of my ass, can someone sane say so?
It would only call chown/chgrp on the files *inside* the undelete dir, and user,group,etc would have to be accounted for in another way. Am I going in the wrong direction?
Mike
Quoted 9 lines · Mike Fedyk
Quoted 8 lines · Martin Dalecki
Quoted 7 lines
Quoted 4 lines
For the educated user it was always a pain
in the you know where, to constantly run out of quota space due to file versioning.Ahh, so we'd need to chown the files to root (or a configurable user and group) to get around the quota issue.
Quoted 6 lines
Quoted 2 lines
Welcome to my kill-file. This just shows that you don't even have basic background.
Thank you.
Now, if I'm talking out of my ass, can someone sane say so?
Your idea should work on deletion, when the inode were about to be destroyed, but ...
Quoted 3 lines
It would only call chown/chgrp on the files *inside* the undelete dir, and user,group,etc would have to be accounted for in another way. Am I going in the wrong direction?
... of course, there still is the problem of hard links.
If you unlink a file, it might still be around under another name.
Consider:
$ ln bigfile newbigfile $ rm bigfile
Under your scheme, maybe bigfile would be moved to the undeletion area ... fine.
The problem would start if the ownership was changed to root, because then 'newbigfile' would also be owned by root and you could no longer access your file ;)
regards,
Rik
Signature · 3 lines
Will hack the VM for food. http://www.surriel.com/ http://distro.conectiva.com/
Quoted 17 lines · Rik van Riel
Quoted 9 lines · Mike Fedyk
Quoted 8 lines · Martin Dalecki
Quoted 7 lines
Quoted 4 lines
For the educated user it was always a pain
in the you know where, to constantly run out of quota space due to file versioning.Ahh, so we'd need to chown the files to root (or a configurable user and group) to get around the quota issue.
Quoted 6 lines
Quoted 2 lines
Welcome to my kill-file. This just shows that you don't even have basic background.
Thank you.
Now, if I'm talking out of my ass, can someone sane say so?
Quoted 9 lines
Your idea should work on deletion, when the inode were about to be destroyed, but ...
Quoted 3 lines
It would only call chown/chgrp on the files *inside* the undelete dir, and user,group,etc would have to be accounted for in another way. Am I going in the wrong direction?
... of course, there still is the problem of hard links.
I had considered hard links. Take a look at my another message from me in this thread and see Daniel's response to it.
Basically, it would only move the files to the undelete area if the link count == 1. If you just decremented the link, then unlink() in glibc would work as it does now.
Mike
Quoted 4 lines · Mike Fedyk
Basically, it would only move the files to the undelete area if the link count == 1. If you just decremented the link, then
unlink()in glibc would work as it does now.
Always racy if done in userspace, unless you introduce a centralised "unlink daemon" (hope no glibc developer reads that, they might be tempted to implement such an abomination...):
proc1 proc2
--------------------
stat()
stat()
unlink()
unlink()*kaboom*, blackhole opens, file is gone.
/If/ you start doing such a mess (personally I don't want undeletion anyways), please do it at least in a correct way.
Andreas
Signature · 3 lines
Andreas Ferber - dev/consulting GmbH - Bielefeld, FRG
---------------------------------------------------------
+49 521 1365800 - [email protected] - www.devcon.netQuoted 1 line · Mike Fedyk
On Tue, Feb 26, 2002 at 06:07:49PM +0100, Martin Dalecki wrote:
[SNIPPED...]
Quoted 7 lines
Quoted 4 lines
Quoted 1 line
group) to get around the quota issue.
Welcome to my kill-file. This just shows that you don't even have basic background.
Thank you.
Quoted 7 lines
Now, if I'm talking out of my ass, can someone sane say so?
It would only call chown/chgrp on the files *inside* the undelete dir, and user,group,etc would have to be accounted for in another way. Am I going in the wrong direction?
Mike
No SUID, no nothing special to accomplish the following:
johnson[1]$ pwd/home/users/johnson
johnson[2]$ rm -r *
johnson[3]$ ls -la
total 8
drwxr-xr-x 2 rjohnson guru 4096 Feb 26 13:03 .
drwxr-xr-x 54 root root 4096 Feb 26 13:04 ..
johnson[4]$ ls -laR /home/users/lost+found/rjohnson total 5428
drwxr-xr-x 17 rjohnson guru 4096 Feb 13 16:54 . drwxrwxrwx 21 root root 4096 Feb 26 05:34 .. -rw-r--r-- 1 rjohnson guru 6 Oct 15 1998 .XF86_S3 -rw------- 1 rjohnson guru 0 Aug 10 1998 .Xauthority -rw-r--r-- 1 rjohnson guru 1557 Dec 31 1997 .acrorc -rw-r--r-- 1 rjohnson guru 273 May 13 1998 .addressbook -rw-r--r-- 1 rjohnson guru 2684 May 13 1998 .addressbook.lu -rw-r--r-- 1 rjohnson guru 0 Feb 13 16:54 .bash_history -rwxr-xr-x 1 rjohnson guru 343 May 17 1999 .bashrc -rw-r--r-- 1 rjohnson guru 2938 Sep 26 1994 .emacs -rw-r--r-- 1 rjohnson guru 47 Aug 5 1996 .forward.SAVED -rw-r--r-- 1 rjohnson guru 92 Feb 26 1997 .gopherrc -rw-r--r-- 1 rjohnson guru 130 Nov 2 1998 .ispell_english -rw-r--r-- 1 rjohnson guru 164 Sep 26 1994 .kermrc -rw-r--r-- 1 rjohnson guru 34 Sep 26 1994 .less -rw-r--r-- 1 rjohnson guru 114 Sep 26 1994 .lessrc -rw-r--r-- 1 rjohnson guru 8 Feb 12 1999 .mosaic-cc-2.7b5 -rw-r--r-- 1 rjohnson guru 1349 Feb 12 1999 .mosaic-hot.html -rw-r--r-- 1 rjohnson guru 1413 Feb 12 1999 .mosaic-hot.html.backup -rw-r--r-- 1 rjohnson guru 282 Oct 20 1995 .mosaic-hotlist-default -rw-r--r-- 1 rjohnson guru 854 Aug 22 1996 .mosaic-hotlist-default.html drwx------ 2 rjohnson guru 4096 Jan 27 1997 .mosaic-personal-annotations -rw-r--r-- 1 rjohnson guru 1718 Feb 12 1999 .mosaic-x-history -rw-r--r-- 1 rjohnson guru 39706 Feb 12 1999 .mosaic-x-history.backup -rw-r--r-- 1 rjohnson guru 5 Feb 12 1999 .mosaicpid -rw-r--r-- 1 rjohnson guru 204 Oct 10 1995 .mymail -rw-r--r-- 1 rjohnson guru 9236 Dec 30 1999 .pine-debug1 -rw-r--r-- 1 rjohnson guru 8978 Feb 19 1999 .pine-debug2 -rw-r--r-- 1 rjohnson guru 8978 Feb 19 1999 .pine-debug3 -rw-r--r-- 1 rjohnson guru 9237 Feb 19 1999 .pine-debug4 -rw-r--r-- 1 rjohnson guru 10682 Dec 30 1999 .pinerc -rw-r--r-- 1 rjohnson guru 90 Jul 15 1995 .plan -rw-r--r-- 1 rjohnson guru 625 Oct 15 1998 .procmailrc -rw-r--r-- 1 rjohnson guru 17 Dec 6 1994 .profile -rw-r--r-- 1 rjohnson guru 454 Jul 15 1995 .project -rw-r--r-- 1 rjohnson guru 35 May 16 1995 .rhosts drwxr-xr-x 2 rjohnson guru 4096 Jan 26 1997 .seyon -rw-r--r-- 1 rjohnson guru 209 Feb 13 16:54 .signature -rw-r--r-- 1 rjohnson guru 532 Nov 19 1996 .signature.SAVED -rw-r--r-- 1 rjohnson guru 275 Sep 10 1997 .signature.bak -rw-r--r-- 1 rjohnson guru 50 Feb 19 1999 .sversionrc drwx------ 2 rjohnson guru 4096 Jan 26 1997 .term -rw-r--r-- 2 rjohnson guru 12288 Sep 11 1996 .vacation.dir -rw-r--r-- 1 rjohnson guru 247 Sep 11 1996 .vacation.msg -rw-r--r-- 2 rjohnson guru 12288 Sep 11 1996 .vacation.pag -rwxr-xr-x 1 rjohnson guru 443 Sep 4 1997 .xinitrc -rw-r--r-- 1 rjohnson guru 1117 Sep 2 1998 BOMBER.CREW -rw-r--r-- 1 rjohnson guru 7697 Jul 27 1998 BusLogic.html -rw-r--r-- 1 rjohnson guru 21 Apr 17 1998 CHECK.KERNEL -rw-r--r-- 1 rjohnson guru 6519 Mar 30 1998 Computers -rw-r--r-- 1 rjohnson guru 442 Sep 28 1998 DELAY.PATCH -rw-r--r-- 1 rjohnson guru 4349 Nov 2 1998 Flying_saucers -rw-r--r-- 1 rjohnson guru 4349 Mar 17 1998 Flying_saucers.bak -rw-r--r-- 1 rjohnson guru 576 May 15 1998 IRQ.PATCH -rw-r--r-- 1 rjohnson guru 69 Apr 24 1998 ME.TXT -rwxr-xr-x 1 rjohnson guru 131 Aug 3 1998 MIT drwx--x--x 2 rjohnson guru 4096 Oct 21 1998 Mail -rw-r--r-- 1 rjohnson guru 135 Jul 11 1998 PATENT -rw-r--r-- 1 rjohnson guru 625 Jun 3 1998 _procmail drwxr-xr-x 2 rjohnson guru 4096 Dec 1 1998 asm -rwxr-xr-x 1 rjohnson guru 1358 Sep 16 1997 backup -rw-r--r-- 1 rjohnson guru 230454 Jun 5 1998 baldie.bmp drwxr-xr-x 3 rjohnson guru 4096 Jun 1 1998 bomb -rwxr-xr-x 1 rjohnson guru 2993 Oct 9 16:06 c.c drwxr-xr-x 2 rjohnson guru 4096 Sep 28 1998 callback -rw-r--r-- 1 rjohnson guru 12900 May 18 1998 chaos.txt -rwxr-xr-x 1 rjohnson guru 57276 Dec 9 1997 chkdev.c -rw-r--r-- 1 rjohnson guru 5276 Nov 17 1998 cpu.speed -rw-r--r-- 1 rjohnson guru 895 Sep 29 1998 delay.c -rw-r--r-- 1 rjohnson guru 964 Sep 30 1998 delay.patch -rwxr-xr-x 1 rjohnson guru 5414 Mar 9 1998 diskio -rw-r--r-- 1 rjohnson guru 3364 Nov 22 1998 evil.IDE -rwxr-xr-x 1 rjohnson guru 5061 Nov 11 1998 fastchk.asm -rw-r--r-- 1 rjohnson guru 957 Oct 31 1997 firewall.sh -rwxr-xr-x 1 rjohnson guru 4717 Oct 1 15:09 fpu -rw-r--r-- 1 rjohnson guru 499 Oct 30 1997 fpu.c -rw-r--r-- 1 rjohnson guru 257507 Jan 28 08:47 friday.ps -rw-r--r-- 1 rjohnson guru 5363 Aug 20 1997 ftp.txt -rwxr-xr-x 1 rjohnson guru 345 Sep 15 1998 get_bomb -rwxr-xr-x 1 rjohnson guru 219539 Dec 6 1994 gwm drwxr-xr-x 2 rjohnson guru 4096 Nov 14 1997 homepage -rwxr-xr-x 1 rjohnson guru 84 Jan 28 1998 hostfile -rw-r--r-- 1 rjohnson guru 86994 Jan 28 1998 hosts.tmp -rwxr-xr-x 1 rjohnson guru 22261 Sep 10 1998 iiclink.asm -rw-r--r-- 1 rjohnson guru 1522 May 25 1998 info.bak -rw-r--r-- 1 rjohnson guru 392 Aug 17 1998 ioapic-2.1.115-A -rw-r--r-- 1 rjohnson guru 30547 May 14 1998 irq.c drwxr-xr-x 2 rjohnson guru 4096 Jan 26 1997 jboot -rw-r--r-- 1 rjohnson guru 675 Nov 29 1995 jboot-1.13.lsm -rw-r--r-- 1 rjohnson guru 8263 Oct 4 1998 jitter.c -rwxr-xr-x 1 rjohnson guru 79 Jun 26 1998 keepalive.sh -rw-r--r-- 1 rjohnson guru 24348 Dec 10 1997 latierra.c -rw-r--r-- 1 rjohnson guru 267 Aug 11 1997 lawsuit drwxr-xr-x 2 rjohnson guru 4096 Mar 19 1997 logs -rwxr-xr-x 1 rjohnson guru 5904 Nov 9 1998 lookups -rw-r--r-- 1 rjohnson guru 1533 Nov 9 1998 lookups.c -rw-r--r-- 1 rjohnson guru 1339467 Aug 31 1998 ls-lR -rw-r--r-- 1 rjohnson guru 1239040 Jul 28 1998 m4-1.4.tar drwx------ 2 rjohnson guru 4096 Oct 5 1999 mail -rwxr-xr-x 1 rjohnson guru 411945 Nov 14 1997 main -rwxr-xr-x 1 rjohnson guru 469 Dec 16 1996 make_bfloppy -rwxr-xr-x 1 rjohnson guru 474 Sep 16 1996 make_boot -rwxr-xr-x 1 rjohnson guru 460 Jun 19 1998 make_floppy -rw-r--r-- 1 rjohnson guru 1175 Mar 20 1998 mem.c -rw-r--r-- 1 rjohnson guru 5835 Nov 9 1998 messages -rwxr-xr-x 1 rjohnson guru 36 Sep 12 1996 modem -rw-r--r-- 1 rjohnson guru 28084 Oct 3 1998 mptable.c -rw-r--r-- 1 rjohnson guru 5715 Aug 22 1996 mwmrc -rwxr-xr-x 1 rjohnson guru 1312 Jan 28 1997 new_backup -rw-r--r-- 1 rjohnson guru 2343 Apr 21 1998 operating.systems -rwxr-xr-x 1 rjohnson guru 718 Jan 2 1996 plot drwxr-xr-x 3 rjohnson guru 4096 Jan 26 1997 pppdc drwxr-xr-x 2 rjohnson guru 4096 Jan 26 1997 ptools -rw-r--r-- 1 rjohnson guru 3955 Jun 18 1998 reboot.c -rwxr-xr-x 1 rjohnson guru 214 Nov 6 1997 restore -rwxr-xr-x 1 rjohnson guru 2185 Jan 6 1997 rights.h -rwxr-xr-x 1 rjohnson guru 7293 Dec 26 1996 showserv -rw-r--r-- 1 rjohnson guru 1765 Aug 13 2001 smile.sh drwxr-xr-x 19 rjohnson guru 4096 Apr 17 1998 test -rw-r--r-- 1 rjohnson guru 788903 Jul 22 1998 test.cdcolor -rwxr-xr-x 1 rjohnson guru 5019 May 25 1998 timer -rw-r--r-- 1 rjohnson guru 1247 Oct 3 1998 timer.c drwxr-xr-x 10 rjohnson guru 4096 Nov 18 1998 tools -rw-r--r-- 1 rjohnson guru 471 Oct 9 16:09 typescript -rw-r--r-- 1 rjohnson guru 426 Jun 24 1998 udelay.patch -rwxr-xr-x 1 rjohnson guru 4137 Dec 5 1996 unix2world -rwxr-xr-x 1 rjohnson guru 171 Jan 27 1998 update_host -rwxr-xr-x 1 rjohnson guru 84707 Jul 10 1998 utility.asm -rwxr-xr-x 1 rjohnson guru 16819 Sep 25 1998 vl12ct.asm -rwxr-xr-x 1 rjohnson guru 3845 Dec 5 1996 world2unix -rwxr-xr-x 1 rjohnson guru 8141 Oct 9 16:08 xxx
All the deleted files, with the correct path(s), are now in the top directory file the file-system ../lost+found directory. They are still owned by the original user, still subject to the same quota. The disk space can't run out because you have simply moved files that didn't exceed the disk space before they were moved.
All one needs is a compile-time switch to enable the following:
If the lost+found directory is writable by the current user, blindly create any paths that may already exist, using the ../lost+found/~ directory as the root, and rename() the file you think you are unlink()ing unless the file exists in /tmp, or is a sym-link, special file, pipe, socket or device.
To enable such a function (after modifing the C library), just make lost+found world-writable. The permissions of its contents remain unchanged as does the owner. If the owner didn't want others in the universe to read his resume.txt, or porno.jpg, they still can't read it and they can't overwrite it.
The sysadmin deletes the contents (probably via crond), every time it has been backed up.
There are lots of advantages to this scheme. The most notable being that it is transparent to normal users. Database programs that do a lot of deleting temporary files that don't exist in /tmp, need to be have their default directory on a file-system with lost+found not world writable.
Cheers,
Dick Johnson
Penguin : Linux version 2.4.1 on an i686 machine (797.90 BogoMips).
111,111,111 * 111,111,111 = 12,345,678,987,654,321
Quoted 7 lines · Richard B. Johnson
All the deleted files, with the correct path(s), are now in the top directory file the file-system ../lost+found directory. They are still owned by the original user, still subject to the same quota. The disk space can't run out because you have simply moved files that didn't exceed the disk space before they were moved.
Ummm... it never occurred to you why someone would delete files in the first place?
-hpa
Quoted 11 lines · H. Peter Anvin
Quoted 7 lines · Richard B. Johnson
All the deleted files, with the correct path(s), are now in the top directory file the file-system ../lost+found directory. They are still owned by the original user, still subject to the same quota. The disk space can't run out because you have simply moved files that didn't exceed the disk space before they were moved.
Quoted 4 lines
Ummm... it never occurred to you why someone would delete files in the first place?
-hpa
Yep. They probably thought they had changed directory to some scratch file-system and they were cleaning it up! Most wildcard deletions are truly accidental like this :
ls .c>* # woops, made a file called '*', I'll fix it..
rm * # Good, now back to work...
ls *.c >files
Cheers,
Dick Johnson
Penguin : Linux version 2.4.1 on an i686 machine (797.90 BogoMips).
111,111,111 * 111,111,111 = 12,345,678,987,654,321
Quoted 8 lines · Richard B. Johnson
johnson[4]$ ls -laR
/home/users/lost+found/rjohnsontotal 5428drwxr-xr-x 17 rjohnson guru 4096 Feb 13 16:54 . drwxrwxrwx 21 root root 4096 Feb 26 05:34 ..-rw-r--r-- 1 rjohnson guru 6 Oct 15 1998 .XF86_S3
:
:
-rwxr-xr-x 1 rjohnson guru 8141 Oct 9 16:08 xxx
A shorter example would have sufficed...
Quoted 4 lines
All the deleted files, with the correct path(s), are now in the top directory file the file-system ../lost+found directory. To enable such a function (after modifing the C library), just make lost+found world-writable.
Making lost+found world-writable is a terrible idea (even world readable is bad) because it exposes potentially sensitive files to the world if it happens that fsck moves a file there after some filesystem problem. It may be that the sensitive file was in a secure directory, and now it is world readable.
I would stronly suggest using some other directory for this purpose, since if you are changing lost+found to be world writable, you could just as easily do "mkdir .undelete".
Cheers, Andreas
--
Andreas Dilger
http://sourceforge.net/projects/ext2resize/
http://www-mddsp.enel.ucalgary.ca/People/adilger/
Quoted 9 lines · Andreas Ferber
Quoted 3 lines · Mike Fedyk
Basically, it would only move the files to the undelete area if the link count == 1. If you just decremented the link, then
unlink()in glibc would work as it does now.Always racy if done in userspace, unless you introduce a centralised "unlink daemon" (hope no glibc developer reads that, they might be tempted to implement such an abomination...):
Quoted 8 lines
proc1 proc2
--------------------stat() stat() unlink() unlink()*kaboom*, blackhole opens, file is gone.
I had previously suggested to Mike (but he seems to have missed it) that you should _always_ move files to the undelete area, regardless of how many links there are. This avoids all of the races, doesn't increase space usage at all (because the undelete area is always in the same fs), and is actually better for the users (because "rm <file>", "unrm <file>" will always work even if <file> has multiple links).
Cheers, Andreas
--
Andreas Dilger
http://sourceforge.net/projects/ext2resize/
http://www-mddsp.enel.ucalgary.ca/People/adilger/
Quoted 8 lines · Mike Fedyk
Quoted 7 lines · Rik van Riel
Your idea should work on deletion, when the inode were about to be destroyed, but ...
Quoted 3 lines
It would only call chown/chgrp on the files *inside* the undelete dir, and user,group,etc would have to be accounted for in another way. Am I going in the wrong direction?
Quoted 6 lines
Quoted 2 lines
... of course, there still is the problem of hard links.
I had considered hard links. Take a look at my another message from me in this thread and see Daniel's response to it.
Correction, that was Andreas Dilger that replied offline, and has been posted by me with his permission.
And now the idea of chgrp/chown is out of the question...
Mike
Quoted 4 lines · Richard B. Johnson
All the deleted files, with the correct path(s), are now in the top directory file the file-system ../lost+found directory. They are still owned by the original user, still subject to the same quota.
And what about:
- Luser rm's "foo.c"
- Luser starts working on new version of "foo.c"
- Luser recognizes, that the old version was better
- Luser rm's new "foo.c"
- Luser tries to unrm the old "foo.c" -> *bang*
Trust me, there /will/ be a luser who tries to do it this way. If teaching lusers were enough, you'd have no need for an unrm at all. Everyone would be using version control for important data, and everything would be fine.
Quoted 2 lines
The disk space can't run out because you have simply moved files that didn't exceed the disk space before they were moved.
But a user will end up unable to /free/ any diskspace. User tries something, generates a /huge/ error log filling up the quota/disk, oops, has to call sysadmin before work can go on... Five minutes later, the fix just tried didn't work, oops, has to call admin again, and so on. Do you /really/ want this?
And how do you want to handle temp files? If you don't exclude them from undeletion, they will fill up your diskspace soon. For the moment I can't think of any mechanism that identifies temp files reliably (without API changes).
Quoted 1 line
All one needs is a compile-time switch to enable the following:
And a system wide configurable switch, and a user configurable switch and so on.
Undeletion has /many/ implications, did you think through all of them?
Just as a personal note: personally I would simply /refuse/ to work on a system where I end up unable to delete even files I /own/, or at least I would end up implementing my own way of deleting files which circumvents undeletion (there will /always/ be a way to do it).
If your employer didn't expressively forbid you to keep private data on your work account, you are allowed to do so, at least here in germany, and you can sue your employer if he takes actions to look into your private data without informing you /before/ doing it (taken strictly, if you are allowed to keep private data on your work account, you even have to be informed explicitly that the data may be backuped and recovered later from backup tapes). So in the end, undeletion is also a matter of privacy, and the ability to undelete may even pose legal problems on a company.
Andreas
Signature · 3 lines
Andreas Ferber - dev/consulting GmbH - Bielefeld, FRG
---------------------------------------------------------
+49 521 1365800 - [email protected] - www.devcon.netQuoted 7 lines · Andreas Ferber
Quoted 4 lines · Richard B. Johnson
All the deleted files, with the correct path(s), are now in the top directory file the file-system ../lost+found directory. They are still owned by the original user, still subject to the same quota.
Quoted 7 lines
And what about:
- Luser rm's "
foo.c"
- Luser starts working on new version of "foo.c"
- Luser recognizes, that the old version was better
- Luser rm's new "foo.c"
- Luser tries to unrm the old "foo.c" -> *bang*
As stated in a later post, a basic versioning could be implemented where a ".1" or similar could be added to the end of the file. This could definitely cause problems with the disk filling up though (see below).
Quoted 5 lines
But a user will end up unable to /free/ any diskspace. User tries something, generates a /huge/ error log filling up the quota/disk, oops, has to call sysadmin before work can go on... Five minutes later, the fix just tried didn't work, oops, has to call admin again, and so on. Do you /really/ want this?
A special binary could be created that would be able to read the contents of the .undelete directory based on the uid of the user running it. The binary could remove the file if desired. The user space implementation would require more thought, but there are several possibilities that I can think of.
Quoted 5 lines
And how do you want to handle temp files? If you don't exclude them from undeletion, they will fill up your diskspace soon. For the moment I can't think of any mechanism that identifies temp files reliably (without API changes).
This is a good point. The kernel space code could just filter out anything being removed from /tmp (and the various .undelete directories). But there would be other files. A low priorotiy cron job or daemon could remedy the others (being configurable in user space). it could also remove files with a high version number if space was low.
Quoted 5 lines
Quoted 1 line
All one needs is a compile-time switch to enable the following:
And a system wide configurable switch, and a user configurable switch and so on.
I am not sure what this is referencing. I was now thinking that a kernel compile option (CONFIG_UNDELETE) could be used for those who wanted this, and those that didn't could leave it out. And for those that do, but only on some mountpoints, the sysadmin could simply have the .undelete just on those mounts that needed the undelete feature. The kernel code would simply remove the file like normal if the .undelete directory was not present.
Quoted 2 lines
Undeletion has /many/ implications, did you think through all of them?
I REALLY doubt it! :-) But it is fun getting into the kernel code and trying some stuff out.
Quoted 6 lines
Just as a personal note: personally I would simply /refuse/ to work on a system where I end up unable to delete even files I /own/, or at least I would end up implementing my own way of deleting files which circumvents undeletion (there will /always/ be a way to do it).
Quoted 10 lines
If your employer didn't expressively forbid you to keep private data on your work account, you are allowed to do so, at least here in germany, and you can sue your employer if he takes actions to look into your private data without informing you /before/ doing it (taken strictly, if you are allowed to keep private data on your work account, you even have to be informed explicitly that the data may be backuped and recovered later from backup tapes). So in the end, undeletion is also a matter of privacy, and the ability to undelete may even pose legal problems on a company.
Valid points which leads me to think that a special suid/sgid user space program would be needed. The .undelete directory would have chmod 700 owned by root (or possibly 770 with 'chown root.undelete' ownership).
Jamie
Signature · 3 lines
Email: [email protected] GPG/PGP ID: 26384A3A Fingerprint: D9FF DF4A 2D46 A353 A289 E8F5 AA75 DCBE 2638 4A3A
Quoted 9 lines · Andreas Ferber
Quoted 2 lines · Richard B. Johnson
The disk space can't run out because you have simply moved files that didn't exceed the disk space before they were moved.
But a user will end up unable to /free/ any diskspace. User tries something, generates a /huge/ error log filling up the quota/disk, oops, has to call sysadmin before work can go on... Five minutes later, the fix just tried didn't work, oops, has to call admin again, and so on. Do you /really/ want this?
This is just being silly. Obviously the user will be able to delete files from the .undelete directory, and a daemon to do automatic cleanup was also proposed. Thinking anything else is just being obtuse.
You could have the unlink() wrapper check that there is still some free space/quota when it is doing the move, and if not it deletes old files until there is free space/quota. The daemon just does this for you in the background to avoid slowing things down.
Cheers, Andreas
--
Andreas Dilger
http://sourceforge.net/projects/ext2resize/
http://www-mddsp.enel.ucalgary.ca/People/adilger/
Quoted 9 lines · Andreas Ferber
Quoted 4 lines · Richard B. Johnson
All the deleted files, with the correct path(s), are now in the top directory file the file-system ../lost+found directory. They are still owned by the original user, still subject to the same quota.
And what about:
Quoted 7 lines
- Luser rm's "
foo.c"
- Luser starts working on new version of "foo.c"
- Luser recognizes, that the old version was better
- Luser rm's new "foo.c"
- Luser tries to unrm the old "foo.c" -> *bang*Trust me, there /will/ be a luser who tries to do it this way.
Yes, users will do that. And this problem is easily solved by keeping a copy of each deleted file based on the date, so you can have several versions of the same file in the undeleted dir.
Quoted 4 lines
If
teaching lusers were enough, you'd have no need for an unrm at all. Everyone would be using version control for important data, and everything would be fine.
Not everyone works with text-only formats.
Quoted 9 lines
Quoted 2 lines
The disk space can't run out because you have simply moved files that didn't exceed the disk space before they were moved.
But a user will end up unable to /free/ any diskspace. User tries something, generates a /huge/ error log filling up the quota/disk, oops, has to call sysadmin before work can go on... Five minutes later, the fix just tried didn't work, oops, has to call admin again, and so on. Do you /really/ want this?
The undelete daemon will have to be quota aware. The unfortunate side affect is that if the user is close to their limit, undelete is effectively disabled because there won't be enough space left in their quota to keep the deleted file.
The only way for undelete to fill up your drives is for the undelete daemon to crash and die. This can be avoided by having init monitor it... or whatever other mechanism you want...
Quoted 5 lines
And how do you want to handle temp files? If you don't exclude them from undeletion, they will fill up your diskspace soon. For the moment I can't think of any mechanism that identifies temp files reliably (without API changes).
The temp files will only make other older files in the undelete dir be purged...
Quoted 7 lines
Quoted 1 line
All one needs is a compile-time switch to enable the following:
And a system wide configurable switch, and a user configurable switch and so on.
Undeletion has /many/ implications, did you think through all of them?
No, but this thread has brought up many considerations.
Quoted 6 lines
Just as a personal note: personally I would simply /refuse/ to work on a system where I end up unable to delete even files I /own/, or at least I would end up implementing my own way of deleting files which circumvents undeletion (there will /always/ be a way to do it).
Yes, statically compiled binaries would work, a library preload, etc.
Quoted 10 lines
If your employer didn't expressively forbid you to keep private data on your work account, you are allowed to do so, at least here in germany, and you can sue your employer if he takes actions to look into your private data without informing you /before/ doing it (taken strictly, if you are allowed to keep private data on your work account, you even have to be informed explicitly that the data may be backuped and recovered later from backup tapes). So in the end, undeletion is also a matter of privacy, and the ability to undelete may even pose legal problems on a company.
That is a configuration issue. All the implementation will need to do is be configurable enough to follow the local policy.
Mike
Hi
Quoted 7 lines
Quoted 4 lines
All the deleted files, with the correct path(s), are now in the top directory file the file-system ../lost+found directory. They are still owned by the original user, still subject to the same quota.
And what about:
Quoted 8 lines
- Luser rm's "
foo.c"
- Luser starts working on new version of "foo.c"
- Luser recognizes, that the old version was better
- Luser rm's new "foo.c"
- Luser tries to unrm the old "foo.c" -> *bang*Trust me, there /will/ be a luser who tries to do it this way. If teaching lusers were enough, you'd have no need for an unrm at all.
You don't consider me a luser, right?
Okay, and I *did* need unrm few times. Few examples:
/dev# rm sbpcd * (simple typo, recovered by immediate powerdown + fsck)
/big$ mp1enc > samotari.mpg (oops, I did it twice, second time by mistake, and
powerswitch was too far away to make it in time)
So yes, unrm is usefull. And it would be even more usefull if it recovered truncated files, too. How many times did you do > instead of >>? I did that mistake many times, its just easy..
Pavel
Signature · 2 lines
Philips Velo 1: 1"x4"x8", 300gram, 60, 12MB, 40bogomips, linux, mutt, details at http://atrey.karlin.mff.cuni.cz/~pavel/velo/index.html.
Quoted 14 lines · Pavel Machek
Hi
Quoted 13 lines
Quoted 4 lines
All the deleted files, with the correct path(s), are now in the top directory file the file-system ../lost+found directory. They are still owned by the original user, still subject to the same quota.
And what about:
- Luser rm's "
foo.c"
- Luser starts working on new version of "foo.c"
- Luser recognizes, that the old version was better
- Luser rm's new "foo.c"
- Luser tries to unrm the old "foo.c" -> *bang*
Quoted 4 lines
Quoted 2 lines
Trust me, there /will/ be a luser who tries to do it this way. If teaching lusers were enough, you'd have no need for an unrm at all.
You don't consider me a luser, right?
Nope.
Some newbees think that Windoze 'send-to-the-wastebasket' is a kernel- level "safe-delete". It's just some ^&$)##*@*) program that slows most of us down.
Even Windows/Professional/2000 (NT) developers knew that it was garbage. If you've figured out how to get to the CMD prompt, just type:
cd \
rm -r *.*
| | |______ They still have dots | |__________ Yes, even "folders" <coff, coff> |_____________ What do you expect for a stolen OS? Yes, `rm` instead of
del, following the Unix pathname tradition.
Cheers,
Dick Johnson
Penguin : Linux version 2.4.18 on an i686 machine (799.53 BogoMips).
Bill Gates? Who?
Quoted 9 lines · Pavel Machek
Quoted 9 lines
And what about:
- Luser rm's "
foo.c"
- Luser starts working on new version of "foo.c"
- Luser recognizes, that the old version was better
- Luser rm's new "foo.c"
- Luser tries to unrm the old "foo.c" -> *bang*
Quoted 3 lines
Quoted 2 lines
Trust me, there /will/ be a luser who tries to do it this way. If teaching lusers were enough, you'd have no need for an unrm at all.
You don't consider me a luser, right?
No, not really. ;-)
Quoted 3 lines
Okay, and I *did* need unrm few times. Few examples:
/dev# rm sbpcd * (simple typo, recovered by immediate powerdown + fsck)
% rm sbpcd *
zsh: sure you want to delete all the files in /dev [yn]? n
rm: remove write-protected file `sbpcd'? n
%
Quoted 6 lines
/big$ mp1enc > samotari.mpg (oops, I did it twice, second time by mistake, and
powerswitch was too far away to make it in time)So yes, unrm is usefull. And it would be even more usefull if it recovered truncated files, too. How many times did you do > instead of >>? I did that mistake many times, its just easy..
% cat > foo
zsh: file exists: foo
% cat >| foo
[Now zsh is quiet. It even "fixes up" the history if you want so, so that you can simply press "Up"+"Enter" if you really want to overwrite the file]
Surely, protection against typos etc. has its value. But do it at the place where does typos happen (ie. at the shell prompt), not by messing with lowlevel stuff like the unlink syscall, which
a) catches only very few ways of destroying a files contents
b) poses a /great/ deal of complexity on you (like having to identify tempfiles, managing disk space etc.)
With some simple checks of the commandline you can catch many of the common typos which end up destroying data. And it comes at nearly no cost, is much more flexible and avoids any problems with tempfiles, quota etc. in the first place.
And for the rest, which are not catched by those checks, that's what backups are for.
Honestly, I also had a few moments where I wished I had an unrm command available, and where it required a bit of work to fix up the mess. But those situations are not that common that I really want to have to manage a full fledged undeletion system all the time, and work around the problems that it imposes.
Also, take the "human factor" into account: Users /will/ get more sloppy with regards to thinking before typing, checking commandlines before hitting enter etc., if they always have a "safety net" behind them. And it /will/ bite them in the ass once they are in a situation (different system, root, ...) where those safety nets are not available.
This is also the reason why I don't like those "rm -i" aliases some distros are setting by default. I have /seen/ luser typing "rm *" when they just wanted to delete some of the files in a directory, because they were used to rm asking for every file if they want to delete it. Now guess what happens if such a luser is dropped into an environment where those aliases are not set...
And yes, I realize that for example those zsh examples above are also somewhat into the same direction. But they are different from the "rm -i" example above in that they don't change the semantics of a command line, so that you are still obliged into checking the command line /before/ submitting it, otherwise you get those annoying questions and still have to resubmit the command.
Andreas
Signature · 3 lines
Andreas Ferber - dev/consulting GmbH - Bielefeld, FRG
---------------------------------------------------------
+49 521 1365800 - [email protected] - www.devcon.netHi!
Quoted 8 lines
Quoted 3 lines
Okay, and I *did* need unrm few times. Few examples:
/dev# rm sbpcd * (simple typo, recovered by immediate powerdown + fsck)
% rm sbpcd *
zsh: sure you want to delete all the files in /dev [yn]? n
rm: remove write-protected file `sbpcd'? n
%
Is it decent enough to do rm -rf /usr/src/linux without asking?
Quoted 7 lines
Quoted 6 lines
/big$ mp1enc > samotari.mpg (oops, I did it twice, second time by mistake, and
powerswitch was too far away to make it in time)So yes, unrm is usefull. And it would be even more usefull if it recovered truncated files, too. How many times did you do > instead of >>? I did that mistake many times, its just easy..
Quoted 6 lines
% cat > foo
zsh: file exists: foo
% cat >| foo
[Now zsh is quiet. It even "fixes up" the history if you want so, so that you can simply press "Up"+"Enter" if you really want to overwrite the file]
This would not help. mp1enc has nasty habit of randomly not starting, so I had to repeat the command above few times. What was wrong was that I repeated it one time too many times.
Quoted 3 lines
Surely, protection against typos etc. has its value. But do it at the place where does typos happen (ie. at the shell prompt), not by messing with lowlevel stuff like the unlink syscall, which
Not sure it is possible. What you want is safety nets in every app.
Quoted 3 lines
a) catches only very few ways of destroying a files contents
b) poses a /great/ deal of complexity on you (like having to identify tempfiles, managing disk space etc.)
Really? ext2 *already* is undelete capable. mc: cd /#undel:hda3. So, where's that great complexity?
Pavel
Signature · 2 lines
Casualities in World Trade Center: ~3k dead inside the building, cryptography in U.S.A. and free speech in Czech Republic.
Quoted 11 lines · Martin Dalecki
Quoted 7 lines
Quoted 4 lines
For the educated user it was always a pain
in the you know where, to constantly run out of quota space due to file versioning.Ahh, so we'd need to chown the files to root (or a configurable user and group) to get around the quota issue.
Welcome to my kill-file. This just shows that you don't even have basic background.
Ok guys,
Maybe you'll respect the ideas coming from Andreas Dilger, as some of you obviously don't like some of my ideas.
Here goes:
Quoted 7 lines · Mike Fedyk
I don't think we need anything very complicated either.
Here's what it probably should do *in libc*:
o trap unlink calls
o if link count >= 2 then act normally
o if link count == 1 then move file (including directory structure from mount point to $mount_point/.deleted/$path/file)
Well, my idea on this is that you don't check the link count at all. Why should it be special if I delete /home/adilger/foo, when it is also linked to /home/adilger/bar? If I want to undelete /home/adilger/foo, then I should be able to. Since it is on the same filesystem, it doesn't take any more space in the filesystem to keep this link, and in fact avoids the race condition entirely.
Only the undelete daemon would do real deletions, everything else would _always_ do something like:
base=echo $file | sed "s#$mntpt/##"
mv $file $mntpt/.undelete/$base[.timestamp/username/etc]
Quoted 9 lines
The undelete daemon (undeleted) would do:
o monitor how full the various deleted directories are (always keep some percentage empty to allow new files to be deleted without overflowing the space configured for undelete)
o enforce configurable setting for how much space .undelete will hold
o delete any single file that will not fit in .undelete's space no matter how new it is
o any other sysadmin notification type of things
Quoted 5 lines
Should the glibc routine interact directly with the undelete daemon so that the case of a lot of deletion of large files will be handled faster? Otherwise, if you delete a lot of files, df won't show the free space getting bigger until undeleted did a rescan of it's undelete dirs and freed the old deleted files.
Well, my take on this would be that since we are only ever moving files over to .undelete, then we are never using up more space than what we are already using, so we do not need to ever synchronously free space to delete a file.
We could always have unlink() write to a socket (e.g. .undelete/daemon or so) telling it which file it just deleted, and the daemon sleeps on this socket until woken to see if it needs to do cleanup. Having the name of the recently deleted file passed in allows it to do things like clean up old copies of that file quickly, etc.
Note also that the "unrm" command would need some smarts so that it can match against variations of $base, because programs may rename a file, make a new copy, and then delete the renamed file (e.g. emacs, vi, etc). It would probably do some sort of regexp on the basename() of the file.
Cheers, Andreas
--
Andreas Dilger
http://sourceforge.net/projects/ext2resize/
http://www-mddsp.enel.ucalgary.ca/People/adilger/
A lot of talk about "daemons" ... seems overkill to me. Any reason not to let each user do this on their own? I've got an rm/unrm program that just stores the "rescued" files in your home directory for a period of time based on the either the name or pathname.
It is not fully "hardened" -- it is possible to conceive of filenames and filename/directory name pairs which will confuse it -- but it is certainly functional for 98% of cases, which has been good enough for my personal use.
Disadvantages:
* the mtime is purposefully changed when the file is deleted to make it easy to tell when a file was "deleted", so you lose that information.
* Directory modes and owners are not maintained.
* File ownerships are maintained only to the extent that a hard link allows you to. If you couldn't do a hard link (cross file systems) then the "saved" file will be owned by you unless you are root.
* There's no way to say "save until X amount is saved then really delete" (whether "X amount" is %age of file system, a fixed amount, or some other criteria). The only criteria is age of deletion.
* (true) deletions are only attempted when a subsequent file is rm'ed. So it's conceivable to delete 300Mb of data that is scheduled to disappear in 30 seconds but not have it go away for three days, because you left town for the weekend and neither you nor your cron scripts ran "rm" again until Monday.
If this sort of program is useful to folks, I'm more than happy to provide it. No doubt it could be enhanced to address some of the shortcomings above. I just got tired many years ago of accidentally typing "rm * .o" and only finding out my typo when I saw "rm: .o: file not found" and it has, for the most part, served with minor modifications for many years.
Rick
Quoted 4 lines · Rick Lindsley
A lot of talk about "daemons" ... seems overkill to me. Any reason not to let each user do this on their own? I've got an rm/unrm program that just stores the "rescued" files in your home directory for a period of time based on the either the name or pathname.
Well, the daemon could be used to do only background cleanup. You could always have the unlink wrapper do cleanup synchronously all the time, but this might make some deletes slow. If you weren't running a daemon, then eventually the unlink wrapper would do all of the cleanups for you. However, it would not be possible with the unlink wrapper for one user to force cleanup of another user's deleted files if the filesystem is running out of space. I think having a daemon do this is far safer than making "rm" an suid program (eek).
The other problem with moving files to the owner's home directory instead of just renaming it to a per-filesystem .undelete directory is that this takes a long time if they are on different filesystems, or (heaven forbid) that the home directory is NFS mounted and you just deleted a 500MB GIMP swap file from /tmp.
Cheers, Andreas
--
Andreas Dilger
http://sourceforge.net/projects/ext2resize/
http://www-mddsp.enel.ucalgary.ca/People/adilger/
Quoted 7 lines · Dan Maas
Quoted 3 lines
but I don't want a Netware filesystem running on Linux, I want a *native* Linux filesystem (i.e. ext3) that has the ability to queue deleted files should I configure it to.
Rather than implementing this in the filesystem itself, I'd first try writing a libc shim that overrides
unlink(). You could copy files to safety, or do anything else you want, before they actually get deleted...
Yep, more portable.
Now the question is: Is there already something written?
Quoted 10 lines · Mike Fedyk
Quoted 7 lines · Dan Maas
Quoted 3 lines
but I don't want a Netware filesystem running on Linux, I want a *native* Linux filesystem (i.e. ext3) that has the ability to queue deleted files should I configure it to.
Rather than implementing this in the filesystem itself, I'd first try writing a libc shim that overrides
unlink(). You could copy files to safety, or do anything else you want, before they actually get deleted...Yep, more portable.
But it only works if everything get linked with the new library.
Quoted 7 lines
Now the question is: Is there already something written?
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
Quoted 1 line · In article <02022518330103.01161@grumpersII> you wrote:
But it only works if everything get linked with the new library.
Nope, just put the new Lib into /etc/ld.so.preload
Actually there is a unlink package out there, which does exactly this.
Greetings
Bernd
Followup to: <02022518330103.01161@grumpersII> By author: Tom Rauschenbach <[email protected]> In newsgroup: linux.dev.kernel
Quoted 7 lines · Mike Fedyk
Quoted 5 lines · Dan Maas
Quoted 4 lines
Quoted 3 lines
but I don't want a Netware filesystem running on Linux, I want a *native* Linux filesystem (i.e. ext3) that has the ability to queue deleted files should I configure it to.
Quoted 8 lines
Quoted 5 lines
Quoted 3 lines
Rather than implementing this in the filesystem itself, I'd first try writing a libc shim that overrides
unlink(). You could copy files to safety, or do anything else you want, before they actually get deleted...Yep, more portable.
But it only works if everything get linked with the new library.
What's a lot worse is that the kernel cannot chose to garbage-collect it. One reason to put undelete in the kernel is that that files in limbo can be reclaimed as the disk space is needed for other users, and you don't risk getting ENOSPC due to the disk being full with ghosts.
-hpa
Signature · 3 lines
<[email protected]> at work, <[email protected]> in private! "Unix gives you enough rope to shoot yourself in the foot." http://www.zytor.com/~hpa/puzzle.txt <[email protected]>
Quoted 16 lines · H. Peter Anvin
Followup to: <02022518330103.01161@grumpersII> By author: Tom Rauschenbach <[email protected]> In newsgroup: linux.dev.kernelQuoted 13 lines · Mike Fedyk
Quoted 10 lines · Dan Maas
Quoted 7 lines
Quoted 3 lines
but I don't want a Netware filesystem running on Linux, I want a *native* Linux filesystem (i.e. ext3) that has the ability to queue deleted files should I configure it to.
Rather than implementing this in the filesystem itself, I'd first try writing a libc shim that overrides
unlink(). You could copy files to safety, or do anything else you want, before they actually get deleted...Yep, more portable.
Quoted 9 lines
Quoted 2 lines
But it only works if everything get linked with the new library.
What's a lot worse is that the kernel cannot chose to garbage-collect it. One reason to put undelete in the kernel is that that files in limbo can be reclaimed as the disk space is needed for other users, and you don't risk getting ENOSPC due to the disk being full with ghosts.
True, and it could to tricks like listing space used for undelete as "free" in addition to dynamic garbage collection.
Though, with a daemon checking the dirs often, or using Daniel's idea of a socket between unlink() in glibc and an undelete daemon could work quite similairly.
Also, there wouldn't be any interaction with filesystem internals, and userspace would probably work better with non-posix type filesystems (vfat, hfs, etc) too.
IOW, there seems to be little gain to having an kernelspace solution.
Mike
Quoted 3 lines · Mike Fedyk
Though, with a daemon checking the dirs often,
Can you say "race condition?"
-hpa
Quoted 7 lines
True, and it could to tricks like listing space used for undelete as "free" in addition to dynamic garbage collection.
Though, with a daemon checking the dirs often, or using Daniel's idea of a socket between
unlink()in glibc and an undelete daemon could work quite similairly.
Quoted 6 lines
Also, there wouldn't be any interaction with filesystem internals, and userspace would probably work better with non-posix type filesystems (vfat, hfs, etc) too.
IOW, there seems to be little gain to having an kernelspace solution.
IMNSHO everyone thinking about undeletion in Linux should be sentenced to 1 year of VMS usage and asked then again if he still think's that it's a good idea...
Quoted 7 lines · H. Peter Anvin
Quoted 3 lines · Mike Fedyk
Though, with a daemon checking the dirs often,
Can you say "race condition?"
Uhh, no.
You have a configurable size for the undelete dirs and you delete a file. Now, that file gets moved to $mountpoint/.undelete. The daemon gets notified through a socket, and it can check to see if it needs to delete any older deleted files to make sure .undelete doesn't get bigger than configured.
We're only scanning the dirs upon daemon startup (reminds me of quota... ;), and all other daemon actions are triggered by unlink() writing to a socket. The worst thing that can happen is not seeing your free space immediately, but a few seconds later.
Mike
Quoted 11 lines · Martin Dalecki
Quoted 11 lines
True, and it could to tricks like listing space used for undelete as "free" in addition to dynamic garbage collection.
Though, with a daemon checking the dirs often, or using Daniel's idea of a socket between
unlink()in glibc and an undelete daemon could work quite similairly.Also, there wouldn't be any interaction with filesystem internals, and userspace would probably work better with non-posix type filesystems (vfat, hfs, etc) too.
Quoted 6 lines
Quoted 2 lines
IOW, there seems to be little gain to having an kernelspace solution.
IMNSHO everyone thinking about undeletion in Linux should be sentenced to 1 year of VMS usage and asked then again if he still think's that it's a good idea...
Can you describe the pitfalls that VMS went through so we can aviod the problems?
I haven't had the chance to use VMS, and don't have any hardware to try it out on. Also, just because one implementation was bad (even long ago, and unix was considered bad then too... ;) does it mean the entire idea is bad.
Mike
Followup to: <[email protected]> By author: Mike Fedyk <[email protected]> In newsgroup: linux.dev.kernel
Quoted 9 lines
Uhh, no.
You have a configurable size for the undelete dirs and you delete a file. Now, that file gets moved to $mountpoint/.undelete. The daemon gets notified through a socket, and it can check to see if it needs to delete any older deleted files to make sure .undelete doesn't get bigger than configured.
Quoted 5 lines
We're only scanning the dirs upon daemon startup (reminds me of quota... ;), and all other daemon actions are triggered by
unlink()writing to a socket. The worst thing that can happen is not seeing your free space immediately, but a few seconds later.
Hence race condition. Also, the solution to hard-reserve space seems to fundamentally defeat the purpose (IMO).
-hpa
Signature · 3 lines
<[email protected]> at work, <[email protected]> in private! "Unix gives you enough rope to shoot yourself in the foot." http://www.zytor.com/~hpa/puzzle.txt <[email protected]>
Quoted 21 lines · Mike Fedyk
Quoted 11 lines · Martin Dalecki
Quoted 11 lines
True, and it could to tricks like listing space used for undelete as "free" in addition to dynamic garbage collection.
Though, with a daemon checking the dirs often, or using Daniel's idea of a socket between
unlink()in glibc and an undelete daemon could work quite similairly.Also, there wouldn't be any interaction with filesystem internals, and userspace would probably work better with non-posix type filesystems (vfat, hfs, etc) too.
Quoted 7 lines
Quoted 3 lines
IOW, there seems to be little gain to having an kernelspace solution.
IMNSHO everyone thinking about undeletion in Linux should be sentenced to 1 year of VMS usage and asked then again if he still think's that it's a good idea...
Quoted 6 lines
Can you describe the pitfalls that VMS went through so we can aviod the problems?
I haven't had the chance to use VMS, and don't have any hardware to try it out on. Also, just because one implementation was bad (even long ago, and unix was considered bad then too... ;) does it mean the entire idea is bad.
Yes I can. The main problem is that most people think that undeletion is a magical way of getting around stiupid users. But the fact is that the very same users very quickly adapt to the the presence of undeletion facilities. And guess whot? They will expect you to instantly recover allways a version of "this" file from the "stone age". So the pain for the sysadmin will certainly not be decreased. Quite contrary for what he expects. For the educated user it was always a pain in the you know where, to constantly run out of quota space due to file versioning.
Quoted 4 lines
So the pain for the sysadmin will certainly not be decreased. Quite contrary for what he expects. For the educated user it was always a pain in the you know where, to constantly run out of quota space due to file versioning.
Netware was somewhat more sensible. Digging out an old file took running a tool which had a little irritation factor. In addition stuff got automatically recycled over time and as disk space was needed.
Quoted 10 lines · Martin Dalecki
Quoted 6 lines · Mike Fedyk
Can you describe the pitfalls that VMS went through so we can aviod the problems?
I haven't had the chance to use VMS, and don't have any hardware to try it out on. Also, just because one implementation was bad (even long ago, and unix was considered bad then too... ;) does it mean the entire idea is bad.
Yes I can. The main problem is that most people think that undeletion is a magical way of getting around stiupid users.
That is one use, but not the only use. It is one feature that is missing on Linux. I don't know what other unix-like systems have, but it'd be nice if Linux had it.
Quoted 6 lines
But the fact is
that the very same users very quickly adapt to the the presence of undeletion facilities. And guess whot? They will expect you to instantly recover allways a version of "this" file from the "stone age". So the pain for the sysadmin will certainly not be decreased. Quite contrary for what he expects.
Yes, I can understand this exactly, but it still doesn't negate the usefulness of undeletion.
Quoted 3 lines
For the educated user it was always a pain
in the you know where, to constantly run out of quota space due to file versioning.
Ahh, so we'd need to chown the files to root (or a configurable user and group) to get around the quota issue.
Mike
Quoted 7 lines
Quoted 4 lines
For the educated user it was always a pain
in the you know where, to constantly run out of quota space due to file versioning.Ahh, so we'd need to chown the files to root (or a configurable user and group) to get around the quota issue.
Welcome to my kill-file. This just shows that you don't even have basic background.
Quoted 12 lines · H. Peter Anvin
Followup to: <[email protected]> By author: Mike Fedyk <[email protected]> In newsgroup: linux.dev.kernelQuoted 9 lines
Uhh, no.
You have a configurable size for the undelete dirs and you delete a file. Now, that file gets moved to $mountpoint/.undelete. The daemon gets notified through a socket, and it can check to see if it needs to delete any older deleted files to make sure .undelete doesn't get bigger than configured.
Quoted 7 lines
Quoted 5 lines
We're only scanning the dirs upon daemon startup (reminds me of quota... ;), and all other daemon actions are triggered by
unlink()writing to a socket. The worst thing that can happen is not seeing your free space immediately, but a few seconds later.Hence race condition.
But an acceptable one (it's a small delay), unless the daemon dies. :(
Quoted 3 lines
Also, the solution to hard-reserve space seems
to fundamentally defeat the purpose (IMO).
Do you really thing we should be moving files from kernel space? Ok, glibc could move the files, that'd be ok.
So, (I don't know) how is the kernel going to support undeletion on all filesystems (ext2/3, reiserfs, vfat, jfs, xfs, and any other writable fs...) in the exact same way (as seen from userspace...)?
Mike
Quoted 7 lines · Mike Fedyk
Quoted 3 lines · Martin Dalecki
For the educated user it was always a pain
in the you know where, to constantly run out of quota space due to file versioning.Ahh, so we'd need to chown the files to root (or a configurable user and group) to get around the quota issue.
Well, I don't agree with changing file ownership, because _any_ way around the quota system will be exploited by users (e.g. deleting files temporarily to gain more space, and hope they aren't destroyed before they need them again). It also opens a huge can of worms security wise, because it may be possible for one user to undelete files belonging to another user if you are not super careful.
No, I would have the unlink wrapper/daemon be quota-aware, and if a user is getting close to filling their quota then it would delete more of that user's files from the undelete directory, just as if the entire fs was getting full or the user had hit their preconfigured limit for maximum undelete size or versions of a file. Since the unlink call will never _increase_ the amount of disk used by a user (it is simply a rename() in disguise) this in itself can't be the cause a quota problem.
The only potential problem would be if the cleanup daemon dies. In that case, a user should still be able to do something like "unrm --purge" to manually clean up his files in the undelete tree (or "unrm -ls <filespec>" to show files and "unrm -d <file...>" to really delete individual ones).
For people who don't want to have undelete at all (for whatever reason) can always have something like "max_undelete=0" in their .unrmrc file, or just not use it in the first place.
Cheers, Andreas
--
Andreas Dilger
http://sourceforge.net/projects/ext2resize/
http://www-mddsp.enel.ucalgary.ca/People/adilger/
Hi!
Quoted 11 lines
Quoted 7 lines
Quoted 3 lines
but I don't want a Netware filesystem running on Linux, I want a *native* Linux filesystem (i.e. ext3) that has the ability to queue deleted files should I configure it to.
Rather than implementing this in the filesystem itself, I'd first try writing a libc shim that overrides
unlink(). You could copy files to safety, or do anything else you want, before they actually get deleted...Yep, more portable.
Now the question is: Is there already something written?
Yep, I did it at one point. Then I realized I often kill files with
Quoted 1 line
file....
Pavel
Signature · 2 lines
Philips Velo 1: 1"x4"x8", 300gram, 60, 12MB, 40bogomips, linux, mutt, details at http://atrey.karlin.mff.cuni.cz/~pavel/velo/index.html.