graceful handling of removing a plugable storage device that is being written to

Cc to BTRFS mailinglist as it triggered the idea of mine again.

Hi!

Today I did it again and removed a BTRFS partition that is written too.= =20
That BTRFS as of Kernel 3.0.3 (debian package) does not like very much.=
I=20
think thats a known issue and I wrote a mail to BTRFS mailing list abou= t=20
it.

the=20

And then cause the expected results IMHO are by no way the ideal result= s:

e=20

But I never saw any other OS that did that.

And I see the problems with high bandwidth writes piling up in memory=20 causing severe memory pressure.

But then could Linux just freeze processes that continue writing to the= =20
drive until it is replugged again? Of course that shouldn=B4t happen to=
the=20
drive / resides on.

And there is a userspace part in it - the possibly udev and dbus drive= n=20
notification to the user.

Yet despite all of this NetBSD has a gsoc 2011 project at least suggest= ed=20
for exactly this behavior:

Graceful USB disk detach/reattach
http://wiki.netbsd.org/projects/gsoc_2011/disk-removal/

They even mention the Amiga in there.

Okay, its only for USB, not for eSATA, but I think it should be made=20 generic for removable devices.

Would that be possible? I gladly file an enhancement request about it o= r=20
help testing it.

I think thats the only approach that makes sense here. USB sticks and=20 harddisks have no means to disallow device removal at any time. Thus th= e=20
OS should offer the user a way to rethink the decision and plug the dev= ice=20
in to prevent data loss. Actually I am surprised that no other operatin= g=20
system except AmigaOS seemed to offer this behavior. Well I am not quit= e=20
sure about MS-DOS writing to disk. Maybe it even did that. But I did no= t=20
use MS-DOS often.=20

All current mainstream operating systems I know of default to loose dat= a=20
in that case. I think there is a better choice. What do you think? Migh= t=20
not be much of a server feature, but important for the desktop.

Ciao,
--=20
Martin 'Helios' Steigerwald - http://www.Lichtvoll.de
GPG: 03B0 0D6C 0040 0710 4AFA B82F 991B EAAC A599 84C7
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" = in
the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html

How do you cope with=20
(1) headless systems (one where there is no udev/dbus notification or d= isplay).
(2) the user walking off in a hurry and never seeing the notification? Should the kernel/user processes freeze indefinitely?

There is also a 3rd scenario - how how one malicious person or process = doing a repeat insert/remove/write and get resource to pile up and cras= h the machine?

It is probably possible/recommended with Amiga because Amiga is seldoml= y run headless?

--
To unsubscribe from this list: send the line "unsubscribe linux-fsdevel= " in
the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html

Am Sonntag, 11. September 2011 schrieb Hin-Tak Leung:

s

This all are important and valid questions, IMHO. Still I think the=20 approach taken by AmigaOS has some merit here.

(1) headless systems:

a) servers usually do not have much to do with removable media. But sti= ll,=20
what about FC or iSCSI LUNs? What should the kernel do here? Frankly, I= =20
don=B4t know. Maybe its best to default to current behavior which impos= es a=20
risk for data loss. But then NFS is used in enterprise environments, to= o,=20
and it does block by default. Indefinetely. I have seen loads of 300 an= d=20
more cause of that behavior which is there to *prevent* data loss on NF= S=20
clients.

b) headless media systems: maybe its best to have to default to current= =20
behavior, when its known that a notification can=B4t be done. But how t= o=20
tell? Maybe best would be a timeout. Then the user even would have a=20 chance to reinsert the media.

(2) I thought about how long to wait / possibly freeze processes as wel= l:=20
Maybe it would be good to let go after a while. But I think that also=20 depends on whether more writes are done. If its an USB stick and the us= er=20
just copied some files to it and removed it prematurely without noticin= g=20
the notification, then I think the kernel could wait indefinetely.

*But* and this brings up a serious issue, I did not think about before:= =20
When the user mounts the USB stick somewhere else and finds out about t= he=20
missing files only by then, there is a real risk for data loss, if the=20 kernel of the machine that stalled the I/O insists on completing the=20 writes if the user inserts the USB stick again.

Thus it seems to me that the kernel would have to check the last mount=20 time and the filesystem state. If the last mount time is newer and/or t= he=20
filesystem is cleanly unmounted, I think the kernel must refuse any fur= ther=20
attempts to complete outstanding writes in order to protect filesystem=20 integrity.

=46rankly, I never tried this on AmigaOS. I know that AmigaOS expects t= he=20
exact same floppy disk to be inserted again. Only the same name isn=B4t= =20
enough. But I have no idea, what AmigaOS would have done, when I insert= ed=20
the disk into another Amiga, did something there and then insert it int= o=20
the Amiga with the notification and pressed "okay". Probably it would h= ave=20
eaten the disk then.

This is a serious issue which makes implementing my suggestion more=20 difficult. The kernel has to make sure not to eat a filesystem in order=
to=20
complete outstanding writes!

(3) I wouldn=B4t worry too much about malicious persons. Why? Cause wit= h=20
current standard ulimit values there are way easier methods to stall a=20 machine to a halt. I have seen more than once during holding Linux=20 performance tuning courses, that running the command "stress" with=20 aggressive parameters effectively offlines a Linux machine. I often do = a=20
check list on how often course participants make one of the Linux serve= rs=20
we work so unresponsive that a reboot is in order. So I think graceful=20 handling of media removal doesn=B4t add much to the existing issues=20 regarding that topic.

Ciao,
--=20
Martin 'Helios' Steigerwald - http://www.Lichtvoll.de
GPG: 03B0 0D6C 0040 0710 4AFA B82F 991B EAAC A599 84C7
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" = in
the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html

he

=B4t

rted

Are you sure the volume name wasn't enough?
=46or most things, it relied on the volume name, that's how you could m= ap
floppy names to hard drive directories using assigns.

nto

have

Most probably.

Gr{oetje,eeting}s,

=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 Geert

--
Geert Uytterhoeven -- There's lots of Linux beyond ia32 -- geert@linux-= m68k.org

In personal conversations with technical people, I call myself a hacker= =2E But
when I'm talking to journalists I just say "programmer" or something li= ke that.
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0 =C2=A0=C2=A0 =C2=A0=C2=A0 -- Linus Torvalds --
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" = in
the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html

Am Sonntag, 11. September 2011 schrieb Geert Uytterhoeven:

e>=20

, when

ap

Yes, I am pretty sure I tried that back then. Difficult to repeat at th= e=20
moment, since I think at least the high density disk drive in my A4000 = is=20
toast since quite a while.

Anyway, if Linux implements something like that it should not only depe= nd=20
on a volume name ;).

--=20
Martin 'Helios' Steigerwald - http://www.Lichtvoll.de
GPG: 03B0 0D6C 0040 0710 4AFA B82F 991B EAAC A599 84C7