SCSI Patches - mostly on/off-line stuff
Quoted 4 lines · Max TenEyck Woodbury
[snip]
3) Add scsi force-device-online and force-device-offline commands to the/proc/scsi/scsiinterface.
[snip]
This patch from Max got me thinking about hot-plugging issues. Now there are 2 things a hotplug-tolerant scsi (pseudo) adapter might do if it detected the removal of
a device, calls that do the equivalent of:
1) echo "scsi remove-single-device <h> <b> <t> <l>" > /proc/scsi/scsi
2) echo "scsi force-device-offline <h> <b> <t> <l>" > /proc/scsi/scsi
Option 1) is pretty absolute and is currently disallowed if there are any open fds on that device. With some work this restriction could be relaxed.
Seen from a scsi upper level driver's point of view (e.g. sd or sg) option 1) would be a detach(device) call. Taking the case of sg (a character device) what should it return on subsequent calls to read(), write(), ioctl() on a fd associated with the detached device? [My
guess: ENODEV]. How about a poll() involving such a fd? [POLLHUP or POLLERR] And if the fd was running async should a SIGPOLL signal be generated by the detach() call? [with 'band' POLL_HUP if rt signal]. Most likely apps would like to be informed of such events. I'm not clear on what a block special such as sd should do in response to a premature detach() on an open fd.
Option 2) allows for the device to be brought back online with its state being maintained. It could be nasty (for the scsi subsystem) if a USB mass storage device was pulled out of one USB port and placed in another.
Currently there is no mid-level call into upper level scsi drivers to indicate that the state of 'offline' has been flipped. [Perhaps there should be to allow upper level drivers to generate signals.] Sg doesn't test the state of a device's offline bit before it tries to dispatch a scsi command to the mid level. [Perhaps it should.] Same questions apply as to what sg and sd should do. Another option in this case is that system commands (e.g. read() and write()) are put into a wait state for up to a minute, say, to give the device a chance to come back online.
Doug Gilbert
_______________________________________________
Linux-hotplug-devel mailing list
[email protected]
http://lists.sourceforge.net/lists/listinfo/linux-hotplug-devel
Quoted 14 lines · Douglas Gilbert
Quoted 4 lines · Max TenEyck Woodbury
[snip]
3) Add scsi force-device-online and force-device-offline commands to the/proc/scsi/scsiinterface.
[snip]This patch from Max got me thinking about hot-plugging issues. Now there are 2 things a hotplug-tolerant scsi (pseudo) adapter might do if it detected the removal of
a device, calls that do the equivalent of:
1) echo "scsi remove-single-device <h> <b> <t> <l>" >/proc/scsi/scsi
2) echo "scsi force-device-offline <h> <b> <t> <l>" >/proc/scsi/scsi
Quoted 3 lines
Option 1) is pretty absolute and is currently disallowed if there are any open fds on that device. With some work this restriction could be relaxed.
It need not be effective at once. Most drivers would probably happy with a flag/call to signal a vanished device and a callback once all fds are gone to free the memory associated. In fact that would be preferable. It's not a drivers business to know what to do in this case. A driver's duty should be to report that condition.
Quoted 6 lines
Seen from a scsi upper level driver's point of view (e.g. sd or sg) option 1) would be a detach(device) call. Taking the case of sg (a character device) what should it return on subsequent calls to
read(),write(),ioctl()on a fd associated with the detached device? [My
guess: ENODEV]. How about apoll()involving such a
Could be confusing, but it's probably right.
Quoted 2 lines
as sd should do in response to a premature
detach()on an open fd.
As a suggestion:
Wait for a few minutes and return errors to open() only. After a timeout return errors. If it is reattached some kind of checksumming might be a good idea.
Quoted 2 lines
Option 2) allows for the device to be brought back online with its state being maintained. It could be
How ? Is there a way to know internal state has been maintained. For sd it might not matter, for sg a disconnected device must be considered gone for good by default.
Those drivers with these problems probably register themselves as host controllers for each device/bridge. These register/unregister calls need the same attention as above.
Quoted 3 lines
that system commands (e.g.
read()andwrite()) are put into a wait state for up to a minute, say, to give the device a chance to come back online.
That choice must be left to the high level drivers. Do really want to use off-line for that ?
Regards
Oliver
_______________________________________________
Linux-hotplug-devel mailing list
[email protected]
http://lists.sourceforge.net/lists/listinfo/linux-hotplug-devel
There was a bit of mail last week touching on SCSI hotplugging.
I'd be interested in knowing if there's any sort of "status quo" or "plan of record" there. Or even a list of open issues, given that I understand there's a lot of interest in fitting SCSI and hotplugging together!
- Dave
Quoted 11 lines
From: Douglas Gilbert <[email protected]>
Sent: Thursday, January 11, 2001 1:56 PMQuoted 4 lines · Max TenEyck Woodbury
[snip]
3) Add scsi force-device-online and force-device-offline commands to the/proc/scsi/scsiinterface.
[snip]This patch from Max got me thinking about hot-plugging issues. ...
_______________________________________________
Linux-hotplug-devel mailing list http://linux-hotplug.sourceforge.net
[email protected]
http://lists.sourceforge.net/lists/listinfo/linux-hotplug-devel
Quoted 7 lines · David Brownell
There was a bit of mail last week touching on SCSI hotplugging.
I'd be interested in knowing if there's any sort of "status quo" or "plan of record" there. Or even a list of open issues, given that I understand there's a lot of interest in fitting SCSI and hotplugging together!
The basic question is whether hot-plugging support is just too much effort for the lk 2.4 scsi subsystem and should be left to lk 2.5? OTOH could there be a series of incremental changes in lk 2.4 to support it?
The areas of interest are:
1) lower level (pseudo) drivers [e.g. usb/storage + sbp2_1395]
2) scsi mid level
3) scsi upper level drivers [sd, sr, st + sg]
4) block + cdrom subsystems
In area 1) the driver authors are keen to get things going. Various approaches have been tried:
- user intervention via rescan-scsi-bus.sh script [found in recent Suse distros in the scsi.spm package]
- scsi_(un)register_host()
- [perhaps onlineúLSE for temporary outage]
In area 2) we usually defer to Eric Youngdale on design issues. The add/remove-single-device procfs technique (as used by rescan-scsi-bus) is only allowed when the device has no open fds. To remove the user intervention requirement, exported C calls equivalent to add/remove-single-device need to be added (or existing calls stretched). As for allowing those calls when there are open fds, there are issues:
a) command "in flight" or could be in error recovery
b) re-entrancy?
c) mid level resource control (e.g. Scsi_Device objects)
d) impact on areas 3) and 4)
In area 3) my last post pointed out that, at least from the sg driver's point of view, it would not be that difficult. Coping with a premature sg_detach() isn't that much different from coping with a premature sg_release() which it already does. Extending from my own narrow area to the other upper level drivers, I assumed it wouldn't be a big deal for them. Both st and sg interface to applications at this level so this is the end of their story.
In area 4) the sd and sr drivers interface to the block and cdrom subsystems. Maybe others could comment on the impact here.
Doug Gilbert
_______________________________________________
Linux-hotplug-devel mailing list http://linux-hotplug.sourceforge.net
[email protected]
http://lists.sourceforge.net/lists/listinfo/linux-hotplug-devel
Quoted 4 lines
The basic question is whether hot-plugging support is just too much effort for the lk 2.4 scsi subsystem and should be left to lk 2.5? OTOH could there be a series of incremental changes in lk 2.4 to support it?
There are hotplugging devices out there.
Some basic solution is needed.
Quoted 10 lines
The areas of interest are:
1) lower level (pseudo) drivers [e.g. usb/storage +sbp2_1395]
2) scsi mid level
3) scsi upper level drivers [sd, sr, st + sg]
4) block + cdrom subsystemsIn area 2) we usually defer to Eric Youngdale on design issues. The add/remove-single-device procfs technique (as used by rescan-scsi-bus) is only allowed when the device has no open fds. To remove the user intervention requirement,
For device addition at least that is always the case. One problem less.
Quoted 7 lines
exported C calls equivalent to add/remove-single-device need to be added (or existing calls stretched). As for allowing those calls when there are open fds, there are issues:
a) command "in flight" or could be in error recovery
b) re-entrancy?
c) mid level resource control (e.g.Scsi_Deviceobjects)
d) impact on areas 3) and 4)
Device removal seems to be the hard case.
How about the following solution:
1. A flag to mark the device hotpluggable, like the emulated flag
2. export the add/remove-device call
3. return proper error codes for the unregister calls (and the calls from 2.)
4. add a call for taking the device offline
IMO we are now at a stage where the number of open fds cannot increase, can it ?
5. add a status of catastrophic failure to the returnable states -> no error handling, get the device offline
Now we have solved the problem of how to deal with "in flight" commands. To the upper layers unplugging is like a dreadful error whch they have to handle anyway.
6. add a callback (w. a private data pointer) to be called when the usage count drops to zero. At this time removal must work and low level drivers can free memory.
This scheme is not the most elegant as it leaves the issues of replugging with the low level drivers, but it is implementable without great changes.
Regards
Oliver
_______________________________________________
Linux-hotplug-devel mailing list http://linux-hotplug.sourceforge.net
[email protected]
http://lists.sourceforge.net/lists/listinfo/linux-hotplug-devel
Quoted 50 lines · Oliver Neukum
Quoted 5 lines
The basic question is whether hot-plugging support is just too much effort for the lk 2.4 scsi subsystem and should be left to lk 2.5? OTOH could there be a series of incremental changes in lk 2.4 to support it?
There are hotplugging devices out there.
Some basic solution is needed.Quoted 10 lines
The areas of interest are:
1) lower level (pseudo) drivers [e.g. usb/storage +sbp2_1395]
2) scsi mid level
3) scsi upper level drivers [sd, sr, st + sg]
4) block + cdrom subsystemsIn area 2) we usually defer to Eric Youngdale on design issues. The add/remove-single-device procfs technique (as used by rescan-scsi-bus) is only allowed when the device has no open fds. To remove the user intervention requirement,
For device addition at least that is always the case. One problem less.
Quoted 7 lines
exported C calls equivalent to add/remove-single-device need to be added (or existing calls stretched). As for allowing those calls when there are open fds, there are issues:
a) command "in flight" or could be in error recovery
b) re-entrancy?
c) mid level resource control (e.g.Scsi_Deviceobjects)
d) impact on areas 3) and 4)Device removal seems to be the hard case.
How about the following solution:
1. A flag to mark the device hotpluggable, like the emulated flag
2. export the add/remove-device call
3. return proper error codes for the unregister calls (and the calls from 2.)
4. add a call for taking the device offlineIMO we are now at a stage where the number of open fds cannot increase, can it ?
5. add a status of catastrophic failure to the returnable states -> no error handling, get the device offline
Now we have solved the problem of how to deal with "in flight" commands. To the upper layers unplugging is like a dreadful error whch they have to handle anyway.
Quoted 6 lines
6. add a callback (w. a private data pointer) to be called when the usage count drops to zero. At this time removal must work and low level drivers can free memory.
This scheme is not the most elegant as it leaves the issues of replugging with the low level drivers, but it is implementable without great changes.
I'd be delighted to see an ongoing exploration of what is necessary and feasible for the 2.4.0 series kernel (this message is a good start of that topic), plus what ought to be undertaken in 2.5.0 in order to create
thorough and clean hotplugging support. Hopefully, some work will be shared. We'd probably do the shared stuff first and then branch off into
the deeper and more dangerous work.
Miles
_______________________________________________
Linux-hotplug-devel mailing list http://linux-hotplug.sourceforge.net
[email protected]
http://lists.sourceforge.net/lists/listinfo/linux-hotplug-devel
As far as the mid-layer is concerned, hotplugging more or less consists
of two independent tasks. Addition of new devices, and removal of devices.
To support addition, the main question is how one detects that a device
has been hotplugged in the first place, and whether this detection mechanism tells you what devices are new, or whether a new bus scan needs to be
initiated. In the event that the low-level driver has some way of being
notified that a new device is on the bus (along with the ID and LUN) then it would mainly be a matter of exposing the appropriate interfaces to low-level drivers so that they can register the detection of new device. In the event that a bus scan would need to be re-initiated, then a little more thought would have to go into it, I suppose. The tricky bit here is going to be the question of what context these functions would be called - the functions for adding devices really weren't designed for the purposes of being called from an interrupt context, for example.
The question of hotunplugging a device is a bit trickier as there are
potential race conditions in the event that the device is in use. Currently as things stand, removal of a device fails if the device is busy (i.e. has open file descriptors) - note that there may be a small window of a race condition between the open() function where we increment the usage count and the device removal, but with a minor amount of work this could be fixed.
Perhaps a better question to examine is whether our device removal
strategies needs to be improved. It would be quite possible of course to fix things so that even if there are open file descriptors and a device is removed that it would still work. I am thinking of a flag of some sort that would say "delete when usage count drops to 0" or some such - not hard to implement either, and would be a bit more robust than what we have now.
Sort of a "pending remove" of the device. This sort of thing is not all
that compatible with the modules code - I believe that module removal can either succeed or fail, but there is no "pending removal" state of a module.
Anyways, my initial question reallly has to do with exactly how one is
notified that a new device has appeared and whether a bus scan needs to be
initiated or not. That will control the degree to which we have to screw
with things in the mid-layer to support this.
-Eric
----- Original Message -----
From: "Douglas Gilbert" <[email protected]>
To: "David Brownell" <[email protected]>
Cc: <[email protected]>; <[email protected]>
Sent: Wednesday, January 17, 2001 7:08 PM
Subject: Re: SCSI Patches - mostly on/off-line stuff
Quoted 48 lines
Quoted 7 lines · David Brownell
There was a bit of mail last week touching on SCSI hotplugging.
I'd be interested in knowing if there's any sort of "status quo" or "plan of record" there. Or even a list of open issues, given that I understand there's a lot of interest in fitting SCSI and hotplugging together!
The basic question is whether hot-plugging support is just too much effort for the lk 2.4 scsi subsystem and should be left to lk 2.5? OTOH could there be a series of incremental changes in lk 2.4 to support it?
The areas of interest are:
1) lower level (pseudo) drivers [e.g. usb/storage +sbp2_1395]
2) scsi mid level
3) scsi upper level drivers [sd, sr, st + sg]
4) block + cdrom subsystemsIn area 1) the driver authors are keen to get things going. Various approaches have been tried:
- user intervention viarescan-scsi-bus.shscript [found in recent Suse distros in the scsi.spm package]
- scsi_(un)register_host()
- [perhaps onlineúLSE for temporary outage]In area 2) we usually defer to Eric Youngdale on design issues. The add/remove-single-device procfs technique (as used by rescan-scsi-bus) is only allowed when the device has no open fds. To remove the user intervention requirement, exported C calls equivalent to add/remove-single-device need to be added (or existing calls stretched). As for allowing those calls when there are open fds, there are issues:
a) command "in flight" or could be in error recovery
b) re-entrancy?
c) mid level resource control (e.g.Scsi_Deviceobjects)
d) impact on areas 3) and 4)In area 3) my last post pointed out that, at least from the sg driver's point of view, it would not be that difficult. Coping with a premature
sg_detach()isn't that much different from coping with a prematuresg_release()which it already does. Extending from my own narrow area to the other upper level drivers, I assumed it wouldn't be a big deal for them. Both st and sg interface to applications at this level so this is the end of their story.
Quoted 8 lines
In area 4) the sd and sr drivers interface to the block and cdrom subsystems. Maybe others could comment on the impact here.
Doug Gilbert
-
To unsubscribe from this list: send the line "unsubscribe linux-scsi" in the body of a message to [email protected]
_______________________________________________
Linux-hotplug-devel mailing list http://linux-hotplug.sourceforge.net
[email protected]
http://lists.sourceforge.net/lists/listinfo/linux-hotplug-devel
Quoted 5 lines
Anyways, my initial question reallly has to do with exactly how one is
notified that a new device has appeared and whether a bus scan needs to be
initiated or not. That will control the degree to which we have to screw
with things in the mid-layer to support this.
[Venkat] Some of the SCSI devices support AEN, which the low level
driver can use to detect any addition in BUS. I not sure whether all the scsi device do support it. If this the scsi case low level driver can call the SCSI ML to add the new device. I dont whether this is a general solution.
_______________________________________________
Linux-hotplug-devel mailing list http://linux-hotplug.sourceforge.net
[email protected]
http://lists.sourceforge.net/lists/listinfo/linux-hotplug-devel
[I was told that FCP did not support AENs but I might be mistaken.]
In general, as SCSI becomes more networked, most approaches will want the initiator to figure out what is the
current list of targets the initiator is connected to, rather than the targets trying to inform interested initiators whether
the targets are coming on-line or going off-line.
Keeping that in mind, a bus rescan is a better way to go (where a bus is 'virtual' for networked SCSI architectures).
-prasenjit
Prasenjit Sarkar
Research Staff Member
IBM Almaden Research
San Jose
Venkatesh Ramamurthy <[email protected]>@vger.kernel.org on 01/18/2001 08:20:03 AM
Sent by: [email protected]
To: "'Eric Youngdale'" <[email protected]>, Douglas Gilbert
<[email protected]>, David Brownell <[email protected]>
cc: [email protected], [email protected]
Subject: RE: SCSI Patches - mostly on/off-line stuff
Quoted 1 line
Anyways, my initial question reallly has to do with exactly how one
is
Quoted 1 line
notified that a new device has appeared and whether a bus scan needs to
be
Quoted 1 line
initiated or not. That will control the degree to which we have to
screw
Quoted 2 lines
with things in the mid-layer to support this.
[Venkat] Some of the SCSI devices support AEN, which the low level
driver can use to detect any addition in BUS. I not sure whether all the scsi device do support it. If this the scsi case low level driver can call the SCSI ML to add the new device. I dont whether this is a general solution.
-
To unsubscribe from this list: send the line "unsubscribe linux-scsi" in the body of a message to [email protected]
_______________________________________________
Linux-hotplug-devel mailing list http://linux-hotplug.sourceforge.net
[email protected]
http://lists.sourceforge.net/lists/listinfo/linux-hotplug-devel
Quoted 10 lines
In general, as SCSI becomes more networked, most approaches will want the initiator to figure out what is the
current list of targets the initiator is connected to, rather than the targets trying to inform interested initiators whether
the targets are coming on-line or going off-line.Keeping that in mind, a bus rescan is a better way to go (where a bus is 'virtual' for networked SCSI architectures).
[ Venkat] The issue is who issues the BUS scanning operations. Is it
done in userspace ? What is the time interval between bus scanning for new detection? How is it going to affect SCSI BUS IO speed? We dont want a too frequent scanning which affects IO performance. Too long an interval really undermines Plug & Play. AEN seems better , but i have to check whether FCP supports it.
_______________________________________________
Linux-hotplug-devel mailing list http://linux-hotplug.sourceforge.net
[email protected]
http://lists.sourceforge.net/lists/listinfo/linux-hotplug-devel
Quoted 3 lines
[I was told that FCP did not support AENs but I might be mistaken.]
[Venkat] Actually FCP does and ordinary SCSI devices does not
support AEN. When a FC device is added or removed , a LIP ( Lopp Initilaization Procedure) is initiated. For non-FC devices there is one version of AEN that allows target initiated event notification.
Quoted 5 lines
In general, as SCSI becomes more networked, most approaches will want the initiator to figure out what is the
current list of targets the initiator is connected to, rather than the targets trying to inform interested initiators whether
the targets are coming on-line or going off-line.
[Venkat] I agree that bus rescan is a better way. If the SCSI
devices are connected though a SAF_TE processor then we can poll the SCSI Bus frequently, as the SAF_TE knows about addition and detection.
_______________________________________________
Linux-hotplug-devel mailing list http://linux-hotplug.sourceforge.net
[email protected]
http://lists.sourceforge.net/lists/listinfo/linux-hotplug-devel
Douglas, thanks for that overview of the structure of the SCSI subsystem -- I needed one, that was better than I'd suspected I'd find!
It seemed to me that something was missing in that stack though; the layers above (4), specifically filesystems, that would also need to know about new devices that were added. When everything is working smoothly, users need to see filesystems get mounted as a direct consequence of hotplugging such storage units.
One way to look at that issue is to ask what user mode notifications will be used to address that part of the hotplug problem. Devfsd is what some folk like, but it's not universally accepted. GUI driven solutions don't seem right in all cases either.
Eric:
Quoted 2 lines
As far as the mid-layer is concerned, hotplugging more or less consists
of two independent tasks. Addition of new devices, and removal of devices.
Exactly. That is, if you will, one of the first architectural requirements. Once that's addressed, everything else starts to fall into place.
As I understand things, USB-to-SCSI adapters (like most usb-storage devices) will also have "bus" add/remove events to deal with. Likewise with cases like hotplugging a SCSI controller on a Cardbus laptop, or on a CompactPCI (or HotplugPCI) enterprise level server. Presumably those cases aren't as troublesome?
You commented that "hot unplugging a device is a bit trickier" ... yes!
The notion of a "pending remove" state has crossed my mind too. "New style" networking drivers seem to have something like this, and USB has analagous issues. Re tying it into the module subsystem, I'll have to try that idea on for size; the module system doesn't really know about "devices" as such, and maybe it should. (Something needs to.)
Quoted 4 lines
Anyways, my initial question reallly has to do with exactly how one is
notified that a new device has appeared and whether a bus scan needs to be
initiated or not. That will control the degree to which we have to screw
with things in the mid-layer to support this.
Reasoning by analogy for a moment ... USB and Cardbus both have kernel threads (khubd, and the cardbus watcher) tasked with detecting such stuff. Also, it's plausible to me to require "add new device" processing to have a thread context to work with, like "remove device"; again, USB and Cardbus both have that requirement. So it seems to me that having some thread(s) involved here will be the typical case. Presumably that means that the mid-layer code would need to accomodate that model.
- Dave
_______________________________________________
Linux-hotplug-devel mailing list http://linux-hotplug.sourceforge.net
[email protected]
http://lists.sourceforge.net/lists/listinfo/linux-hotplug-devel
Quoted 4 lines
One way to look at that issue is to ask what user mode notifications will be used to address that part of the hotplug problem. Devfsd is what some folk like, but it's not universally accepted. GUI driven solutions don't seem right in all cases either.
devfsd is at least already here. The existing /sbin/hotplug architecture can't handle it. A new kind of event would be needed.
Even so, you'll probably won't make the user happy if it works only on plugging in and not when he puts in a new medium.
A look at supermount might be in order.
Quoted 5 lines
As I understand things, USB-to-SCSI adapters (like most usb-storage devices) will also have "bus" add/remove events to deal with. Likewise with cases like hotplugging a SCSI controller on a Cardbus laptop, or on a CompactPCI (or HotplugPCI) enterprise level server. Presumably those cases aren't as troublesome?
Yes, a new bus is scanned anyway.
To a certain extent the notion of bus vs. device becomes moot, when devices are identified by globally unique numbers. Should a fibre channel drive be given other names if it's shifted to another controller ?
To let deeds follow words I've attached a patch that adds error reporting and locking to the bus removal functions.
Regards
Oliver
drivers/scsi/scsi.c +30 -18
@@ -53,6 +53,7 @@#include <linux/interrupt.h> #include <linux/delay.h> #include <linux/init.h> +#include <linux/smp_lock.h> #define __KERNEL_SYSCALLS__
@@ -1376,7 +1377,7 @@ }static int scsi_register_host (Scsi_Host_Template *); -static void scsi_unregister_host (Scsi_Host_Template *); +static int scsi_unregister_host (Scsi_Host_Template *);/* * Function: scsi_release_commandblocks()
@@ -1963,14 +1964,8 @@/* * Similarly, this entry point should be called by a loadable module if it * is trying to remove a low level scsi driver from the system. -* -* Note - there is a fatal flaw in the deregister module function. -* There is no way to return a code that says 'I cannot be unloaded now'. -* The system relies entirely upon usage counts that are maintained, -* and the assumption is that if the usage count is 0, then the module -* can be unloaded. */ -static void scsi_unregister_host (Scsi_Host_Template * tpnt) +static int scsi_unregister_host (Scsi_Host_Template * tpnt) {int online_status;int pcount0, pcount;
@@ -1982,6 +1977,9 @@struct Scsi_Host *shpnt;char name[10 ];/* host_no>=10^9? I don't think so. */ +/* get the big kernel lock, so we don't race with open() */ +lock_kernel (); +/* * First verify that this host adapter is completely free with no pending * commands
@@ -1992,7 +1990,7 @@if (SDpnt->host->hostt == tpnt && SDpnt->host->hostt->module &&GET_USE_COUNT (SDpnt->host->hostt->module)) -return ; + goto err_out;/* * FIXME(eric) - We need to find a way to notify the * low level driver that we are shutting down - via the
@@ -2044,7 +2042,7 @@ } SDpnt->online = online_status;printk (KERN_ERR "Device busy???\n" ); -return ; + goto err_out; }/* * No, this device is really free. Mark it as such, and
@@ -2070,7 +2068,7 @@/* If something still attached, punt */ if (SDpnt->attached) {printk (KERN_ERR "Attached usage count = %d\n" , SDpnt->attached); -return ; + goto err_out; }devfs_unregister (SDpnt->de); }
@@ -2178,6 +2176,13 @@ } }MOD_DEC_USE_COUNT ; + +unlock_kernel (); +return 0 ; + +err_out: +unlock_kernel (); +return -1 ; }static int scsi_unregister_device (struct Scsi_Device_Template *tpnt);
@@ -2259,12 +2264,13 @@struct Scsi_Host *shpnt;struct Scsi_Device_Template *spnt;struct Scsi_Device_Template *prev_spnt; - + +lock_kernel ();/* * If we are busy, this is not going to fly. */ if (GET_USE_COUNT (tpnt->module) !=0 ) -return 0 ; + goto error_out;/* * Next, detach the devices from the driver.
@@ -2301,11 +2307,15 @@ prev_spnt->next = spnt->next;MOD_DEC_USE_COUNT ; +unlock_kernel ();/* * Final cleanup for the driver is done in the driver sources in the * cleanup function. */ return 0 ; +error_out: +unlock_kernel (); +return -1 ; }
@@ -2342,22 +2352,24 @@/* Reverse the actions taken above */ -void scsi_unregister_module (int module_type,void *ptr) +int scsi_unregister_module (int module_type,void *ptr) { +int retval =0 ; +switch (module_type) {case MODULE_SCSI_HA : -scsi_unregister_host ((Scsi_Host_Template *) ptr); + retval =scsi_unregister_host ((Scsi_Host_Template *) ptr);break ;case MODULE_SCSI_DEV : -scsi_unregister_device ((struct Scsi_Device_Template *) ptr); -break ; + retval =scsi_unregister_device ((struct Scsi_Device_Template *)ptr); +break ;/* The rest of these are not yet implemented. */ case MODULE_SCSI_CONST :case MODULE_SCSI_IOCTL :break ;default : } -return ; +return retval; }#ifdef CONFIG_PROC_FS
LIP is not an AEN (as defined by the scsi standards) and therefore is not propagated to the scsi layer.
In addition, LIP is also specific to FC-AL and the event for switched FC networks is different.
Moreover, a LIP contains no extra information.
Apart from that, I agree with Venkat about his conclusions.
Prasenjit Sarkar
Research Staff Member
IBM Almaden Research
San Jose
_______________________________________________
Linux-hotplug-devel mailing list http://linux-hotplug.sourceforge.net
[email protected]
http://lists.sourceforge.net/lists/listinfo/linux-hotplug-devel
David Brownell wrote:
<snip>
Quoted 5 lines
The notion of a "pending remove" state has crossed my mind too. "New style" networking drivers seem to have something like this, and USB has analagous issues. Re tying it into the module subsystem, I'll have to try that idea on for size; the module system doesn't really know about "devices" as such, and maybe it should. (Something needs to.)
This "pending remove" would be a flag that could be unset at any point before the device removal occurs, right?
m.
_______________________________________________
Linux-hotplug-devel mailing list http://linux-hotplug.sourceforge.net
[email protected]
http://lists.sourceforge.net/lists/listinfo/linux-hotplug-devel
Quoted 7 lines · David Brownell
It seemed to me that something was missing in that stack though; the layers above (4), specifically filesystems, that would also need to know about new devices that were added. When everything is working smoothly, users need to see filesystems get mounted as a direct consequence of hotplugging such storage units.
Quoted 5 lines
One way to look at that issue is to ask what user mode notifications will be used to address that part of the hotplug problem. Devfsd is what some folk like, but it's not universally accepted. GUI driven solutions don't seem right in all cases either. [snip]
Devfs has the useful property that only devices that are present and have their drivers ready appear in the /dev hierarchy. In the case of SCSI devices, devfs maintains this dynamically via calls in the _attach() and _detach() functions within the upper level drivers. The new directory notification capability in lk 2.4 (see Documentation/dnotify.txt) could then be used to detect changes in the /dev directory (or below).
A user mode notification daemon could read /proc/scsi/scsi periodically to detect the arrival and departure of SCSI (pseudo) devices. To this end, the output read from /proc/scsi/sg/devices would be easier to parse.
If a cdrom drive has just be introduced, it would relatively easy to work out which /dev/sr<n> device name it was. I guess existing cdrom applications would then be smart enough to work out whether the drive was empty, had a music cd, or a data cdrom (which fs type, etc).
Currently there is a boolean flag: 'emulated' that sg can yield via an ioctl and is available in /proc/scsi/sg/hosts . I assume it was introduced for the first pseudo scsi driver:
ide-scsi. Seems to me that this flag could be expanded to be the major device number of the primary protocol stack that a pseudo SCSI device belongs to (e.g. IDE, USB, IEEE1394 etc). This would allow a user mode notification daemon to cross reference the primary protocol stack.
Doug Gilbert
_______________________________________________
Linux-hotplug-devel mailing list http://linux-hotplug.sourceforge.net
[email protected]
http://lists.sourceforge.net/lists/listinfo/linux-hotplug-devel
Quoted 15 lines · Douglas Gilbert
Quoted 7 lines · David Brownell
It seemed to me that something was missing in that stack though; the layers above (4), specifically filesystems, that would also need to know about new devices that were added. When everything is working smoothly, users need to see filesystems get mounted as a direct consequence of hotplugging such storage units.
Quoted 5 lines
One way to look at that issue is to ask what user mode notifications will be used to address that part of the hotplug problem. Devfsd is what some folk like, but it's not universally accepted. GUI driven solutions don't seem right in all cases either. [snip]
Quoted 8 lines
Devfs has the useful property that only devices that are present and have their drivers ready appear in the /dev hierarchy. In the case of SCSI devices, devfs maintains this dynamically via calls in the
_attach()and_detach()functions within the upper level drivers. The new directory notification capability in lk 2.4 (seeDocumentation/dnotify.txt) could then be used to detect changes in the /dev directory (or below).
Wouldn't we need to implement two SCSI hotplugging solutions, then -- one for when devfs is available and another when it isn't? If so, there might still be enough benefit to devfs users to justify the extra work. Would there be any problem getting identical behavior from both implementations? Would we consider supporting only one configuration (devfs only)? That would seem like a bad idea.
Perhaps we'd have a situation where hotplugging works a little better when the installation uses devfs. That would provide some incentive to distributions to ship with devfs preconfigured.
Miles
_______________________________________________
Linux-hotplug-devel mailing list http://linux-hotplug.sourceforge.net
[email protected]
http://lists.sourceforge.net/lists/listinfo/linux-hotplug-devel
Well, now that I think of it ... this may be a good way to capture the two different kinds of "remove". Think of "remove" as breaking a binding between device and driver, and these scenarios:
- One "remove" is done by removing the hardware. That can't really be reversed ... gotta clean up any messy device and "higher level" state, errors all around.
- Another is done by sysadmin request. Hardware still there, driver still there ... but they're not bound. (Maybe it's install-new-driver time, say, or to make sure hardware removal won't cause trouble.)
In that latter case there's a lot of flexibility. Why are you thinking a "remove" might want to get undone?
- Dave
----- Original Message -----
From: Miles Lane <[email protected]>
To: David Brownell <[email protected]>
Cc: <[email protected]>; <[email protected]>
Sent: Thursday, January 18, 2001 11:45 AM
Subject: Re: SCSI Patches - mostly on/off-line stuff
Quoted 10 lines
David Brownell wrote:
<snip>
Quoted 5 lines
The notion of a "pending remove" state has crossed my mind too. "New style" networking drivers seem to have something like this, and USB has analagous issues. Re tying it into the module subsystem, I'll have to try that idea on for size; the module system doesn't really know about "devices" as such, and maybe it should. (Something needs to.)
Quoted 4 lines
This "pending remove" would be a flag that could be unset at any point before the device removal occurs, right?
m.
_______________________________________________
Linux-hotplug-devel mailing list http://linux-hotplug.sourceforge.net
[email protected]
http://lists.sourceforge.net/lists/listinfo/linux-hotplug-devel
Quoted 7 lines
Quoted 4 lines
One way to look at that issue is to ask what user mode notifications will be used to address that part of the hotplug problem. Devfsd is what some folk like, but it's not universally accepted. GUI driven solutions don't seem right in all cases either.
devfsd is at least already here. The existing
/sbin/hotplugarchitecture can't handle it. A new kind of event would be needed.
A new event type doesn't change the architecture ... if that's all that's needed, the architecture is unchanged!
To some folk, it won't even matter if the event comes from a devfsd thread or a kernel thread. Implementation detail. Except maybe to the US Patent Office ... sigh. (One iteration of USB hotplugging did just that: devfsd called the same policy scripts as the kernel thread version.)
Quoted 3 lines
Even so, you'll probably won't make the user happy if it works only on plugging in and not when he puts in a new medium.
A look at supermount might be in order.
Yep, lots of tools to choose from. But again, not everyone runs supermount (or wants to) -- Yet Another Daemon, as I recall the situation.
- Dave
_______________________________________________
Linux-hotplug-devel mailing list http://linux-hotplug.sourceforge.net
[email protected]
http://lists.sourceforge.net/lists/listinfo/linux-hotplug-devel
Quoted 3 lines
directory notification capability in lk 2.4 (see
Documentation/dnotify.txt) could then be used to detect changes in the /dev directory (or below).
Why ? Devfs will call devfsd on its own.
Quoted 2 lines
A user mode notification daemon could read
/proc/scsi/scsiperiodically to detect the arrival and departure of SCSI
No ! This must not be done. Parsing /proc/scsi/scsi or for that matter any other file in /proc by a user space demon is a race condition. There's nothing you can do to change that. There's no way you could guarantee that the information gained by parsing is still valid when you use it. The coordination of a new device to a major/minor (or name) must be made in kernel space and cannot be broken until the user mode helper returns.
Either devfs is used to do that or new event types are added to the /sbin/hotplug mechanism. For SCSI you'd need it in the bus scanning code and in the partition recognition code. As this is a larger undertaking, this would indicate IMHO that it'd be wiser to use the infrastructure contained in devfs. But there are other possibilities, however it must be done in kernel space.
Regards
Oliver
_______________________________________________
Linux-hotplug-devel mailing list http://linux-hotplug.sourceforge.net
[email protected]
http://lists.sourceforge.net/lists/listinfo/linux-hotplug-devel
Quoted 5 lines
Quoted 2 lines
devfsd is at least already here. The existing
/sbin/hotplugarchitecture can't handle it. A new kind of event would be needed.A new event type doesn't change the architecture ... if that's all that's needed, the architecture is unchanged!
Ok, so we don't call an extension a change ;-)
But the required changes to other subsystems would be larger. You'd need to call the helper at
- SCSI bus scanning
- Partition detection
- disk change detection
- module insertation/removal
at least. Possibly at other places too.
And you'd begin to get some issues, like
disk change -> helper script -> open device file -> disk change -> helper script -> etc ...
This is starting to get messy, IMHO.
I am not sure whether you can run these scripts in paralel. And by the way some of this is needed for scsi<->usb bridges.
Quoted 5 lines
To some folk, it won't even matter if the event comes from a devfsd thread or a kernel thread. Implementation detail. Except maybe to the US Patent Office ... sigh. (One iteration of USB hotplugging did just that: devfsd called the same policy scripts as the kernel thread version.)
Devfs can give you a name to work on.
The kernel thread'll give you major/minor at best. But that's a small difference.
Quoted 7 lines
Quoted 3 lines
Even so, you'll probably won't make the user happy if it works only on plugging in and not when he puts in a new medium.
A look at supermount might be in order.Yep, lots of tools to choose from. But again, not everyone runs supermount (or wants to) -- Yet Another Daemon, as I recall the situation.
It's a proper filesystem without a demon.
Why would a requirement to use a filesystem to enjoy a feature be unreasonable ? If you can require usbdevfs you can require supermount, too, IMHO.
Regards
Oliver
_______________________________________________
Linux-hotplug-devel mailing list http://linux-hotplug.sourceforge.net
[email protected]
http://lists.sourceforge.net/lists/listinfo/linux-hotplug-devel
Quoted 9 lines · David Brownell
Well, now that I think of it ... this may be a good way to capture the two different kinds of "remove". Think of "remove" as breaking a binding between device and driver, and these scenarios:
- One "remove" is done by removing the hardware. That can't really be reversed ... gotta clean up any messy device and "higher level" state, errors all around.
Quoted 7 lines
- Another is done by sysadmin request. Hardware still there, driver still there ... but they're not bound. (Maybe it's install-new-driver time, say, or to make sure hardware removal won't cause trouble.)
In that latter case there's a lot of flexibility. Why are you thinking a "remove" might want to get undone?
Would you clarify how you envision a user working with a pending remove? Is it an implementation where a long period of time might pass before the the device is actually removed?
Could something like this happen?
- I start compiling XFree86 on a removable device and tell the system to unmount the media and remove the device when the compile is complete, a "pending remove" is set.
- During the compile, I realize that I want to make some additional source modifications, so I want to cancel the device removal and continue accessing the device.
I may be misunderstanding the sequence of events and how that might impact a user.
Miles
_______________________________________________
Linux-hotplug-devel mailing list http://linux-hotplug.sourceforge.net
[email protected]
http://lists.sourceforge.net/lists/listinfo/linux-hotplug-devel
Quoted 8 lines
Quoted 3 lines
In that latter case there's a lot of flexibility. Why are you thinking a "remove" might want to get undone?
Would you clarify how you envision a user working with a pending remove? Is it an implementation where a long period of time might pass before the the device is actually removed?
If I understand this correctly, "pending" is a state which indicates that a physical device has been unplugged, but the connection between that device's node (and fds) and the driver still exists. At some point (no more open fds) the driver would be free to break the node <-> driver relation. If indeed the device is replugged in the pending period the driver could go back from "pending" to "active". (If there's no state to be lost)
For the user any action on an fd should either yield an error or block. Probably open should fail.
However I'd like to see the decision to go from pending to gone in the common core where possible, eg. in the scsi upper layer. The same should be true for the decision when to return an error to user space.
HTH
Oliver
_______________________________________________
Linux-hotplug-devel mailing list http://linux-hotplug.sourceforge.net
[email protected]
http://lists.sourceforge.net/lists/listinfo/linux-hotplug-devel
I see what you're thinking. No, I mostly see these distinctions as relating to code paths the kernel has to deal with; ideally, users don't see "pending".
We need that stuff to behave well (it took a while for USB to do so, and the networking code is still rolling in such changes) before we should expose it to users. IMO.
----- Original Message -----
From: Miles Lane <[email protected]>
To: David Brownell <[email protected]>
Cc: <[email protected]>; <[email protected]>
Sent: Thursday, January 18, 2001 3:25 PM
Subject: Re: SCSI Patches - mostly on/off-line stuff
Quoted 29 lines
Quoted 9 lines · David Brownell
Well, now that I think of it ... this may be a good way to capture the two different kinds of "remove". Think of "remove" as breaking a binding between device and driver, and these scenarios:
- One "remove" is done by removing the hardware. That can't really be reversed ... gotta clean up any messy device and "higher level" state, errors all around.
Quoted 7 lines
- Another is done by sysadmin request. Hardware still there, driver still there ... but they're not bound. (Maybe it's install-new-driver time, say, or to make sure hardware removal won't cause trouble.)
In that latter case there's a lot of flexibility. Why are you thinking a "remove" might want to get undone?
Would you clarify how you envision a user working with a pending remove? Is it an implementation where a long period of time might pass before the the device is actually removed?
Could something like this happen?
- I start compiling XFree86 on a removable device and tell the system to unmount the media and remove the device when the compile is complete, a "pending remove" is set.
Quoted 8 lines
- During the compile, I realize that I want to make some additional source modifications, so I want to cancel the device removal and continue accessing the device.
I may be misunderstanding the sequence of events and how that might impact a user.
Miles
_______________________________________________
Linux-hotplug-devel mailing list http://linux-hotplug.sourceforge.net
[email protected]
http://lists.sourceforge.net/lists/listinfo/linux-hotplug-devel
Quoted 3 lines · Venkatesh Ramamurthy
[Venkat] I agree that bus rescan is a better way. If the SCSI
devices are connected though aSAF_TEprocessor then we can poll the SCSI Bus frequently, as theSAF_TEknows about addition and detection.
Polling is not practical, because for some bus changes the kernel must know immediately of the change and start using a new bus configuration - for example in 1394 if the root device bus location has changed. Also the user expects to see a 1394 device appear immediately after it is attached. So the rescan for 1394 must be event driven.
To help answer Eric's question about types of events that need to be handled, for 1394 all device (node) IDs may change after a bus reset which automatically occurs every time a device is added or removed and for all 1394 host adapters generates an interrupt. Compare to parallel SCSI where there is no automatic notification (or detection) of a device being removed and the SCSI host adapter may optionally interrupt for a bus reset. For 1394 an OS must reference device GUIDs to determine a mapping from old device ids (pre-bus reset) to new (post-bus reset) device ids. So event driven rescanning is definitely needed for 1394. 1394 also has a generation count which if passed to upper software layers and always included in requests to the lower layer can be used by the lower layer to validate requests. For other busses this generation count would have to be manufactured (e.g. maybe for parallel SCSI increment a counter after a bus reset).
So I think rescanning should occur in the kernel automatically after every bus reset for every different kind of peripheral bus. Every peripheral bus, even parallel SCSI, I think has a bus reset: parallel SCSI, FC, 1394 - yes, ATA - ?, others - ?
Signature · 7 lines
Bob Frey [email protected] _______________________________________________ Linux-hotplug-devel mailing list http://linux-hotplug.sourceforge.net [email protected] http://lists.sourceforge.net/lists/listinfo/linux-hotplug-devel
So I think rescanning should occur in the kernel automatically after
every
bus reset for every different kind of peripheral bus. Every
peripheral bus,
even parallel SCSI, I think has a bus reset: parallel SCSI, FC, 1394
- yes,
ATA - ?, others - ? [Venkat] Bus reset is a very costly operation in SCSI. After doing a
SCSI BUS RESET all devices go to UNIT ATTENTION . All the devices need to cleared from this condition. So in order to detect new devices are we going to do a SCSI BUS RESET every now and then??? Since SCSI dos not support target initiated event notification, we have no choice left. Some kernel thread or a user daemon can initiate a bus scanning.
_______________________________________________
Linux-hotplug-devel mailing list http://linux-hotplug.sourceforge.net
[email protected]
http://lists.sourceforge.net/lists/listinfo/linux-hotplug-devel