SCSI Hotplug for lk 2.5

SCSI Hotplug for lk 2.5

The direction the lk 2.5 series is moving is to do things like device scanning, persistent mapping and other administrative tasks in the user space, where practical.
To this end we need driverfs to:
- work out a device naming strategy
- add its class hierarchy
- allow drivers to set and show parameters (and perhaps state)
- settle down
While this is happening perhaps we can discuss the design of the SCSI Hotplug logic.

The hotplug subsystem provides an established mechanism for drivers in the kernel to inform user space programs primarily that a new device has been "hot plugged" or "hot unplugged". It is used extensively by the USB subsystem in the lk 2.4 series. This idea might be extended to provide user space alerts for important state changes in a device (e.g. removable media inserted).

Here are five categories of SCSI events that are candidates for hotplug alerts:
1) scsi (lower level) driver addition or removal
2) scsi host (initiator) addition or removal
3) scsi device (target/lun) addition or removal
4) removable media addition or removal
5) unit attention or "asynchronous event notification"
I believe hotplug alerts for 2) and 3) are required; for 1) is questionable and for 4) + 5) would be "nice to have".

The existing /sbin/hotplug subsystem would be called from the kernel space with a first "command line" parameter of "scsi" with the following environment variables:

	ACTION		add | remove [ | event {for UA or AEN} ]
	CATEGORY	DRIVER | HOST | TARGET | MEDIA | EVENT
	TARGET_PATH	{for categories TARGET, MEDIA and EVENT}
	HOST_PATH	{for categories MEDIA, TARGET, and HOST}
	DRIVER_PATH	{for all categories}
	PERIPHERAL_TYPE	{for categories MEDIA and TARGET}
	SCSI_LEVEL	{for categories MEDIA and TARGET}
	SCSI_PROTOCOL	SPI | FC | USB_MASS_STORAGE | SBP | SAS ...
	.....
	VENDOR
	PRODUCT
	REVISION

There is a trade-off between how much information is passed to /sbin/hotplug as environment variables and how much is available from driverfs and other sources. Timing of the alerts is important:
the ACTION=add should be as late as possible so that all driverfs information is available (and upper level drivers have attached to the device ?).

ACTION=remove (apart from media) comes with no additional driverfs (or other kernel) information because the host/device is gone :-) In the scsimon driver (see note c) ) there is the notion of ADDITION_SEQNO and REMOVAL_SEQNO to track attaches and detaches (in this case of hosts and targets). Sequence numbers are easy (and SMP safe) to implement in the kernel and could act as a key for user space programs that are tracking the state of host and device attachments.

Comments?

Notes:
a) SCSI host and device hotplug alerts could be added relatively
easily to the scsi mid level
b) The "..._PATH" environment variables would be relative

	paths into the driverfs tree. For example:
		TARGET_PATH="root/pci0/00:0c.0/scsi1/1:0:0:0"
		DRIVER_PATH="bus/scsi/drivers/aic7xxx"

c) scsimon is an optional upper level scsi subsystem driver
described at http://www.torque.net/scsi/scsimon.html
Linus has made it clear that he thinks this driver does too much in the kernel space (e.g. holding trailing device state). It has an example of generating hotplug alerts, see sm_call_policy() [which should use snprintf() in lk 2.5]
d) James Bottomley would like to do as much scanning and
device identification as possible in the user space. I still think the mid level needs to do a 36 byte INQUIRY (the LLDDs may want to see a longer inquiry) and perhaps looks for INQUIRY VPD page=0x83
e) for multipath devices I would expect a hotplug alert for
each path a device is discovered on (perhaps this is not practical)
f) this document was written when lk 2.5.29 was current.
lk 2.5.30 has just been released with driverfs patches that address some of the issues mentioned above

Doug Gilbert
1st August 2002

How about new paths to a device? It seems that the FibreChannel people need that.

Merge this with CATEGORY ?

Ideally you'd want reduced functionality without driverfs mounted.

Yes, after that.

This is very bad. You may very well have to change permissions of a device node.

Regards
Oliver

It is not only fibrechannel - dual initiated busses, dual ported devices, maybe iSCSI and maybe other low level drivers can have mutiple paths to the actual device.

Did you see Doug's:

e) for multipath devices I would expect a hotplug alert for
each path a device is discovered on (perhaps this is not practical)

With my current multi-path code, a hotplug per path is possible, and not difficult to add. The device alert (as Doug describes) already includes a path, we probably would also need some flag that says this is the first path; on removal, you would want a flag saying if this was the last path.

User level code would have to notice (for example) that a device was already found and mounted, and that this is just another path to the device.

-- Patrick Mansfield

On a related note....

It would be really nice if a low-level driver could signal, somehow, to the VFS layer that it was a hot-unpluggable device, and thus precautions should be taken (i.e. perfer mounting with -o sync, etc.)

Matt

Matthew Dharm                              Home: [email protected] 
Maintainer, Linux USB Mass Storage Driver

C:  They kicked your ass, didn't they?
S:  They were cheating!
					-- The Chief and Stef
User Friendly, 11/19/1997

I don't think this s wise. By definition all SAN devices are 'hot-unpluggable', and you sure don't want to mount them sync only.

Douglas -

I'm still trying to figure out what events we might need for user versus kernel scanning or some combination. I'm re-reading James original OLS paper about user scanning, and I've been thinking about how to scan from user level. It might be best to do work on user level scanning after we have better driverfs support, and always have initrd.

I've been trying to re-code scsi_scan.c with user level scanning in mind (not such a big deal, since we already have proc scanning), especially so that we can scan a single target, rather than a single LUN via the proc interface. We probably need a target hotplug event, this would be best handled by adding a device_register() for each target; this is not so easy for multi-path, since target id's can be different for the same target (even if the linux scsi id was the FC ID), so it would be easier to just generate a hotplug event that could kick off scanning.

I haven't done anything with the device_list[] (in terms of hotplug or moving it to user space).

My understanding is that driverfs will automatically generate hotplug events on each device_register() call. These have been added to scsi, so the two main events are already handled, and we need not make other changes.

Multi-path will also hook into driverfs, and driverfs would then generate hotplug events for them.

-- Patrick Mansfield