move scsi parts of dm hw handlers to scsi layer

Currently dm-multipath has what it calls hw_handlers that contain hw details and currently those handlers can be used to failover storage works, LSI/Engenio RDAC mode, and EMC Clariion boxes. This code is a little strange in dm-multipath and they are doing scsi no good (think about problems initializing scsi disks/paths when they are in a passive
path that requires explicit/manual activation and we retry over and over
and over).

The patch below begins to push the scsi hw handler code down to the scsi
layer. I only began to covert dm-emc.c and it only hooks in at the sense
decoding in scsi_error.c. I wanted to make sure I was going about the module loading and binding correctly. With a new target bus we could do
some driver model stuff instead, but I was not sure if that was appropriate for this?

Next up would be to get Jens's cmd type tree and work on the message passing code.

And this patch currently does not work since I do not have a Clariion box and I do not know what to match for the {vendor, model} tuple. I think it was something like DCG or DGC and the model had different RAID
strings depending on how it was setup and could have some other string if it did not use RAID. If you have a Clariion or you work for EMC and happen to know this info could you pass those strings to me?

Quoted 10 lines

The patch below begins to push the scsi hw handler code down to the scsi
layer. I only began to covert dm-emc.c and it only hooks in at the sense
decoding in scsi_error.c. I wanted to make sure I was going about the module loading and binding correctly. With a new target bus we could do
some driver model stuff instead, but I was not sure if that was appropriate for this?

Why don't we use scsi_devinfo for this?

We have to have some sort of device table anyway as these handlers are far from being generic, so any sense code which triggers action on one device might be perfectly ok for others.

Easiest way would be to have one BLIST flag for each hardware handler and merge the respective hardware handler into that target if the blacklist entry matches.

Quoted 11 lines

Next up would be to get Jens's cmd type tree and work on the message passing code.

And this patch currently does not work since I do not have a Clariion box and I do not know what to match for the {vendor, model} tuple. I think it was something like DCG or DGC and the model had different RAID
strings depending on how it was setup and could have some other string if it did not use RAID. If you have a Clariion or you work for EMC and happen to know this info could you pass those strings to me?

IIRC this is

'DGC' 'DISK'
'DGC' 'RAID 10'
'DGC' 'RAID 5'

Hrmph. There is one bit which doesn't quite work out.
While the hardware handler knows how to handle error codes and how to switch
paths for a specific device, it doesn't know _when_ to switch it. I don't think it's a clever idea to switch paths whenever you encounter an
passive path. Seems like you could do a nice ping-pong that way ...

Quoted 11 lines · Hannes Reinecke

Quoted 10 lines

The patch below begins to push the scsi hw handler code down to the scsi
layer. I only began to covert dm-emc.c and it only hooks in at the sense
decoding in scsi_error.c. I wanted to make sure I was going about the module loading and binding correctly. With a new target bus we could do
some driver model stuff instead, but I was not sure if that was appropriate for this?

Why don't we use scsi_devinfo for this?

I was adding my fields when I noticed this comment:

* Do not add to this list, use the command line or proc interface to add
* to the scsi_dev_info_list. This table will eventually go away.

Quoted 3 lines

We have to have some sort of device table anyway as these handlers are far from being generic, so any sense code which triggers action on one device might be perfectly ok for others.

When I was looking for the history of that commet, I thought I read that we are supposed to be moving to some userspace approach that pushes that info down via some magic interface.

Quoted 30 lines · Mike Christie

Quoted 13 lines · Hannes Reinecke

Quoted 4 lines

The patch below begins to push the scsi hw handler code down to the
scsi
layer. I only began to covert dm-emc.c and it only hooks in at the

Quoted 9 lines

sense
decoding in scsi_error.c. I wanted to make sure I was going about the
module loading and binding correctly. With a new target bus we could
do
some driver model stuff instead, but I was not sure if that was appropriate for this?

Quoted 1 line

Why don't we use scsi_devinfo for this?

I was adding my fields when I noticed this comment:

* Do not add to this list, use the command line or proc interface to add
* to the scsi_dev_info_list. This table will eventually go away.

Quoted 5 lines

We have to have some sort of device table anyway as these handlers are
far from being generic, so any sense code which triggers action on one
device might be perfectly ok for others.

Quoted 6 lines

When I was looking for the history of that commet, I thought I read that
we are supposed to be moving to some userspace approach that pushes that
info down via some magic interface.

I added this comment at the wrong place. I meant to say I thought we are
supposed to be moving away from the kernel devinfo list to some userspace one that gets sent down via the module_param or some new magic
interface.

Am Fr 21.07.2006 13:55 schrieb Mike Christie <[email protected]>:

Quoted 35 lines

Quoted 28 lines · Mike Christie

Quoted 13 lines · Hannes Reinecke

Quoted 4 lines

The patch below begins to push the scsi hw handler code down to the
scsi
layer. I only began to covert dm-emc.c and it only hooks in at the

Quoted 9 lines

sense
decoding in scsi_error.c. I wanted to make sure I was going about the
module loading and binding correctly. With a new target bus we could
do
some driver model stuff instead, but I was not sure if that was appropriate for this?

Quoted 1 line

Why don't we use scsi_devinfo for this?

I was adding my fields when I noticed this comment:

* Do not add to this list, use the command line or proc interface to add
* to the scsi_dev_info_list. This table will eventually go away.

Quoted 5 lines

We have to have some sort of device table anyway as these handlers are
far from being generic, so any sense code which triggers action on one
device might be perfectly ok for others.

Quoted 6 lines

When I was looking for the history of that commet, I thought I read that
we are supposed to be moving to some userspace approach that pushes that
info down via some magic interface.

Quoted 6 lines

I added this comment at the wrong place. I meant to say I thought we are
supposed to be moving away from the kernel devinfo list to some userspace one that gets sent down via the module_param or some new magic
interface.

Or so they claim. I seem to remember some discussion about it; the net result was the scsi_devinfo will stay with us for the time being.

Otherwise you'll end up having to configure your kernel / module during startup. With parameters which are static anyway. Can't say I like it. And the tricky bit is that these information has to be present prior to any initialisation, so you basically have to feed it during modprobe time. Not really clever.

IIRC this is

'DGC' 'DISK'
'DGC' 'RAID 10'
'DGC' 'RAID 5'

Hrmph. There is one bit which doesn't quite work out.
While the hardware handler knows how to handle error codes and how to switch
paths for a specific device, it doesn't know _when_ to switch it. I don't think it's a clever idea to switch paths whenever you encounter an
passive path. Seems like you could do a nice ping-pong that way ...

Cheers,

Hannes

-
To unsubscribe from this list: send the line "unsubscribe linux-scsi" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html

Quoted 44 lines · Hannes Reinecke

Am Fr 21.07.2006 13:55 schrieb Mike Christie <[email protected]>:

Quoted 35 lines

Quoted 28 lines · Mike Christie

Quoted 13 lines · Hannes Reinecke

Quoted 4 lines

The patch below begins to push the scsi hw handler code down to the
scsi
layer. I only began to covert dm-emc.c and it only hooks in at the

Quoted 9 lines

sense
decoding in scsi_error.c. I wanted to make sure I was going about the
module loading and binding correctly. With a new target bus we could
do
some driver model stuff instead, but I was not sure if that was appropriate for this?

Quoted 1 line

Why don't we use scsi_devinfo for this?

I was adding my fields when I noticed this comment:

* Do not add to this list, use the command line or proc interface to add
* to the scsi_dev_info_list. This table will eventually go away.

Quoted 5 lines

We have to have some sort of device table anyway as these handlers are
far from being generic, so any sense code which triggers action on one
device might be perfectly ok for others.

Quoted 6 lines

When I was looking for the history of that commet, I thought I read that
we are supposed to be moving to some userspace approach that pushes that
info down via some magic interface.

Quoted 6 lines

I added this comment at the wrong place. I meant to say I thought we are
supposed to be moving away from the kernel devinfo list to some userspace one that gets sent down via the module_param or some new magic
interface.

Quoted 9 lines

Or so they claim. I seem to remember some discussion about it; the net result was the scsi_devinfo will stay with us for the time being.

Otherwise you'll end up having to configure your kernel / module during startup. With parameters which are static anyway. Can't say I like it. And the tricky bit is that these information has to be present prior to any initialisation, so you basically have to feed it during modprobe time. Not really clever.

He He fun :)

Sticking what we need in devinfo is a lot easier. And I think it makes sense since the devices info we want to bind with is in there already. If nobody says anything, I will send the next version of the path with devinfo integration.

Quoted 53 lines · Mike Christie

Quoted 43 lines · Hannes Reinecke

Am Fr 21.07.2006 13:55 schrieb Mike Christie <[email protected]>:

Quoted 35 lines

Quoted 28 lines · Mike Christie

Quoted 13 lines · Hannes Reinecke

Quoted 4 lines

The patch below begins to push the scsi hw handler code down to the
scsi
layer. I only began to covert dm-emc.c and it only hooks in at the

Quoted 9 lines

sense
decoding in scsi_error.c. I wanted to make sure I was going about the
module loading and binding correctly. With a new target bus we could
do
some driver model stuff instead, but I was not sure if that was appropriate for this?

Quoted 1 line

Why don't we use scsi_devinfo for this?

I was adding my fields when I noticed this comment:

* Do not add to this list, use the command line or proc interface to add
* to the scsi_dev_info_list. This table will eventually go away.

Quoted 5 lines

We have to have some sort of device table anyway as these handlers are
far from being generic, so any sense code which triggers action on one
device might be perfectly ok for others.

Quoted 6 lines

When I was looking for the history of that commet, I thought I read that
we are supposed to be moving to some userspace approach that pushes that
info down via some magic interface.

Quoted 6 lines

I added this comment at the wrong place. I meant to say I thought we are
supposed to be moving away from the kernel devinfo list to some userspace one that gets sent down via the module_param or some new magic
interface.

Quoted 9 lines

Or so they claim. I seem to remember some discussion about it; the net result was the scsi_devinfo will stay with us for the time being.

Otherwise you'll end up having to configure your kernel / module during startup. With parameters which are static anyway. Can't say I like it. And the tricky bit is that these information has to be present prior to any initialisation, so you basically have to feed it during modprobe time. Not really clever.

Quoted 6 lines

He He fun :)

Sticking what we need in devinfo is a lot easier. And I think it makes sense since the devices info we want to bind with is in there already. If nobody says anything, I will send the next version of the path with devinfo integration.

I think Patrick added the comment and the interface so he can add the history. One can use the module or proc interface to pass update devinfo information in (the sysfs migration never was done). Well it has the drawback stated above it can address the issue that certain devices can be supported without a kernel recompile.

Post storage summit I started creating a hardware handler in SCSI, but for some reason that I do not recall I started working on SDEV state model change integrated into devinfo. The thought being that devices would come up in a standby state and then all the varied commands to determine path state could be executed from user space.

Well it did solve the issue I was trying to address (passive paths generating errors on startup), it would need user space assistance with all the plus / minus issues that brings.

-----Original Message-----
From: [email protected] [mailto:[email protected]] On Behalf Of Hannes Reinecke
Sent: Friday, July 21, 2006 7:41 AM
To: Mike Christie
Cc: [email protected]; [email protected]; [email protected]
Subject: Re: [PATCH RFC] move scsi parts of dm hw handlers to scsi layer

Am Fr 21.07.2006 13:20 schrieb Mike Christie <[email protected]>:

Quoted 4 lines

Currently dm-multipath has what it calls hw_handlers that contain hw details and currently those handlers can be used to failover storage works, LSI/Engenio RDAC mode, and EMC Clariion boxes. This code is a little strange in dm-multipath and they are doing scsi no

good (think

Quoted 6 lines

about problems initializing scsi disks/paths when they are in a passive
path that requires explicit/manual activation and we retry over and over
and over).

Yes, good idea. And it would even solve the problem of getting all this
weird
error messages during booting as now these handlers are independently of
of
the multipathing configuration and we're now able to handle weird path
configurations natively.

Quoted 5 lines

The patch below begins to push the scsi hw handler code down to the scsi
layer. I only began to covert dm-emc.c and it only hooks in at the sense
decoding in scsi_error.c. I wanted to make sure I was going

about the

Quoted 5 lines

module loading and binding correctly. With a new target bus we could do
some driver model stuff instead, but I was not sure if that was appropriate for this?

Why don't we use scsi_devinfo for this?
We have to have some sort of device table anyway as these handlers are far from being generic, so any sense code which triggers action on one device might be perfectly ok for others.

Easiest way would be to have one BLIST flag for each hardware handler and merge the respective hardware handler into that target if the blacklist entry matches.

Quoted 4 lines

Next up would be to get Jens's cmd type tree and work on the message passing code.

And this patch currently does not work since I do not have

a Clariion

Quoted 4 lines

box and I do not know what to match for the {vendor, model} tuple. I think it was something like DCG or DGC and the model had different RAID
strings depending on how it was setup and could have some

other string

Quoted 1 line

if it did not use RAID. If you have a Clariion or you work

for EMC and

Quoted 2 lines

happen to know this info could you pass those strings to me?

IIRC this is

'DGC' 'DISK'
'DGC' 'RAID 10'
'DGC' 'RAID 5'

Hrmph. There is one bit which doesn't quite work out. While the hardware handler knows how to handle error codes and how to switch
paths for a specific device, it doesn't know _when_ to switch it. I don't think it's a clever idea to switch paths whenever you encounter
an
passive path. Seems like you could do a nice ping-pong that way ...

Cheers,

Hannes

-
To unsubscribe from this list: send the line "unsubscribe linux-scsi" in
the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html

Quoted 5 lines

IIRC this is

'DGC' 'DISK'
'DGC' 'RAID 10'
'DGC' 'RAID 5'

And 'DGC' 'LUNZ'

What about supporting a wildcard model string?

Quoted 32 lines · Mike Christie

Quoted 31 lines · Hannes Reinecke

Am Fr 21.07.2006 13:55 schrieb Mike Christie <[email protected]>:

Quoted 28 lines

Quoted 15 lines · Mike Christie

Hannes Reinecke wrote:

I was adding my fields when I noticed this comment:

* Do not add to this list, use the command line or proc interface to add
* to the scsi_dev_info_list. This table will eventually go away.

Quoted 5 lines

We have to have some sort of device table anyway as these handlers are
far from being generic, so any sense code which triggers action on one
device might be perfectly ok for others.

Quoted 6 lines

When I was looking for the history of that commet, I thought I read that
we are supposed to be moving to some userspace approach that pushes that
info down via some magic interface.

I added this comment at the wrong place. I meant to say I thought we are
supposed to be moving away from the kernel devinfo list to some userspace one that gets sent down via the module_param or some new magic
interface.

Quoted 8 lines

Quoted 8 lines

Or so they claim. I seem to remember some discussion about it; the net result was the scsi_devinfo will stay with us for the time being.

Otherwise you'll end up having to configure your kernel / module during startup. With parameters which are static anyway. Can't say I like it. And the tricky bit is that these information has to be present prior to any initialisation, so you basically have to feed it during modprobe time. Not really clever.

Quoted 6 lines

He He fun :)

Sticking what we need in devinfo is a lot easier. And I think it makes sense since the devices info we want to bind with is in there already. If nobody says anything, I will send the next version of the path with devinfo integration.

I think Patrick added the comment and the interface so he can add the history. One can use the module or proc interface to pass update devinfo information in (the sysfs migration never was done). Well it has the drawback stated above it can address the issue that certain devices can be supported without a kernel recompile.

Post storage summit I started creating a hardware handler in SCSI, but for some reason that I do not recall I started working on SDEV state model change integrated into devinfo. The thought being that devices would come up in a standby state and then all the varied commands to determine path state could be executed from user space.

Well it did solve the issue I was trying to address (passive paths generating errors on startup), it would need user space assistance with all the plus / minus issues that brings.

Quoted 30 lines · Mike Anderson

Quoted 9 lines · Mike Christie

He He fun :)

Sticking what we need in devinfo is a lot easier. And I think it makes
sense since the devices info we want to bind with is in there already.
If nobody says anything, I will send the next version of the path with
devinfo integration.

I think Patrick added the comment and the interface so he can add the
history. One can use the module or proc interface to pass update devinfo
information in (the sysfs migration never was done). Well it has the drawback stated above it can address the issue that certain devices can be
supported without a kernel recompile.

Post storage summit I started creating a hardware handler in SCSI, but for
some reason that I do not recall I started working on SDEV state model
change integrated into devinfo. The thought being that devices would come
up in a standby state and then all the varied commands to determine path
state could be executed from user space.

Quoted 5 lines

Well it did solve the issue I was trying to address (passive paths generating errors on startup), it would need user space assistance with all
the plus / minus issues that brings.

Yeah, I am fine with all that and have no complaints. How can I complain
when I always pop up with the do it in userspace comment :)

My main focus for this was not really the startup. That was a side benefit to having the sense and that info in the kernel for failover requests and IO that is executed when the path is passive or active.

So today, we could use your userspace start up for bring up but we still
need something to do failover and process those results and possibly process the results of other IO (similar to the emc first IO to the path
case in the patch). For the latter, I think the EMC check_sense tests I
am doing in my patch are already handled by other checks in scsi-ml so the sense stuff in scsi_emc_clariion.c is not exactly needed. So maybe we do not need hw handlers for that. As I have said before, I keep running into this problem where vendors say yes we need it but we do not
have many good examples.

But if scsi-ml's sense decoding is ok by default and we do not need to override it, this just leaves explicit/manual failover. If we throw that
to userspace along with your strart up code then I guess we do not need
kernel hw handlers at all.

Did you guys end up doing a hw handler that runs in userspace for xdr or
something? Has it worked out? Is there some common code that can be reused today by any chance? Or is this one of those cases where we are going too far with putting stuff in userspace do you think?

Quoted 73 lines · Mike Anderson

Quoted 53 lines · Mike Christie

Quoted 43 lines · Hannes Reinecke

Am Fr 21.07.2006 13:55 schrieb Mike Christie <[email protected]>:

Quoted 35 lines

Quoted 28 lines · Mike Christie

Quoted 13 lines · Hannes Reinecke

Quoted 4 lines

The patch below begins to push the scsi hw handler code down to the
scsi
layer. I only began to covert dm-emc.c and it only hooks in at the

Quoted 9 lines

sense
decoding in scsi_error.c. I wanted to make sure I was going about the
module loading and binding correctly. With a new target bus we could
do
some driver model stuff instead, but I was not sure if that was appropriate for this?

Quoted 1 line

Why don't we use scsi_devinfo for this?

I was adding my fields when I noticed this comment:

* Do not add to this list, use the command line or proc interface to add
* to the scsi_dev_info_list. This table will eventually go away.

Quoted 5 lines

We have to have some sort of device table anyway as these handlers are
far from being generic, so any sense code which triggers action on one
device might be perfectly ok for others.

Quoted 6 lines

When I was looking for the history of that commet, I thought I read that
we are supposed to be moving to some userspace approach that pushes that
info down via some magic interface.

Quoted 6 lines

I added this comment at the wrong place. I meant to say I thought we are
supposed to be moving away from the kernel devinfo list to some userspace one that gets sent down via the module_param or some new magic
interface.

Quoted 9 lines

Or so they claim. I seem to remember some discussion about it; the net result was the scsi_devinfo will stay with us for the time being.

Otherwise you'll end up having to configure your kernel / module during startup. With parameters which are static anyway. Can't say I like it. And the tricky bit is that these information has to be present prior to any initialisation, so you basically have to feed it during modprobe time. Not really clever.

Quoted 6 lines

He He fun :)

Sticking what we need in devinfo is a lot easier. And I think it makes sense since the devices info we want to bind with is in there already. If nobody says anything, I will send the next version of the path with devinfo integration.

I think Patrick added the comment and the interface so he can add the history. One can use the module or proc interface to pass update devinfo information in (the sysfs migration never was done). Well it has the drawback stated above it can address the issue that certain devices can be supported without a kernel recompile.

Post storage summit I started creating a hardware handler in SCSI, but for some reason that I do not recall I started working on SDEV state model change integrated into devinfo. The thought being that devices would come up in a standby state and then all the varied commands to determine path state could be executed from user space.

Quoted 4 lines

Well it did solve the issue I was trying to address (passive paths generating errors on startup), it would need user space assistance with all the plus / minus issues that brings.

Yeah, I am fine with all that and have no complaints. How can I complain when I always pop up with the do it in userspace comment :)

My main focus for this was not really the startup. That was a side benefit to having the sense and that info in the kernel for failover requests and IO that is executed when the path is passive or active.

So today, we could use your userspace start up for bring up but we still need something to do failover and process those results and possibly process the results of other IO (similar to the emc first IO to the path case in the patch). For the latter, I think the EMC check_sense tests I am doing in my patch are already handled by other checks in scsi-ml so the sense stuff in scsi_emc_clariion.c is not exactly needed. So maybe we do not need hw handlers for that. As I have said before, I keep running into this problem where vendors say yes we need it but we do not have many good examples.

But if scsi-ml's sense decoding is ok by default and we do not need to override it, this just leaves explicit/manual failover. If we throw that to userspace along with your strart up code then I guess we do not need kernel hw handlers at all.

Did you guys end up doing a hw handler that runs in userspace for xdr or something? Has it worked out? Is there some common code that can be reused today by any chance? Or is this one of those cases where we are going too far with putting stuff in userspace do you think?