dm-mpath and scsi persistent reservation
Subsystems (2)
Hi,
the dm-mpath multipathing driver enqueues write ios returned by the scsi layer with a "reservation conflict" error (on assymetric storage controllers, like Clariion, where queue_if_no_path is enabled).
This is wrong, and a potent data-corrupter : wio sent to a reserved scsi device should never be retried on this same device.
Can someone advise on a viable solution for this problem ? I understand the Device Mapper being a block remapper, it is not expected to receive scsi errors directly ... but maybe some kind of translation might be acceptable. Or is there hope in the request-based remapper ?
Jens, James, Alasdair, ... as maintainers of the involved subsystems, would you care to give some advise on the issue.
Regards,
cvaroqui
Christophe,
The SCSI Hardware handler is created for devices with special needs. It is available in 2.6.27. One can be written for any device to translate the scsi sense code to a different error code that the dm understands. Is this something that coule help this situation ?
Have a look at drivers/scsi/device_handler directory.
Hope this helps,
chandra
Quoted 23 lines · Christophe Varoqui
Hi,
the dm-mpath multipathing driver enqueues write ios returned by the scsi layer with a "reservation conflict" error (on assymetric storage controllers, like Clariion, where
queue_if_no_pathis enabled).This is wrong, and a potent data-corrupter : wio sent to a reserved scsi device should never be retried on this same device.
Can someone advise on a viable solution for this problem ? I understand the Device Mapper being a block remapper, it is not expected to receive scsi errors directly ... but maybe some kind of translation might be acceptable. Or is there hope in the request-based remapper ?
Jens, James, Alasdair, ... as maintainers of the involved subsystems, would you care to give some advise on the issue.
Regards,
cvaroqui
--
To unsubscribe from this list: send the line "unsubscribe linux-scsi" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html
It seems to me the device handler infrastructure proposes to translate scsi error codes from requests generated by the device handler itself. I don't know how we can detect a reservation conflict from a device handler without submitting a dangerous write io.
I don't see how we could use a device handler to translate an scsi error code from a write io submitted to the multipath device map. Do you ?
Regards,
cvaroqui
Quoted 32 lines
The SCSI Hardware handler is created for devices with special needs. It is available in 2.6.27. One can be written for any device to translate the scsi sense code to a different error code that the dm understands. Is this something that coule help this situation ?
Have a look at
drivers/scsi/device_handlerdirectory.Hope this helps,
chandra
Quoted 20 lines · Christophe Varoqui
Hi,
the dm-mpath multipathing driver enqueues write ios returned by the scsi layer with a "reservation conflict" error (on assymetric storage controllers, like Clariion, where
queue_if_no_pathis enabled).This is wrong, and a potent data-corrupter : wio sent to a reserved scsi device should never be retried on this same device.
Can someone advise on a viable solution for this problem ? I understand the Device Mapper being a block remapper, it is not expected to receive scsi errors directly ... but maybe some kind of translation might be acceptable. Or is there hope in the request-based remapper ?
Jens, James, Alasdair, ... as maintainers of the involved subsystems, would you care to give some advise on the issue.
James Bottomley
maintainerQuoted 4 lines · Christophe Varoqui
It seems to me the device handler infrastructure proposes to translate scsi error codes from requests generated by the device handler itself. I don't know how we can detect a reservation conflict from a device handler without submitting a dangerous write io.
For SCSI-2 reservations, Test Unit Ready will do this for you.
For SCSI-3, you're right, it's more complex. You actually have to use the PR IN commands to read the reservations if you don't want to test what they'll actually do with an I/O.
Quoted 2 lines
I don't see how we could use a device handler to translate an scsi error code from a write io submitted to the multipath device map. Do you ?
Well, there is a problem. Reservation Conflict should be treated as a device error and passed straight up ... it shouldn't really have any effect on dm mp because a path switch is unlikely to fix any issues. So dm mp shouldn't be intercepting this type of error at all.
James
Quoted 8 lines · James Bottomley
Quoted 2 lines
I don't see how we could use a device handler to translate an scsi error code from a write io submitted to the multipath device map. Do you ?
Well, there is a problem. Reservation Conflict should be treated as a device error and passed straight up ... it shouldn't really have any effect on dm mp because a path switch is unlikely to fix any issues. So dm mp shouldn't be intercepting this type of error at all.
I think what Christophe was asking for is something like this:
[RFC PATCH 1/4] convert block layer drivers to blkerr
http://marc.info/?l=linux-scsi&m=112487427230642&w=2
[RFC PATCH 2/4] convert dm to blkerr error values
http://marc.info/?l=linux-scsi&m=112487427306501&w=2
[RFC PATCH 3/4] convert dm-multipath to blkerr error
http://marc.info/?l=linux-scsi&m=112487431524436&w=2
[RFC PATCH 4/4] convert scsi to blkerr error values
http://marc.info/?l=linux-scsi&m=112487431524350&w=2
something that allows lower layers to give the upper layers some extra info. In the patches the scsi layer would return a fatal device error, and device mapper multipath would see that and just fail the IO instead of retrying on a new path.
I do not like my implementation in those patches, but I did not have time in the past to rework them. I can now though if you guys have any comments. I am really struggling on the definition of the block layer errors codes and what info they should convey. For example I was not sure if they should they give a hint about if the error is fatal or retryable like in the patches above or should they describe what happened?
Quoted 2 lines · Christophe Varoqui
It seems to me the device handler infrastructure proposes to translate scsi error codes from requests generated by the device handler itself. I
No, the handler doesn't generate the requests.
Device handler's check_sense() function is called from the scsi_error.c:scsi_check_sense() function whenever a sense code is returned from the device (on normal I/Os).
When in the sense function, the device handler can do any action (like closing the other path(s)) and returning an appropriate error code (such that dm doesn't retry the I/O on other paths).
Quoted 5 lines
don't know how we can detect a reservation conflict from a device handler without submitting a dangerous write io.
I don't see how we could use a device handler to translate an scsi error code from a write io submitted to the multipath device map. Do you ?
That is what it actually does, it intercepts the sense code returned from the device on normal I/Os.
Quoted 39 lines
Regards,
cvaroquiQuoted 6 lines
The SCSI Hardware handler is created for devices with special needs. It is available in 2.6.27. One can be written for any device to translate the scsi sense code to a different error code that the dm understands. Is this something that coule help this situation ?
Quoted 26 lines
Have a look at
drivers/scsi/device_handlerdirectory.Hope this helps,
chandra
Quoted 20 lines · Christophe Varoqui
Hi,
the dm-mpath multipathing driver enqueues write ios returned by the scsi layer with a "reservation conflict" error (on assymetric storage controllers, like Clariion, where
queue_if_no_pathis enabled).This is wrong, and a potent data-corrupter : wio sent to a reserved scsi device should never be retried on this same device.
Can someone advise on a viable solution for this problem ? I understand the Device Mapper being a block remapper, it is not expected to receive scsi errors directly ... but maybe some kind of translation might be acceptable. Or is there hope in the request-based remapper ?
Jens, James, Alasdair, ... as maintainers of the involved subsystems, would you care to give some advise on the issue.
--
To unsubscribe from this list: send the line "unsubscribe linux-scsi" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html
Quoted 8 lines · Chandra Seetharaman
Quoted 3 lines · Christophe Varoqui
It seems to me the device handler infrastructure proposes to translate scsi error codes from requests generated by the device handler itself. I
No, the handler doesn't generate the requests.
Quoted 8 lines
Device handler's
check_sense()function is called from the scsi_error.c:scsi_check_sense() function whenever a sense code is returned from the device (on normal I/Os).When in the sense function, the device handler can do any action (like closing the other path(s)) and returning an appropriate error code (such that dm doesn't retry the I/O on other paths).
Thanks for the clarification. I shouldn't have stopped after reading the hp_sw handler, which does not implement a .check_sense
So now, device handlers indeed look promising. I still see difficulties though :
1/ reservation conflict is a scmd status, and as such short-cuts the scsi_check_sense function altogether. Sould we hook device handlers to status parser too ?
2/ should we create an trivial device handler and load multi-handlers multipath maps or should we make this device handler implicitly always active ?
Regards,
cvaroqui
--
To unsubscribe from this list: send the line "unsubscribe linux-scsi" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html
Quoted 7 lines · Christophe Varoqui
Quoted 5 lines
It seems to me the device handler infrastructure proposes to translate scsi error codes from requests generated by the device handler itself. I don't know how we can detect a reservation conflict from a device handler without submitting a dangerous write io.
Quoted 6 lines
For SCSI-2 reservations, Test Unit Ready will do this for you.
For SCSI-3, you're right, it's more complex. You actually have to use the PR IN commands to read the reservations if you don't want to test what they'll actually do with an I/O.
The PR-IN "READ FULL STATUS" looked promising indeed, bu does not work on any storage hardware I tried. Always ends up there (in sg_persist.c) :
293 else if (SG_LIB_CAT_ILLEGAL_REQ == res)
294 fprintf(stderr, "PR in: bad field in cdb including "
295 "unsupported service action\n");
Other PR-INs would list the registration keys and reservation, but how would we know from the kernel or from userspace multipath which keys are associated with host's I_T nexus ? The key would likely have been registered by a userspace clusterware for fencing purpose.
PR-OUT "REGISTER" with params rk=0 sark=0 may trigger a significative difference when send over a registered I_T or not (based on SPC-3 - Table 33 — Register behaviors for a REGISTER service action).
Did you have something different in mind ?
Quoted 9 lines
Quoted 3 lines
I don't see how we could use a device handler to translate an scsi error code from a write io submitted to the multipath device map. Do you ?
Well, there is a problem. Reservation Conflict should be treated as a device error and passed straight up ... it shouldn't really have any effect on dm mp because a path switch is unlikely to fix any issues. So dm mp shouldn't be intercepting this type of error at all.
Well, a path switch might be a valid behaviour considering persistent reservations are per I_T nexus ... the io may succeed if submited from another intiator for example. But anyway, if all paths failed the io should not be queued.
Regards,
cvaroqui
--
To unsubscribe from this list: send the line "unsubscribe linux-scsi" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html