Possible bug on SRP device delete
Hello all,
on the SCST-devel mailing list I posed the following question regarding the removal of mapped SRP disks. I have been suggested to the RDMA mailing list as it might be an ib_srp bug:
Quoted 26 lines
Quoted 19 lines
5) If I echo one line of ibsrpdm -c to
/sys/class/infiniband_srp/srp-mthca0-1/add_targetso to add a srp disk
to my system, what happens is that anothersrp_hostgets created. If I
later remove the drive with "echo 1 >/sys/block/sdX/device/delete", the/sys/class/srp_hosts/srp_hostXYremains. Now after having remapped and
deleted a few times the disk I am totally polluted of srp_host's. Another serious problem is thatsrp_daemon(from srptools) thinks the
disk is still connected because it sees the previoussrp_host, and won't
reconnect it (won't recreate the sdX device).
So what is the proper way to unmap a srp drive also deleting thesrp_host?Quoted 3 lines
Seems like an
ib_srpbug to me. Please report this on the linux-rdma mailing list
(http://vger.kernel.org/vger-lists.html#linux-rdma).Will do. What part seems a bug, the fact
srp_hostXYdoes not go away together with the device?
Quoted 6 lines
Please keep in mind that with the SRP protocol multiple LUNs share one RDMA channel. That RDMA channel won't be closed and reopened because a single host is deleted.
ib_srphowever should reconnect the RDMA channel if /eh_host_reset_handler/ is invoked on /ib_srp/. Notes:
- IMHOib_srpshould be modified such that it reports SCSI commands as
failed as soon as an IB error completion has been received instead of waiting until these commands time out. That would makeib_srprecover much more quickly from cable reconnects or SRP target restarts.
- Please don't usesg_resetonib_srp, or you will hit this:
https://bugzilla.kernel.org/show_bug.cgi?id=13893.
I can't currently understand the whole explanation, but please have a look if you think there is a bug.
Also I would like to know, as per my original question, what was the correct procedure for also removing the srp_host. There should be some way to remove it, shouldn't it?
Thank you
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html
Quoted 19 lines · On Mon, 2010-12-13 at 17:32 +0100, torn5 wrote:
I have been suggested to the RDMA mailing list as it might be an
ib_srpbug:Quoted 17 lines
Quoted 17 lines
5) If I echo one line of ibsrpdm -c to
/sys/class/infiniband_srp/srp-mthca0-1/add_targetso to add a srp disk
to my system, what happens is that anothersrp_hostgets created. If I
later remove the drive with "echo 1 >/sys/block/sdX/device/delete", the/sys/class/srp_hosts/srp_hostXYremains. Now after having remapped and
deleted a few times the disk I am totally polluted of srp_host's. Another serious problem is thatsrp_daemon(from srptools) thinks the
disk is still connected because it sees the previoussrp_host, and won't
reconnect it (won't recreate the sdX device).
So what is the proper way to unmap a srp drive also deleting thesrp_host?
There currently no way to disconnect a SRP connection without removing the module. It's an issue, and I've been talking with some people about addressing it, though it hasn't been terribly high on the priority list.
As was mentioned, if you force a host reset, it will drop the connection. Unfortunately, it will also try to reconnect, so this may not be terribly useful.
While it may be possible to use slave_alloc()/slave_destroy() from the scsi_host_template to keep a count of the number of active devices and tear down the connection, I'm not sure this is a good behavior to have, either. I think it may be a bit surprising, and it may also cause srp_daemon to keep re-adding a target that does not present devices to your host.
Quoted 5 lines
Quoted 5 lines
Notes:
- IMHOib_srpshould be modified such that it reports SCSI commands as failed as soon as an IB error completion has been received instead of waiting until these commands time out. That would makeib_srprecover much more quickly from cable reconnects or SRP target restarts.
This is another area that is being worked currently. I don't think it will be ready for 2.6.38, but hopefully for 2.6.39.
Quoted 2 lines
Quoted 2 lines
- Please don't use
sg_resetonib_srp, or you will hit this:
https://bugzilla.kernel.org/show_bug.cgi?id=13893.
I have a tested patch that fixes this, and will try to post it later tonight or tomorrow.
Dave
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html