I want scsi_target_block() in interrupt context

I'm working on an implementation of SCSI RDMA Protocol (SRP) on InfiniBand, and I'm running into an issue managing the number of commands that get queued.

The setup is like this: each SRP scsi_host can connect to multiple target ports. Each target port can have multiple LUNs behind it. To support this, for each connection, I create a device, set its parent to the scsi_host's shost_gendev, and then call scsi_scan_target to discover the LUNs behind the target port.

This mostly works fine, except the number of outstanding commands is limited per connection. This doesn't fit that well with the Linux SCSI stack -- can_queue for the scsi_host doesn't work, since a single scsi_host can have multiple connections, and cmd_per_lun doesn't work since a single connection can have many LUNs.

What seems like it _would_ be exactly what I need would be calling scsi_target_block() on the device representing the connection. Unfortunately, scsi_target_block() uses all sorts of device model stuff that takes semaphores and hence potentially sleeps. This means that I can't block a connection while holding a lock to prevent races.

So I don't see a race-free way I can ensure that I never queue up too many commands on a given connection.

Any advice?

Thanks a lot,
Roland

Just allocate one scsi_host per connection.

Christoph> Just allocate one scsi_host per connection.

OK, that solves the queuing problem nicely.

Right now, I create a scsi_host for each local IB port. For each scsi_host, I have a writable sysfs attribute that userspace can put the addresses of remote ports to connect to. Any preference for how to implement the mechanism for userspace to pass in a remote address to connect to?

Also, with my current scheme, I have a nice hierarchy of devices from host port to target port to LUN. Having this topology available seems nice -- are we losing anything useful for multipathing etc. by not having a device for the host port?

Thanks,
Roland

for iscsi and tcp connections, open-iscsi did the socket connection stuff in userspace then passed the fd down to the driver which used sockfd_lookup().

Mike> for iscsi and tcp connections, open-iscsi did the socket Mike> connection stuff in userspace then passed the fd down to the Mike> driver which used sockfd_lookup().

How does userspace pass the fd down? Through a character device?

Thanks,
Roland

open-iscsi uses a netlink socket for most of its interface due to how it pushed a lot of code to userspace. You could just do sysfs I bet, or are you having a problem of not having some place to to hang the initial setup attributes.

Mike> open-iscsi uses a netlink socket for most of its interface Mike> due to how it pushed a lot of code to userspace. You could Mike> just do sysfs I bet, or are you having a problem of not Mike> having some place to to hang the initial setup attributes.

Yeah, that's the question -- without a scsi_host, what's the natural place to put an attribute in sysfs?

- R.

Don't know. For iscsi-sfnet we tried puttting it a lot of difference places: our own class, made a virtual driver model bus, on our /sys/module dir. configfs ended up working ok.