replace ioctl for sysfs take 2
Quoted 4 lines · AJ Lewis
Do we want all these attributes in the
scsi_hostclass, or should we put them in the iscsi devices directory? So in/sys/bus/iscsi-sfnet/devices/iscsi1- it seems to me these attributes are specific to the iscsi target, and don't need to be in thescsi_hostclass.
I think all attributes that are common over software- and hardware based iscsi implementations should go into a iscsi transport class. Of course it's hard to without a hardware driver to work with. I remember Qlogic said they wanted to submit their driver some time in the future, maybe one of their developers is listening on linux-scsi.
Quoted 3 lines · Christoph Hellwig
Quoted 2 lines · AJ Lewis
Do we want all these attributes in the
scsi_hostclass, or should we put them in the iscsi devices directory? So in/sys/bus/iscsi-sfnet/devices/iscsi1-
Woah, woah, woah. What's this "iscsi-sfnet" directory in sysfs? That's just wrong. No way is that crap going into the kernel. We have to have *one* iscsi infrastructure, not a dozen.
Signature · 6 lines
"Next the statesmen will invent cheap lies, putting the blame upon the nation that is attacked, and every man will be glad of those conscience-soothing falsities, and will diligently study them, and refuse to examine any refutations of them; and thus he will by and by convince himself that the war is just, and will thank God for the better sleep he enjoys after this process of grotesque self-deception." -- Mark Twain
Quoted 10 lines · Matthew Wilcox
Quoted 4 lines · Christoph Hellwig
Quoted 2 lines · AJ Lewis
Do we want all these attributes in the
scsi_hostclass, or should we put them in the iscsi devices directory? So in/sys/bus/iscsi-sfnet/devices/iscsi1-Woah, woah, woah. What's this "iscsi-sfnet" directory in sysfs? That's just wrong.
It is only a pseudo bus for the driver from Christoph's suggestion, that's just to make probing and removing of the driver's devices easier (it looks like a normal scsi driver now).
No way is that crap going into the kernel. We have to have
Quoted 2 lines
*one* iscsi infrastructure, not a dozen.
Don't worry, there is no way we are going to put transport, scsi or anything like that there. This driver will use the scsi-ml infrastucture for sysfs and this includes using the transport classes. I think this comment from AJ, was just in error when he was unaware of scsi-ml's transport classes.
Quoted 11 lines · Mike Christie
Quoted 9 lines · Matthew Wilcox
Quoted 4 lines · Christoph Hellwig
Quoted 2 lines · AJ Lewis
Do we want all these attributes in the
scsi_hostclass, or should we put them in the iscsi devices directory? So in/sys/bus/iscsi-sfnet/devices/iscsi1-
Quoted 7 lines
Quoted 2 lines
Woah, woah, woah. What's this "iscsi-sfnet" directory in sysfs? That's just wrong.
It is only a pseudo bus for the driver from Christoph's suggestion, that's just to make probing and removing of the driver's devices easier (it looks like a normal scsi driver now).
Oh yeah, just to add so you do not have to read through the entire thread. As you know struct device_driver's need a bus to attach devices through, so a scsi_host could have a pci_driver, parent pci_dev and the pci bus for this purpose. Since this driver is software the iscsi-sfnet bus and iscsi-sfnet driver are a virtual bus and driver to emulate what the pci bus does for normal drivers.
For my comment where I state we look like a normal scsi driver I just mean in our struct device_driver->probe we do the scsi_alloc/add_host() and in our struct device_driver-?reomve we do the scsi_remove/put_host().
Quoted 10 lines
No way is that crap going into the kernel. We have to have
Quoted 2 lines
*one* iscsi infrastructure, not a dozen.
Don't worry, there is no way we are going to put transport, scsi or anything like that there. This driver will use the scsi-ml infrastucture for sysfs and this includes using the transport classes. I think this comment from AJ, was just in error when he was unaware of scsi-ml's transport classes.
Quoted 31 lines · Mike Christie
Quoted 12 lines · Mike Christie
Quoted 10 lines · Matthew Wilcox
Quoted 4 lines · Christoph Hellwig
Quoted 2 lines · AJ Lewis
Do we want all these attributes in the
scsi_hostclass, or should we put them in the iscsi devices directory? So in/sys/bus/iscsi-sfnet/devices/iscsi1-Quoted 8 lines
Quoted 2 lines
Woah, woah, woah. What's this "iscsi-sfnet" directory in sysfs? That's just wrong.
It is only a pseudo bus for the driver from Christoph's suggestion, that's just to make probing and removing of the driver's devices easier (it looks like a normal scsi driver now).
Oh yeah, just to add so you do not have to read through the entire thread. As you know struct device_driver's need a bus to attach devices through, so a
scsi_hostcould have apci_driver, parentpci_devand the pci bus for this purpose. Since this driver is software the iscsi-sfnet bus and iscsi-sfnet driver are a virtual bus and driver to emulate what the pci bus does for normal drivers.
Quoted 3 lines
For my comment where I state we look like a normal scsi driver I just mean in our struct
device_driver->probewe do the scsi_alloc/add_host() and in our struct device_driver-?reomve we do the scsi_remove/put_host().
For bus usage and virtual drivers like this and scsi_debug, would it be better to just have a single virtual bus for all virtual drivers to share, or should we instead modify the scsi_bus so that it can handle ULD and virtual LLD struct device_driver attachments. Today, the scsi_bus seems to be used for upper layer drivers becuase they are the only ones needing to attach UL struct device_drivers to scsi devices, but instead of scsi_debug or iscsi-sfnet implementing its own virtual bus the scsi_bus could manage a low level virtual driver's scsi_host parent's devices.
Also, with James's patch we have scsi_devices and scsi_targets off the scsi_bus, does scsi_host's shost_gendev belong there too? What is the purpose of the scsi_bus?
Quoted 17 lines
Quoted 11 lines
No way is that crap going into the kernel. We have to have
Quoted 2 lines
*one* iscsi infrastructure, not a dozen.
Don't worry, there is no way we are going to put transport, scsi or anything like that there. This driver will use the scsi-ml infrastucture for sysfs and this includes using the transport classes. I think this comment from AJ, was just in error when he was unaware of scsi-ml's transport classes.
-
To unsubscribe from this list: send the line "unsubscribe linux-scsi" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html
Quoted 10 lines · Mike Christie
For bus usage and virtual drivers like this and
scsi_debug, would it be better to just have a single virtual bus for all virtual drivers to share, or should we instead modify thescsi_busso that it can handle ULD and virtual LLD structdevice_driverattachments. Today, thescsi_busseems to be used for upper layer drivers becuase they are the only ones needing to attach UL structdevice_driversto scsi devices, but instead ofscsi_debugor iscsi-sfnet implementing its own virtual bus thescsi_buscould manage a low level virtual driver'sscsi_hostparent's devices.
Quoted 3 lines
Also, with James's patch we have
scsi_devicesandscsi_targetsoff thescsi_bus, does scsi_host'sshost_gendevbelong there too? What is the purpose of thescsi_bus?>
Using a virtual bus only makes sense when you actually have multiple virtual drivers that need to attach. Thus, we only have one virtual scsi bus (scsi_bus_type) and it's only used for the generic device embedded in struct scsi_device. We use it only to attach the various ULDs. The host and target devices just have a null bus.
I'm not quite sure what you want to use another bus type for? It sounds like you really want to be using a device class instead. The difference is
- classes provide (potentially multiple) interfaces to existing devices;
- busses are used for keeping track of attached devices and matching them to their drivers.
Unless you really have a use for device matching and driver attachment, you probably just want to stick with classes.
James
Quoted 4 lines · James Bottomley
Quoted 2 lines · Mike Christie
For bus usage and virtual drivers like this and
scsi_debug, would it be better to just have a single virtual bus for all virtual drivers to share, or should we instead modify thescsi_busso that it can handle ULD and virtual LLD structdevice_driverattachments. Today, thescsi_busseems to be used for upper layer drivers becuase they are the only ones needing to attach UL structdevice_driversto scsi devices, but instead ofscsi_debugor iscsi-sfnet implementing its own virtual bus thescsi_buscould manage a low level virtual driver'sscsi_hostparent's devices.
Quoted 8 lines
Quoted 1 line
Also, with James's patch we have
scsi_devicesandscsi_targetsoff thescsi_bus, does scsi_host'sshost_gendevbelong there too? What is the purpose of thescsi_bus?>Using a virtual bus only makes sense when you actually have multiple virtual drivers that need to attach. Thus, we only have one virtual scsi bus (
scsi_bus_type) and it's only used for the generic device embedded in structscsi_device. We use it only to attach the various ULDs. The host and target devices just have a null bus.
In that first target patch you sent it had the target dev's bus set to scsi_bus_type. That caused my confusion as to what the bus was for. I see the new patch with this removed. Nevermind.
Quoted 11 lines
I'm not quite sure what you want to use another bus type for? It sounds like you really want to be using a device class instead. The difference is
- classes provide (potentially multiple) interfaces to existing devices;
- busses are used for keeping track of attached devices and matching them to their drivers.Unless you really have a use for device matching and driver attachment, you probably just want to stick with classes.
We just needed something to track the driver's scsi_hosts. It was also used becuase we used a struct device_driver to hang setup attributes (non iscsi related attrs needed to setup our driver's devices without an ioctl) off of like scsi_debug (this is why I asked about a single virtual bus for both of us to share).
Since we followed the suggestion to do a host per transport endpoint when we rmmod the driver we just need some way to loop over every scsi_host and free them up. We had a linked list of scsi_hosts. Christoph suggested a class or bus instead of the linked list, and had preferred the bus. Now you are suggesting to use a class. It wouldn't be ok to go back to the single host would it? In that case we would not need a class, bus, or linked list?
Quoted 17 lines · Mike Christie
Quoted 11 lines · Matthew Wilcox
Quoted 5 lines · Christoph Hellwig
Quoted 3 lines · AJ Lewis
Do we want all these attributes in the
scsi_hostclass, or should we put them in the iscsi devices directory? So in/sys/bus/iscsi-sfnet/devices/iscsi1-Woah, woah, woah. What's this "iscsi-sfnet" directory in sysfs? That's just wrong.
It is only a pseudo bus for the driver from Christoph's suggestion, that's just to make probing and removing of the driver's devices easier (it looks like a normal scsi driver now).
Quoted 8 lines
Quoted 3 lines
No way is that crap going into the kernel. We have to have *one* iscsi infrastructure, not a dozen.
Don't worry, there is no way we are going to put transport, scsi or anything like that there. This driver will use the scsi-ml infrastucture for sysfs and this includes using the transport classes. I think this comment from AJ, was just in error when he was unaware of scsi-ml's transport classes.
Yeah - that was me being ignorant of the transport classes. Speaking of which, is anyone working on a transport class for iscsi? I see a patch for one in the archives
( http://marc.theaimsgroup.com/?l=linux-scsi&m=108033504301079&w=2 ), but it looks like the discussion sort of died off and nothing ever got accepted. I'd be willing to put together a patch for an iscsi transport class if no one else is working on one right now.
Regards,
Signature · 8 lines
AJ Lewis Voice: 612-638-0500 Red Hat Inc. E-Mail: [email protected] 720 Washington Ave. SE, Suite 200 Minneapolis, MN 55414 Current GPG fingerprint = D9F8 EDCE 4242 855F A03D 9B63 F50C 54A8 578C 8715 Grab the key at: http://people.redhat.com/alewis/gpg.html or one of the many keyservers out there...
Quoted 3 lines · Mike Christie
In that first target patch you sent it had the target dev's bus set to
scsi_bus_type. That caused my confusion as to what the bus was for. I see the new patch with this removed. Nevermind.
er, yes, cut and paste error ...
Quoted 5 lines
We just needed something to track the driver's
scsi_hosts. It was also used becuase we used a structdevice_driverto hang setup attributes (non iscsi related attrs needed to setup our driver's devices without an ioctl) off of likescsi_debug(this is why I asked about a single virtual bus for both of us to share).
Ah, OK, I see. So you mean add a bus to the scsi_host->shost_gendev. That's certainly feasible. With clever matching we could get all LLDs that wanted this functionality to add a dummy driver which they could then use to traverse their attached hosts.
Quoted 7 lines
Since we followed the suggestion to do a host per transport endpoint when we rmmod the driver we just need some way to loop over every
scsi_hostand free them up. We had a linked list ofscsi_hosts. Christoph suggested a class or bus instead of the linked list, and had preferred the bus. Now you are suggesting to use a class. It wouldn't be ok to go back to the single host would it? In that case we would not need a class, bus, or linked list?
A host is the analogue of a bus. In iSCSI that's really the other end point. Using abstractions incorrectly (like a single host for the entire iSCSI system) is bound to end up with problems due to the concept mismatch.
James
Quoted 2 lines · James Bottomley
A host is the analogue of a bus.
That's not a very helpful analogy, since only SPI and FC-AL resemble a bus, and all of the newer SCSI transports are switched fabrics.
Quoted 3 lines
In iSCSI that's really the other end point. Using abstractions incorrectly (like a single host for the entire iSCSI system) is bound to end up with problems due to the concept mismatch.
I have trouble understanding your viewpoint. Your answers to the following questions will hopefully clear things up.
Do you think Linux hosts should be used in a similar way by all switched SCSI transports (e.g. FC-SW, iSCSI, SAS)? If not, why not?
Do you think switched SCSI transports should allocate one Linux host for each I_T nexus?
Do you think switched SCSI transports should allocate one Linux host for each (SAM-2 or SAM-3) SCSI initiator port?
Do you think switched SCSI transports should allocate one Linux host for each (SAM-2 or SAM-3) SCSI initiator device?
Signature · 2 lines
Scott M. Ferris, [email protected]
Quoted 6 lines · Scott M. Ferris
Quoted 2 lines · James Bottomley
A host is the analogue of a bus.
That's not a very helpful analogy, since only SPI and FC-AL resemble a bus, and all of the newer SCSI transports are switched fabrics.
Well, I'm at a bit of a loss to make it plainer ... a host is somewhere you plug your bus this is an obvious concept even for switched fabrics.
The mid layer concept of the host is designed around managing the resources that come with such a physical attachment.
However, since the iSCSI driver is virtual, it thinks more in terms of connected endpoints, so a bus becomes a representation of this virtual connection. But the question of where you want your resources managed seems to have the appropriate answer of "per connection". What you definitely don't want is to have the entire universe of iSCSI devices treated as a single set of resources to manage.
Quoted 6 lines
Quoted 3 lines
In iSCSI that's really the other end point. Using abstractions incorrectly (like a single host for the entire iSCSI system) is bound to end up with problems due to the concept mismatch.
I have trouble understanding your viewpoint. Your answers to the following questions will hopefully clear things up.
I doubt it, but I'll try.
Quoted 2 lines
Do you think Linux hosts should be used in a similar way by all switched SCSI transports (e.g. FC-SW, iSCSI, SAS)? If not, why not?
Yes.
Quoted 2 lines
Do you think switched SCSI transports should allocate one Linux host for each
I_Tnexus?
No.
Quoted 2 lines
Do you think switched SCSI transports should allocate one Linux host for each (SAM-2 or SAM-3) SCSI initiator port?
Yes.
Quoted 2 lines
Do you think switched SCSI transports should allocate one Linux host for each (SAM-2 or SAM-3) SCSI initiator device?
No.
James
Quoted 8 lines · James Bottomley
Quoted 2 lines · Scott M. Ferris
That's not a very helpful analogy, since only SPI and FC-AL resemble a bus, and all of the newer SCSI transports are switched fabrics.
Well, I'm at a bit of a loss to make it plainer ... a host is somewhere you plug your bus this is an obvious concept even for switched fabrics.
So a host is where you plug your (non-existent) bus? I think it would be clearer to use terminology from SAM.
Quoted 8 lines
Quoted 6 lines
Quoted 3 lines
In iSCSI that's really the other end point. Using abstractions incorrectly (like a single host for the entire iSCSI system) is bound to end up with problems due to the concept mismatch.
I have trouble understanding your viewpoint. Your answers to the following questions will hopefully clear things up.
I doubt it, but I'll try.
Thanks.
Quoted 4 lines
Quoted 2 lines
Do you think Linux hosts should be used in a similar way by all switched SCSI transports (e.g. FC-SW, iSCSI, SAS)? If not, why not?
Yes.
I'm glad we agree on that.
Quoted 4 lines
Quoted 2 lines
Do you think switched SCSI transports should allocate one Linux host for each
I_Tnexus?No.
This is exactly why I ask these questions. The iSCSI driver developers just implemented this, because they thought this is what you and Christoph were asking for. Apparently it's not what you wanted.
Quoted 5 lines
Quoted 3 lines
Do you think switched SCSI transports should allocate one Linux host for each (SAM-2 or SAM-3) SCSI initiator port?
Yes.
Thank you. This is a much clear statement of your intent.
Quoted 4 lines
Quoted 2 lines
Do you think switched SCSI transports should allocate one Linux host for each (SAM-2 or SAM-3) SCSI initiator device?
No.
Should all drivers that currently use one host for each SCSI initiator device, and a channel for each initiator port on each device, be modified to use a host for each initiator port?
Documentation/scsi/scsi_mid_low_api.txt says that a host corresponds to a SCSI initiator device. Could someone change that to say SCSI initiator port instead, since that seems to be the new goal?
Signature · 2 lines
Scott M. Ferris, [email protected]
Quoted 2 lines · Scott M. Ferris
So a host is where you plug your (non-existent) bus? I think it would be clearer to use terminology from SAM.
Actually, no. A host is a SAM implementation, not SAM itself. I think the where are the resources question is the clearest way to look at it.
Quoted 9 lines
Quoted 4 lines
Quoted 2 lines
Do you think switched SCSI transports should allocate one Linux host for each
I_Tnexus?No.
This is exactly why I ask these questions. The iSCSI driver developers just implemented this, because they thought this is what you and Christoph were asking for. Apparently it's not what you wanted.
Well, that seems to be the correct thing to do. Most of the time (all of the time?) your virtual connection is to a single storage target, so its identical to a bus with one device on it, isn't it?
Quoted 8 lines
Quoted 4 lines
Quoted 2 lines
Do you think switched SCSI transports should allocate one Linux host for each (SAM-2 or SAM-3) SCSI initiator device?
No.
Should all drivers that currently use one host for each SCSI initiator device, and a channel for each initiator port on each device, be modified to use a host for each initiator port?
I've made no secret about wanting to dump our channel abstraction. However, I'm not going to force a change like this.
James
Quoted 42 lines · Scott M. Ferris
Quoted 9 lines · James Bottomley
Quoted 2 lines · Scott M. Ferris
That's not a very helpful analogy, since only SPI and FC-AL resemble a bus, and all of the newer SCSI transports are switched fabrics.
Well, I'm at a bit of a loss to make it plainer ... a host is somewhere you plug your bus this is an obvious concept even for switched fabrics.
So a host is where you plug your (non-existent) bus? I think it would be clearer to use terminology from SAM.
Quoted 8 lines
Quoted 6 lines
Quoted 3 lines
In iSCSI that's really the other end point. Using abstractions incorrectly (like a single host for the entire iSCSI system) is bound to end up with problems due to the concept mismatch.
I have trouble understanding your viewpoint. Your answers to the following questions will hopefully clear things up.
I doubt it, but I'll try.
Thanks.
Quoted 4 lines
Quoted 2 lines
Do you think Linux hosts should be used in a similar way by all switched SCSI transports (e.g. FC-SW, iSCSI, SAS)? If not, why not?
Yes.
I'm glad we agree on that.
Quoted 3 lines
Quoted 2 lines
Do you think switched SCSI transports should allocate one Linux host for each
I_Tnexus?
Quoted 7 lines
Quoted 1 line
No.
This is exactly why I ask these questions. The iSCSI driver developers just implemented this, because they thought this is what you and Christoph were asking for. Apparently it's not what you wanted.
Thank you for clarifying this. So we should go back to a single linux host, right? The iscsi session is an I_T nexus, so the only way to store the session state in the host is to allocate a session per host.
Quoted 24 lines
Quoted 5 lines
Quoted 3 lines
Do you think switched SCSI transports should allocate one Linux host for each (SAM-2 or SAM-3) SCSI initiator port?
Yes.
Thank you. This is a much clear statement of your intent.
Quoted 4 lines
Quoted 2 lines
Do you think switched SCSI transports should allocate one Linux host for each (SAM-2 or SAM-3) SCSI initiator device?
No.
Should all drivers that currently use one host for each SCSI initiator device, and a channel for each initiator port on each device, be modified to use a host for each initiator port?
Documentation/scsi/scsi_mid_low_api.txtsays that a host corresponds to a SCSI initiator device. Could someone change that to say SCSI initiator port instead, since that seems to be the new goal?
Quoted 9 lines · Mike Christie
Quoted 5 lines
This is exactly why I ask these questions. The iSCSI driver developers just implemented this, because they thought this is what you and Christoph were asking for. Apparently it's not what you wanted.
Thank you for clarifying this. So we should go back to a single linux host, right? The iscsi session is an
I_Tnexus, so the only way to store the session state in the host is to allocate a session per host.
Sometimes, but not always. I believe the goal is now one host for each initiator port (with iSCSI, combination of InitiatorName and ISID).
If you're trying to create multiple iSCSI sessions to different target portals within the same SCSI target port (i.e. to TargetAddresses of the same TargetName with the same TargetPortalGroupTag), you'd need to use a different SCSI initiator port (typically by using a different ISID), which means you'd need to allocate multiple hosts.
Signature · 2 lines
Scott M. Ferris, [email protected]
Quoted 8 lines · Mike Christie
Quoted 4 lines
This is exactly why I ask these questions. The iSCSI driver developers just implemented this, because they thought this is what you and Christoph were asking for. Apparently it's not what you wanted.
Thank you for clarifying this. So we should go back to a single linux host, right? The iscsi session is an
I_Tnexus, so the only way to store the session state in the host is to allocate a session per host.
No; for iscsi, the host and target for I_T is the right thing to do.
The reason why doesn't lie in terminology or a specification, it lies in the code.
When the cisco-iscsi driver was first presented, it had a single host for everything and a huge connection management resource array hanging off that. This was clearly wrong beacuse there was a huge overhead managing the array. The correct approach was to make the host correspond to a single element of that array and use the mid-layer host management functions instead of home grown resource management functions. Doing it this way makes both the management and presentation of the information more logical and the code cleaner.
James
Quoted 17 lines · James Bottomley
Quoted 8 lines · Mike Christie
Quoted 4 lines
This is exactly why I ask these questions. The iSCSI driver developers just implemented this, because they thought this is what you and Christoph were asking for. Apparently it's not what you wanted.
Thank you for clarifying this. So we should go back to a single linux host, right? The iscsi session is an
I_Tnexus, so the only way to store the session state in the host is to allocate a session per host.No; for iscsi, the host and target for
I_Tis the right thing to do.The reason why doesn't lie in terminology or a specification, it lies in the code.
Quoted 8 lines
When the cisco-iscsi driver was first presented, it had a single host for everything and a huge connection management resource array hanging off that. This was clearly wrong beacuse there was a huge overhead managing the array. The correct approach was to make the host correspond to a single element of that array and use the mid-layer host management functions instead of home grown resource management functions. Doing it this way makes both the management and presentation of the information more logical and the code cleaner.
Ok. If the new magic iscsi transport specific target code handles target data like the scsi_host does for hostdata then would it be better to put our session struct in the target data? I think this also simplifies refcounting. Most session stuff makes sense hanging off the target so this might be easier, right?
Quoted 4 lines
James
Quoted 5 lines · Mike Christie
Ok. If the new magic iscsi transport specific target code handles target data like the
scsi_hostdoes for hostdata then would it be better to put our session struct in the target data? I think this also simplifies refcounting. Most session stuff makes sense hanging off the target so this might be easier, right?
Possibly, but only if the way its handled would be the same on all iscsi initiator drivers. Otherwise, if it's specific to the driver implementation, it would be in hostdata. Regardless, you'd still have one host per one of these entities.
The transport class is to abstract out common code from multiple drivers, so what goes in there must genuinely be common. Even if you put the common session data in the target, you probably still end up having driver specific data in the host.
James
Quoted 8 lines · James Bottomley
Quoted 1 line · Mike Christie
Ok. If the new magic iscsi transport specific target code handles target data like the
scsi_hostdoes for hostdata then would it be better to put our session struct in the target data? I think this also simplifies refcounting. Most session stuff makes sense hanging off the target so this might be easier, right?Possibly, but only if the way its handled would be the same on all iscsi initiator drivers. Otherwise, if it's specific to the driver implementation, it would be in hostdata.
Yes, ok. That is what I am striving for. By us hanging things like target name on the target instead of on the scsi device or scsi host will help us conform to other HW iscsi drivers.
Regardless, you'd still have
Quoted 1 line
one host per one of these entities.
One host per I_T nexus/session? But I thought the only reason against using a single host for all nexii (ignoring the port/channel discussion), was that the driver would need its own refcounting.
If the target has some place for driver target data (with the common iscsi code going in the appropriate transport class stucts) couldn't we just do sometihng similar to how a scsi_device has the hostdata pointers and drivers seem to assume when slave_destroy is called they can just free what was allocated in slave_alloc. And when the driver does its final scsi_host_put it assumes the hostdata will get freed when all the handles are released. If the midlayer could provide any similar lifetime management functionality for targets the iscsi driver would not need any private structures that are refcounted internally since everything could fall under the host, device, or target's managemnet.
Quoted 10 lines
The transport class is to abstract out common code from multiple drivers, so what goes in there must genuinely be common. Even if you put the common session data in the target, you probably still end up having driver specific data in the host.
James
Quoted 13 lines · Mike Christie
Quoted 9 lines · James Bottomley
Quoted 1 line · Mike Christie
Ok. If the new magic iscsi transport specific target code handles target data like the
scsi_hostdoes for hostdata then would it be better to put our session struct in the target data? I think this also simplifies refcounting. Most session stuff makes sense hanging off the target so this might be easier, right?Possibly, but only if the way its handled would be the same on all iscsi initiator drivers. Otherwise, if it's specific to the driver implementation, it would be in hostdata.
Quoted 8 lines
Yes, ok. That is what I am striving for. By us hanging things like target name on the target instead of on the scsi device or scsi host will help us conform to other HW iscsi drivers.
Regardless, you'd still have
Quoted 1 line
one host per one of these entities.
One host per
I_Tnexus/session? But I thought the only reason against using a single host for all nexii (ignoring the port/channel discussion), was that the driver would need its own refcounting.
Oh yeah by ignoring Scott's comments I just did not want to complicate things as to what should be done when the isid is the same but the target port is different. In that case are we supposed to reuse scsi hosts.
Quoted 19 lines
If the target has some place for driver target data (with the common iscsi code going in the appropriate transport class stucts) couldn't we just do sometihng similar to how a
scsi_devicehas the hostdata pointers and drivers seem to assume whenslave_destroyis called they can just free what was allocated inslave_alloc. And when the driver does its finalscsi_host_putit assumes the hostdata will get freed when all the handles are released. If the midlayer could provide any similar lifetime management functionality for targets the iscsi driver would not need any private structures that are refcounted internally since everything could fall under the host, device, or target's managemnet.Quoted 11 lines
The transport class is to abstract out common code from multiple drivers, so what goes in there must genuinely be common. Even if you put the common session data in the target, you probably still end up having driver specific data in the host.
James
-
To unsubscribe from this list: send the line "unsubscribe linux-scsi" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html
Quoted 7 lines · Mike Christie
Quoted 3 lines
Possibly, but only if the way its handled would be the same on all iscsi initiator drivers. Otherwise, if it's specific to the driver implementation, it would be in hostdata.
Yes, ok. That is what I am striving for. By us hanging things like target name on the target instead of on the scsi device or scsi host will help us conform to other HW iscsi drivers.
You're thinking of the qlogic one? They have an entire network stack onboard and they do have resource restrictions per card. A virtual driver doesn't.
Quoted 6 lines
Regardless, you'd still have
Quoted 1 line
one host per one of these entities.
One host per
I_Tnexus/session? But I thought the only reason against using a single host for all nexii (ignoring the port/channel discussion), was that the driver would need its own refcounting.
It's not just refcounting, it's resource management. It seems to me, given that you can allocate resources per connection that per connection is the most efficient way to manage them.
Quoted 10 lines
If the target has some place for driver target data (with the common iscsi code going in the appropriate transport class stucts) couldn't we just do sometihng similar to how a
scsi_devicehas the hostdata pointers and drivers seem to assume whenslave_destroyis called they can just free what was allocated inslave_alloc. And when the driver does its finalscsi_host_putit assumes the hostdata will get freed when all the handles are released. If the midlayer could provide any similar lifetime management functionality for targets the iscsi driver would not need any private structures that are refcounted internally since everything could fall under the host, device, or target's managemnet.
Theoretically, I suppose. The mid-layer really prefers to think in terms of hosts and devices. It doesn't really have much use for targets.
James
Scott M. Ferris wrote:
<snip>
Quoted 8 lines
Should all drivers that currently use one host for each SCSI initiator device, and a channel for each initiator port on each device, be modified to use a host for each initiator port?
Documentation/scsi/scsi_mid_low_api.txtsays that a host corresponds to a SCSI initiator device. Could someone change that to say SCSI initiator port instead, since that seems to be the new goal?
Scott,
Yes, I can change that if we all agree. Randy Dunlap asked me some time back to define what was meant by a "linux SCSI host" and that was the best I could come up with at the time.
Interesting to see you mention SAS. There could be lots of fun for the linux SCSI subsystem here:
- addressable switching elements (called "expanders") in the SCSI domain ** service delivery subsystem. These are not SCSI devices (in the SAM sense) but are pretty close. They are controlled by a packetized protocol called SMP whose initiator end sits on a host and target end sits on an expander. [SMP has no concept of logical units.]
- discovery is either dead simple (for example when a disk is directly connected to a HBA's phy [like SATA]) or something that needs a breadth first, hierarchial descent through expanders to end devices. The latter type of discovery is probably better done from the user space.
- in SCSI, devices can have multiple ports and in SAS a port can be made up of multiple phys (i.e. a wide link). The SCSI subsystem can ignore phys and just talk about ports but users can see and touch phys (e.g. those SATA red cables). Also expander routing is done at the link level.
It would be great to see a transport class architecture that could cope with this. Think about how a SMP target device (i.e. an expander's control port) will be represented. I don't think struct scsi_device is appropriate ...
** the term "bus" seems to have lost favour in SAM and been
replaced by the term "domain".
A few related matters:
Steven Fairchild dropped a proposal for SAS storage management called "Common Storage Management Interface for SAS" in the t10 "New Documents and Drafts" section. The url is:
http://www.t10.org/ftp/t10/document.04/04-284r0.pdf
A linux interface and driver is proposed.
A draft standard titled "SCSI-ATA translation" (SAT) is moving along briskly. The primary target areas for this translator are in a SAS host above an SAS domain when some of the "SCSI" devices are SATA disks and SCSI-(p/s)ATA bridges.
Doug Gilbert
Quoted 6 lines · James Bottomley
Quoted 4 lines · Mike Christie
Quoted 3 lines
Possibly, but only if the way its handled would be the same on all iscsi initiator drivers. Otherwise, if it's specific to the driver implementation, it would be in hostdata.
Quoted 7 lines
Quoted 1 line
Yes, ok. That is what I am striving for. By us hanging things like target name on the target instead of on the scsi device or scsi host will help us conform to other HW iscsi drivers.
You're thinking of the qlogic one? They have an entire network stack onboard and they do have resource restrictions per card. A virtual driver doesn't.
I just meant that I am trying to get the iscsi transport class to have common code across virtual and non virtual drivers. I was not striving for virtual and non-virtual to alloc hosts alike. I was the one that converted the iscsi driver to host per session and caused this mess :(
The reason I kept asking below is becuase I misunderstood the resource comment, and was unsure if I did the right thing and was supposed to convert the driver back to a single host.
Quoted 26 lines
Quoted 5 lines
Regardless, you'd still have
Quoted 1 line
one host per one of these entities.
One host per
I_Tnexus/session? But I thought the only reason against using a single host for all nexii (ignoring the port/channel discussion), was that the driver would need its own refcounting.It's not just refcounting, it's resource management. It seems to me, given that you can allocate resources per connection that per connection is the most efficient way to manage them.
Quoted 1 line
If the target has some place for driver target data (with the common iscsi code going in the appropriate transport class stucts) couldn't we just do sometihng similar to how a
scsi_devicehas the hostdata pointers and drivers seem to assume whenslave_destroyis called they can just free what was allocated inslave_alloc. And when the driver does its finalscsi_host_putit assumes the hostdata will get freed when all the handles are released. If the midlayer could provide any similar lifetime management functionality for targets the iscsi driver would not need any private structures that are refcounted internally since everything could fall under the host, device, or target's managemnet.Theoretically, I suppose. The mid-layer really prefers to think in terms of hosts and devices. It doesn't really have much use for targets.
James
-
To unsubscribe from this list: send the line "unsubscribe linux-scsi" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html
Quoted 21 lines · Mike Christie
Quoted 19 lines · Mike Christie
Quoted 10 lines · James Bottomley
Quoted 1 line · Mike Christie
Ok. If the new magic iscsi transport specific target code handles target data like the
scsi_hostdoes for hostdata then would it be better to put our session struct in the target data? I think this also simplifies refcounting. Most session stuff makes sense hanging off the target so this might be easier, right?Possibly, but only if the way its handled would be the same on all iscsi initiator drivers. Otherwise, if it's specific to the driver implementation, it would be in hostdata.
Yes, ok. That is what I am striving for. By us hanging things like target name on the target instead of on the scsi device or scsi host will help us conform to other HW iscsi drivers.
Regardless, you'd still have
Quoted 9 lines
Quoted 5 lines
Quoted 1 line
one host per one of these entities.
One host per
I_Tnexus/session? But I thought the only reason against using a single host for all nexii (ignoring the port/channel discussion), was that the driver would need its own refcounting.Oh yeah by ignoring Scott's comments I just did not want to complicate things as to what should be done when the isid is the same but the target port is different. In that case are we supposed to reuse scsi hosts.
Ignore that question and mistake in iscsi and scsi concepts and iscsi vocab. Don't remember what I was trying to ask or thinking about.
Quoted 33 lines
Quoted 15 lines
If the target has some place for driver target data (with the common iscsi code going in the appropriate transport class stucts) couldn't we just do sometihng similar to how a
scsi_devicehas the hostdata pointers and drivers seem to assume whenslave_destroyis called they can just free what was allocated inslave_alloc. And when the driver does its finalscsi_host_putit assumes the hostdata will get freed when all the handles are released. If the midlayer could provide any similar lifetime management functionality for targets the iscsi driver would not need any private structures that are refcounted internally since everything could fall under the host, device, or target's managemnet.Quoted 8 lines
The transport class is to abstract out common code from multiple drivers, so what goes in there must genuinely be common. Even if you put the common session data in the target, you probably still end up having driver specific data in the host.
Quoted 4 lines
James
Quoted 5 lines
-
To unsubscribe from this list: send the line "unsubscribe linux-scsi" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html-------------------------------------------------------
This SF.Net email is sponsored by BEA Weblogic Workshop FREE Java Enterprise J2EE developer tools!
Get your free copy of BEA WebLogic Workshop 8.1 today.
http://ads.osdn.com/?ad_id=5047&alloc_id=10808&op=click
_______________________________________________
linux-iscsi-devel mailing list [email protected]
https://lists.sourceforge.net/lists/listinfo/linux-iscsi-devel
On Wed, 08 Sep 2004 12:33:51 +1000 Douglas Gilbert wrote:
| Scott M. Ferris wrote: | <snip> | > | > Should all drivers that currently use one host for each SCSI initiator | > device, and a channel for each initiator port on each device, be | > modified to use a host for each initiator port? | > | > Documentation/scsi/scsi_mid_low_api.txt says that a host corresponds | > to a SCSI initiator device. Could someone change that to say SCSI | > initiator port instead, since that seems to be the new goal? | | Scott, | Yes, I can change that if we all agree. Randy Dunlap asked me some | time back to define what was meant by a "linux SCSI host" and that | was the best I could come up with at the time.
Sorry to be dense here, but what is an "initiator port" in this context? Is it a physical connector or something else? A logical connector? (whatever that means)
| Interesting to see you mention SAS. There could be lots of fun | for the linux SCSI subsystem here: | - addressable switching elements (called "expanders") in the | SCSI domain ** service delivery subsystem. These are not | SCSI devices (in the SAM sense) but are pretty close. They | are controlled by a packetized protocol called SMP whose | initiator end sits on a host and target end sits on an | expander. [SMP has no concept of logical units.] | - discovery is either dead simple (for example when a disk is | directly connected to a HBA's phy [like SATA]) or something | that needs a breadth first, hierarchial descent through | expanders to end devices. The latter type of discovery is | probably better done from the user space. | - in SCSI, devices can have multiple ports and in SAS a port can | be made up of multiple phys (i.e. a wide link). The SCSI | subsystem can ignore phys and just talk about ports but | users can see and touch phys (e.g. those SATA red cables). | Also expander routing is done at the link level. | | It would be great to see a transport class architecture that | could cope with this. Think about how a SMP target device | (i.e. an expander's control port) will be represented. | I don't think struct scsi_device is appropriate ... | | | ** the term "bus" seems to have lost favour in SAM and been | replaced by the term "domain". | | | A few related matters: | | Steven Fairchild dropped a proposal for SAS storage management
"dropped" as in dropped (oh, as in deleted) or as in "added", "put", or "placed" ? Seems to be "added"...
To me, this all boils down to "communication is hard" or "it's too easy to be ambiguous".
| called "Common Storage Management Interface for SAS" in the | t10 "New Documents and Drafts" section. The url is: | http://www.t10.org/ftp/t10/document.04/04-284r0.pdf | A linux interface and driver is proposed. | | A draft standard titled "SCSI-ATA translation" (SAT) is moving | along briskly. The primary target areas for this translator are | in a SAS host above an SAS domain when some of the "SCSI" devices | are SATA disks and SCSI-(p/s)ATA bridges. | | Doug Gilbert
--
~Randy
Quoted 4 lines · Mike Christie
I just meant that I am trying to get the iscsi transport class to have common code across virtual and non virtual drivers. I was not striving for virtual and non-virtual to alloc hosts alike. I was the one that converted the iscsi driver to host per session and caused this mess :(
I'm not sure that's a good idea; virtual and physical drivers are very different things and the transport classes may end up being different to support their different needs. I'd do the transport class for what you have currently, which is a virtual driver, and when the physical one comes along we'll see if it's possible to reuse pieces. As long as the code itself is nicely clean and modular this shouldn't be too hard; so anywhere you're doing something that could easily be more generally abstracted, do it. Anywhere where an abstraction would be a contortion of what you want to do, don't bother.
Quoted 3 lines
The reason I kept asking below is becuase I misunderstood the resource comment, and was unsure if I did the right thing and was supposed to convert the driver back to a single host.
A host to the mid-layer is the place in which driver instance resources are managed. We expect to have things like an overall command limit per host, and manage host and device blocking caused by resource issues.
For a physical iscsi driver, one host per card is easy and obvious; each instance of the driver controlling a single card. For a virtual iscsi driver, the driver instances could be many things, but it seems logical to me that your driver instances are per connection, thus you have one host per iSCSI connection. Its certainly possible to write a virtual iSCSI driver where there's a single set of resources for everything, and thus a single host, but I really don't think that would end up being a very efficient driver.
James
Quoted 3 lines
Sorry to be dense here, but what is an "initiator port" in this context? Is it a physical connector or something else? A logical connector? (whatever that means)
The context is ISCSI; the ISCSI spec rigorously defines the term "SCSI initiator port." But it defines it quite abstractly, which is why we can have discussions like this one -- whether in a Linux implementation a SCSI initiator port should map to a Linux SCSI host or channel or whatever.
The SCSI initiator port is what in simpler times was a physical connector.
In ISCSI it is just the initiator-side endpoint of an ISCSI
(initator-target) session. It is more abstract than a physical connector, and even more abstract than a TCP port. A SCSI initiator port is identified by an ISCSI initiator device name (= node name) and ISID. A SCSI initiator port belongs to a particular SCSI initiator device. Multiple SCSI initiator ports can go to a single SCSI initiator device. An ISCSI target reaches a SCSI initiator device via a SCSI initiator port. An ISCSI target reaches a SCSI initiator port via TCP ports, but not in any simple mapping.
If you're asking how an ISCSI initiator port is _implemented_ in a Linux-based initiator, that's what's under discussion. There's more than one way to do it, and different people have done it different ways.
--
Bryan Henderson IBM Almaden Research Center
San Jose CA Filesystems
Quoted 15 lines · Randy.Dunlap
On Wed, 08 Sep 2004 12:33:51 +1000 Douglas Gilbert wrote:
| Scott M. Ferris wrote: | <snip> | > | > Should all drivers that currently use one host for each SCSI initiator | > device, and a channel for each initiator port on each device, be | > modified to use a host for each initiator port? | > | > Documentation/scsi/scsi_mid_low_api.txt says that a host corresponds | > to a SCSI initiator device. Could someone change that to say SCSI | > initiator port instead, since that seems to be the new goal? | | Scott, | Yes, I can change that if we all agree. Randy Dunlap asked me some | time back to define what was meant by a "linux SCSI host" and that | was the best I could come up with at the time.
Quoted 3 lines
Sorry to be dense here, but what is an "initiator port" in this context? Is it a physical connector or something else? A logical connector? (whatever that means)
Probably the best place to start is the diagram at
http://www.t10.org under the "Architecture" tab near the top of the left hand pane. The focus of that diagram is the Architectural Model (SAM) of which
http://www.t10.org/ftp/t10/drafts/sam3/sam3r13.pdf
is the latest available draft. The transports are along the bottom of that diagram and the command sets are along the top.
The concepts of initiator, target, port and delivery subsystems are defined (abstractly) in SAM-3.
It then falls to the various transports (e.g. FCP, iSCSI and SPI) to put "meat" on these abstractions and add some of their own terms (e.g. "logical connector" from iSCSI). Transports that are not exclusively used by SCSI command sets (e.g. iSCSI uses IP) come with their own architectural models and jargon.
SCSI has been around for over 20 years (see wikipedia.org) and for the first 15 years was synonymous with the "SCSI Parallel Interface" (SPI) but that is no longer the case. That change has been very slow getting out to the wider technical community (as reflected by the SCSI entry in wikipedia.org)
<snip/>
Quoted 5 lines
| | A few related matters: | | Steven Fairchild dropped a proposal for SAS storage management"dropped" as in dropped (oh, as in deleted) or as in "added", "put", or "placed" ? Seems to be "added"...
Sorry, just sloppy usage. Authors put proposals on that site for comment.
Doug Gilbert
Quoted 8 lines · Douglas Gilbert
Probably the best place to start is the diagram at
http://www.t10.org under the "Architecture" tab near the top of the left hand pane. The focus of that diagram is the Architectural Model (SAM) of which
http://www.t10.org/ftp/t10/drafts/sam3/sam3r13.pdf
is the latest available draft. The transports are along the bottom of that diagram and the command sets are along the top.
Just FYI in case you haven't seen it, there's an updated draft at:
http://www.t10.org/ftp/t10/drafts/sam3/sam3r13a.pdf
though I can't find the revision history in it anywhere, so I don't know how it differs from sam3r13...
Signature · 8 lines
AJ Lewis Voice: 612-638-0500 Red Hat Inc. E-Mail: [email protected] 720 Washington Ave. SE, Suite 200 Minneapolis, MN 55414 Current GPG fingerprint = D9F8 EDCE 4242 855F A03D 9B63 F50C 54A8 578C 8715 Grab the key at: http://people.redhat.com/alewis/gpg.html or one of the many keyservers out there...