Skip to contentSkip to conversation

scsi: ufs & ums-* & esp_scsi: fix module reference counting

PATCH Superseded Akinobu Mita 6 participants 16 messages May 4, 2015 – May 6, 2015 3 messages not in the archive

Subsystems (4)

v6 of 7 Superseded Latest is v7 →

lore
Share & tools
lore
#0 lore raw

v6 cover scsi: ufs & ums-* & esp_scsi: fix module reference counting

While accessing a scsi_device, the use count of the underlying LLDD module is incremented. The module reference is retrieved through .module field of struct scsi_host_template.

This mapping between scsi_device and underlying LLDD module works well except ufs, unusual usb storage drivers, and sub drivers for esp_scsi. These drivers consist with core driver and actual LLDDs, and scsi_host_template is defined in the core driver. So the actual LLDDs can be unloaded even if the scsi_device is being accessed.

This patch series first adds ability to adjust module reference for scsi host by LLDDs and then fixes actual LLDDs by adjusting module reference after scsi host allocation.

* v6:
- Rebased as v5 doesn't apply cleanly to the latest tree anymore.

* v5:
- Discard v4 changes and restore to v3. Because v4 shows that moving owner module field from scsi_host_template to Scsi_Host requires a lot of changes and introduces complication to existing drivers which don't have the module reference mismatch issue.
- Rebased to the 4.0-rc1

* v4:
- Patch series is almost rewritten as module reference field in struct scsi_host_template has been unused anymore. So Acked-by: and
Reviewed-by: tags that have been received are deleted.

* v3:
- Add fix for ESP SCSI drivers

* v2:
- Pass correct module reference to usb_stor_probe1() instead of touching all ums-* drivers, suggested by Alan Stern

Akinobu Mita (4):
scsi: add ability to adjust module reference for scsi host
scsi: ufs: adjust module reference for scsi host
usb: storage: adjust module reference for scsi host
scsi: esp_scsi: adjust module reference for scsi host

 drivers/scsi/am53c974.c          |  3 +--
 drivers/scsi/esp_scsi.c          | 16 +++++++++++++---
 drivers/scsi/esp_scsi.h          | 11 +++++++----
 drivers/scsi/hosts.c             |  1 +
 drivers/scsi/jazz_esp.c          |  3 +--
 drivers/scsi/mac_esp.c           |  3 +--
 drivers/scsi/scsi.c              |  4 ++--
 drivers/scsi/sun3x_esp.c         |  3 +--
 drivers/scsi/sun_esp.c           |  3 +--
 drivers/scsi/ufs/ufshcd-pci.c    |  1 +
 drivers/scsi/ufs/ufshcd-pltfrm.c |  1 +
 drivers/scsi/ufs/ufshcd.c        |  1 -
 drivers/usb/storage/usb.c        |  8 +++++---
 drivers/usb/storage/usb.h        |  7 +++++--
 include/scsi/scsi_host.h         |  1 +
 15 files changed, 41 insertions(+), 25 deletions(-)
Cc Vinayak Holikatti <[email protected]>
Cc Dolev Raviv <[email protected]>
Cc Sujit Reddy Thumma <[email protected]>
Cc Subhash Jadavani <[email protected]>
Cc Christoph Hellwig <[email protected]>
Cc "James E.J. Bottomley" <[email protected]>
Cc Matthew Dharm <[email protected]>
Cc Greg Kroah-Hartman <[email protected]>
Cc Alan Stern <[email protected]>
Cc "David S. Miller" <[email protected]>
Cc Hannes Reinecke <[email protected]>
Signature · 1 line
1.9.1

A message that is not in the archive (sent off-list or not yet received): 1430750769-11405-1-git-send-email-akinobu.mita-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org

#2 lore raw

PATCH v6 3/4 usb: storage: adjust module reference for scsi host

While accessing a unusual usb storage (ums-alauda, ums-cypress, ...), the module reference count is not incremented. Because these drivers allocate scsi hosts with usb_stor_host_template defined in usb-storage module. So these drivers always can be unloaded.

This fixes it by passing correct module reference to usb_stor_probe1() so that .module field in struct Scsi_Host can be adjusted.

Cc Matthew Dharm <[email protected]>
Cc Greg Kroah-Hartman <[email protected]>
Cc Alan Stern <stern-nwvwT67g6+6dFdvTe/[email protected]>
Cc Christoph Hellwig <[email protected]>
Cc "James E.J. Bottomley" <[email protected]>
---
 drivers/usb/storage/usb.c | 8 +++++---
 drivers/usb/storage/usb.h | 7 +++++--
 2 files changed, 10 insertions(+), 5 deletions(-)
drivers/usb/storage/usb.c +5 -3
index 6c10c88..711cdee 100644
@@ -921,10 +921,11 @@ static unsigned int usb_stor_sg_tablesize(struct usb_interface *intf)
 }
 
 /* First part of general USB mass-storage probing */
-int usb_stor_probe1(struct us_data **pus,
+int __usb_stor_probe1(struct us_data **pus,
 		struct usb_interface *intf,
 		const struct usb_device_id *id,
-		struct us_unusual_dev *unusual_dev)
+		struct us_unusual_dev *unusual_dev,
+		struct module *owner)
 {
 	struct Scsi_Host *host;
 	struct us_data *us;
@@ -947,6 +948,7 @@ int usb_stor_probe1(struct us_data **pus,
 	 */
 	host->max_cmd_len = 16;
 	host->sg_tablesize = usb_stor_sg_tablesize(intf);
+	host->module = owner;
 	*pus = us = host_to_us(host);
 	mutex_init(&(us->dev_mutex));
 	us_set_lock_class(&us->dev_mutex, intf);
@@ -979,7 +981,7 @@ BadDevice:
 	release_everything(us);
 	return result;
 }
-EXPORT_SYMBOL_GPL(usb_stor_probe1);
+EXPORT_SYMBOL_GPL(__usb_stor_probe1);
 
 /* Second part of general USB mass-storage probing */
 int usb_stor_probe2(struct us_data *us)
drivers/usb/storage/usb.h +5 -2
index 307e339..0cb74ba 100644
@@ -194,10 +194,13 @@ extern int usb_stor_reset_resume(struct usb_interface *iface);
 extern int usb_stor_pre_reset(struct usb_interface *iface);
 extern int usb_stor_post_reset(struct usb_interface *iface);
 
-extern int usb_stor_probe1(struct us_data **pus,
+extern int __usb_stor_probe1(struct us_data **pus,
 		struct usb_interface *intf,
 		const struct usb_device_id *id,
-		struct us_unusual_dev *unusual_dev);
+		struct us_unusual_dev *unusual_dev,
+		struct module *owner);
+#define usb_stor_probe1(pus, intf, id, unusual_dev) \
+	__usb_stor_probe1(pus, intf, id, unusual_dev, THIS_MODULE)
 extern int usb_stor_probe2(struct us_data *us);
 extern void usb_stor_disconnect(struct usb_interface *intf);
 
Signature · 6 lines
1.9.1

--
To unsubscribe from this list: send the line "unsubscribe linux-usb" in
the body of a message to [email protected]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
#3 lore raw

PATCH v6 1/4 scsi: add ability to adjust module reference for scsi host

In reply to Akinobu Mita

While accessing a scsi_device, the use count of the underlying LLDD module is incremented. The module reference is retrieved through .module field of struct scsi_host_template.

This mapping between scsi_device and underlying LLDD module works well except ufs, unusual usb storage drivers, and sub drivers for esp_scsi. These drivers consist with core driver and actual LLDDs, and scsi_host_template is defined in the core driver. So the actual LLDDs can be unloaded even if the scsi_device is being accessed.

This adds .module field in struct Scsi_Host and let the module reference be retrieved though it instead of struct scsi_host_template. This allows the actual LLDDs adjust module reference.

Cc Vinayak Holikatti <[email protected]>
Cc Dolev Raviv <[email protected]>
Cc Sujit Reddy Thumma <[email protected]>
Cc Subhash Jadavani <[email protected]>
Cc Christoph Hellwig <[email protected]>
Cc "James E.J. Bottomley" <[email protected]>
Cc Matthew Dharm <[email protected]>
Cc Greg Kroah-Hartman <[email protected]>
Cc Alan Stern <[email protected]>
Cc "David S. Miller" <[email protected]>
Cc Hannes Reinecke <[email protected]>

---
* Changes from v5
- Rebased as v5 doesn't apply cleanly to the latest tree anymore.

 drivers/scsi/hosts.c     | 1 +
 drivers/scsi/scsi.c      | 4 ++--
 include/scsi/scsi_host.h | 1 +
 3 files changed, 4 insertions(+), 2 deletions(-)
drivers/scsi/hosts.c +1 -0
index 8bb173e..21f1442 100644
@@ -411,6 +411,7 @@ struct Scsi_Host *scsi_host_alloc(struct scsi_host_template *sht, int privsize)
 	 */
 	shost->max_cmd_len = 12;
 	shost->hostt = sht;
+	shost->module = sht->module;
 	shost->this_id = sht->this_id;
 	shost->can_queue = sht->can_queue;
 	shost->sg_tablesize = sht->sg_tablesize;
drivers/scsi/scsi.c +2 -2
index 3833bf5..f7534a9 100644
@@ -982,7 +982,7 @@ int scsi_device_get(struct scsi_device *sdev)
 		goto fail;
 	if (!get_device(&sdev->sdev_gendev))
 		goto fail;
-	if (!try_module_get(sdev->host->hostt->module))
+	if (!try_module_get(sdev->host->module))
 		goto fail_put_device;
 	return 0;
 
@@ -1003,7 +1003,7 @@ EXPORT_SYMBOL(scsi_device_get);
  */
 void scsi_device_put(struct scsi_device *sdev)
 {
-	module_put(sdev->host->hostt->module);
+	module_put(sdev->host->module);
 	put_device(&sdev->sdev_gendev);
 }
 EXPORT_SYMBOL(scsi_device_put);
include/scsi/scsi_host.h +1 -0
index e113c75..8742bfd 100644
@@ -620,6 +620,7 @@ struct Scsi_Host {
 	 */
 	unsigned short max_cmd_len;
 
+	struct module *module;
 	int this_id;
 	int can_queue;
 	short cmd_per_lun;
Signature · 1 line
1.9.1
#4 lore raw

PATCH v6 2/4 scsi: ufs: adjust module reference for scsi host

In reply to Akinobu Mita

While accessing a UFS device, the module reference count for core driver (ufshcd) is incremented but not incremented for the actual glue driver (ufshcd-pci or ufshcd-pltfrm). Because these drivers allocate scsi hosts with scsi_host_template defined in ufshcd module. So these drivers always can be unloaded.

This fixes it by adjusting module reference after scsi host allocation.

Cc Vinayak Holikatti <[email protected]>
Cc Dolev Raviv <[email protected]>
Cc Sujit Reddy Thumma <[email protected]>
Cc Subhash Jadavani <[email protected]>
Cc Christoph Hellwig <[email protected]>
Cc "James E.J. Bottomley" <[email protected]>
---
 drivers/scsi/ufs/ufshcd-pci.c    | 1 +
 drivers/scsi/ufs/ufshcd-pltfrm.c | 1 +
 drivers/scsi/ufs/ufshcd.c        | 1 -
 3 files changed, 2 insertions(+), 1 deletion(-)
drivers/scsi/ufs/ufshcd-pci.c +1 -0
index d15eaa4..9fe21b4 100644
@@ -142,6 +142,7 @@ ufshcd_pci_probe(struct pci_dev *pdev, const struct pci_device_id *id)
 		return err;
 	}
 
+	hba->host->module = THIS_MODULE;
 	INIT_LIST_HEAD(&hba->clk_list_head);
 
 	err = ufshcd_init(hba, mmio_base, pdev->irq);
drivers/scsi/ufs/ufshcd-pltfrm.c +1 -0
index e6b29b1..d08e0bb 100644
@@ -322,6 +322,7 @@ static int ufshcd_pltfrm_probe(struct platform_device *pdev)
 		goto out;
 	}
 
+	hba->host->module = THIS_MODULE;
 	hba->vops = get_variant_ops(&pdev->dev);
 
 	err = ufshcd_parse_clock_info(hba);
drivers/scsi/ufs/ufshcd.c +0 -1
index 996030b..c499f16 100644
@@ -4237,7 +4237,6 @@ static void ufshcd_async_scan(void *data, async_cookie_t cookie)
 }
 
 static struct scsi_host_template ufshcd_driver_template = {
-	.module			= THIS_MODULE,
 	.name			= UFSHCD,
 	.proc_name		= UFSHCD,
 	.queuecommand		= ufshcd_queuecommand,
Signature · 1 line
1.9.1
#5 lore raw

PATCH v6 4/4 scsi: esp_scsi: adjust module reference for scsi host

In reply to Akinobu Mita

While accessing a scsi device on host adapter supported by sub driver for the ESP chip (mac_esp, am53c974, sun_esp, jazz_esp, sun3x_esp), the module reference count is not incremented. Because these drivers allocate scsi hosts with scsi_esp_template defined in ESP SCSI driver core module. So these drivers always can be unloaded.

This fixes it by passing correct module reference to newly introduced scsi_esp_host_alloc() so that .module field in struct Scsi_Host can be adjusted.

Cc "David S. Miller" <[email protected]>
Cc Hannes Reinecke <[email protected]>
Cc Christoph Hellwig <[email protected]>
Cc "James E.J. Bottomley" <[email protected]>
---
 drivers/scsi/am53c974.c  |  3 +--
 drivers/scsi/esp_scsi.c  | 16 +++++++++++++---
 drivers/scsi/esp_scsi.h  | 11 +++++++----
 drivers/scsi/jazz_esp.c  |  3 +--
 drivers/scsi/mac_esp.c   |  3 +--
 drivers/scsi/sun3x_esp.c |  3 +--
 drivers/scsi/sun_esp.c   |  3 +--
 7 files changed, 25 insertions(+), 17 deletions(-)
drivers/scsi/am53c974.c +1 -2
index beea30e..c1ba885 100644
@@ -400,7 +400,6 @@ static void dc390_check_eeprom(struct esp *esp)
 static int pci_esp_probe_one(struct pci_dev *pdev,
 			      const struct pci_device_id *id)
 {
-	struct scsi_host_template *hostt = &scsi_esp_template;
 	int err = -ENODEV;
 	struct Scsi_Host *shost;
 	struct esp *esp;
@@ -417,7 +416,7 @@ static int pci_esp_probe_one(struct pci_dev *pdev,
 		goto fail_disable_device;
 	}
 
-	shost = scsi_host_alloc(hostt, sizeof(struct esp));
+	shost = scsi_esp_host_alloc(sizeof(struct esp));
 	if (!shost) {
 		dev_printk(KERN_INFO, &pdev->dev,
 			   "failed to allocate scsi host\n");
drivers/scsi/esp_scsi.c +13 -3
index 065b25d..4484c90 100644
@@ -2675,8 +2675,7 @@ static const char *esp_info(struct Scsi_Host *host)
 	return "esp";
 }
 
-struct scsi_host_template scsi_esp_template = {
-	.module			= THIS_MODULE,
+static struct scsi_host_template scsi_esp_template = {
 	.name			= "esp",
 	.info			= esp_info,
 	.queuecommand		= esp_queuecommand,
@@ -2696,7 +2695,18 @@ struct scsi_host_template scsi_esp_template = {
 	.skip_settle_delay	= 1,
 	.use_blk_tags		= 1,
 };
-EXPORT_SYMBOL(scsi_esp_template);
+
+struct Scsi_Host *__scsi_esp_host_alloc(int privsize, struct module *owner)
+{
+	struct Scsi_Host *shost;
+
+	shost = scsi_host_alloc(&scsi_esp_template, privsize);
+	if (shost)
+		shost->module = owner;
+
+	return shost;
+}
+EXPORT_SYMBOL(__scsi_esp_host_alloc);
 
 static void esp_get_signalling(struct Scsi_Host *host)
 {
drivers/scsi/esp_scsi.h +7 -4
index 84dcbe4..9309e08 100644
@@ -544,9 +544,8 @@ struct esp {
 
 /* A front-end driver for the ESP chip should do the following in
  * it's device probe routine:
- * 1) Allocate the host and private area using scsi_host_alloc()
- *    with size 'sizeof(struct esp)'.  The first argument to
- *    scsi_host_alloc() should be &scsi_esp_template.
+ * 1) Allocate the host and private area using scsi_esp_host_alloc()
+ *    with size 'sizeof(struct esp)'.
  * 2) Set host->max_id as appropriate.
  * 3) Set esp->host to the scsi_host itself, and esp->dev
  *    to the device object pointer.
@@ -573,7 +572,11 @@ struct esp {
  * 13) Check scsi_esp_register() return value, release all resources
  *     if an error was returned.
  */
-extern struct scsi_host_template scsi_esp_template;
+extern struct Scsi_Host *__scsi_esp_host_alloc(int privsize,
+						struct module *owner);
+#define scsi_esp_host_alloc(privsize)	\
+	__scsi_esp_host_alloc(privsize, THIS_MODULE)
+
 extern int scsi_esp_register(struct esp *, struct device *);
 
 extern void scsi_esp_unregister(struct esp *);
drivers/scsi/jazz_esp.c +1 -2
index 9aaa74e..9c268f7 100644
@@ -131,13 +131,12 @@ static const struct esp_driver_ops jazz_esp_ops = {
 
 static int esp_jazz_probe(struct platform_device *dev)
 {
-	struct scsi_host_template *tpnt = &scsi_esp_template;
 	struct Scsi_Host *host;
 	struct esp *esp;
 	struct resource *res;
 	int err;
 
-	host = scsi_host_alloc(tpnt, sizeof(struct esp));
+	host = scsi_esp_host_alloc(sizeof(struct esp));
 
 	err = -ENOMEM;
 	if (!host)
drivers/scsi/mac_esp.c +1 -2
index 14c0334..ad4ad71 100644
@@ -483,7 +483,6 @@ static struct esp_driver_ops mac_esp_ops = {
 
 static int esp_mac_probe(struct platform_device *dev)
 {
-	struct scsi_host_template *tpnt = &scsi_esp_template;
 	struct Scsi_Host *host;
 	struct esp *esp;
 	int err;
@@ -495,7 +494,7 @@ static int esp_mac_probe(struct platform_device *dev)
 	if (dev->id > 1)
 		return -ENODEV;
 
-	host = scsi_host_alloc(tpnt, sizeof(struct esp));
+	host = scsi_esp_host_alloc(sizeof(struct esp));
 
 	err = -ENOMEM;
 	if (!host)
drivers/scsi/sun3x_esp.c +1 -2
index e26e81d..adb4368 100644
@@ -196,13 +196,12 @@ static const struct esp_driver_ops sun3x_esp_ops = {
 
 static int esp_sun3x_probe(struct platform_device *dev)
 {
-	struct scsi_host_template *tpnt = &scsi_esp_template;
 	struct Scsi_Host *host;
 	struct esp *esp;
 	struct resource *res;
 	int err = -ENOMEM;
 
-	host = scsi_host_alloc(tpnt, sizeof(struct esp));
+	host = scsi_esp_host_alloc(sizeof(struct esp));
 	if (!host)
 		goto fail;
 
drivers/scsi/sun_esp.c +1 -2
index 7b6d4c2..1449bea 100644
@@ -489,12 +489,11 @@ static const struct esp_driver_ops sbus_esp_ops = {
 static int esp_sbus_probe_one(struct platform_device *op,
 			      struct platform_device *espdma, int hme)
 {
-	struct scsi_host_template *tpnt = &scsi_esp_template;
 	struct Scsi_Host *host;
 	struct esp *esp;
 	int err;
 
-	host = scsi_host_alloc(tpnt, sizeof(struct esp));
+	host = scsi_esp_host_alloc(sizeof(struct esp));
 
 	err = -ENOMEM;
 	if (!host)
Signature · 1 line
1.9.1
maintainer
#6 lore raw
In reply to Akinobu Mita
Quoted 10 lines · Akinobu Mita

While accessing a scsi_device, the use count of the underlying LLDD module is incremented. The module reference is retrieved through .module field of struct scsi_host_template.

This mapping between scsi_device and underlying LLDD module works well except ufs, unusual usb storage drivers, and sub drivers for esp_scsi. These drivers consist with core driver and actual LLDDs, and scsi_host_template is defined in the core driver. So the actual LLDDs can be unloaded even if the scsi_device is being accessed.

Quoted 3 lines

This patch series first adds ability to adjust module reference for scsi host by LLDDs and then fixes actual LLDDs by adjusting module reference after scsi host allocation.

This series is still missing an ack from ufs. However, as we're on series 6, final warning: object now or I'll take it without ufs ack.

However, it does also strike me that these three drivers have problems because they're using the wrong initialisation pattern: the template is supposed to be in the bus connector for compound drivers not in the core. To see how the pattern is supposed to work look at the 53c700 series of drivers. They have a tiny template in the bus connector part and then the function NCR_700_detect() fills in the rest.

53c700 is possibly not the best example because it fills in the template for every host rather than once at module init time, but you get the idea. how difficult would it be to convert these things to use the correct pattern rather than hacking it in the mid layer?

James

A message that is not in the archive (sent off-list or not yet received): 1430752523.2177.15.camel-d9PhHud1JfjCXq6kfMZ53/[email protected]

#8 lore raw
Quoted 25 lines · James Bottomley
Quoted 10 lines · Akinobu Mita

While accessing a scsi_device, the use count of the underlying LLDD module is incremented. The module reference is retrieved through .module field of struct scsi_host_template.

This mapping between scsi_device and underlying LLDD module works well except ufs, unusual usb storage drivers, and sub drivers for esp_scsi. These drivers consist with core driver and actual LLDDs, and scsi_host_template is defined in the core driver. So the actual LLDDs can be unloaded even if the scsi_device is being accessed.

Quoted 3 lines

This patch series first adds ability to adjust module reference for scsi host by LLDDs and then fixes actual LLDDs by adjusting module reference after scsi host allocation.

This series is still missing an ack from ufs. However, as we're on series 6, final warning: object now or I'll take it without ufs ack.

However, it does also strike me that these three drivers have problems because they're using the wrong initialisation pattern: the template is supposed to be in the bus connector for compound drivers not in the core. To see how the pattern is supposed to work look at the 53c700 series of drivers. They have a tiny template in the bus connector part and then the function NCR_700_detect() fills in the rest.

Quoted 5 lines

53c700 is possibly not the best example because it fills in the template for every host rather than once at module init time, but you get the idea. how difficult would it be to convert these things to use the correct pattern rather than hacking it in the mid layer?

Probably not _that_ difficult.
I'll give esp_scsi a shot.

Cheers,

Hannes

Signature · 8 lines
Dr. Hannes Reinecke		      zSeries & Storage
[email protected]			      +49 911 74053 688
SUSE LINUX Products GmbH, Maxfeldstr. 5, 90409 Nürnberg
GF: J. Hawn, J. Guild, F. Imendörffer, HRB 16746 (AG Nürnberg)
--
To unsubscribe from this list: send the line "unsubscribe linux-usb" in
the body of a message to [email protected]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
#9 lore raw
In reply to a message not in the archive
Quoted 4 lines · James Bottomley

However, it does also strike me that these three drivers have problems because they're using the wrong initialisation pattern: the template is supposed to be in the bus connector for compound drivers not in the core.

Why is it supposed to be done that way? Isn't that less efficient? It means you have to have a separate copy of the template for each bus connector driver, instead of letting them all share a common template in the core.

Alan Stern

--
To unsubscribe from this list: send the line "unsubscribe linux-usb" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html

maintainer
#10 lore raw
Quoted 11 lines · Alan Stern
Quoted 4 lines · James Bottomley

However, it does also strike me that these three drivers have problems because they're using the wrong initialisation pattern: the template is supposed to be in the bus connector for compound drivers not in the core.

Why is it supposed to be done that way? Isn't that less efficient? It means you have to have a separate copy of the template for each bus connector driver, instead of letting them all share a common template in the core.

Well, no it doesn't. The way 53c700 implements it is that there is a common template in the core. The drivers just initialise their variant fields (for 53c700 it's name, proc_name and this_id) and the core fills in the rest. Admittedly wd33c93 doesn't quite get this right, that's why I cited 53c700.

James

A message that is not in the archive (sent off-list or not yet received): 1430775664.2177.36.camel-d9PhHud1JfjCXq6kfMZ53/[email protected]

#12 lore raw

2015-05-05 6:41 GMT+09:00 James Bottomley <James.Bottomley-JuX6DAaQMKPCXq6kfMZ53/[email protected]>:

Quoted 13 lines · Alan Stern
Quoted 11 lines · James Bottomley
Quoted 5 lines

However, it does also strike me that these three drivers have problems because they're using the wrong initialisation pattern: the template is supposed to be in the bus connector for compound drivers not in the core.

Why is it supposed to be done that way? Isn't that less efficient? It means you have to have a separate copy of the template for each bus connector driver, instead of letting them all share a common template in the core.

Quoted 5 lines

Well, no it doesn't. The way 53c700 implements it is that there is a common template in the core. The drivers just initialise their variant fields (for 53c700 it's name, proc_name and this_id) and the core fills in the rest. Admittedly wd33c93 doesn't quite get this right, that's why I cited 53c700.

I have converted usb-storage and ums-alauda in the attached patch. Each ums-* driver needs to expand module_usb_driver() macro as we need to initialize their scsi host template in module_init().

Alan, how do you feel the change like this?

From 4c6aa0b56c153afe331a98969a857f16d99f96b6 Mon Sep 17 00:00:00 2001
From: Akinobu Mita <[email protected]>
Date: Tue, 5 May 2015 22:30:04 +0900
Subject: [PATCH] usb: storage: fix module reference for scsi host

---
 drivers/usb/storage/alauda.c   | 21 +++++++++++++++++++--
 drivers/usb/storage/scsiglue.c | 12 +++++++++++-
 drivers/usb/storage/scsiglue.h |  3 ++-
 drivers/usb/storage/usb.c      | 25 +++++++++++++++++++++----
 drivers/usb/storage/usb.h      |  3 ++-
 5 files changed, 55 insertions(+), 9 deletions(-)
drivers/usb/storage/alauda.c +19 -2
index 4b55ab6..0baa484 100644
@@ -42,6 +42,7 @@
 #include "transport.h"
 #include "protocol.h"
 #include "debug.h"
+#include "scsiglue.h"
 
 MODULE_DESCRIPTION("Driver for Alauda-based card readers");
 MODULE_AUTHOR("Daniel Drake <[email protected]>");
@@ -1232,6 +1233,8 @@ static int alauda_transport(struct scsi_cmnd *srb, struct us_data *us)
 	return USB_STOR_TRANSPORT_FAILED;
 }
 
+static struct scsi_host_template alauda_host_template;
+
 static int alauda_probe(struct usb_interface *intf,
 			 const struct usb_device_id *id)
 {
@@ -1239,7 +1242,8 @@ static int alauda_probe(struct usb_interface *intf,
 	int result;
 
 	result = usb_stor_probe1(&us, intf, id,
-			(id - alauda_usb_ids) + alauda_unusual_dev_list);
+			(id - alauda_usb_ids) + alauda_unusual_dev_list,
+			&alauda_host_template);
 	if (result)
 		return result;
 
@@ -1266,4 +1270,17 @@ static struct usb_driver alauda_driver = {
 	.no_dynamic_id = 1,
 };
 
-module_usb_driver(alauda_driver);
+static int __init alauda_driver_init(void)
+{
+	usb_stor_host_template_init(&alauda_host_template, "ums-alauda",
+				    THIS_MODULE);
+
+	return usb_register(&alauda_driver);
+}
+module_init(alauda_driver_init);
+
+static void __exit alauda_driver_exit(void)
+{
+	usb_deregister(&alauda_driver);
+}
+module_exit(alauda_driver_exit);
drivers/usb/storage/scsiglue.c +11 -1
index 0e400f3..05b8fe2 100644
@@ -540,7 +540,7 @@ static struct device_attribute *sysfs_device_attr_list[] = {
  * this defines our host template, with which we'll allocate hosts
  */
 
-struct scsi_host_template usb_stor_host_template = {
+static struct scsi_host_template usb_stor_host_template = {
 	/* basic userland interface stuff */
 	.name =				"usb-storage",
 	.proc_name =			"usb-storage",
@@ -592,6 +592,16 @@ struct scsi_host_template usb_stor_host_template = {
 	.module =			THIS_MODULE
 };
 
+void usb_stor_host_template_init(struct scsi_host_template *sht,
+				 const char *name, struct module *owner)
+{
+	memcpy(sht, &usb_stor_host_template, sizeof(*sht));
+	sht->name = name;
+	sht->proc_name = name;
+	sht->module = owner;
+}
+EXPORT_SYMBOL_GPL(usb_stor_host_template_init);
+
 /* To Report "Illegal Request: Invalid Field in CDB */
 unsigned char usb_stor_sense_invalidCDB[18] = {
 	[0]	= 0x70,			    /* current error */
drivers/usb/storage/scsiglue.h +2 -1
index ffa1cca..1909601 100644
@@ -43,6 +43,7 @@ extern void usb_stor_report_device_reset(struct us_data *us);
 extern void usb_stor_report_bus_reset(struct us_data *us);
 
 extern unsigned char usb_stor_sense_invalidCDB[18];
-extern struct scsi_host_template usb_stor_host_template;
+extern void usb_stor_host_template_init(struct scsi_host_template *sht,
+					const char *name, struct module *owner);
 
 #endif
drivers/usb/storage/usb.c +21 -4
index 6c10c88..0f76899 100644
@@ -924,7 +924,8 @@ static unsigned int usb_stor_sg_tablesize(struct usb_interface *intf)
 int usb_stor_probe1(struct us_data **pus,
 		struct usb_interface *intf,
 		const struct usb_device_id *id,
-		struct us_unusual_dev *unusual_dev)
+		struct us_unusual_dev *unusual_dev,
+		struct scsi_host_template *sht)
 {
 	struct Scsi_Host *host;
 	struct us_data *us;
@@ -936,7 +937,7 @@ int usb_stor_probe1(struct us_data **pus,
 	 * Ask the SCSI layer to allocate a host structure, with extra
 	 * space at the end for our private us_data structure.
 	 */
-	host = scsi_host_alloc(&usb_stor_host_template, sizeof(*us));
+	host = scsi_host_alloc(sht, sizeof(*us));
 	if (!host) {
 		dev_warn(&intf->dev, "Unable to allocate the scsi host\n");
 		return -ENOMEM;
@@ -1073,6 +1074,8 @@ void usb_stor_disconnect(struct usb_interface *intf)
 }
 EXPORT_SYMBOL_GPL(usb_stor_disconnect);
 
+static struct scsi_host_template usb_stor_host_template;
+
 /* The main probe routine for standard devices */
 static int storage_probe(struct usb_interface *intf,
 			 const struct usb_device_id *id)
@@ -1113,7 +1116,8 @@ static int storage_probe(struct usb_interface *intf,
 			id->idVendor, id->idProduct);
 	}
 
-	result = usb_stor_probe1(&us, intf, id, unusual_dev);
+	result = usb_stor_probe1(&us, intf, id, unusual_dev,
+				 &usb_stor_host_template);
 	if (result)
 		return result;
 
@@ -1137,4 +1141,17 @@ static struct usb_driver usb_storage_driver = {
 	.soft_unbind =	1,
 };
 
-module_usb_driver(usb_storage_driver);
+static int __init usb_storage_driver_init(void)
+{
+	usb_stor_host_template_init(&usb_stor_host_template, "usb-storage",
+				    THIS_MODULE);
+
+	return usb_register(&usb_storage_driver);
+}
+module_init(usb_storage_driver_init);
+
+static void __exit usb_storage_driver_exit(void)
+{
+	usb_deregister(&usb_storage_driver);
+}
+module_exit(usb_storage_driver_exit);
drivers/usb/storage/usb.h +2 -1
index 307e339..a00ec4f 100644
@@ -197,7 +197,8 @@ extern int usb_stor_post_reset(struct usb_interface *iface);
 extern int usb_stor_probe1(struct us_data **pus,
 		struct usb_interface *intf,
 		const struct usb_device_id *id,
-		struct us_unusual_dev *unusual_dev);
+		struct us_unusual_dev *unusual_dev,
+		struct scsi_host_template *sht);
 extern int usb_stor_probe2(struct us_data *us);
 extern void usb_stor_disconnect(struct usb_interface *intf);
 
Signature · 1 line
1.9.1
maintainer
#13 lore raw
Quoted 21 lines · Akinobu Mita

2015-05-05 6:41 GMT+09:00 James Bottomley
<[email protected]>:

Quoted 13 lines · Alan Stern
Quoted 11 lines · James Bottomley
Quoted 5 lines

However, it does also strike me that these three drivers have problems because they're using the wrong initialisation pattern: the template is supposed to be in the bus connector for compound drivers not in the core.

Why is it supposed to be done that way? Isn't that less efficient? It means you have to have a separate copy of the template for each bus connector driver, instead of letting them all share a common template in the core.

Quoted 5 lines

Well, no it doesn't. The way 53c700 implements it is that there is a common template in the core. The drivers just initialise their variant fields (for 53c700 it's name, proc_name and this_id) and the core fills in the rest. Admittedly wd33c93 doesn't quite get this right, that's why I cited 53c700.

Quoted 3 lines

I have converted usb-storage and ums-alauda in the attached patch. Each ums-* driver needs to expand module_usb_driver() macro as we need to initialize their scsi host template in module_init().

Actually, I was more thinking something like this (mirroring 53c700). It hides more of the internals rather than exposing them.

James

---

drivers/usb/storage/alauda.c +8 -1
index 4b55ab6..6cfc5c2 100644
@@ -1232,6 +1232,12 @@ static int alauda_transport(struct scsi_cmnd *srb, struct us_data *us)
 	return USB_STOR_TRANSPORT_FAILED;
 }
 
+static struct scsi_host_template aluda_template = {
+	.name = "aluda",
+	.proc_name = "aluda",
+	.module = THIS_MODULE,
+};
+
 static int alauda_probe(struct usb_interface *intf,
 			 const struct usb_device_id *id)
 {
@@ -1239,7 +1245,8 @@ static int alauda_probe(struct usb_interface *intf,
 	int result;
 
 	result = usb_stor_probe1(&us, intf, id,
-			(id - alauda_usb_ids) + alauda_unusual_dev_list);
+			(id - alauda_usb_ids) + alauda_unusual_dev_list,
+			&aluda_template);
 	if (result)
 		return result;
 
drivers/usb/storage/scsiglue.c +27 -27

85 lines not on this page · raw message

drivers/usb/storage/scsiglue.h +1 -1
index ffa1cca..ae0ed71 100644
@@ -41,8 +41,8 @@
 
 extern void usb_stor_report_device_reset(struct us_data *us);
 extern void usb_stor_report_bus_reset(struct us_data *us);
+extern void usb_stor_template_init(struct scsi_host_template *tpnt);
 
 extern unsigned char usb_stor_sense_invalidCDB[18];
-extern struct scsi_host_template usb_stor_host_template;
 
 #endif
drivers/usb/storage/usb.c +10 -3
index 5600c33..020d7ed 100644
@@ -920,7 +920,8 @@ static unsigned int usb_stor_sg_tablesize(struct usb_interface *intf)
 int usb_stor_probe1(struct us_data **pus,
 		struct usb_interface *intf,
 		const struct usb_device_id *id,
-		struct us_unusual_dev *unusual_dev)
+		struct us_unusual_dev *unusual_dev,
+		struct scsi_host_template *tpnt)
 {
 	struct Scsi_Host *host;
 	struct us_data *us;
@@ -928,11 +929,13 @@ int usb_stor_probe1(struct us_data **pus,
 
 	dev_info(&intf->dev, "USB Mass Storage device detected\n");
 
+
 	/*
 	 * Ask the SCSI layer to allocate a host structure, with extra
 	 * space at the end for our private us_data structure.
 	 */
-	host = scsi_host_alloc(&usb_stor_host_template, sizeof(*us));
+	usb_stor_template_init(tpnt);
+	host = scsi_host_alloc(tpnt, sizeof(*us));
 	if (!host) {
 		dev_warn(&intf->dev, "Unable to allocate the scsi host\n");
 		return -ENOMEM;
@@ -1069,6 +1072,10 @@ void usb_stor_disconnect(struct usb_interface *intf)
 }
 EXPORT_SYMBOL_GPL(usb_stor_disconnect);
 
+static struct scsi_host_template storage_template = {
+	.module = THIS_MODULE,
+};
+
 /* The main probe routine for standard devices */
 static int storage_probe(struct usb_interface *intf,
 			 const struct usb_device_id *id)
@@ -1109,7 +1116,7 @@ static int storage_probe(struct usb_interface *intf,
 			id->idVendor, id->idProduct);
 	}
 
-	result = usb_stor_probe1(&us, intf, id, unusual_dev);
+	result = usb_stor_probe1(&us, intf, id, unusual_dev, &storage_template);
 	if (result)
 		return result;
 
drivers/usb/storage/usb.h +2 -1
index 307e339..918cee8 100644
@@ -197,7 +197,8 @@ extern int usb_stor_post_reset(struct usb_interface *iface);
 extern int usb_stor_probe1(struct us_data **pus,
 		struct usb_interface *intf,
 		const struct usb_device_id *id,
-		struct us_unusual_dev *unusual_dev);
+		struct us_unusual_dev *unusual_dev,
+		struct scsi_host_template *tpnt);
 extern int usb_stor_probe2(struct us_data *us);
 extern void usb_stor_disconnect(struct usb_interface *intf);
 
maintainer
#14 lore raw
In reply to James Bottomley
Quoted 13 lines · James Bottomley
Quoted 11 lines · Alan Stern
Quoted 4 lines · James Bottomley

However, it does also strike me that these three drivers have problems because they're using the wrong initialisation pattern: the template is supposed to be in the bus connector for compound drivers not in the core.

Why is it supposed to be done that way? Isn't that less efficient? It means you have to have a separate copy of the template for each bus connector driver, instead of letting them all share a common template in the core.

Quoted 5 lines

Well, no it doesn't. The way 53c700 implements it is that there is a common template in the core. The drivers just initialise their variant fields (for 53c700 it's name, proc_name and this_id) and the core fills in the rest. Admittedly wd33c93 doesn't quite get this right, that's why I cited 53c700.

You seem to be agreeing with me, even though you think you are disagreeing.

"... there is a common template in the core." -- that's one scsi_host_template structure.

"The drivers just initialize their variant fields ... and the core fills in the rest." -- that's an additional scsi_host_template structure for each driver.

The total comes to N+1 scsi_host_templates, where N is the number of drivers.

Now consider the way usb-storage does things.

There is a common template in the core. That's one scsi_host_template structure.

The drivers use the core's template. They have _no_ variant fields other than .owner. That's why I thought Akinobu's idea of putting the owner field in the Scsi_Host structure was a good idea.

The total comes to 1 scsi_host_template. Is that not better than N+1?

Consider the patch Akinobu just posted. In addition to a whole bunch of new code, it adds this:

Quoted 2 lines
drivers/usb/storage/alauda.c +0 -0

...

drivers/usb/storage/alauda.c static int alauda_transport(struct scsi_cmnd *srb, struct us_data *us) · line 1236

Quoted 5 lines
drivers/usb/storage/alauda.c +1 -0
@@ -1232,6 +1233,8 @@ static int alauda_transport(struct scsi_cmnd *srb, struct us_data *us)
 	return USB_STOR_TRANSPORT_FAILED;
 }
 
+static struct scsi_host_template alauda_host_template;

alauda.c was the only driver modified in that patch, and it gained an new scsi_host_template.

For the case where the variants are identical in all respects other than .owner, doesn't it make sense to allow them to share a single scsi_host_template?

The original design of the SCSI stack envisioned multiple drivers, each in control of multiple SCSI hosts. The idea was that scsi_host_template would be associated with the driver and Scsi_Host with the individual host.

Now the kernel has evolved, and we have multiple drivers, some of which contain multiple subdrivers, each in control of multiple SCSI hosts. In this situation we should be flexible enough to allow the scsi_host_template to be associated with either the driver or the subdriver (decision to be made by the driver). When the only variant field is .owner, it makes sense to associate the scsi_host_template with the driver, not force it to be associated with the subdriver.

The cost is duplication of the .owner field in every Scsi_Host. The savings is a reduction in the number of scsi_host_templates.

Alan Stern

maintainer
#15 lore raw
Quoted 78 lines · Alan Stern
Quoted 13 lines · James Bottomley
Quoted 11 lines · Alan Stern
Quoted 4 lines · James Bottomley

However, it does also strike me that these three drivers have problems because they're using the wrong initialisation pattern: the template is supposed to be in the bus connector for compound drivers not in the core.

Why is it supposed to be done that way? Isn't that less efficient? It means you have to have a separate copy of the template for each bus connector driver, instead of letting them all share a common template in the core.

Quoted 5 lines

Well, no it doesn't. The way 53c700 implements it is that there is a common template in the core. The drivers just initialise their variant fields (for 53c700 it's name, proc_name and this_id) and the core fills in the rest. Admittedly wd33c93 doesn't quite get this right, that's why I cited 53c700.

You seem to be agreeing with me, even though you think you are disagreeing.

"... there is a common template in the core." -- that's one scsi_host_template structure.

"The drivers just initialize their variant fields ... and the core fills in the rest." -- that's an additional scsi_host_template structure for each driver.

The total comes to N+1 scsi_host_templates, where N is the number of drivers.

Now consider the way usb-storage does things.

There is a common template in the core. That's one scsi_host_template structure.

The drivers use the core's template. They have _no_ variant fields other than .owner. That's why I thought Akinobu's idea of putting the owner field in the Scsi_Host structure was a good idea.

The total comes to 1 scsi_host_template. Is that not better than N+1?

Consider the patch Akinobu just posted. In addition to a whole bunch of new code, it adds this:

Quoted 2 lines
drivers/usb/storage/alauda.c +0 -0

...

drivers/usb/storage/alauda.c static int alauda_transport(struct scsi_cmnd *srb, struct us_data *us) · line 1236

Quoted 5 lines
drivers/usb/storage/alauda.c +1 -0
@@ -1232,6 +1233,8 @@ static int alauda_transport(struct scsi_cmnd *srb, struct us_data *us)
 	return USB_STOR_TRANSPORT_FAILED;
 }
 
+static struct scsi_host_template alauda_host_template;

alauda.c was the only driver modified in that patch, and it gained an new scsi_host_template.

For the case where the variants are identical in all respects other than .owner, doesn't it make sense to allow them to share a single scsi_host_template?

The original design of the SCSI stack envisioned multiple drivers, each in control of multiple SCSI hosts. The idea was that scsi_host_template would be associated with the driver and Scsi_Host with the individual host.

Now the kernel has evolved, and we have multiple drivers, some of which contain multiple subdrivers, each in control of multiple SCSI hosts. In this situation we should be flexible enough to allow the scsi_host_template to be associated with either the driver or the subdriver (decision to be made by the driver). When the only variant field is .owner, it makes sense to associate the scsi_host_template with the driver, not force it to be associated with the subdriver.

Quoted 2 lines

The cost is duplication of the .owner field in every Scsi_Host. The savings is a reduction in the number of scsi_host_templates.

So your essential objection is the host template duplication? I know it's a couple of hundred bytes, but surely its dwarfed by all the other stuff you have to duplicate ... the module size of each of these is around 0.25MB, so a couple of hundred bytes would seem a bit insignificant.

James

maintainer
#16 lore raw
Quoted 8 lines · James Bottomley
Quoted 2 lines

The cost is duplication of the .owner field in every Scsi_Host. The savings is a reduction in the number of scsi_host_templates.

So your essential objection is the host template duplication? I know it's a couple of hundred bytes, but surely its dwarfed by all the other stuff you have to duplicate ... the module size of each of these is around 0.25MB, so a couple of hundred bytes would seem a bit insignificant.

Agreed, the size is relatively insignificant. I drop any objection on that account. (But turning things around, what's wrong with duplicating the .owner field in the Scsi_Host structure when we're already duplicating this_id, can_queue, sg_tablesize, and a large bunch of others?)

On the other hand, your code to populate the template copies is a mess. I like Akinobu's approach much better, although I would change a few details:

Declare usb_stor_host_template as "const", and define a new, separate usb_stor_template in usb.c for actual use.

In usb_stor_host_template_init(), do a plain structure assignment rather than memcpy.

Alan Stern

maintainer
#17 lore raw
Quoted 11 lines · Alan Stern
Quoted 8 lines · James Bottomley
Quoted 2 lines

The cost is duplication of the .owner field in every Scsi_Host. The savings is a reduction in the number of scsi_host_templates.

So your essential objection is the host template duplication? I know it's a couple of hundred bytes, but surely its dwarfed by all the other stuff you have to duplicate ... the module size of each of these is around 0.25MB, so a couple of hundred bytes would seem a bit insignificant.

Quoted 5 lines

Agreed, the size is relatively insignificant. I drop any objection on that account. (But turning things around, what's wrong with duplicating the .owner field in the Scsi_Host structure when we're already duplicating this_id, can_queue, sg_tablesize, and a large bunch of others?)

It's really only the pattern wrongness. However, as Rusty says (at length) the best APIs are those that you can't get wrong and having the owner in the template gives that (it's a Level 10 interface). Duplicating the owner in the host gets us to the point where following convention you get it right but nothing prevents abuse (it's a Level 4 interface). I mean, it could be worse (his scale goes down to -10) but I'd rather not move down if we don't have to.

Quoted 2 lines

On the other hand, your code to populate the template copies is a mess. I like Akinobu's approach much better

Unfortunately with that approach, values in the host template can't be overridden.

Quoted 8 lines

, although I would change a few
details:

Declare usb_stor_host_template as "const", and define a new, separate usb_stor_template in usb.c for actual use.

In usb_stor_host_template_init(), do a plain structure assignment rather than memcpy.

OK, so I'm happy with that: it solves the "can't be overridden" problem if you mean a driver should

declare a local template

call usb_stor_host_template_init() on the local template

override whatever they want (or nothing)

call usb_stor_probe1 with the new template.

I also note that you can

#define usb_stor_host_template_init(t, n) __usb_stor_host_template_init((t), (n), THIS_MODULE)

And make it just work for normal usage.

James

#18 lore raw

2015-05-06 6:42 GMT+09:00 James Bottomley
<[email protected]>:

Quoted 50 lines · Alan Stern
Quoted 11 lines · James Bottomley
Quoted 9 lines
Quoted 3 lines

The cost is duplication of the .owner field in every Scsi_Host. The savings is a reduction in the number of scsi_host_templates.

So your essential objection is the host template duplication? I know it's a couple of hundred bytes, but surely its dwarfed by all the other stuff you have to duplicate ... the module size of each of these is around 0.25MB, so a couple of hundred bytes would seem a bit insignificant.

Quoted 5 lines

Agreed, the size is relatively insignificant. I drop any objection on that account. (But turning things around, what's wrong with duplicating the .owner field in the Scsi_Host structure when we're already duplicating this_id, can_queue, sg_tablesize, and a large bunch of others?)

It's really only the pattern wrongness. However, as Rusty says (at length) the best APIs are those that you can't get wrong and having the owner in the template gives that (it's a Level 10 interface). Duplicating the owner in the host gets us to the point where following convention you get it right but nothing prevents abuse (it's a Level 4 interface). I mean, it could be worse (his scale goes down to -10) but I'd rather not move down if we don't have to.

Quoted 2 lines

On the other hand, your code to populate the template copies is a mess. I like Akinobu's approach much better

Unfortunately with that approach, values in the host template can't be overridden.

Quoted 8 lines

, although I would change a few
details:

Declare usb_stor_host_template as "const", and define a new, separate usb_stor_template in usb.c for actual use.

In usb_stor_host_template_init(), do a plain structure assignment rather than memcpy.

OK, so I'm happy with that: it solves the "can't be overridden" problem if you mean a driver should

declare a local template

call usb_stor_host_template_init() on the local template

override whatever they want (or nothing)

Quoted 7 lines

call usb_stor_probe1 with the new template.

I also note that you can

#define usb_stor_host_template_init(t, n) __usb_stor_host_template_init((t), (n), THIS_MODULE)

And make it just work for normal usage.

I have just posted a formal patch. usb_stor_host_template_init() is now called in the newly added module_usb_stor_driver() helper macro which is almost same as module_usb_driver() so that it doesn't need to specify THIS_MODULE for each call.